Etienne, François Anne-Marie Michel

José Marcio

Martin

Tarek, David, Leonardo Valentin

Marcelo Bob

Dan Lucas

Aline Davi

I want to express my gratefulness for having friends I can count on during the hard times

Introduction

The past decade has seen an insurgence of deep learning in the literature and in a large variety of applications. Particularly in computer vision tasks: image classification, segmentation, labeling, etc. much of this progress was due to Convolutional Neural Networks (CNNs). One of the reasons CNNs are particularly interesting for computer vision tasks is their translationequivariance, which is the property of an operator commuting with translations, i.e., translating an input and applying the operator gives the same results as applying the operator to the input and then translating its output. Translation-invariant operators (i.e. their output does not change if you translate the input) are also interesting and can be obtained from a translation equivariant operator by applying some sort of pooling, e.g. taking the supremum of the values of the equivariant output. Convolutions are intrinsically translation-equivariant, and because fully convolutional neural networks (i.e. CNNs without fully connected layers) are built with only convolutions and (in general) point-wise operators, they are either translation-equivariant or translation-invariant.

Equivariance is not a property limited to translations. Indeed, equivariance to other kinds of transformations is frequently sought in computer vision, particularly when the transformations in question form a group or semigroup. It has been a topic in the literature of image processing for quite some time: Scale-spaces [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF][START_REF] Heijmans | Morphological scale-spaces, scale-invariance and lie groups[END_REF][START_REF] Lowe | Object recognition from local scale-invariant features[END_REF][START_REF] Witkin | Scale-space filtering: A new approach to multi-scale description[END_REF][START_REF] Alvarez | Axioms and fundamental equations of image processing[END_REF][START_REF] Pauwels | An extended class of scale-invariant and recursive scale space filters[END_REF] are essentially operators equivariant to a semigroup of scalings, the approach of group morphology [START_REF] Roerdink | Group morphology[END_REF] builds general morphological operators that are equivariant to some group of transformations, and diffusion schemes [START_REF] Weickert | Anisotropic Diffusion in Image Processing[END_REF] are often equivariant to rotations. be associated to equivariant operators on a space of functions. More recently, the method of moving frames has been thoroughly explored [START_REF] Fels | Moving coframes: II. regularization and theoretical foundations[END_REF][START_REF] Hubert | Smooth and algebraic invariants of a group action: local and global constructions[END_REF] and also applied to problems in image processing such as feature detection [START_REF] Tuznik | Affine differential invariants for invariant feature point detection[END_REF], object recognition [START_REF] Calabi | Differential and numerically invariant signature curves applied to object recognition[END_REF] and curve evolution [START_REF] Faugeras | Cartan's moving frame method and its application to the geometry and evolution of curves in the euclidean, affine and projective planes[END_REF].

The object of this thesis is the derivation of group and semigroup equivariant neural networks. There are two main frameworks which were used to study this problem of equivariance in the thesis: The first one is an extension of the scale-equivariant networks proposed in [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], which is the main topic of Chapters 2 and 3. The second one is based on the method of moving frames, which we use to define a new class of group equivariant layers for CNNs and is the main topic of Chapters 4, 5 and 6. What follows is a brief summary of each chapter:

Chapter 1 introduces some important notions that will be used throughout the thesis, namely groups, group equivariance and invariance. We proceed to review the notion of geometric deep learning and discuss the group convolution and steerable networks, which are the most used forms of group equivariant layers for neural networks.

Chapter 2 explores our work with scale-equivariant architectures. Particularly we extend the scale semigroup equivariant network proposed by [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF]. That network uses the Gaussian scale-space to first lift the input image to a suitable multi-scale representation and apply linear scale-equivariant operator to obtain an equivariant network. We find sufficient conditions for other scale-spaces to be used as the first layer and apply those scale-spaces, especially morphological scale-spaces, to image classification and segmentation. A scale-equivariant network is illustrated in Figure 1. We show that morphological scale-spaces can have an advantage over CNNs and Gaussian scale-space based networks when shape is the only available information, as illustrated in Figure 1 (b) and (c). The approaches presented in this chapter were published in Sangalli et al. (2021).

Chapter 3 continues to work on scale-equivariant networks for image segmentation. We build upon the U-Net segmentation model, which attains state-of-the-art results in a variety of segmentation tasks, but is not a priori scale-equivariant, and experiments show empirically that it is not scale-equivariant in practice as shown in Figure 3. We propose the Scale Equivariant U-Net (SEU-Net), illustrated in Figure 2, in order to have a scale-equivariant network similar to U-Net in terms of global architecture. The blocks of downsampling and upsampling of the U-Net are carefully analyzed in order to define their counterparts in the SEU-Net. The SEU-Net is tested in segmentation tasks and is shown to perform well compared to the U-Net and to Scale-Equivariant architectures that do not perform operations such as upsampling in the equivariant part of the network. The contents of this chapter were published in Sangalli et al. (2022b).

Chapter 4 introduces neural networks based on moving frames. It begins by a review of the method of moving frames and how it can be used to obtain differential invariants. The method is applied to obtain differential invariants of images under the action of the group SE (2) i.e. the group of rotations and translations in the plane. In essence, the method finds an orientation at each point of the image based on the local geometry, and the orientation is used to rotate each neighborhood being processed as illustrated in Figure 4. Because of their ability of describing the local geometry at a given scale we use Gaussian derivatives to compute the differential invariants and implement a SE(2)-equivariant architecture, called SE2DINNet. We perform a series of ablation studies in the MNIST dataset and obtain competitive results in the MNIST-Rot dataset. This chapter explores the works published in Sangalli et al. (2022a) Chapter 5 extends the approach of the previous chapter to SE (3)-equivariant networks on volumes. However, because of numerical issues we cannot extend those approaches directly. We note that the computation of the differential invariants can be decomposed in two steps: computation of a moving frame followed by the application of the moving frame to partial derivatives of the image (referred to as n-jets), as illustrated in Figure 5(a). In this chapter we show that keeping only the first moving frame also defines an equivariant network. This leads to the definition of an alternative neural network based on moving frames that, instead of computing the moving frames at each feature map as the previous approach we compute the moving frame at a single image, assumed to be the input image, and apply that moving frame to subsequent layers. A network constructed from this principle is illustrated in Figure 5 (b). In that way, before the activation function the equivariant layers based on the moving frame become linear with respect to the output of the previous layer. The approach is tested on databases of low-resolution medical images for classification and improves most of the benchmarks on those datasets. This chapter correspond to the results published in Sangalli et al. (2023).

Introduction morphological lifting • • • projection f 1 (γ 0 , •) f 1 (γ 1 , •) f 1 (γ 2 , •) f 1 (γ 3 , •) f L (γ 0 , •) f L (γ 1 , •) f L (γ 2 , •) f L (γ 3 , •) f l = σ(f l G h l + b l ) (a)
Introduction Λ H × W × C in H × W × S × C Π H × W × Cout
Chapter 6 discusses some potential applications of neural networks based on moving frames that were not discussed in other chapters because they were not as developed as the previous ones. Particularly we explore (i) a contrast-invariant neural network based on a layer invariant to an affine transformation on the gray-levels of the input image, (ii) a scale, rotation and translation invariant network based on applying a multi-scale Gaussian derivative network to the differential invariants from Chapter 4 and (iii) a SE(3)-equivariant neural network for point clouds based on the method of moving frames. 4: Illustration of the idea behind our moving frame approach for equivariant networks (a) as well as the illustration of our implementation used in Chapter 4. The idea is to use the local structure of the image to obtain a rotation equivariant quantity (e.g. the direction of the gradient, shown in red) and apply filters (shown in blue) like in a convolution but turned with respect to that quantity. When the image is rotated, the red arrows are equal and so the filters. By computing a linear combination of differential invariants, as the first 1 × 1 convolution in (b) we can achieve an effect similar to the illustration in example (a). , where the moving frame is computed at every SE2DIN block, and the approach used in Chapter 5, where the moving frame is computed in the first layer and used subsequent layers. Similar diagrams can be shown in the case the network uses residual connections.

Introduction

Introduction

Introduction en Français

La dernière décennie a été marquée par une montée en popularité de l'apprentissage profond dans la littérature, dans une grande variété d'applications. Par exemple, dans certaines tâches de vision par ordinateur : classification, segmentation, détection d'objets etc. Une grande partie de ce progrès est due aux Réseaux de Neurones Convolutifs (CNNs). L'une des raisons pour lesquelles les CNNs sont particulièrement intéressants est leur équivariance par rapport aux translations, la propriété de commuter avec les translations, c'est-à-dire que la translation de l'entrée suivie de l'application d'un opérateur donne la même sortie qu'appliquer l'opérateur suivi de la translation. Les opérateurs invariants par translation (leur sortie ne change pas si leur entrée est transformée par une translation) sont également intéressants et peuvent être obtenus par un opérateur équivariant par translation suivi d'une fonction de réduction, comme par exemple le supremum de tous les pixels de l'image. Les convolutions sont intrinsèquement équivariantes par translation, et il en va de même pour les réseaux entièrement convolutifs (CNNs sans couches totalement connectés) cars ils sont construits uniquement à partir de convolutions et d'opérateurs ponctuels.

En général, l'équivariance n'est pas limitée aux translations. En effet, des opérateurs équivariants pour d'autres transformations sont souvent recherchés en vision par ordinateur, notamment dans le cas où ces transformations constituent un groupe ou un semi-groupe. L'équivariance est un sujet d'intérêt dans la littérature du traitement d'images depuis quelques années: Les espaces d'échelle [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF][START_REF] Heijmans | Morphological scale-spaces, scale-invariance and lie groups[END_REF][START_REF] Lowe | Object recognition from local scale-invariant features[END_REF][START_REF] Alvarez | Axioms and fundamental equations of image processing[END_REF] sont des opérateurs équivariants par l'action d'un semi-groupe d'échelles, la morphologie de groupes [START_REF] Roerdink | Group morphology[END_REF] construit des opérateurs morphologiques équivariants par une transformation de groupe, et les schémas de diffusion [START_REF] Weickert | Anisotropic Diffusion in Image Processing[END_REF] sont fréquemment équivariants par rotation.

Les avancées récentes dans la littérature de l'apprentissage profond vont au-delà de l'équivariance par translation et cherchent l'équivariance -ou l'invariance -par rapport à d'autres transformations et sur d'autres domaines: les réseaux équivariants par permutation pour les ensembles [START_REF] Zaheer | Deep sets[END_REF], l'équivariance par rotation par 90 • , translations et réflections dans le domaine des images (Cohen and Welling, 2016a), l'équivariance par rapport aux rotations et aux translations en 2D [START_REF] Worrall | Harmonic networks: Deep translation and rotation equivariance[END_REF][START_REF] Cohen | Steerable CNNs[END_REF][START_REF] Shen | Pdo-econvs: Partial differential operator based equivariant convolutions[END_REF] et 3D (Weiler et al., 2018a;[START_REF] Shen | PDO-s3DCNNs: Partial differential operator based steerable 3D CNNs[END_REF][START_REF] Worrall | Cubenet: Equivariance to 3d rotation and translation[END_REF][START_REF] Thomas | Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds[END_REF][START_REF] Thomas | Rotation-invariant point convolution with multiple equivariant alignments[END_REF], l'équivariance par changement d'échelle [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF][START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF] et l'équivariance par transformation de Lorentz pour la physique [START_REF] Bogatskiy | Lorentz group equivariant neural network for particle physics[END_REF]. Équivariance par rapport aux transformations peut améliorer un modèle et réduire le nombre d'échantillons d'entraînement nécessaires pour l'apprentissage si la symétrie par rapport à la famille de transformations est présente dans la base de données.

Dans la littérature mathématique on trouve de nombreuses contributions sur l'équivariance et l'invariance. En particulier, la méthode du repère mobile proposée par Élie Cartan [START_REF] Cartan | La méthode du repere mobile, la théorie des groupes continus, et les espaces généralisés[END_REF] est une méthode pour trouver des invariants différentiels dans une variété. Chaque invariant différentiel peut être associé à un opérateur équivariant dans l'espace de fonctions. Plus récemment, la méthode du repère mobile a été soigneusement explorée [START_REF] Fels | Moving coframes: II. regularization and theoretical foundations[END_REF][START_REF] Hubert | Smooth and algebraic invariants of a group action: local and global constructions[END_REF] et appliquée à certains problèmes de traitement d'images, comme la détection de caractéristiques [START_REF] Tuznik | Affine differential invariants for invariant feature point detection[END_REF], la reconnaissance d'objets [START_REF] Calabi | Differential and numerically invariant signature curves applied to object recognition[END_REF] et l'évolution de courbes [START_REF] Faugeras | Cartan's moving frame method and its application to the geometry and evolution of curves in the euclidean, affine and projective planes[END_REF].

Cette thèse porte sur la conception de réseaux de neurones équivariants par rapport à un groupe ou semi-groupe. L'équivariance a été étudiée dans de deux cadres. Le premier est une extension des réseaux équivariants par changement d'échelle proposés par [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], et il est le principal sujet des chapitres 2 et 3. Le deuxième utilise la méthode du repère mobile pour définir des réseaux équivariants par l'action d'un groupe de Lie. Le deuxième cadre est abordé dans les Chapitres 4, 5 et 6. Ce qui suit est un résumé de chaque chapitre.

Dans le Chapitre 1 nous introduisons des notions importantes qui nous seront utiles tout au long de la thèse : groupes, équivariance et invariance. Ensuite nous effectuons une revue des notions d'apprentissage profond géométrique et nous discutons des convolutions de groupes et des réseaux orientables, qui sont les couches équivariantes par l'action d'un groupe les plus utilisées dans la littérature.

Le Chapitre 2 explore notre travail sur les architectures équivariantes par changement d'échelle. En particulier nous généralisons le réseau équivariant par un semi-groupe d'échelles proposé par [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF]. Cette approche utilise l'espace d'échelle Gaussien pour obtenir une représentation multi-échelle adéquate à partir de l'image d'entrée, ce qui est appelé dépliement ici. Puis, des opérateurs linéaires et équivariants par changement d'échelle sont appliqués pour obtenir un réseau équivariant. Nous trouvons des conditions suffisantes pour utiliser d'autres espaces d'échelles comme dépliement, et nous en utilisons certains, en particulier des espaces morphologiques, dans des tâches de classification et segmentation d'images. Un réseau équivariant par changement d'échelle est illustré sur la Figure 6. Nous montrons que les espaces d'échelle morphologiques peuvent avoir des avantages sur les CNNs et les réseaux basés sur les espaces d'échelle Gaussiens lorsque la forme est la seule information disponible, comme illustré dans les Figures 6 (b) et (c). Les approches présentées dans ce chapitre ont été publiées dans Sangalli et al. (2021).

Le Chapitre 3 poursuit le travail sur les réseaux équivariants par changement d'échelle pour la segmentation d'images. Nous nous appuyons sur le modèle de segmentation U-Net, qui atteint l'état-d'art dans une variété de tâches de segmentation, mais n'est pas équivariant par changement d'échelle ni à priori, ni en pratique, ce qui est confirmé par des expériences, comme montré dans la Figure 8. Nous proposons un U-Net Équivariant par Changement d'échelle (SEU-Net), illustré dans la Figure 7, un réseau équivariant par changement d'échelle qui est comparable à U-Net pour son architecture globale. Les blocs de sous-échantillonnage et suréchantillonnage de U-Net sont méticuleusement analysés pour définir ses analogues dans le SEU-Net. Le SEU-Net est testé dans des tâches de segmentation et il est montré qu'il obtient une bonne performance en comparaison à U-Net et à des architectures qui ne font pas de suréchantillonnage dans la partie équivariante du réseau. Le contenu de ce chapitre été publié dans Sangalli et al. (2022b). Le Chapitre 4 introduit des réseaux de neurones basés sur la méthode du repère mobile. Nous appliquons la méthode pour trouver des invariants différentiels de l'action de SE (2), le groupe des rotations et translations en deux dimensions. En essence, pour chaque point de l'image, la méthode trouve une orientation basée sur la géométrie locale, puis l'orientation est utilisée pour faire tourner chaque voisinage en train d'être traité, comme illustré par la Figure 9. Nous utilisons les dérivées Gaussiennes pour calculer les invariants différentiels grâce à leur capacité de décrire la géométrie locale, et nous implémentons une architecture (appelée SE2DINNet) équivariante par rapport à l'action de SE (2). Nous faisons une série d'études d'ablation et nous obtenons des résultats compétitifs dans la base de données MNIST-Rot. Ce chapitre explore les travaux publiés dans Sangalli et al. (2022a) Le Chapitre 5 étend l'approche du chapitre précèdent aux réseaux équivariants par l'action de SE (3) pour les volumes. Toutefois, en raisons des problèmes numériques causés par la forme polynomiale des invariants, nous ne pouvons pas étendre ces approches directement. Nous précisons que le calcul des invariants peut être décomposé en deux parties: le calcul du repère mobile suivi par l'application de ce repère mobile aux dérivées (appelées n-jets), comme illustré par la Figure 10(a). Dans ce chapitre nous montrons que nous pouvons garder seulement le repère mobile calculé dans la première couche d'un réseau et l'utiliser pour définir un réseau équivariant. Cela nous ramène à une définition d'un réseau équivariant alternatif basé sur les repères mobiles. Un réseau construit à partir de ce principe est illustré dans la Figure 10(b). De cette façon, avant l'application de la fonction d'activation les couches de ce réseau sont linéaires par rapport à la sortie de la couche précédente. L'approche est testée pour la classification de bases d'images médicales en faible résolution, et il améliore les résultats précédents sur ces bases de données. Ce chapitre correspond aux résultats publiés dans Sangalli et al. (2023).

dépliement morphologique • • • projection f 1 (γ 0 , •) f 1 (γ 1 , •) f 1 (γ 2 , •) f 1 (γ 3 , •) f L (γ 0 , •) f L (γ 1 , •) f L (γ 2 , •) f L (γ 3 , •) f l = σ(f l G h l + b l ) (a)
Λ H × W × C in H × W × S × C Π H × W × Cout H/2 × W/2 × S × 2C
Le Chapitre 6 examine des applications potentielles des réseaux de neurones basés sur les repères mobiles. En particulier, dans le Chapitre 6 nous explorons : (i) un réseau invariant par changement de contraste basé sur une couche invariante par transformation affine des niveaux de gris de l'image d'entrée, (ii) un réseau invariant par changement d'échelle, rotation et translation basé sur l'application des dérivées Gaussiennes multi-échelles aux invariants différentiels du Chapitre 4 et (iii) un réseau de neurones équivariant par l'action de SE(3) pour les nuages de points basé sur la méthode du repère mobile. The success of convolutional neural networks is partially determined by its ability to consider local information and to exploit the translational symmetry intrinsic to many computer vision tasks. Translational symmetry is not, however, the only interesting symmetry present in computer vision tasks. Consider the case of a classification problem like in Figure 1.1, it would be desirable that the image be classified as "cat" independently of perspective changes. "Cat" "Cat" "Dog" Figure 1.1: An image and its transformations in a classification task. In a fully convolutional network, a correct classification of an image implies the correct classification of all its translated versions (disregarding border effects). Since CNNs are not equivariant to rotations and scalings, for example, a correct classification of an image does not imply correct classification of its rotated and scaled counterpart.

More general than invariance is the property of equivariance. Bluntly speaking an operator φ is equivariant to a family of transformations if it commutes with each transformation of the family. A case of particular interest is when φ is equivariant with respect to a family of group actions (or semigroup actions, as will be seen in Chapters 2 and 3). Figure 1.2 shows an example of equivariance. Segmentation is a task where equivariance can be particularly important, as transforming an input image geometrically should have the same effect on its segmentation maps. In the case of image classification, for example, invariance can be interesting if the data contains the relevant symmetries, as illustrated in Figure 1.1. Invariant networks can be obtained from equivariant layers followed by pooling functions. The field of Geometric Deep Learning [START_REF] Bronstein | Geometric deep learning: Grids, groups, graphs, geodesics, and gauges[END_REF] is a field of deep learning that incorporates symmetry in the neural networks. Group-Equivariant Neural Networks, in particular, are neural networks that are equivariant to certain group transformations. Despite being quite recent, many approaches to group equivariant neural networks exist for different groups: For images we have networks equivariant to 2D rotations (Cohen and Welling, 2016a;[START_REF] Worrall | Harmonic networks: Deep translation and rotation equivariance[END_REF][START_REF] Weiler | Learning steerable filters for rotation equivariant CNNs[END_REF] and scale [START_REF] Zhu | Scale-equivariant neural networks with decomposed convolutional filters[END_REF][START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF][START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF]; for point clouds there are 3D rotations equivariant networks [START_REF] Thomas | Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds[END_REF], for sets there are permutation equivariant networks [START_REF] Guttenberg | Permutation-equivariant neural networks applied to dynamics prediction[END_REF], physicsinformed networks equivariant to Lorentz transformations [START_REF] Bogatskiy | Lorentz group equivariant neural network for particle physics[END_REF] to name a few.

This chapter serves as an introduction to many of the topics that will be viewed during this thesis. It is organized as follows: A brief review of the definition and some properties of smooth manifolds and their tangent spaces is given in Section 1.2. Groups and group equivariance, one of the main topics in this thesis, are introduced and discussed in Section 1.3. Geometric deep learning, the area that concerns, among other things, equivariant neural networks, is discussed in Section 1.4. Fiber bundles are a concept from topology and differential geometry that will be useful in some parts of this thesis and is therefore introduced in Section 1.5. The theoretical framework behind many of the group equivariant neural networks, the group convolution, is introduced in Section 1.6, including the framework of classical convolution, the

Smooth Manifolds

Later in this chapter and in following chapters many objects are described in terms of smooth manifolds [START_REF] Lee | Smooth manifolds[END_REF]. This section serves merely to recall the definition and introduce some of the notations used when discussing manifolds, specially those topics which will be useful in the rest of the thesis. Essentially manifolds are objects that locally look like an Euclidean space. A topological manifold is a space which has the local topology of a Euclidean space: Definition 1 (Topological manifold). A topological manifold M of dimension d is a topological space which is locally homeomorphic to R d , i.e. for every p ∈ M there exists an open set U ⊆ M containing p and an open set V ⊆ R d such that U and V are homeomorphic.

A topological manifold alone is not sufficient to allow for things such as differentiation. For that, additional structure will be necessary. A smooth manifold M is a topological manifold with the extra structure that allows the use of differential operations on it. In order to define smooth manifolds we start by defining charts and atlases. A smooth manifold may sometimes be described in terms of local coordinates. Given a chart (U, φ), a point p ∈ U is written in local coordinates as x = φ(p) = (x 1 (p), x 2 (p), . . . , x d (p)).

Let f : M → R be a function. We say that f is smooth at point p ∈ M if there is a chart (U, φ) with p ∈ U , such that f • φ -1 is smooth, i.e. infinitely continuously differentiable, at φ(p), as a function from R d to R. If f is smooth at all p ∈ M then f is smooth, also referred to as a smooth map. The set of smooth functions is denoted C ∞ (M, R).

Given that differentiability is defined, a way to compute derivatives in M is possible. Indeed, there are several definitions of derivative that apply to smooth manifolds, but here we only look at the tangent space sense of derivative, which generalizes the notion of directional derivative of scalar functions in R d to the manifold M. Given a smooth manifold M and a point p ∈ M, a linear map v :

C ∞ (M, R) → R satisfying the Leibniz rule, ∀f 1 , f 2 ∈ C ∞ (M, R) v(f 1 f 2 ) = f 1 (p)v(f 2 ) + v(f 1 )f 2 (p) (1.1)
is called a derivation at p. For all p ∈ M, the set of derivations at p forms an d-dimensional real vector space which is denoted T p M and is called the tangent space at p and its elements can also be called tangent vectors. In the Euclidean space, an operator satisfying (1.1) is the derivative along an specific direction, and this definition generalizes this to derivatives in general smooth manifolds.

Given a chart (U, φ) describing local coordinates x = φ(p) = (x 1 (p), . . . , x n (p)), we define

∂ ∂x i p for p ∈ M for all i ∈ {1, . . . , d} and f ∈ C ∞ (M, R), as ∂ ∂x i p f = ∂ ∂x i φ(p) (f • φ -1 ) (1.2)
where A vector field is a mapping that maps each point p ∈ M to a vector in T p M. A more precise definition of the vector field will be given when talking about fiber bundles. In that case, we can define ∂ ∂x i , for i ∈ {1, . . . , d} by associating each p ∈ M to ∂ ∂x i p .

Group Equivariance

Groups are objects that arise with the study of symmetry. One can define groups as follows:

Definition 4 (Group). A tuple (G, •) where G is a set and " • " : G × G → G a function is a group if • ∀a, b, c ∈ G, (a • b) • c = a • (b • c) (Associativity) • ∃e ∈ G, ∀a ∈ G e • a = a • e = a (Neutral Element) • ∀a ∈ G, ∃a -1 ∈ G, a • a -1 = a -1 • a = e (Inverse)
Groups are often used to model transformations on objects, because the composition of transformations defines an associative product, and they may be defined in terms of symmetries of objects, for example given the four vertices of a square p 0 = (-1, -1), p 1 = (1, -1), p 2 = Figure 1.4: Illustration of the group p4. It can be seen as rotations by multiples of 90 • on these points. As such it has a cyclic structure: if r is a rotation by 90 • , its elements are p4 = {e, r, r 2 , r 3 } where e is the identity.

(-1, 1), p 3 = (1, 1), the transformations that map the set {p 0 , p 1 , p 2 , p 3 } to itself and preserve orientation (i.e. in this example orientation is the relative ordering when passing through the vertices in counter-clockwise direction) are described at Figure 1.4 by the group p4, which is also isomorphic to Z 4 = Z/4Z, the integers modulo four.

Let us also discuss groups with additional structure that will be seen during the thesis. A group G which is also a topological space, for with the operators of product (a, b) → a • b and inverse a → a -1 are continuous is a topological group. If additionally G is a smooth manifold [START_REF] Lee | Smooth manifolds[END_REF] and the product and inverse are smooth maps, then G is a Lie Group.

Definition 5 (Lie Group). A Lie group is a group (G, •) such that G is a smooth manifold and the product (g, h) → g • h and g → g -1 are both smooth functions.

A Lie group can model a continuous and smooth transformation between equivalent objects, an example could be obtained by the rotations of the circle, as they can be obtained by continuously rotating the circle, in contrast to the rotations p4 that need to be discretized.

The main way which groups are used, specially within the context of computer vision, it through group actions.

Definition 6 (Group Action). Given a set X and a group G, a group action action is a map π : G × X → X such that π(e, x) = x for all x ∈ X where e is the identity in G, and satisfies one of the following properties:

• ∀g, h ∈ G, ∀x ∈ X π(g, π(h, x)) = π(g • h, x) in which case it is called a left group action; • ∀g, h ∈ G, ∀x ∈ X π(g, π(h, x)) = π(h•g, x)
in which case it is called a right group action.

We often denote a group action π using subscripts, π g (x) := π(g, x), ∀g ∈ G, x ∈ X, and π g is also used to denote the function x → π g (x). In that sense, π is a group action if and only if the operators π g , g ∈ G together with the composition • form a group such that the map g → π g is a group homomorphism. If G is a Lie Group and X a smooth manifold, π is a Lie group action if it is a group action and a smooth map. If the group action being used is clear from the context we denote simply π g

(x) = g • x, ∀x ∈ X, g ∈ G.
Group actions model a wide range of geometric transformations e.g. translations, rotations, or the operators in Chapter 2 called homothecies or scalings, etc. For example, a very interesting Lie group is the group of invertible d × d real matrices GL(d) equipped with the matrix multiplication rule. Any subset of invertible matrices which is closed under multiplication and inversion is also a group and if it has a smooth manifold structure it is also a Lie group. This gives rise to many interesting Lie groups, for example: The elements of SO( 2) for example may be written as matrices of the form .3) which are referred to as rotations matrices. The group p4 is isomorphic to a subgroup of SO( 2), which we identify p4

∀θ ∈ R, R θ = cos θ -sin θ sin θ cos θ , ( 1
= {R 0 , R π 2 , R π , R3π 2
} and we can verify it maps the vertices of the square to the to the vertices of the square and preserves orientations, but it is not a Lie subgroup.

Groups which contain translations as well as invertible linear transformations can also be written as a subgroup of GL(d) suppose that G R d is the product of a group of matrices G ⊆ GL(d) and translations, then its composition has the form

(A, v) • (B, w) = (AB, Aw + v).
(1.4)

However, if we identify its elements with the d

+ 1 × d + 1 matrices (A, v) = A v 0 1 , (1.5)
we end with the same composition group, meaning that G R d can be viewed as a subgroup of GL(d + 1). Interesting groups that can be represented this way are

• affine transformations Aff(d) = GL(d) R d ;
• equi-affine transformations Equi-Aff(d) = SL(d) R d of area-preserving transformations; Another matrix group is a group of scalings which can be written as S = R >0 = (0, +∞) and is already given matrix form for 1 × 1 matrices, but can be a matrix of arbitrary dimension by writing S ∼ = {s • I d |s > 0}. Furthermore S can be composed with translations R d to form a group of scales and translations S R d . In many computer vision tasks, symmetry to transformations can be found in the data e.g. in segmentation tasks, the prediction of an image spatially shifted by v ∈ Z 2 at the pixel position x should result in the same class as the pixel in xv in the original image. In particular, the action of a group on Z d also defines an action on the space of functions mapping This action is defined for all f ∈ Y X by

(g • f )(x) = π Y X g [f ](x) = f (π X g -1 [x]) = f (g -1 • x). (1.6)
For example, the translations on X induce translations on Y X , as rotations by p4 define rotations by multiples of 90 • in images. We can also view f as a discretization of a signal on X = R d . In that way, we can apply the action of Lie groups such as Aff(d), SO(d) or S on the points in X, mapping the grid X to another grid in R d and use interpolation to approximate a transformed image in X. Examples of the aforementioned actions are shown in Figure 1.5. By ensuring that a neural network treats transformed samples in an equivalent manner we can effectively reduce the search space and improve generalization. To include this constraint in an operator we require that it is G-equivariant.

Definition 7 (Equivariance). Let G be a group acting on sets X and Y with actions denoted by1 π X : G × X → X and π Y : G × Y → Y , respectively. We say that an operator φ :

X → Y is G-equivariant with respect to π X and π Y if for all g ∈ G ∀g ∈ G, φ • π X g = π Y g • φ.
(1.7)

In most cases the actions π X and π Y are clear from the context and we say simply that φ is G-equivariant.

The example in Figure 1.2 illustrates the concept of equivariant operators. Note that it is not necessary for the actions on the input and output domains to match.

Invariance is a particular case of equivariance and can be obtained when ∀g ∈ G, π g = id, i.e. an operator φ is G-invariant with respect to π if

φ • π g = id • φ = φ.
(1.8)

Again if π is clear from the context we say that φ is G-invariant.

The main concern in this thesis is about constructing, training and evaluating neural networks that are equivariant with respect to some transformation.

Geometric Deep Learning

Group equivariant neural networks generally fall under the broader field of Geometric Deep Learning. We assume that inputs are signals f : Ω → X defined on some domain Ω. The domain Ω can be several different things, which are assumed to fall into one of the so-called 5 G's [START_REF] Bronstein | Geometric deep learning: Grids, groups, graphs, geodesics, and gauges[END_REF]: Grids, Graphs, Groups, Geodesics and Gauges. A neural network in this framework will be a function of the φ : X Ω → Y , where X and Y are finite-dimensional vector spaces.

The models presented in this thesis follow the Geometric Deep Learning Blueprint (Bronstein et al., 2021) (GDLBP), which characterizes some of the fundamental building blocks of geometric deep learning.

It is assumed that we aim for the network φ to be equivariant to the action of a group G. The group G acts on the spaces X, Y and Ω with actions denoted by π. On spaces of functions of these sets the action is given by (1.6).

The blocks characterized by GDLBP are:

• Linear G-equivariant layer: A linear operator B :

X Ω → Y Ω which satisfies B • π g = π g • B • Equivariant Non-Linearity: A pointwise function σ : X → Y satisfying σ • π g = π g • σ • Local Pooling: P : X Ω → X Ω such that Ω ⊆ Ω • G-invariant layer: (also referred to as projection) An operator A : Y Ω → Y such that A • π g = A
Assuming that besides the projection all of the parts above are G-equivariant, a G-equivariant function can then be constructed by composition of these blocks, particularly

φ = σ l • B l • P l-1 • • • • • P 1 • σ 1 • B 1 .
(1.9) and a G-invariant function is given by A • φ.

Fiber Bundles

The general theory behind group convolutional neural networks can be based on considering spaces as fiber bundles [START_REF] Cohen | A general theory of equivariant CNNs on homogeneous spaces[END_REF]. Fiber bundles are objects that locally look like the Cartesian product of two topological spaces, but globally their topology is not necessarily that of the Cartesian product. Formally a fiber bundle is given by Definition 8.

Definition 8 (Fiber bundle). A fiber bundle is a tuple (E, B, p, F ) where E, B and F are topological spaces and p : E → B is a continuous surjection satisfying the condition of local triviality: for every x ∈ B there is an open neighborhood U ⊆ B containing x and a homeomorphism ϕ : p -1 (U ) → U × F , referred to as trivialization, such that for every y ∈ p -1 (U )

proj 1 • ϕ(y) = p(y) (1.10)
where proj 1 is the projection in the first coordinate.

The spaces E, B and F are referred to, respectively, as the total space, the base space and the canonical fiber.

The fiber bundle is such that locally, E "looks like" a Cartesian product of B and F . As examples let us consider the cylinder and the Möbius band. The cylinder can be written as the product

S 1 × [-1, 1] where S 1 = {x ∈ R 2 | x = 1}
is the circle. The base space and canonical fiber are respectively S 1 and [-1, 1] and as a projection we can use p = proj 1 and in that case the trivialization is just the identity. The cylinder is a trivial fiber bundle, because it coincides with a Cartesian product. Now for the Möbius band, let the total space be E = [0, 1]×[-1, 1]/ ∼ where the equivalence relation ∼ is given by (x, y) ∼ (x , y ) if and only if (x, y) = (x , y ), or x = 0, x = 1, y = -y , or x = 1, x = 0, y = -y for all x ∈ R. The quotient space E can be interpreted as giving a half-twist to one vertical end of [0, 1] × [-1, 1] and gluing them together, giving rise to the example in Figure 1.6(b). We can define the projection p : E → S 1 , as q(x, y) → e 2iπx where q maps (x, y) ∈ [0, 1] × [-1, 1] to its equivalence class. This projection is well defined because it maps 0 and 1 to the same point on the circle. To find a local trivialization notice that given a point (x, y) [START_REF] Sangalli | Scale equivariant neural networks with morphological scale-spaces[END_REF] such that q is a homeomorphism restricted to U and then the inverse of q can be used to construct a trivialization.

∈ [0, 1] × [-1, 1] one can find a small enough open set U ⊂ [0, 1] × [-1,
Definition 9 (Section of a fiber bundle). A section on a fiber bundle is a function assigning to each x ∈ B an element in its fiber own p -1 (x). Formally, a section is a continuous function with the property p • s = id B .

A prominent example of a fiber bundle is the tangent bundle. In a manifold M, associated to each point x ∈ M is the tangent space T x M . Keeping in mind that the tangent spaces are isomorphic between themselves, their disjoint union T M = x∈M T x M forms the total space of a fiber bundle. The projection function p is simply defined as p(z) = x for all z ∈ T x M, x ∈ M, the base is M and the fiber is any vector space V ∼ = T x M isomorphic to T x M for any x ∈ M. More generally we can define a vector bundle if we use an arbitrary vector space V as the fiber. This will play an interesting role in group convolutions as the application of a group representation on V defines a group representation of the sections of the total space E. A vector field can now be more precisely defined as a section in T M. Fiber bundles will be used later in this chapter to talk about the group convolution in homogeneous spaces. In Chapter 4 we use the jet-bundle, which is a fiber bundle that contains differential information of a manifold, but the study of fiber bundles is not necessary in that case because in our cases of interest the jet-bundle reduces to a Cartesian product.

Chapter 1. Group Equivariant Networks

Group Convolution

As outlined in the GDLBP, a linear-equivariant operator is used as a building block in many equivariant neural networks approaches. In the case of regular CNN this operator is the convolution, but more generally the group convolution can be defined for group-equivariant neural networks. We begin this section by analyzing some aspects of the convolution in Euclidean spaces before we examine the general case of group-convolution.

Special Case: Classical Convolution

Linear operators and have a range of interesting properties for a neural network. Moreover, in our case we are particularly interested in having equivariant neural networks. Therefore let us start our analysis by examining the most-used example of a group-equivariant operator in deep learning: the convolution.

A convolution is an operator defined for signals on a domain E, in which we assume either

E = R d or E = Z d . Let us analyze, for instance the convolution of a multivariate signal f : E → R C by a filter w : E → R C×C , which is defined as (f * w)(x) = R d w(x -y) • f (y)dy, (1.11) if E = R d and (f * w)(x) = y∈E w(x -y) • f (y), (1.12) if E = Z d
, where in both cases • stands for matrix product and the output is a multivariate signal f * w :

E → R C . Alternatively, for each output coordinate o, 1 ≤ o ≤ C (f * w)(x) o = C i=1 R d w(x -y) i,o • f (y) i dy, (1.13) for E = R d , (f * w)(x) o = C i=1 y∈E w(x -y) i,o • f (y) i , (1.14)
for E = Z d . Note that these definitions hold provided the functions y → w(x -y) • f (y) and y → w(x -y) i,o • f (y) i are integrable for almost every x ∈ R d , and the sums

y∈E w(x -y) • f (y) and y∈E w(x -y) i,o • f (y) i converge for all x ∈ R d .
A convolution can be viewed as a the filter w being placed through a translation at every point in the domain, followed by the application of the filter through a scalar product.

Let us focus on the discrete case E = Z d . In the continuous case, computations are similar, albeit requiring a bit more formalism. Let us define the p-norms, for p > 1 as

f p = y∈E |f (y)| p 1 p and f ∞ = sup y∈E |f (y)| and p = {f : E → R| f p < ∞}.
From now on we assume that 1 ≤ i ≤ C and 1 ≤ o ≤ C , f i ∈ p and w i,o ∈ q for some p > 1 or p = ∞ and q such that 1 p + 1 q = 1. By Hölder's Inequality, the sums in (1.14) converge for all y ∈ Z d .

We define translation of signals on E by vectors v ∈ E as the action (1.6), i.e.

∀v, x ∈ E, π v [f ](x) = f (x -v), (1.15)
where we use the same symbol to denote the actions on both (R C ) E and (R C ) E . It is straightforward to see that for a fixed w, the operator defined by (1.12) is linear and translation equivariant, moreover the converse is also true:

if ϕ : (R C ) E → (R C ) E is such that • ϕ is linear • π v • ϕ = ϕ • π v
then ϕ is a convolution by some filter w. What follows is a proof.

Proof. Assume that ϕ is linear and translation-equivariant. In the case where E = Z d , we can characterize a signal f by computing its impulse responses, i.e. linear functionals given as convolutions by the signals δ y e i where δ y is given by

δ y (x) = 1, if x = y 0, if x = y (1.16
) and e i is the i-th vector of the canonical bases on R m i.e. equal to 1 at the i-th coordinate and 0 at the others. Applying this to f gives us (f * δ y e

i )(x) = f (x -y) i , therefore for all x ∈ E f (x) = C i=1 (f * δ 0 )(x)e i = C i=1 y∈E δ 0 (x -y)e i • f (y) = C i=1 y∈E δ y (x)e i • f (y).
(1.17)

For all x ∈ E, 1 ≤ i ≤ C, 1 ≤ j ≤ C let h ij (x, y) = ϕ(e i δ y ) j (x) = ϕ(δ y
) by linearity we obtain the following (1.18) where h(x, y) i denotes the i-th row of h(x, y). Since ϕ is translation-equivariant, we have h i (x, y) = ϕ(e i , δ y (x)) j = ϕ(e i π y [δ 0 ](x)) j = ϕ(e i , δ 0 )(x -y) j = h ij (0, xy), (1.19) defining w(y) := h(0, y) we obtain

ϕ(f )(x) = ϕ C i=1 y∈E (e i δ y ) • f (y) (x) = C i=1 y∈E ϕ(e i δ y )f (y) (x) = C i=1 y∈E h(x, y) i • f (y) = y∈E h(x, y) • f (y),
ϕ(f )(x) = y∈E h(x, y) • f (y) = y∈E w(x -y) • f (y) = (f * w)(x).
(1.20)

Haar Measure

The group convolution will be defined by means of an integral for signals on the group, like the convolution is defined as an integral of signals on E. For such we define the Haar Measure. Given a group G, a set S ⊆ G and g ∈ G let us define the left-translation of S by g as

g • S = {g • s|s ∈ S}. (1.21)
Let G be a topological group and σ a σ-algebra generated by the Borel sets of G. A measure

µ : σ → R + is called left-invariant if ∀S ∈ σ we have 2 µ(g • S) = µ(S).
(1.22)

Haar's theorem states that, up to a multiplicative constant, there is a unique left-invariant measure that is finite on compact sets, outer regular on Borel sets and inner regular on open sets. Such a measure is called Haar measure. In the following subsection, integration in the group G is performed with respect to a Haar measure. The Lebesgue measure is an example of Haar measure of R d , invariant to translation. When G = GL(n) is a matrix group, the Haar measure can be obtained by dµ(A) = 1 |det A| d dA, where dA here denotes the Lebesgue measure of GL(n). Note that it must also be the case for subgroups of GL(n).

Group Convolution

The group convolution generalizes the convolution in the case G = E is a more general group. Particularly let us assume that G is a locally compact topological group. Indeed, we begin with a signal f : G → R C and wish to compute an operator that is both linear and G-equivariant, where the group action of G on signals like f is given by

∀g, h ∈ G, π g [f ](h) = (g • f )(h) := f (g -1 h).
(1.23)

The group convolution operators can be deduced from the Haar measure using a definition similar to the convolution. Assuming that f : E → R and w : E → R are signals on the group, by looking at (1.11) substitute the expression xy for the translation group by the expression x • y -1 in a more general group. 3 A convolution-like operator for a topological group G can be written in terms of the Haar measure .24) and indeed we can show that it is equivariant with respect to transformations [g

(f * w)(g) = G w(h -1 g)f (h)dh = G w(h -1 )f (gh)dh, ( 1
• f ](g) = f (g -1 g) ([g • f ] * w)(g) = G w(h -1 g)[g • f ](h)dh = G w(h -1 g)f (g -1 h)dh = G w(h -1 g -1 g)f (h)dh = (f * w)(g -1 g) = [g • (f * w)](g).
(1.25)

In the case where G is a discrete group the Haar measure is the counting measure and the integral in (1.24) becomes a sum

(w * f )(g) = g ∈G w(g -1 g)f (g ).
(1.26)

Circling back to where we started, if we replace G by Z d and x = g, y = g in (1.26) then g -1 g becomes xy and we obtain (1.12). Analogously if we assume G = R d in the continuous definition (1.11), it becomes the classical continuous convolution. While the convolution can be viewed as a filter being translated over the Euclidean domain and applied via a scalar product, the group convolution can be viewed as a filter being transformed according to the group composition law before begin applied via a scalar product.

As an example of the group convolution consider one of the first group-equivariant convolutional networks that is equivariant to something besides translations in the literature, the discrete rotations and translations equivariant neural network 4 proposed by Cohen and Welling (2016a). The group p4 is identified by the group of 90 • rotations on the plane, making the group G = p4 × Z 2 a group of translations and rotations on the (discrete) plane Z 2 . The group multiplication on G is given, for (R

1 , v 1 ), (R 2 , v 2 ) ∈ p4 × Z 2 = G (R 1 , v 1 ) • (R 2 , v 2 ) = (R 1 • R 2 , R 1 v 2 + v 1 )
, (1.27) where R 1 , R 2 are identified by rotation matrices of angles multiples of 90 w(P -1 R, x -P -1 y)f (P, y).

(1.28)

So in comparison with the Z 2 convolution, which translates a filter by every possible vector in Z 2 , the p4 × Z 2 , convolutions translate and rotate by multiples of 90 • the filters and because filters are in p4 × Z 2 the rotation is accompanied by a cyclic translation in the first coordinate. An issue is that input and output signals of neural networks are not often in a domain G when G = Z d , but usually they have the form f : Z d → R C or f : R d → R C . In the previous example, inputs are supposed to be images on Z 2 and because G = p4 × Z 2 we can apply the so-called lifting to these operators, which we will talk about in more detail in the next chapter, but in this case a valid way to compute the network would be to use a signal f : G → R C such that ∀P ∈ p4, x ∈ Z 2 , f (P, x) = f (x) as input. In this way we can apply a group convolutional neural network to an input f : Z 2 → R C . To show that this architecture is equivariant first we must know how the group acts on f , which is ((P, y) • f )(x) = (P x + y). We can verify that the lifting is equivariant ḡ • f = g • f , and moreover if the network is constructed only from group convolutions and point-wise functions it becomes evident that the result is equivariant as the composition of equivariant operators is equivariant. The visualization of the group action on feature maps of a network like this is seen in Figure 1 in Cohen and Welling (2016a).

Steerable Filters

When discussing steerable networks, group actions generally have additional structure. Specifically we have a group representation acting on the function spaces. A group representation is a mapping of group elements into linear maps i.e. τ : G → GL(V ), where GL(V ) is the set of linear automorphisms of a vector space V , such that τ (g) • τ (h) = τ (g • h). We can check that a group representation defines a group action (g, y) → τ (g)y for g ∈ G and y ∈ R C . In the following we assume that either

E = Z d or E = R d . A direct sum of group representations τ 1 : G → GL(V 1 ), τ 2 : G → GL(V 2 ) is a new representation in GL(V 1 ⊕ V 2 )
given by a block diagonal matrix as follows .29) Steerable CNNs work by defining equivariant filter banks. More specifically, they are usually defined by means of a classical convolution by an equivariant filter bank. This leads to equivariance to the restrictions to groups G that are the product of (E, +) with a group (H, •) that fixes the origin 0. In this sense, we view a filter bank as a convolutional filter w : E → R C×C . Equivariant filter banks refer to filter banks with the following property: for all h ∈ H, f :

τ (g) = τ 1 (g) ⊕ τ 2 (g) = τ 1 (g) 0 0 τ 2 (g) . ( 1
E → R C , x ∈ E τ (h)(f * w)(x) = (π h f * w)(x) (1.30)
where τ is a representation of H and π is the action (1.6), which is also a representation. More specifically τ is a representation on R q and π is a representation for the space of functions (R

C ) E . A subspace W ⊆ V is called invariant if the representation τ (g) maps W to W i.e. τ (g)•W = W for all g ∈ G.
A representation τ is called irreducible if it only has no non-trivial invariant subspaces, i.e. if the only invariant subspaces are V and {0}. The term irrep is used for short to refer to an irreducible representation.

If G is finite or if G is a semisimple Lie group we have that any representation can be broken into a direct sum of irreps and a change of basis, that is, for some A ∈ GL(V ), and for all g ∈ G A

• τ (g) • A -1 = ϕ 1 (g) ⊕ ϕ 2 (g) ⊕ • • • ⊕ ϕ k (g) =      ϕ 1 (g) ϕ 2 (g) . . . ϕ k (g)      (1.31)
where ϕ i , i ∈ {1, . . . , k} are irreps. Steerable CNNs profit from (1.31) to create equivariant operators based on multiple operators filter banks that are equivariant with respect to multiple irreps, i.e. the representations in the left-hand-side of (1.30) are given by different irreps.

Let us consider a simplified example to illustrate how a steerable network may be computed from the definitions above. Let us work again to construct a p4×Z 2 -equivariant linear operator.

In that way we have H = p4. The input actions of p4 are rotations on E and the action of G = p4 × Z 2 is the action π g defined in the last subsection. Let us define a representation of p4 in R 3 given by τ (R) = ϕ 1 (R) ⊕ ϕ 2 (R), where ϕ 1 (R) = I 1 = 1 is the identity in R and ϕ 2 (R) = R used R as a rotation matrix by multiples of 90 • in R 2 . We can make a steerable filter basis by making a filter 2,3,4,5,6 where the filters are given by functions with support in a 3 × 3 cross in Z 2 given by.

w i = (w i 1 , w i 2 , w i 3 ) : Z 2 → R 3 i = 1,
w 1 1 = 0 1 0 1 1 1 0 1 0 w 1 2 = 0 0 0 0 0 0 0 0 0 w 1 3 = 0 0 0 0 0 0 0 0 0 w 2 1 = 0 0 0 0 1 0 0 0 0 w 2 2 = 0 0 0 0 0 0 0 0 0 w 2 3 = 0 0 0 0 0 0 0 0 0 w 3 1 = 0 0 0 0 0 0 0 0 0 w 3 2 = 0 0 0 0 0 1 0 0 0 w 3 3 = 0 1 0 0 0 0 0 0 0 w 4 1 = 0 0 0 0 0 0 0 0 0 w 4 2 = 0 1 0 0 0 0 0 0 0 w 4 3 = 0 0 0 1 0 0 0 0 0 w 5 1 = 0 0 0 0 0 0 0 0 0 w 5 2 = 0 0 0 1 0 0 0 0 0 w 5 3 = 0 0 0 0 0 0 0 1 0 w 6 1 = 0 0 0 0 0 0 0 0 0 w 6 2 = 0 0 0 0 0 0 0 1 0 w 6 3 = 0 0 0 0 0 1 0 0 0 (1.32) Given some w = 6 i=1 α i w i , the convolution f * w satisfies (π R,v f * w)(x) = τ (R)(f * w)(x -v).
( 1.33) This is illustrated in Figure 1.7. Some more details are needed in order to define a full steerable CNN, as pointwise non-linearties may not be equivariant to the representation τ , but we do not go into detail here.

f * w π R f (x) = f (R -1 x) τ (R)(f * w)(R -1 x) π R (f ) * w Figure 1
.7: Illustration of the equivariance of steerable convolutions. The feature maps f * w in R 3 can be seen as a scalar, represented by the gray level of the images on the right, and a vector, represented by a red arrow (only shown for some pixels). Rotating the image f by a multiple of 90 • will also rotate the feature maps, but leaves the scalar values unchanged, and the vector part is rotated as well.

Group Convolution in Homogeneous Spaces

General theories formalizing group-equivariant neural networks [START_REF] Cohen | A general theory of equivariant CNNs on homogeneous spaces[END_REF][START_REF] Kondor | On the generalization of equivariance and convolution in neural networks to the action of compact groups[END_REF] rely on the definition of group convolution on homogeneous spaces and can unify the concepts discussed above. A set X, acted upon by G is called an homogeneous space if for every x, y ∈ X there exists g ∈ G such that g • x = y (in this case we also say that the action of G is transitive). A particular example of an homogeneous space is the set of cosets G/H = {gH|g ∈ H} for a given normal subgroup H ≤ G, i.e. a subgroup H such that ∀h ∈ H, g ∈ G, g • h • g -1 ∈ H. In this section we review the approach of [START_REF] Cohen | A general theory of equivariant CNNs on homogeneous spaces[END_REF] to define an equivariant linear map on functions of homogeneous spaces.

We can represent a group as a fiber bundle using H and G/H as fiber and base, respectively. Let H ⊆ G be a subgroup, the map p : G → G/H given by p(g) = gH is a projection and each fiber is homeomorphic to H [START_REF] Cohen | A general theory of equivariant CNNs on homogeneous spaces[END_REF]. This is called the principal H-bundle.

Chapter 1. Group Equivariant Networks

An associated vector bundle is defined by choosing a representation (V, τ ) of H, we take G × V , such that h • (g, v) = (hg, τ (h -1 ) • v) and computing the quotient (G × V )/H = A. This is equivalent to a fiber bundle with base G/H and a fiber V , and thus sections s : G/H → A can be viewed as a stack of feature maps. The last ingredient necessary to define equivariant operators is to represent sections as Mackey functions. A Mackey function can be written as a function f : G → V that is subject to the condition f (gh) = τ (h -1 )f (g). Mackey functions encode sections of A. In [START_REF] Cohen | A general theory of equivariant CNNs on homogeneous spaces[END_REF] it is shown that every bounded, linear and Gequivariant operator on a homogeneous space can be written as a group convolution like in (1.24) applied to a Mackey function.

Now to put things into perspective, let us assume for instance that G = p4 × Z 2 . The group Z 2 is an homogeneous space with respect to G, because its elements are related by translations. Let us define by H the stabilizer of the origin o = (0, 0) at Z 2 , which is defined by H = {g ∈ G|g • o = o}. We have that the stabilizer H is exactly p4 and thus G/H ∼ = Z 2 . If we choose V = R 3 with the representation from last section we end with the same kinds of equivariant operators on Mackey functions that are obtained extending f : Z 2 → V to a function f : G → V that is constant along the orbits.

If, on the other hand, we choose H = {e} and V = R n with τ (g) = id, we end with the group convolution defined in Section 1.6.3.

Conclusions

This chapter serves as an introduction to some themes central to the thesis, namely groups, group actions, group-equivariant and group-invariant operators. We relate those concepts to the field of geometric deep learning, showing how one can obtain an equivariant neural network from blocks of linear equivariant operators, pointwise non-linearities and so on. In this chapter the group convolution is also reviewed. Group convolutions, including steerable networks, are the main operators which are used to define equivariant networks in the literature. In Chapters 2 and 3 we use the approach of [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] which is a generalization of the group convolution, the semi-group convolution, for scale-equivariant networks. In Chapters 4, 5 and 6, however, we steer away from group convolutions and we propose a new approach based on the method of moving frames.

Résumé en Français

Ce chapitre est une introduction à quelques sujets centraux de la thèse, en particulier les groupes, les actions de groupe et les opérateurs équivariants ou invariants par l'action d'un groupe. Nous montrons les liens entre ces sujets et l'apprentissage profond géométrique, en montrant qu'il est possible de définir un réseau équivariant à partir de blocs d'opérateurs équivariants linéaires, de non-linéarités appliquées ponctuellement et d'autres opérateurs standard de l'apprentissage profond comme la Batch Normalization. Dans ce chapitre la convolution de groupe est également revue. La convolution de groupe, y compris les convolutions orientables, est le principal opérateur utilisé pour obtenir des réseaux équivariants dans la littérature. Dans les chapitres 2 et 3 nous utilisons l'approche de Worrall and Welling (2019), les convo-lutions de semi-groupe, une généralisation des convolutions de groupe. Dans les chapitres 4, 5 et 6, nous n'utilisons pas les convolutions de groupe mais nous proposons une méthode pour obtenir de couches équivariantes basée sur la méthode du repère mobile.

Chapter 2

Scale-Equivariant Networks

Introduction

Scale transformations are naturally found in many computer vision tasks, since distance from the camera causes a scale variation in the captured images. This makes scale-equivariance a particularly interesting property for some networks. Generally speaking, convolutional neural networks are not equivariant either in theory or in practice, as illustrated in Figure 3.1 for the case of the U-Net. Scale-spaces are methods from image processing that allow for multi-scale representation of images. This chapter focuses on studying scale-equivariant networks based on scale-spaces.

In contrast to the group convolution discussed in Chapter 1, Deep Scale-Spaces [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] introduce neural networks equivariant to the action of semigroups, instead of groups. Semigroup actions can model non-invertible transformations and in [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] the authors focused on equivariance to downsampling in discrete domains. Focusing on downsampling is a way to address equivariance to scalings without creating spurious information through interpolation. In their architecture, the first layer is based on a Gaussian scale-space operator, and subsequent layers of this network are equivariant to the action of a semigroup of scalings and translations. Effectively, these operators are equivariant to rescaling of a discrete image. There are other scale-spaces with similar mathematical properties to the Gaussian scale-space, in particular the morphological scale-spaces.

In this chapter we generalize the approach of semigroup scale-equivariant neural networks by finding a sufficient condition in which scale-spaces, in the sense of [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF], can be used as the first layer, or the so-called lifting, in an equivariant network. Furthermore we investigate architectures built using morphological scale-spaces liftings. We observe that the morphological scale-spaces networks compare favorably to the Gaussian one in tasks of classification and segmentation of images at scales previously unseen by the network, in contrast to [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], in which the experiments test the overall performance of the network, but where the train and test sets objects follow the same scale distribution.

The chapter is organized as follows: We begin by introducing the groups and semigroups of scalings, particularly the semigroup which we seek equivariance to, in Section 2.2. In Section 2.3 we apply the group convolution (1.24) to the case of the scalings group and in Section 2.4 we review the approach of [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] to define scale-cross-correlations and we also discuss their advantages, such as their increased receptive fields compared to CNNs. We talk about the lifting and projection layers in Sections 2.5 and 2.5.2, respectively, which are layers that are used to map images to and from signals on semigroup in an equivariant (or approximately equivariant) way. In Section 2.5.2 we also introduce the scale-dropout applied before the projection layer to increase robustness and encourage the use of multi-scale information. In Section 2.5.3 we discuss the conditions necessary for a lifting layer and we propose a class of liftings based on scale-spaces. Particularly this includes the Gaussian and morphological scale-spaces. We validate our methods experimentally in Section 2.6, where we test scale-equivariant methods in tasks of classification and segmentation. For the classification task we use the Large Scale MNIST dataset [START_REF] Jansson | Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges[END_REF] and for the segmentation we use a synthetic dataset of shapes. In both cases the goal is to measure how well a model performs when trained for images in specific scales and tested in images with objects in unseen scales. In both cases the equivariant models outperform CNN baselines, including the U-Net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] for the segmentation task and the models with morphological lifting, particularly with the quadratic dilations lifting, seem to have some advantages when shape matters more than texture. We end the chapter with some concluding remarks in Section 2.7.

Scalings Group and Semigroup

Throughout the rest of the chapter, the transformations of interest are representations of a group or semigroup of scalings and translations. The group of scalings can be identified by the positive real numbers using real multiplication, denoted (S, •), where S = R >0 = {s ∈ R|s > 0} and • is the real multiplication. Adding translations operators amounts to computing the direct product of a group of scalings and a group of translations, S R 2 . An element of that group is denoted by (s, x), s ∈ S, x ∈ R 2 , and the product • is defined by (s, x) • (t, y) = (st, sy + x).

(2.1)

This group acts on images f : R 2 → R n by means of a re-scaling followed by a translation, i.e. we define the action R s,x , (s, x) ∈ S × R 2 as, for all (s, x)

∈ S × R 2 , y ∈ R 2 R s,x [f ](y) = f (sy + x), (2.2)
and similarly, for functions f :

S × R 2 → R n we define the action R s,x , (s, x) ∈ S × R 2 as R s,x [f ](t, y) = f (st, sy + x). (2.3)
In this chapter many of the approaches are based on representing scales as semigroups. A semigroup is a tuple (G, •) of a set G and a function • : G × G → G with the property of associativity. In other words it is an object obtained by removing the properties of inverse and neutral element from the definition of group. Similarly to groups, we can define semigroup actions on a set X as π : G × X → X which can be written as families of functions π g (x) := π(g, x) ∀g ∈ G, x ∈ X which are homorphic to the semigroup structure i.e. ∀g,

h ∈ G π g • π h = π g•h (left semigroup action) or ∀g, h ∈ G π g • π h = π h•g (right semigroup action).
To model the semigroup approach, we consider some semigroups that are sub-semigroups of S R 2 , i.e. we consider a semigroup S × E ⊆ S × R 2 that is closed under the product "•". A first approach would be to take S ≥1 R 2 , where S ≥1 = R ≥1 = {s ∈ R|s ≥ 1}. A discrete semigroup of scalings can also be obtained by S γ = {γ n |n ∈ N} for some γ > 1. In the case where γ is an integer, both S γ × R 2 and S γ × Z 2 form sub-semigroups of S R 2 , but for a general γ > 1, only S γ ×R 2 is closed under the product (2.1). In all of these cases, the operators R s,x and R s,x , when restricted to values in S × E, form semigroup actions on functions defined on S × E and E, respectively.

Scale Group Convolution

Before we investigate the case of semigroup equivariant networks for which most applications in this chapter were developed, let us first look at the group convolution of the scalings group in order to gain some intuition into the form of the semigroup convolution. With the goal of defining a linear, scale-equivariant operator, such as outlined in Section 1.4 by the GDLBP, we first examine a special case of networks equivariant to the group S R 2 . Moreover, we assume the normal subgroup H to simply be the identity {(1, 0)}, meaning that the group representation acts only on the domain of the input functions f : S R 2 .

Assume that we have a function f : S R 2 → R and a filter h : S R 2 → R. We compute the group convolution of a single-channel image by a single channel filter in order to simplify the computations, as in the multi-channel case it is only a question of applying multiple singlechannel convolutions. We find that the left-invariant Haar measure in S × R 2 can be obtained by

dµ((s, x)) =   det   s 0 x 1 0 s x 2 0 0 1     -3 dsdx 1 dx 2 = s -6 dsdx 1 dx 2 (2.4)
and the convolution (1.24) becomes

(f * h)(s, x) = S×E 1 s 6 f (s , x )h (s , x ) -1 • (s, x) ds dx (2.5) = S×E 1 s 6 f (s , x )h(s -1 s, s -1 x -s -1 x )ds dx = S E 1 s 6 f (s , x )h(s -1 s, s -1 x -s -1 x )dx ds = S E 1 s 6 f (ss , sx + x)h(s -1 , -s -1 x )dx ds = S E 1 s 6 f (s, x) • (s , x ) h (s , x ) -1 dx ds .
It is equivariant to the action (2.3), which we can verify:

(R t,y f * h)(s, x) = S×E 1 s 6 f (ts , tx + y)h(s -1 s, s -1 x -s -1 x )ds dx = S×E t 2 s 6 f (s , tx + y)h(s -1 ts, s -1 tx -s -1 tx )ds dx = S×E 1 s 6 f (s , x + y)h(s -1 ts, s -1 tx -s -1 x )ds dx = S×E 1 s 6 f (s , x )h(s -1 ts, s -1 tx -s -1 x + s -1 y)ds dx = S×E 1 s 6 f (s , x )h((s , x ) -1 (t, y)(s, x))ds dx = R t,y (f * h)(s, x).

Semigroup Convolution

Let's first stress the interest of extending the equivariance setting to semigroups. In image processing, important examples of semigroup actions are scale-spaces. As we will present in more details in Section 2.5.3, semigroup actions on images may be the convolution with a Gaussian kernel (Gaussian scale-space) or the application of a morphological operator such as erosion, dilation, opening or closing (morphological scale-spaces). Scale-spaces highlight the multi-scale nature of images and have shown great efficiency as image representations [START_REF] Lowe | Object recognition from local scale-invariant features[END_REF]. Besides, they are naturally complementary to the scaling operation. For example, the Gaussian blurring acts as a low-pass filter and allows the subsampling (or downscaling on a discrete domain) of an image to avoid aliasing artifacts.

Hence, equivariance of linear operators to semigroups seems highly desirable, as it is natural to expect that the same information at different scales produce the same responses up to some shift due to scale change. Since we cannot reproduce the results of the group convolution, a new expression has to be derived. In this case, however, we constrain the problem to only discrete semigroups, as that is the object that is most interesting at this stage. Taking inspiration from the group convolution, we notice that (1.26) can also be written, for u ∈ G, H(f

)(u) = g∈G h(g)f (u • g) if we change the function w for its conjugate h, that is h(g) = w(g -1 )
, and thanks to a change in variables. Now this expression can be applied to semigroups, considering the semigroup right action

R u (f )(g) = f (u • g). We get that operators H defined by ∀u ∈ G, H(f )(u) = g∈G R u (f )(g)h(g) (2.6) are indeed equivariant to the semigroup action R t , t ∈ G, since H(R t (f ))(u) = g∈G R u (R t (f ))(g)h(g) = g∈G R tu (f )(g)h(g) = R t (H(f ))(u). (2.7)
This class of semigroup equivariant operators is the semigroup cross-correlation proposed in [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF]. We also write f G h := H, remarking however that, contrary to the Figure 2.1: The Scale-cross-correlation performs dilated convolutions at each scale, where the dilation rate is equal to the current scale. Each grid represents how filters are applied to the same feature map in S γ × Z 2 at a different scale level. In the present example we assume that γ = 2. We show a filter being applied to feature maps at scales γ 0 , γ 1 and γ 2 .

group case, this operation is not symmetrical in f and h even when the law • on G is commutative.

Implementation for Integer Scalings

Following [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], assuming that the base of the scale-semigroup is an integer γ, the operator (2.6) applied to a signal f at a point (γ k , x) becomes

H(f )(γ k , x) = (f G h)(γ k , x) = l≥0 y∈Z 2 f (γ k+l , γ k y + x)h(γ l , y). (2.8)
We can also try to obtain a discrete operator from (2.5). That operator is given by

H(f )(γ k , x) = (f G h)(γ k , x) = l≥0 y∈Z 2 γ -6l f (γ k+l , γ k y + x)h(γ l , y).
(2.9) and it differs from (2.8) by the term γ -6l multiplying the arguments of the sum. In fact both approaches as equivalent as we can compute the cross-correlations by h(γ l , y) = γ -6l h(γ l , y), therefore, we keep the first approach (2.8) because it is simpler. In Figure 2.1, the shape of the filters for different scales of the same image is shown. Note that it is the same filter being applied with different dilation rates at each scale.

The operations here were defined for single channel images on G, but they can easily be applied to multichannel images. Let the input f = (f 1 , . . . , f n ) ∈ (R C ) G be a signal with n channels. Assuming the output has C channels, the filter is of the form h : G → R C×C . In this case we can compute a multi-channel scale-cross-correlation operator at channel o ∈ {1, . . . , mb} as

(f G h) o (γ k , x) := C c=1 (f c G h c,o )(γ k , x).
(2.10) We can verify that it is equivariant

(R g f G h) o (γ k , x) = C c=1 R g (f ) c G h c,o (γ k , x) = C c=1 R g (f c ) G h c,o (γ k , x) = C c=1 R g (f c G h c,o )(γ k , x) = R g C c=1 (f c G h c,o ) (γ k , x) = R g (f G h) o (γ k , x).
(2.11)

Improved Receptive Field

In a convolutional, or semigroup-convolutional neural network, the receptive field is the region which is effectively used to compute an output point. More precisely, if f out is the output of the network, then the pixel f out (x) is computed only from the pixels of the input f falling inside the receptive field. For example, in a CNN consisting of 4 layers of 3 × 3 convolutions, the last layer will use a 3 × 3 square from the previous layer in the convolutions, and to compute each pixels in that 3 × 3 square another 3 × 3 square from the previous layer will be used, but since the squares overlap that results in a 5 × 5 square from layer 2 and a similar argument shows that it uses a 7 × 7 from layer 1. This network is illustrated in Figure 2.2. More generally, a CNN of L layers, with each of these layers computing a convolution by a K × K filter has as a receptive field a square of size L(K -1) + 1. Other operations such as subsampling also influence the receptive field.

The advantage of having a large receptive field is that more of the information on the image is used at a time and more information can be extracted for objects which are far from each other in the image. Note that in (2.8), because of the multiplicative constant γ k in the spatial component, the receptive fields of networks consisting of scale semigroup correlations are much

• • • input f : Z 2 → R C Λf = f 1 : G → R C f L : G → R cC Πf L : Z 2 → R C f 1 (γ 0 , •) f 1 (γ 1 , •) f 1 (γ 2 , •) f 1 (γ 3 , •) Λ Π f l = σ(f l G h l + b l ) Figure 2.
3: Illustration of a scale-crosscorrelation architecture using a lifting and a projection layer. The lifting, denoted Λ will map a 2D signal to a signal on G and the projection will map a signal on G to a 2D signal. Scales truncated at four. larger than those of a CNN of corresponding size. Indeed, a network consisting of L scale cross-correlations with filters with dimensions P × K × K (i.e. the support of the filter is a grid

{(p, k 1 , k 2 )|p ∈ {γ 0 , γ 1 , . . . , γ P -1 }, 1 ≤ k 1 , k γ ≤ K}) at a scale coordinate s has a receptive field of size greater than γ s K + γ s+P -1 (L -1)(K -1)
, which is significantly larger than the size of the receptive field of a CNN with the same number of layers and the same kernel size even for s = 1 or s = 2. At s = 0, P = 1 it is equivalent to a CNN. In other words, an architecture built on scale cross-correlations attains the same receptive field as a deep CNN with much smaller depth and number of parameters.

Lifting and Projection

The operators of the previous section are defined on the set of functions with the semigroup as a domain, F = (R C ) G , but images input to networks are functions f : Z 2 → R C . In this section we review the lifting and projections layers -operators which map images to functions on the semigroup and vice-versa. They operate by equivariantly mapping the inputs from Z 2 to the semigroup and vice-versa, as illustrated in Figure 2.3.

Lifting

So far, in the general case, we have considered functions f : G → R C defined in the semi-group G, but the input to CNNs are images defined on a discrete set X, which may be different from the semigroup we seek equivariance to. In particular in this chapter we consider the semigroup product of translations and scalings.

In general, if the semigroup G = S X is a semidirect product of two semigroups S and X then in theory the issue is easily overcome because applying (2.6) to a signal f : X → R instead of a signal defined on G does not change the equivariance property, provided we have an appropriate semigroup action R u on f , but the output will be a signal on G. So in order to apply equivariant operators a signal f on X on we must lift it to a signal G at some point. Accordingly, the task is divided into two steps. First, we define (2.6) which holds as it is, like in Section 2.4.1. Second, we introduce a lifting operator Λ to map a function f defined on X into a function Λf defined on G, as it is done in Section 2.5.3. The operator H becomes then:

∀u ∈ G, H(f )(u) = g∈G R u (Λf )(g)h(g).
Now we easily check that it is sufficient for the lifting operator to be equivariant to G, that is, R u • Λ = Λ • R u , to have equivariance of H. Indeed, in that case by omitting parentheses for readability,

H(R t f )(u) = g∈G R u ΛR t f (g)h(g) = g∈G R u R t Λf (g)h(g) = R t (Hf )(u).
(2.12)

The advantage of this two-step approach is the richness of operators induced by the variety of possible liftings, as exposed in the rest of this section.

Projection

Now we have taken a signal f : X → R C and mapped it to a signal on G where we can apply equivariant operators, obtaining a signal φ(f

) : G → R C such that φ(R u f ) = R u (φ(f )).
However, an output of this form is not suitable for most applications and applying densely connected neural network layer to change the output form would destroy the equivariance.

As the last part of the GDLBP, we must apply a projection layer, its form depends on the application: the output should lie in X like in a segmentation task, so the projection Π :

(R C ) G → (R C ) X must be an equivariant operator, i.e. Π • R u = R u • Π; the output should be a vector, like in a classification task, so the projection Π : (R C ) G → R C must be invariant, i.e. Π • R u = Π .
It turns out that in practice the projections are only approximately equivariant when G is not a group, but in later sections this approximation will be verified empirically.

In general a permutation invariant operator (e.g. sum, average or supremum) can be applied to the scale-dimension to define the projection layer, i.e. if α : R N → R is permutation invariant we can define

Π(f )(x) = α(f (γ 0 , x, γ 1 x, γ 2 x, . . . )) (2.13) or Π(f )(x) = α((f (γ k , x) k≥0,x∈Z 2 ) (2.14)
where we assume that f has a finite support in order for the maximum and sum to be well defined.

In practice to perform the projection we apply a max-projection along the scale dimension, defined by ∀z ∈ Z 2 , Π[f ](z) = sup k∈N {f (γ k , z)}. To be consistent with the lifting, we would like to have

R γ k ,z • Π = Π • R γ k ,z . Instead, we have R γ k ,z Πf (y) = sup l∈N f (γ l , γ k y + z), and ΠR γ k ,z f (y) = sup l∈N f (γ l+k , γ k y + z) = sup l≥k f (γ l , γ k y + z) so that R γ k ,z Πf (y) = max{ΠR γ k ,z f (y), max 0≤l<k f (γ l , γ k y + z)}.
The previous expression will be equivariant if the scale where the maximum is attained is smaller than k, but in general we can only hope for approximate equivariance for small enough k. The approximate equivariance will be empirically verified in experiments in Section 2.6. Note that other projections (e.g. sum or average) have the same flaw, as this is intrinsic to the semigroup-equivariant approach, even though it was omitted previously in the literature.

Scale Dropout

The projection layer produces an approximately equivariant/invariant feature map, but it is not guaranteed to use the multiscale information at the last layer. In order to incentive the use of multiple scales by the classification layers and to produce more robust results with respect to scale changes we proposed a special version of dropout [START_REF] Srivastava | Dropout: A simple way to prevent neural networks from overfitting[END_REF], which we call Scale Dropout, applied before the projection layer.

The Scale Dropout works by dropping (i.e. setting to 0) all values of a feature map at randomly chosen scales. Formally, given a feature map f :

S γ × Z 2 → R C , the scale dropout with rate r ∈ [0, 1] is computed as ScaleDropout r (f )(s, x) = X(s)f (s, x) (2.15)
where X(s), for every s ∈ S γ is a Bernoulli variable of parameter r, i.e. P X(s) = 1 = 1 -r and P X(s) = 0 = r.

Scale-Spaces as Lifting Operators

We focus on the semigroup product of scalings and translations

G = S Z 2 = (S × Z 2 , •).
A lifting Λ in the space of scalings is characterized by the property

Λ • R t,z = R t,z • Λ, (2.16)
With the goal of defining a suitable and general lifting, scale-spaces will serve as a theoretical framework.

Many studies have been made on defining scale-spaces [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF][START_REF] Witkin | Scale-space filtering: A new approach to multi-scale description[END_REF][START_REF] Pauwels | An extended class of scale-invariant and recursive scale space filters[END_REF][START_REF] Alvarez | Axioms and fundamental equations of image processing[END_REF]. In here, following the definitions of [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF], a scale-space can be viewed as an operator that commutes with a scaling. A scaling is a family

{S t : V R 2 → V R 2 |t > 0}, where V = R C , of
operators on images such that:

S 1 = id, ∀t, s > 0 S t S s = S ts (2.17)
where id is the identity transform. Therefore a scaling is a group action of (R + * , ×). An example is the family S p , p ≥ 0, given by (S p t f )(x) = f 1 t p x , where p controls the rate of scaling. We have that R t,0 = S p t -1 p .

Let S be a scaling and + a commutative operation such that (R + * , +) is a semigroup. Then a (S, +) scale-space is a family {T (t)|t > 0} of operators such that, for all t, s > 0 [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF]:

T (t)T (s) = T (t +s), T (t)S(t) = S(t)T (1). (2.18)
The property T (t)S(s) = S(s)T (t/s), for all t, s > 0, is a direct consequence of the second property. Here, in addition to (2.18), we assume that the scale-space T (t) is translationequivariant for all t > 0 (i.e.

T (t)(L z f ) = L z (T (t)f ) where L z (f )(x) = f (x + z)).
Thanks to the second property in (2.18), a (S p , +) scale-space T defines an operator on images f :

R 2 → R C ∀(s, x) ∈ S × E (Λf )(s, x) = (T (s -1 p )f )(x) (2.19) such that, for all (t, z) ∈ S × Z 2 , R (t,z) • Λ = Λ • R (t,z)
, where R (s,x) is applied to an image on a continuous domain. So, in order for Λ to be our lifting operator we assume that the input f is a function on a continuous domain. In practice, we discretize Λ and the input images.

With this, the morphological scale-spaces, as well as the Gaussian scale-space, being (S 1 2 ,0 , +) scale-spaces, can be used as the lifting operators.

Gaussian Scale-Space

The Gaussian scale-space is a (S 1 2 , +) scale-space defined by the family

T G (t)[f ](x, y) = u(x, y, t), where ∂ t u = ∆u, u(x, y, 0) = f (x, y), ∀(x, y) ∈ R 2 (2.20) For all images f ∈ R R 2 and points x ∈ R 2 .
The solution to the previous PDE, which is the scale-space T G can be computed as the convolution

(T G (t)f )(x) = (f * G t )(x) = R 2 f (y)G t (x -y)dy = 1 2πt R 2 f (y) exp x -y 2 2t dy, (2.21)
where

G t (x) = (2πt) -1 exp -x 2 2t
. This was the scale-space considered in Worrall and Welling (2019). There, it was assumed that image f has a maximum spatial frequency content. They model this by assuming that there exists a signal f 0 and a constant s 0 > 0 such that f = (f 0 * G s 0 ). An example of the Gaussian scale-space representation of an image is given in Figure 2.4.

Morphological Scale-Spaces

Similarly to the Gaussian scale-space, we can obtain morphological scale-spaces using partial differential equations [START_REF] Brockett | Evolution equations for continuous-scale morphological filtering[END_REF][START_REF] Van Den Boomgaard | The morphological structure of images: The differential equations of morphological scale-space[END_REF]. The quadratic morphological erosions and dilations, for example, can be regarded as morphological counterparts to the Gaussian scale-space [START_REF] Van Den Boomgaard | The morphological structure of images: The differential equations of morphological scale-space[END_REF] and are obtained by the PDE

∂ t u = ±|∇u| 2 u(x, y, 0) = f (x, y) ∀(x, y) ∈ R 2 (2.22)
Λ G f (γ 0 , •), . . . Λ G f (γ 4 , •) using a base γ = 2.
The solution to the equation above generate the families of quadratic erosions and dilations. They can be written as dilations and erosions by the structuring functions q t (x) = -x 2 4ct t > 0, i.e.

ε qt (f )(x) = ε t (f )(x) = inf y∈R 2 (f (x + y) -q t (y)) , and, (2.23)
δ qt (f )(x) = δ t (f )(x) = sup y∈R 2 (f (x -y) + q t (y)) , (2.24)
form (S The choice of the quadratic structuring functions as the natural counterpart of the Gaussian kernels can be generalized to the notion of poweroid structuring function [START_REF] Jackway | Properties of multiscale morphological smoothing by poweroids[END_REF]. The corresponding multiscale structuring function [START_REF] Diop | Multiscale image analysis based on robust and adaptive morphological scale-spaces[END_REF][START_REF] Schmidt | Morphological counterparts of linear shift-invariant scalespaces[END_REF]

b p,t (x) = - p -1 p p/(p-1)
x p/(p-1)

t 1/(p-1) (2.25)
is a scale-space and a solution of the Hamilton-Jacobi equation

∂ t u = ±|∇u| p u(x, y, 0) = f (x, y) ∀(x, y) ∈ R 2 (2.26)
Besides the previous case of parabolic structuring function for p = 2, note that for a fixed t > 0, when p → 1,then b p,t → 0, i.e., flat structuring element of radius t, and when p → ∞ one gets b p,t → x /t, i.e., conical structuring function.

The families of dilations and erosions δ bp,t and ε bp,t for some p ≥ 1 form (S 1 p , +) scale-spaces and thus constitute liftings. [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF], where a ∨ b = max{a, b}.

Combining Liftings Linear Combination

From two liftings Λ 1 and Λ 2 we have that the linear combination αΛ 1 + βΛ 2 also constitutes a lifting, as we can see by the linearity of R t,z and R t,z

(αΛ 1 + βΛ 2 )R t,z = (αΛ 1 R t,z + βΛ 2 R t,z ) = (αR t,z Λ 1 + βR t,z Λ 2 ) = R t,z (αΛ 1 + βΛ 2 ).
In particular this opens the way to convex combinations of two liftings. To motivate the interest in these kinds of combinations, consider the scale-space given by

L α t = αδ t + (1 -α)ε t (2.27)
where δ t and ε t are flat dilations by a disk of radius t and α ∈ [0, 1]. In the case where α = 0 we have an erosion scale-space and in the case α = 1 we have a dilation scale-space. In the case α = 0.5 we have a self-dual scale-space, that is L

0.5 t [f ] = -L 0.5 t [-f ].
If it is not known a priori what kind of scale-space would be better suited for the data and neural network architecture, α can be optimized by gradient descent along with the weights of the network, interpolating between those scale-spaces.

Composition

The composition of different liftings also yields a lifting. Consider T s and U s two scale-spaces and Λ

[f ](s, x) = (T s -1 p • U s -1 p f )(x) then Λ[R t,z f ](s, x) = (T s -1 p U s -1 p R t,z f )(x) = (T s -1 p R t,z U (st) -1 p f )(x) = R t,z (T (st) -1 p U (st) -1 p f )(x) = (T (st) -1 p U (st) -1 p f )(t -1 x + z) = Λ[f ](st, s -1 x + z) = R t,z (Λ[f ])(s, x).
The Lasry-Lions operators [START_REF] Angulo | Lipschitz regularization of images supported on surfaces using riemannian morphological operators[END_REF] can be obtained as compositions of the form δ qct • ε qt and ε qct δ qt , with c ∈ [0, 1]. The operators can vary between an erosion, or dilation, for c = 0 to an opening, or closing, for c = 1. For intermediate values, they possess interesting regularization properties [START_REF] Angulo | Lipschitz regularization of images supported on surfaces using riemannian morphological operators[END_REF]. This parameter could as well be learned by the network.

Now consider the operator

ψ (k,α,c,p) t = αδ b p,ckt ε b p,kt + (1 -α)ε b p,ckt δ b p,kt . (2.28)
By changing the parameters (k, α, c, p), we can control the base size of the structuring element, the closeness to an extensive or anti-extensive morphological operator, its closeness to a morphological filter or to an additive operator and if the structuring elements are flat, smooth or conical. In other words, it can interpolate between morphological dilations, erosions, openings and closings and self-dual operators, with structuring elements of different sizes that can be flat or smooth. The rescalings are obtained from the MNIST images which have size 28 × 28. In all scales, MNIST Large Scale images have size 112 × 112, and images are either padded or cropped to match that shape.

Experiments

Synthetic Shapes Dataset

In order to first assess the classification performance of scale-equivariant networks with different types of lifting operators we perform a segmentation experiment on a synthetic dataset. We are specifically interested in the case where the networks are trained in a dataset to segment objects divided into fixed scales, but in the test set objects can be scaled independently of one another, with the re-scaling factor following some probability distribution. Unlike in the classification case, it would be difficult to define a data augmentation technique that when applied to the training set mimics the behavior of the test set.

The dataset used for that consists of 224 × 224 synthetic binary images of shapes such as Figure 2.9 which are divided in three classes: disks, stars and the background. In the training set, only one scale is present, like in Figure 2.9(a). We construct test sets where each shape is re-scaled by a factor uniformly sampled from the interval [2 -i , 2 i ], in which we use i = 1, 2. Figure 2.9(b) shows an example of a test image. The train set contains 10000 images and the test sets contain 500 images each. The experiment is repeated ten times, each time generating a different training/test set pair.

Image Classification

Firstly we begin by examining properties of the scale-equivariant networks when applied to a task of image classification. Specifically we want to see what effect changing each module of the networks produces in the result based on unseen scales.

In order to separately evaluate different aspects of scale-equivariant neural networks, we begin by first defining a base architecture consisting of L = 5 layers consisting of scale-cross correlation layers, or convolution layers as a baseline. The layers have 12, 16, 24, 32 and 64 filters, based on the architecture used by [START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF]. We always use scales varying in a range of size 4. We also fix the use of max projection at the projection layer. We change other aspects of the network in order to test its components:

• Lifting: We train several different liftings in the model: Identity lifting, quadratic dilation scale-space, Gaussian scale-space and a convex combination of quadratic dilations and erosions (where the paremeter α of the convex combination is optimized as a network weight).

• Varying scale dimension: Filters can have a different size in the scale dimension, we test sizes 1 and 2. As was done by [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], only some layers can have a scale greater than one. So we train a layer with a scale dimension of 1 in every layer and one using a scale dimension of 2 the second and fourth layers and 1 at the other layers.

• Scale Dropout: Tested models appear both with and without the usage of scale dropout.

When present, its drop rate is set to 25%.

We compare our results with a CNN with the same number of layers and filters per layer, 3 × 3 filters and strided convolutions at layers two and four. We train the CNN baseline with and without scale jittering. Scale jittering refers to rescaling an image according to a random value sampled uniformly from the interval [s -1 , s], for some s > 0, and padding or cropping the image to have the same size as before. When cropped, the location of the cropping box is random. We use a value of s equal to 4.

The accuracies obtained from different liftings are shown in Figure 2.7. First off we note that there does not seem to be a single lifting that performs better than the rest, instead it seems that the quadratic dilation and the identity liftings perform better in different situations. More specifically when training in tr1, the identity lifting has a better accuracy, in tr4 it is the dilation lifting, and in tr2 it depends on the test scale. Moreover, it seems that dropout consistently improved the performance of equivariant models.

We can see that the augmented CNN trained at scale 2 has a better generalization than the rest of the models, but when trained at scale 4 it underperforms other models for most scales, indicating that the equivariant models are more robust to the training set scale than the augmented CNN.

Accuracies obtained from models of different scale dimensions are shown in Figure 2.8. From the figure it becomes apparent that increasing the scale dimension to two does not necessarily improve performance or generalization. This result is consistent with the one in [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] in the Cityscapes experiment, but in that case scale-equivariance was not explicitly tested.

Image segmentation

The architecture chosen for the equivariant models consists simply of six layers of semigroup cross-correlations. Because the scale cross-correlation has a naturally large receptive field, subsampling is not necessary. The output of the network is a three-channel image with the scores for each class, and the class is chosen as the coordinate with the greatest score. To quantitatively evaluate the models we use the Intersection over Union(IoU), or Jaccard index, between the prediction and the ground truth which can be written as

IoU(GT, Pred) = TP TP + FP + FN , (2.29)
where GT is the ground truth image, Pred is the prediction by the network, TP is the number of true positives, FP of false positives and FN of false negatives. When more than two classes are present we compute the IoU for each class individually and then compute the mean.

As baselines, we compare the models to a CNN with the same number of layers and a similar size and number of parameters, and also to a U-Net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] architecture. In Chapter 3 the comparison between segmentation by a U-Net and by scale-equivariant architectures will be studied more in-depth.

In Table 2.1 we compare the IoU obtained from different models. We see that CNN performs badly, compared to the equivariant models, even in the training set scale. This is partially attributed to the fact that the receptive field of the CNN is not as large, although having the same number of layers and a similar number of parameters. As expected, the equivariant models outperformed the CNN architectures and the U-Net architecture in the generalization to other scales.

To analyze why the dilation is suited to this particular dataset, we can analyze the effect of applying a discrete re-scaling, i.e. a subsampling to the objects processed by the scalespaces. In Fig. 2.10 we see the difference between a Gaussian and dilation lifting followed by a subsampling operator. Indeed, the persistence of concavities of the star shapes makes it easier to distinguish the objects in the last images.

Conclusions

In this chapter the semigroup cross-correlation approach to scale-equivariant networks was reviewed, in particular the scale-cross-correlation, which is the special case of the semigroup cross-correlation when applied to a semigroup of scales and translations. Some interesting properties were discussed, for example the large receptive field that these models have compared to matched CNNs. We also talked about the lifting and projection layers.

Because input images to neural networks are normally defined as signals on Z 2 and the scale-cross-correlation is defined for signals on a semigroup which is the Cartesian product of Z 2 with a semigroup of scales, it is necessary to map images to the semigroup, and because the end results should be equivariant, this has to be done in an equivariant manner. The lifting layer does that. We extend the approach to lifting given by [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], where only the Gaussian scale-space was used, to a general class of scale spaces, particularly it allows for both morphological and the Gaussian scale-spaces. We also show that linear combinations and compositions of liftings are also liftings, allowing for example for the Lasry-Lions lifting.

The projection layer maps signals on the semigroup back to signals on Z 2 in a way that is approximately equivariant. We also introduce the scale dropout layer before the projection layer. The scale dropout improves robustness of the model by randomly dropping scales during training and encouraging the use of multi-scale information.

In the experimental section, two tasks were studied: classification and segmentation. Classification is done in the Large Scale MNIST dataset, a dataset obtained by rescaling images of MNIST, and a synthetic dataset based on the segmentation of different shapes and the background. In both tasks the goal was to train the model with images at a certain scale range and test it in scales unseen during training. In both tasks the images have a clearly defined polarity in their pixels, for which we expected the morphological scale-spaces to be particularly adapted. Our experiments confirmed this hypothesis, in particular for the quadratic dilation scale-space, as it can help preserve the shapes of objects after a re-scaling. Even when compared to data augmentation in the MNIST Large Scale experiment, the equivariant models were shown to have a higher degree of robustness with respect to the training scale.

Résumé en Français

Dans ce chapitre, la théorie des réseaux équivariants basés sur la corrélation-croisée sur les semigroupes a été revue. En particulier, nous utilisons la corrélation-croisée d'échelle, c'est-à-dire la corrélation-croisée sur un semi-groupe appliquée à un semi-groupe d'échelles et translations. Quelques propriétés intéressantes sont discutées, comme le champ réceptif de ces modèles, qui est significativement plus grand que celui des CNNs similaires.

Les images d'entrée aux réseaux de neurones sont normalement définies comme des signaux sur Z 2 et la corrélation croisée d'échelle est définie par des signaux sur un semi-groupe qui est le produit Cartésien de Z 2 et un semi-group d'échelle. Cela nous oblige à mapper les images à ce produit Cartésien. Comme les résultats finaux doivent être équivariants, cela doit être fait de façon équivariante. C'est à cela que sert la couche de dépliement. Nous généralisons l'approche de dépliement proposée par [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], où seulement l'espace d'échelle Gaussien a été utilisée, à une classe générale d'espaces d'échelle. En particulier cela nous permet d'utiliser les espaces d'échelle morphologiques, comme par exemple les espaces d'échelle de dilatations, ainsi comme l'espace d'échelle Gaussien. Nous montrons aussi que les combinaisons linéaires et compositions de dépliements sont aussi des dépliements, ce qui nous permet d'utiliser, par exemple, un dépliement basé sur l'opérateur de Lasry-Lions [START_REF] Angulo | Lipschitz regularization of images supported on surfaces using riemannian morphological operators[END_REF].

La couche de projection mappe un signal sur le semi-groupe à un signal sur Z 2 de façon approximativement équivariante. Nous présentons aussi la couche de dropout d'échelle avant la couche de projection. Le dropout d'échelle améliore la robustesse du modèle en mettant à zéro les sorties correspondantes à certaines échelles, choisies aléatoirement, de la dernière couche avant la projection pendant l'apprentissage du réseau, ce qui encourage l'utilisation d'information multi-échelle.

Dans la section expérimentale, deux tâches ont été étudiées : la classification et la segmentation. La classification est faite dans la base de données MNIST en grande échelle [START_REF] Jansson | Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges[END_REF], qui est obtenue en changent l'échelle des images de MNIST (LeCun et al., 2010), et la segmentation sur une base de données synthétique. Dans les deux tâches les images ont une polarité clairement définie, pour laquelle nous nous attendions que les espaces d'échelle morphologiques soient particulièrement adaptés. Nos expériences confirment cette hypothèse, en particulier pour l'espace d'échelle des dilatations quadratiques, car il peut aider à mieux préserver les formes des objets après un changement d'échelle. Même comparé à l'augmentation de données dans la base MNIST en grande échelle, nous avons montré des résultats qui indiquent que les modèles équivariants ont un plus grand taux de robustesse au changement de l'échelle de l'ensemble d'apprentissage.

Chapter 3

Scale Equivariant U-Net

Introduction

In the previous chapter a strong argument was made for scale-equivariant networks when symmetry to changes in scale are present in the network, however, a larger architecture was not fully investigated. Moreover, the behaviour of common layers such as pooling and upsampling in a scale-equivariant setting were ignored. In this chapter a larger architecture that uses both aforementioned operators is studied. The SResNet was proposed in [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] in order to perform scale-equivariant segmentation and obtained good results in the cityscapes dataset, but it was not tested in re-scaled images, meaning that it is difficult to determine the contribution of scale-equivariance. Moreover it restricts the upsampling operator to be applied outside of semigroup domain, i.e. after projection, meaning that it is restricted to apply an upsampling by a large factor at the last layer to compensate for its downsamplings.

The U-Net architecture [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] has become famous for its great performance in semantic segmentation. It is a fully convolutional neural network, i.e. a CNN without any dense layer. Thanks to this, its output is equivariant to a certain subgroup of translations. However, architectures like U-Net are not scale equivariant a priori, and experiments show they are not in practice (Sangalli et al., 2021) as illustrated by Figure 3.1. Having a scaleequivariant counterpart of such an architecture is desirable as scale symmetry is frequently present in semantic segmentation data. For this reason in this chapter the Scale-Equivariant U-Net (SEU-Net) is introduced.

The SEU-Net is obtained by substituting the building blocks of the U-Net by their scaleequivariant counterparts. The counterpart to the convolution, a linear scale-equivariant operator, are the scale-cross correlations. Counterparts of subsampling and upsampling operators, both which are important components of the U-Net, are studied in this chapter.

We begin the chapter by disscussing related work in Section 3.2 followed by a review the architecture of U-Net in Section 3.3. In Section 3.4 the SEU-Net is proposed and reviewed and its building blocks are analysed in regards to scale-equivariance. Pooling and upsampling are given special attention. Later The SEU-Net is tested and compared against the U-Net and the SResNet in Section 3.5. More specifically the SEU-Net is tested in segmentation tasks, where the training images are at a fixed scale and the test images are re-scaled to scales unseen during training. The datasets used for the experiments were a dataset of strand images for plane manufacture, the Oxford-IIIT dataset for natural image segmentation and the DIC-C2DH-HeLa dataset for cell segmentation. We also compare the SEU-Net with a scale-augmented U-Net in both the Oxford-IIIT Pet and the DIC-C2DH-HeLa datasets and obtain positive results as the SEU-Net can obtain a higher degree of scale generalization without losing performance at scale 1, compared to the U-Net with augmentation. In the last experiment the SEU-Net is also shown to have a good interation with scale augmentation. We end the chapter in Section 3.6

Related Work

As discussed in the last chapter, scale-equivariance and scale-invariance are topics already discussed in the deep learning literature [START_REF] Zhu | Scale-equivariant neural networks with decomposed convolutional filters[END_REF][START_REF] Ghosh | Scale steerable filters for locally scale-invariant convolutional neural networks[END_REF][START_REF] Jansson | Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges[END_REF][START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF][START_REF] Sosnovik | Scale-equivariant steerable networks[END_REF]. The experimental benchmarks found in those papers are interesting as a first way to measure equivariance, but tend to be based on very simple tasks, such as the classification of re-scaled digits from the MNIST dataset or low resolution images of clothes from the Fashion-MNIST dataset. In [START_REF] Sosnovik | Disco: accurate discrete scale convolutions[END_REF], combinations of base filters are optimized to minimize the equivariance error of discrete scale convolutions. This is applied to classification, tracking and geometry estimation, but not Chapter 3. Scale Equivariant U-Net segmentation.

In [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] the semigroup equivariant models were applied to classification and semantic segmentation of datasets of large images, achieving better results compared to matched non-equivariant architectures. Yet, the role of scale-equivariance was not isolated, as the performance of the models was not measured for inputs on scales unseen in the training set. Later on, this approach was revisited by Sangalli et al. (2021) (Chapter 2), where the Gaussian scale-space originally used was generalized to other scale-spaces and the models were tested in experiments where the networks are trained in one fixed scale and tested on unseen scales, albeit on synthetic or simple datasets. In all these approaches, the authors either avoided pooling and upsampling in their architectures, or used them but did not discuss their impact on scale equivariance.

While scale-equivariance has been a topic in the literature for some time, as far as we know a scale-equivariant U-Net has not yet been proposed, contrary to the rotation-equivariance case [START_REF] Chidester | Enhanced rotationequivariant u-net for nuclear segmentation[END_REF]. Moreover, the current benchmarks for scale-equivariance were either based on simple datasets like MNIST or did not explicitly measure the equivariance in their segmentation or classification experiments, by training the networks on one fixed scale and testing on unseen scales. Here we propose semantic segmentation experiments based on natural data which measure the equivariance of the predictions.

U-Net

The U-Net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF], illustrated in Figure 3.2(a), is a CNN architecture for semantic segmentation. It consists of of an auto-encoder structure and as such can be divided into two main parts: the encoder and the decoder. The encoder and decoder are linked by blocks referred to as skip connections. The encoder and decoder can be further decomposed into four main blocks: convolution blocks, pooling, upsampling and skip connections.

Here we given a description of the U-Net. First off, to parametrize it we use to integer parameters, the depth D and the number of channels at the first layer C. The encoder consists of L blocks B ↓ l l = 1, . . . , L, of two convolutions followed by pooling or more precisely

B ↓ l [f ] := Pool • σ • BN 2 l h 2 l * σ • BN 1 l (h 1 l * B ↓ l-1 [f ]) (3.1)
where σ denotes a non-linearity such as ReLU or leaky ReLU, BN i l denotes a batch normalization layer [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF] and h i l denote convolutional filters for i ∈ {1, 2} l ∈ {1, . . . , D} and Pool denotes a subsampling functions. The filters h i l both have output dimension given by 2 l C. In the original paper they used 3 × 3 convolutional filters and a 2 × 2 max-pooling with stride two as subsampling the first encoder block is the identity B ↓ 0 [f ] := f . The decoder blocks are applied in reverse order. The first decoder block is just the last encoder block

B ↑ L [f ] := B ↓ L [f ]
and subsequent decoder blocks are the concatenation of the upsampling of the previous decoder block with the encoder block at the same level, followed by two convolutions, i.e. for l ∈ {0, . . . , L -1}

B ↑ l [f ] := σ • BN 2 l w 2 l * σ • BN 1 l (w 1 l * conc[B ↓ l [f ], Upsamp(B ↑ l+1 )) (3.2)
where conc :

(R C 1 ) Ω × (R C 2 ) Ω → (R C 1 +C 2 ) Ω denotes concatenation, or equivalently the skip connections, i.e. conc[f, g](x) := (f 1 (x), . . . , f C 1 (x), g 1 (x), •, g C 2 (x)), (3.3)
and Upsamp denotes some kind of upsampling (2 × 2 transposed convolution with stride two in the original paper). The convolution filters w i l , i ∈ {1, 2}, l ∈ {0, . . . , L -1} have output dimension 2 l C.

Finally the output of the network is given by the application of a convolutional layer at B ↑ 0 to compute the logits of the segmentation layer.

Because of its skip connections, the U-Net effectively removes the spatial dimension bottleneck of classical auto-encoders, where a high resolution segmentation map must be entirely recovered from a low resolution feature map at the output of the encoder.

Keep in mind that the description we gave in this section refers to a specific type of U-Net architecture. In practice the encoder architecture can be changed to be different kinds of architectures, including modern and powerful classification architectures such as a ResNet (Abedalla et al., 2020). This practice is commonly referred to as changing the backbone of the U-Net. In this chapter we use the formulation we provided.

Scale-Equivariant U-Net

In this section we aim to propose the Scale-Equivariant U-Net (SEU-Net), i.e. a U-Net architecture made with scale-equivariant components. The resulting model is approximately scale-equivariant and should have an increased generalization capacity compared to the U-Net. The architecture proposed for the SEU-Net is described in Figure 3.2(b).

In the framework of the previous chapter, a network can be written as Γ = Π • Σ • Λ, where Λ and Π are the lifting and projection respectively, and Σ is the core part of the network mapping the lifted space to itself. We already saw that Λ

• R γ k ,z = R γ k ,z • Λ and we assume R γ k ,z •Π ≈ Π•R γ k ,z . Hence, to build a (approximately) scale-equivariant network, it is sufficient to have Σ • R γ k ,z = R γ k ,z • Σ.
In order to remove some choice of hyperparameters, lifting and projections layers are fixed being a Gaussian lifting and a max-projection.

In particular, a way to render the U-Net scale-equivariant is to design scale-equivariant versions of its components in Σ. Convolutions are already rendered equivariant by scale-crosscorrelations, and since batch normalization and pixelwise activations preserve scale-equivariance, we can easily render the convolution blocks equivariant. Skip connections are also equivariant as is.

The rest of this section is dedicated to the remaining components: pooling and upsampling.

Pooling

Classical pooling operators naively applied scale by scale do not result in scale-equivariant poolings in the lifting space. For example, the max-pooling MP[f ](x) = max y∈N f (rx + y) with strides r ∈ N and neighborhood N ⊆ Z 2 (usually a r × r square). Its naive extension to the lifted space MP

[f ](γ k , x) = MP[f (γ k , •)](x) ∀k ∈ N does not commute with R γ k ,x .
Strided convolutions however, generalize well to this scenario, written as the subsampling operator D t [f ](γ k , x) = f (γ k , tx) following a scale-cross-correlation. We can verify that it is scale-equivariant:

H × W × C H/2 × W/2 × 2C H/4 × W/4 × 4C H/8 × W/8 × 8C H/16 × W/16 × 16C 24C 8C 12C 4C 6C 2C 3C C (a) U-Net Λ H × W × C in H × W × S × C Π H × W × Cout H/2 × W/2 × S × 2C H/4 × W/4 × S × 4C H/8 × W/8 × S × 8C H/16 × W/16 × S × 16C (b) SEU-Net
D t [R γ k ,x f ](γ l , y) = (R γ k ,x f )(γ l , ty) = f (γ l+k , γ k ty) = D t [f ](γ l+k , γ k y) = R γ k ,x [D t f ](γ l , y
). We use strides as the subsampling in our networks, with a stride of t = 2.

Besides the strided scale-crosscorrelations we used, we can define another class of pooling operators, inspired by classical max-pooling. Let us place ourselves in a slightly different case of pooling a function in a continuous domain f :

S × R 2 → R, with S × R 2 acting on it by R γ l ,z f (γ k , x) = f (γ k+l , γ l x + z), k, l ∈ N, x ∈ R 2 , z ∈ Z 2 .
We define the pooling of f as an operator F followed by a downsampling

D γ l [f ](γ k , z) = f (γ k , γ l z) P [f ] = D γ l F f. (3.4)
If F commutes with R γ k ,x , then so does P . We consider three pooling functions: F id = id (strides) and two dilation scale-spaces [START_REF] Heijmans | Algebraic framework for linear and morphological scale-spaces[END_REF]):

• The max-pooling of scale-semigroup-valued images is given by a re-scaled max-pooling

F max [f ](γ k , z) = sup y∈N k ×N k {f (z -y)} (3.5)
where

N k = {γ k x|x ∈ N } and N is for example a γ l -sided square in R 2 .
• The quadratic dilation (quadpool) scale-space is a morphological counterpart to the Gaussian scale-space (Van Den Boomgaard and Smeulders, 1994) defined by

F quad [f ](γ k , z) = sup y∈R 2 f (z -y) - y 2 cγ 2k , (3.6)
where c > 0 is some constant.

In contrast to the strided scale-cross-correlations given by F id , the functions F max and F quad are scale-equivariant only in this continuous setting, their discretized versions are not actually equivariant. Nonetheless, a network employing scale-cross-correlations and these poolings would be equivariant when applied to signals in the domain S × R 2 .

Upsampling

Upsampling blocks are a well established part of modern neural network architectures for segmentation and other tasks. In order to extend upsampling to a scale-equivariant setting, we look at the case where f is defined on a continuous domain. In that case, the downsampling D γ l has an inverse U γ l which is the natural upsampling.

In the discrete case the problem becomes more complicated as downscaling is not invertible, but for k, l ∈ N we can define an upsampling U γ l as an operator satisfying ∀x ∈ Z 2

U γ k [f ](γ l , γ k x) = f (γ l , x) and U γ lk = U γ k • U γ l . (3.7)
With this, we have

D γ k • U γ k = id. For all k, U γ k (f ) values are only restricted in the points y ∈ kZ 2 = {kx|x ∈ Z 2 }
, and the values on the other pixels can be defined in several ways (e.g. copies, interpolation) as long as it satisfies (3.7). Now, assume

U γ l R γ k ,x f = R γ k ,γ l x U γ l f, (3.8)
for any f , and disregarding the approximation in pooling, then Σ

• R γ k ,x = R γ k ,x • Σ. Indeed let ψ i = D γ l L i • • • D γ l L 1 , i = 1, .
. . , m denote the part of a SEU-Net of height m from the layer after lifting until the i-th downsampling block, where L j , j = 1, . . . , m, are blocks that commute with R γ k ,x (constructed by scale-cross-correlations, pointwise activations and batch normalization). Denote φ m = L m+1 ψ m and φ i = L i C(U γ l φ i+1 , ψ i ), i = m, . . . , 1 where C denotes concatenation.

With the above hypothesis, we have

φ i R γ k ,x f = R γ k ,γ li x φ i f . In particular, R γ k ,x φ 0 f = φ 0 R γ k ,x f , and we notice that φ 0 is precisely Σ. The sufficient condition U γ l R γ k ,x f = R γ k ,γ l x U γ l f is not verified in general: We can show that U γ l R γ k ,x f = R γ k ,γ l x U γ l f
for at least one lifted image f , one couple of integers (k, l) and a point x ∈ Z 2 . Note that U γ l is an upsampling defined in the associated paper.

Given any k ∈ N, take l = k, x = (0, 0) and any two lifted images f 1 and f 2 that coincide on certain points,

f 1 (s, γ k y) = f 2 (s, γ k y) ∀s ∈ S, y ∈ Z 2 ,
and are different elsewhere, as illustrated in Figure 3.

3. Let us show that U γ l R γ k ,x f i = R γ k ,γ l x U γ l f i either for i = 1 or i = 2 or both.
The set of points where f 1 and f 2 coincide implies in particular that R γ k ,0

f 1 = R γ k ,0 f 2 . Then we have R γ k ,0 U γ k f 1 = R γ k ,0 U γ k f 2 , as R γ k ,0 U γ k f i : (s, y) → f i (γ k s, y), and f 1 (γ k s, y) = f 2 (γ k s, y) for y / ∈ kZ 2 .
Note that R γ k ,0 U γ k is nothing else than an upsampling followed by a downsampling, as in Figure 3

.3. Since, on the other hand, R γ k ,0 f 1 = R γ k ,0 f 2 , we get U γ k R γ k ,0 f 1 = U γ k R γ k ,0 f 2 . Hence, either R k,0 U k f 1 = U k R k,0 f 1 or R k,0 U k f 2 = U k R k,0 f 2 or both, proving our point.
The sufficient condition (3.8) is not verified in general, but Proposition 1 shows a case where it is valid. and, on the other hand

Proposition 1. For N ∈ N * and i ∈ {1, . . . , N }, let U i = {U γ n i +l f i |l ∈ N}, where each f i : G → R n is a function on G and each n i an integer. Let n 0 ≤ min{n i |i = 1, . . . , N } and U = N i=1 U i . Then for all f ∈ U, and k, l ∈ N such that k -l ≤ n 0 , U γ l R γ k ,x f = R γ k ,γ l x U γ l f. (3.9) Proof. First, consider k < m R γ k ,0 • U γ k f (γ p , y) = (U γ k f )(γ k γ p , γ k y) = f (γ k γ p , y) so R γ k ,0 U γ m f (γ p , y) = U γ m-k f (γ p+m , y). Now, let f = U γ m f i ∈ F , k ≤ min{n i |i = 1, . . . , N } ≤ m, we have U γ R γ k ,x f (γ p , y) = U γ R γ k ,x U γ m f i (γ p , y) = U γ R 1,x R γ k ,0 U γ m f i (γ p , y) = R 1,γx U γ U γ m-k f i (γ p+m r, y) = U γ m-k+1 f i (γ m+p , y + γx)
R γ k ,γx U γ f (γ p , y) = R γ k ,γx U γ U γ m f i (γ p , y) = R 1,γx R γ k ,0 U γ m+1 f i (γ p , y) = R 1,γx U γ m-k+1 f i (γ p+k , y) = U γ m-k+1 f i (γ p+m , y + γx) = U γ R γ k ,x f (γ p , y), implying U γ R γ k ,x f = R γ k ,γx U γ f
. Repeated application gives us the desired result.

This property states that upsampling behaves as an equivariant operator as long as the input image is an upsampling of some image in a base scale. It can be interpreted as saying that the downscaling should not destroy information of the images in U. We model this by constraining the scaling factors of the downscaling actions and assuming that the objects of interest in an image are sufficiently big. We would like to point out that this hypothesis is never verified but reasonable for most of the datasets for semantic segmentation.

Before moving on to the experimental part, let us sum up the theoretical properties of a SEU-Net Γ = Π•Σ•Λ. By our construction we can hope for an approximated scale-equivariance Γ • R γ k ,z ≈ R γ k ,z • Γ. Three approximations prevent from exact equivariance. The first one is the approximated equivariance of the projection operator Π, which is intrinsic to the lifting approach. The second one is the discretization of the pooling operators. The last one is the assumption to guarantee an equivariant upsampling, which is never verified in practice. We will see in our experiments that the SEU-Net shows a high degree of scale-equivariance despite these approximations. Furthermore, note that each of these approximations is intrinsic to the problem. If the problem was formulated in the domain S γ × R 2 instead of G, all of these inaccuracies, except for the one coming from the projection layer, would be avoided on paper, but they would all be present at implementation. The same is true for every scale-equivariant network using an upsampling satisfying our definition.

Experiments

Oxford IIIT Pet

The Oxford-IIIT Pet1 dataset [START_REF] Parkhi | Cats and dogs[END_REF] consists of pictures containing cats and dogs. The relevant labeling for this paper, the trimaps, is the segmentation of the images into three classes: the animal, the background and the boundaries of the animal.

DIC-HeLa Cells

Another dataset to test the segmentation networks is a cell segmentation dataset, namely the DIC-C2DH-HeLa dataset [START_REF] Ulman | An objective comparison of celltracking algorithms[END_REF] of HeLa cells on a flat glass recorded by differential interference contrast (DIC). We used 83 images for train/validation and 83 for testing. Columns (a) and (b) in Figure 3.12 show examples images images from the test set and its label in different scales.

Results

Tomographic Images

During training, we extract random 256 × 256 patches of each image as input. As before, we resize test images creating a different test set for each s ∈ {2 i2 |i ∈ {-4, -3, . . . , 3}}. Both U-Net and SEU-Net have height four and four filters in their first layer. The lifting contains six scales and filters have scale dimension equal to one. We use the Mean Squared Error (MSE) to measure the performance and as the loss function and train it with the Adam optimizer.

With the aim of rendering the network more robust to change in scale, we propose a scale dropout applied before projection. We show the results with dropout only for the tomographic images experiment as it did not improve the results in the Pets dataset. For every scale coordinate s ∈ S γ , the dropout layer sets to 0 all pixels f (s, x), x ∈ Z 2 with some fixed probability p. We assume that the feature map f in this case is the output of a ReLU activation, so its minimum value is 0. Effectively this implies that the max-projection will ignore that scale. In all the equivariant models in the tomographic images we apply a dropout with probability p = 0.5.

The overall results are shown in improvement compared to the U-Net, but not enough to be better than strided scale-crosscorrelation pooling.

Oxford IIIT Pet

The compared U-Net and SEU-Net are both of height four and contain eight filters in the first layer. Both the U-Net and SEU-Net have height four and contain sixteen filters in the first layer and use 3 × 3 filters. The SEU-Net truncates at four scales, and filters have depth one in the scales dimension (their values is different from zero in one scale value). The networks are trained using the Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] optimizer with categorical cross-entropy loss. Training the U-Net, SResNet and SEU-Net takes approximately 24, 73, and 97 seconds per epoch respectively, on a Tesla P100-SXM2-16Gb GPU.

Comparison with data augmentation. We also performed scale jittering in the U-Net to compare the effect of the equivariant network with the effect of data augmentation. Scale jittering is performed by rescaling the image by a randomly chosen scale α and either random cropping or padding to the original image. We trained a U-Net with scale jittering in the loss does not improve for 30 epochs. We use a batch size of 8.

Results. The overall results in terms of the IoU are shown in Figure 3.8. Firstly we notice that the SEU-Net increases performance compared to both SResNet and U-Net. SResNet, however, does not consistently generalize better than the U-Net. Dropout improves the quality of SEU-Net, particularly for more extreme scales, indeed, for larger scales the augmented U-Net has a better IoU than the SEU-Net without scale dropout, but not than the one with scale dropout. The augmented U-Net loses performance scale 1, it would probably need to be larger to retain the same performance. We show examples of the predictions of the U-Net and SEU-Net in Figure 3.11. We also did a comparison of the models with different kinds of scale-space poolings in Figure 3.9. The results of poolings other than strided cross-correlations is generally poor compared with strides.

Cell Segmentation

We also evaluate the models in a medical image segmentation dataset, namely the DIC-C2DH-HeLa dataset [START_REF] Ulman | An objective comparison of celltracking algorithms[END_REF] of HeLa cells on a flat glass recorded by differential interference contrast (DIC). We used 83 images for train/validation and 83 for testing. Figure 3.12 (a) and (b) shows an example from the test set with its labels at different scales.

Models are trained with the AdamW optimizer [START_REF] Loshchilov | Decoupled weight decay regularization[END_REF]. Like in the previous experiment, we first train the models in the training set with the original scale distrubtion and test in the test set re-scaled by different values.

Training Details. We also perform scale jittering, but now for both U-Net and SEU-Net. For U-Net we trained models with scale jittering with ranges 4 (α is chosen each step from the interval [ 1 4 , 4]) and 1.5 (α is chosen from the interval [ 2 3 , 3 2 ]) and for SEU-Net we used only the range 1.5 jittering. All models, except for the U-Net with jittering 4 are trained for 200 epochs. The augmented U-Net with jittering 4 is trained for four times as many epochs. To train all models we apply data augmentation consisting of, rotations by a uniformly sampled angles in [-10 (b) Consistency Figure 3.9: Overall results in terms of IoU and Consistency for each scale of the Pet dataset. This time comparing different pooling functions and using the non-augmented U-Net's performance for reference.

and vertical flipping and elastic transformations. Learning rate starts at 10 -3 , weight decay starts at 10 -4 and both are reduced by exponential decay such that they are divided by 10 every 100 epochs (the decay stops at epoch 300 for the U-Net with size 4 jittering). We use a batch size of 1.

Results. Figure 3.10 (a) shows the IoU of different models on the re-scaled test sets. Figure 3.12 shows some segmentation examples. Again, the SEU-Net outperforms the U-Net. The poor results of the SResNet for smaller scales is possibly due to the cell images containing more high-frequency information, compared to the pets images. In contrast to the previous experiment, dropout did not seem to significantly increase performance of the SEU-Net, neither in the train scale nor the test scales. Moreover the SResNet results were greatly decreased due to dropout. This is likely a result of the agumented dataset being more difficult to segment than the original and not being representative of the dataset at base scale. The gain in generalization is only better than the SEU-Net for the smallest scales. The jittering with range 1.5 does not have a very noticeable effect. On the other hand the SEU-Net with 1.5 jittering has a noticeable gain in generalization to larger scales.

Discussion

In all three experiments the SEU-Net is shown to have a greater scale generalization capability than the U-Net. Furthermore, when looking at the comparisons in Figures 3.8 and 3.10(b) we see that the U-Net with augmentation attains a greater performance than the U-Net but at the cost of having a worse performance at the scale 1. We can conjecture that in order for the U-Net to perform better at both all scales it would be necessary to increase its size, i.e. the number of parameters, while simultaneously using jittering with range 4. The SEU-Net, on the other hand, has approximately the same number of parameters as the U-Net and does not need to have its size increased to perform well in multiple scales without losing its performance in the scale 1. Besides, both scale dropout and data augmentation by small scale jitterings were shown to increase the generalization and performance of the SEU-Net at virtually no additional cost. The SResNet does not seem to have the same nice properties, and the main difference between the SEU-Net and the SResNet is the presence of upsampling inside the equivariant pipeline.

Conclusions

In this chapter the non-equivariance to scale of the U-Net was pointed out. Moreover we point some possible problems that could arise from constructing equivariant networks for segmentation without closely examining the downsampling and upsampling layers. For example, the SResNet is a network build on scale-cross-correlations, but instead of performing gradual upsamplings it performs an upsampling by a large factor at the last layer.

Here the SEU-Net was proposed in order to render the U-Net invariant. In order to define the SEU-Net, each building block of the U-Net was studied and a scale-equivariant counterpart scale-equivariant is proposed. The relevant building blocks are: convolutions, pixelwise activations, batch-normalization, subsampling and upsampling. In the SEU-Net, convolutions are substituted by scale-cross-correlations. Pixelwise activations and batch normalization are already equivariant by virtue of being applied in a pixelwise manner. Subsampling and upsampling needed further study to assert their scale-equivariance.

The naive extension of the max-pooling to the signals on the semigroup was shown not to be scale-equivariant, but strided scale-cross-correlations are indeed scale-equivariant, moreover, a class of approximately scale-equivariant operators, which are discretizations of scale-equivariant operators, was proposed as pooling functions.

Upsampling was shown to not be scale-equivariant in general, but sufficient conditions for it to be scale-equivariant were shown. In that way, images were the objects of interest are sufficiently big for a given subsampling can garantee the equivariance of the upsampling.

In the experimental section the SEU-Net was tested in three tasks, namely the segmentation of tissue strands, the segmentation of natural images of the Oxford-IIIT Pet dataset and the segmentation of cells from the DIC-HeLa dataset. The SEU-Net was tested using multiples sets of parameters depending on the pooling function and the use of dropout. Moreover it was compared with two other models, the SResNet and the U-Net. Experimental results show that the SEU-Net can greatly improve the generalization to new scales and even the performance in the training scale. Moreover, the results lead us to conjecture that the U-Net with scale jittering would need to have more parameters to have a good performance in all the range of scales, while the SEU-Net achieves good results without increasing its size. The results suggest that implementing that the improvement comes not only from the scale-equivariant cross-correlations, but also from the SEU-Net global architecture and applying the pooling and upsampling operators inside the equivariant pipeline. The proposed scale dropout was also shown to have the potential to increase scale-equivariant models' performance. In future works it would be interesting to study an equivariant regularization term such as in [START_REF] Sosnovik | Disco: accurate discrete scale convolutions[END_REF] in addition to the scale-dropout.

Résumé en Français

Dans ce chapitre l'absence d'équivariance par rapport au changement d'échelle du réseau de segmentation U-Net a été soulignée. Par ailleurs, nous avons montré des problèmes qui peuvent survenir lorsqu'on propose un réseau équivariant par changement d'échelle pour la segmentation sans examiner les opérations de sous-échantillonnage et suréchantillonnage. Le SResNet [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF], par exemple, est un réseau basé sur la couche de corrélation-croisée mais au lieu de faire des suréchantillonnages graduels, le suréchantillonnage est fait une seule fois avec un facteur d'échelle très élevé dans la dernière couche.

Ici nous avons proposé le U-Net Équivariant par Changement d' Échelle (SEU-Net). Pour obtenir le SEU-Net, chaque bloc de U-Net est étudié et une alternative équivariante est proposée. Les blocs pertinents sont : convolutions, activations ponctuelles, normalisation par lot, sous-échantillonnage et suréchantillonnage. Dans le SEU-Net, les convolutions sont substituées par des corrélations-croisées d'échelle. Les activations et les normalisations par lot sont déjà équivariantes en vertu d'être appliquées ponctuellement. Le sous-échantillonnage et le suréchantillonnage ont eu besoin d'une étude plus approfondie pour pouvoir évaluer leur équivariance.

L'extension naïve du max-pooling aux signaux sur un semi-groupe n'est pas équivariante, mais la corrélation-croisée d'échelle avec strides (i.e. en appliquant les filtres un pixel sûr deux) l'est. Par ailleurs, une classe d'opérateurs approximativement équivariants par changement d'échelle a été proposée pour le sous-échantillonnage, donnée par la discrétisation de certains opérateurs équivariants par changement d'échelle.

Nous avons montré que le suréchantillonnage n'est pas équivariant par changement d'échelle en général, mais il existe des conditions suffisantes pour qu'il soit équivariant. Nous pouvons garantir l'équivariance si les objets d'intérêt son suffisamment grands pour que le sous-échantillonnage ne détruise pas d'information.

Dans la section expérimentale le SEU-Net a été testé sur trois tâches: la segmentation de torons de tissu, la segmentation d'images naturelles dans la base de données Oxford-IIIT Pet [START_REF] Parkhi | Cats and dogs[END_REF] et la segmentation de cellules de la base de données DIC-Hela [START_REF] Ulman | An objective comparison of celltracking algorithms[END_REF]. Le SEU-Net a été testé en utilisant plusieurs configurations d'hyperparamètres des fonctions de sous-échantillonnage et probabilité de dropout d'échelle. Il a été comparé avec deux autres modèles : U-Net et le SResNet. Les résultats expérimentaux montrent que le SEU-Net peut augmenter considérablement la généralisation à des échelles qui ne sont pas représentées dans les données d'entraînement (et même la performance à l'échelle d'apprentissage) par rapport aux résultats donnés par les deux autres modèles. Par ailleurs, les résultats nous amènent à conjecturer que U-Net entraîné avec augmentation par changement d'échelle a besoin d'un nombre plus élevé de paramètres pour obtenir une bonne performance sur différentes échelles, alors que le SEU-Net obtient des bons résultats sur différentes échelles sans augmenter le nombre de paramètres. Les résultats suggèrent que l'amélioration ne vient pas seulement des corrélations-croisées d'échelle, mais aussi de l'architecture globale du SEU-Net qui applique le sous-échantillonnage et le suréchantillonnage avant de la projection. Dans des travaux futurs, il serait intéressant d'étudier une méthode de régularisation équivariante, comme dans [START_REF] Sosnovik | Scale-equivariant steerable networks[END_REF], en plus du dropout d'échelle. Chapter 4

Differential Invariants Blocks

Introduction

Transformation equivariance can be introduced to a neural network to make use of the symmetry intrinsic to data features in many deep learning tasks. Convolutional Neural Networks (CNN) for example, reduce the number of parameters and improve the results in computer vision compared to densely connected neural networks thanks to benefiting from the translation symmetry present in computer vision tasks. Since their introduction as a general framework (Cohen and Welling, 2016a), group equivariant networks have been extended to work with other transformation groups, most notably translations combined with rotations (Cohen and Welling, 2016a;[START_REF] Worrall | Harmonic networks: Deep translation and rotation equivariance[END_REF][START_REF] Shen | Pdo-econvs: Partial differential operator based equivariant convolutions[END_REF][START_REF] Marcos | Rotation equivariant vector field networks[END_REF][START_REF] Weiler | Learning steerable filters for rotation equivariant CNNs[END_REF][START_REF] Weiler | General E(2)-equivariant steerable CNNs[END_REF] or scalings [START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF]. Many approaches to group equivariant neural networks involve first lifting the data to a higher dimensional space obtained by sampling elements of the group and performing group convolutions there (Cohen and Welling, 2016a;[START_REF] Kondor | On the generalization of equivariance and convolution in neural networks to the action of compact groups[END_REF]. This effectively introduces equivariance to the network, but it has the downside of increasing the size and computational cost according to the group dimension, as well as requiring the group transformations to be discretized. Steerable CNNs [START_REF] Cohen | Steerable CNNs[END_REF][START_REF] Worrall | Harmonic networks: Deep translation and rotation equivariance[END_REF] constrain filters in a way that the convolution is equivariant, so it does not require to use signals on the group, but they support less parameters than a CNN with the same number of feature maps.

We propose to use adaptive filters that are rotated to match the local geometry of the image. Since they match the local geometry a rotation of the image implies a rotation of the filters, thus the application of the filters will define an equivariant operator, as illustrated in Figure 4.1. The method of moving frames [START_REF] Fels | Moving coframes: I. a practical algorithm[END_REF][START_REF] Olver | Generating differential invariants[END_REF] formalizes this concept and gives us tools to define such steerable kernels.

The method of moving frames was introduced by [START_REF] Cartan | La méthode du repere mobile, la théorie des groupes continus, et les espaces généralisés[END_REF] and can be applied to derive differential invariants on smooth manifolds. Given a manifold M and a Lie group G acting on it, the method is based on finding, for each point, a transformation from the group, in a way that elements in the same group orbit (i.e. elements related by a group transformation) are mapped to the same point. Given an operator F on M, by first transforming elements according to the moving frame it is possible to obtain an invariant operator ı[F ] corresponding to F . If F happens to be a differential operator, its invariantization is a so-called differential invariant. In this chapter we are mainly concerned with applying the method of moving frames to derive networks equivariant to the orientation-preserving isometries of the plane and the space. We apply these ideas to define SE(2)-equivariant networks for images.

The rest of the chapter is organized as follows: In Section 4.2 we discuss related works in the literature. Section 4.3 reviews the method of moving frames and the concepts that will be necessary for the derivation of the SE(2)-equivariant networks. Section 4.4 computes differential invariants of arbitrary order for SE(2) acting on images. In Section 4.5 introduces the main contribution of the chapter, the networks based on Gaussian derivatives, differential invariants and the method of moving frames in SE (2). In Section 4.6 we perform experiments to validate the use of Gaussian derivatives and investigate pooling method, and we subsequently test the method in the MNIST-Rot dataset, obtaining competitive results. Section 4.7 concludes the chapter.

Related Work

Group equivariant networks have been obtained in many ways, must many involve sampling group elements, lifting elements to a higher dimensional space (Cohen and Welling, 2016a;[START_REF] Shen | Pdo-econvs: Partial differential operator based equivariant convolutions[END_REF] using discretized group elements. The disadvantage of that approach is that computational cost is increased according to the number of samples. Our proposed approach does not require lifting the data to another space and equivariant operations are performed on the same space as the input data, instead. Moreover, we achieve equivariance to continuous rotations. In the literature, equivariance to continuous rotations was achieved using steerable filters [START_REF] Worrall | Harmonic networks: Deep translation and rotation equivariance[END_REF][START_REF] Weiler | Learning steerable filters for rotation equivariant CNNs[END_REF] but in contrast to our method, it supports less parameters than a CNN with the same quantity of filters.

In computer vision, differential invariants form a basis for many of the filtering schemes, specially in the theory of scale-spaces [START_REF] Weickert | Anisotropic Diffusion in Image Processing[END_REF][START_REF] Heijmans | Morphological scale-spaces, scale-invariance and lie groups[END_REF]. They have also been applied to the design of feature descriptors [START_REF] Tuznik | Affine differential invariants for invariant feature point detection[END_REF] and for the automatic learning of PDEs for image restoration [START_REF] Liu | Learning pdes for image restoration via optimal control[END_REF]. Moreover, convolutional neural networks have been compared with numerical schemes for solving PDEs [START_REF] Ruthotto | Deep neural networks motivated by partial differential equations[END_REF] and this idea was applied to the construction of equivariant neural networks [START_REF] Shen | Pdo-econvs: Partial differential operator based equivariant convolutions[END_REF] albeit not with the use of differential invariants.

Moving Frames and Differential Invariants 4.3.1 The Method of Moving Frames

The method of moving frames is a powerful tool that uses the differential properties of Lie groups and manifolds to derive operators invariants under the group action. In the following G denotes a Lie group, M a manifold and we denote the action of G on M as π g [z] = g • z, for g ∈ G and z ∈ M. A moving frame is a G-equivariant operator associating to each point in M a transformation in G, as formalized by the following definition

In this chapter wee shall implicitly consider local Lie group actions when we consider group actions. A local Lie group action of G on M is a smooth map π :

U → M where U ⊆ G × M is an open subset of G × M containing {e} × M, it satisfies one of the following • ∀g, h ∈ G, x ∈ M such that (g • h, x) ∈ U , π[g, π[h, x]] = π[g • h, x] (left action); • ∀g, h ∈ G, x ∈ M such that (h • g, x) ∈ U , π[h, π[g, x]] = π[g • h, x] (right action).
Similarly to group action we use the notation π g (x) := π(g, x) for all (g, x) ∈ U .

Definition 10 (Moving frame Fels and Olver (1998)). A moving frame is a G-equivariant map ρ : M → G, which specifically verifies ρ(g • z) = ρ(z) • g -1 for all g ∈ G, z ∈ M.

From the above definition we can deduce

ρ(g • z) • g • z = ρ(z) • g -1 • g • z = ρ(z) • z, (4.1)
which means that the mapping z → ρ(z) • z is constant over the orbits

O z = {g • z|g ∈ G}. (4.2)
From a moving frame one can obtain an invariant operator from an arbitrary differentiable operator. We use ı to denote the invariantization of an operator between manifolds F : M → N defined as

ı[F ](z) := F (ρ(z) • z). (4.3) We have that ı[F ] is invariant with respect to G, i.e. ı[F ](g • z) = ı[F ](z)
which can be derived directly from (4.1)

ı[F ](g • z) = F (ρ(g • z) • g • z) = F (ρ(z) • z) = ı[F ](z). (4.4)
To the end of reviewing properties of moving frames we recall some properties of group actions that will be useful for the discussion.

Definition 11 (Free and regular group action). Let π g be the action of a Lie Group G on a manifold M. We say that π g is • semi-regular if its orbits all have the same dimension;

• regular if it is semi-regular and every point z ∈ M has arbitrarily small neighborhoods U such that the orbits intersect U in pathwise connected subsets;

• free if the only group element that fixes all points z ∈ M is the identity, i.e. ∀g ∈ G, ∀z ∈ M, if g • z = z then g = e where e is the identity element.

• locally free if there exists an open neighborhood U of the identity such that for all ∀g ∈ U ∀z ∈ M, g • z = z =⇒ g = e.

Theorem 1. Fels and Olver (1998) A moving frame ρ exists in a neighborhood U ⊆ M containing z ∈ M if and only if G acts locally freely and regularly in U .

When G acts freely and semi-regularly near z ∈ M, a moving frame can be obtained by first defining a cross-section.

Definition 12 (Cross section). Let G act semi-regularly in M and have s-dimensional orbits. An embedded submanifold K ⊆ M of dimension m -s is a cross-section to the group orbits if it intersects each orbit transversally i.e. the tangent spaces of the intersecting orbit and the tangent space of the cross-section generate the tangent space of the intersection point in M. If this intersection is unique we call K a regular cross-section. Now let K be a regular cross-section to the orbits on an open set U ⊆ M containing z ∈ M and assume that G acts locally freely and regularly on U . Then, because K is regular on U the orbit O z intersects K in a single point, and because π g is free, there exists a single g z ∈ G such that g z • z ∈ K. Now define ρ : M → G as the function mapping each z to the g z the group element that takes z to the cross section ρ(z)

• z = g • z ∈ K. For some h ∈ G, we have both ρ(z) • z ∈ K and ρ(h • z) • h • z ∈ K, moreover, h • z ∈ O z , therefore, because the intersection is unique, we get ρ(z) • z = ρ(h • z) • h • z.
As shown in the equation below, this implies that ρ is a moving frame, by Definition 10. Indeed,

ρ(z) • z = ρ(h • z) • h • z ⇐⇒ h -1 • ρ(h • z) -1 ρ(z) • z = z ⇐⇒ h -1 • ρ(h • z) -1 = ρ(z) -1 ⇐⇒ ρ(h • z) -1 = h • ρ(z) -1 ⇐⇒ ρ(h • z) = ρ(z) • h -1 (4.5)
where the equivalence from the second line to the third is due to freeness.

In order to simplify the calculations of the moving frame, K is usually a coordinate crosssection i.e. K is a set defined by points z ∈ M where r coordinates are set to constants, or, formally the cross-section a set of the form

K = {z ∈ M|z k 1 = c 1 , z k 2 = c 2 , . . . , z kr = c r }, (4.6)
where k i = k j if i = j, {k 1 , . . . , k r } ⊆ {1, . . . , m} and c i ∈ R for all i ∈ {1, • • • r}. In the case where K is a coordinate cross-section, then the moving frame can be obtained by finding, for each z ∈ M, a g ∈ G that solves the system of equations

w 1 (g, z) = c 1 w 2 (g, z) = c 2 . . . w r (g, z) = c r (4.7)
where

w i (g, z) = proj k i (g • z), i = 1, . . . , r, g ∈ G z ∈ M.
Example 1. As an example of a moving frame, consider the manifold M = R 2 \ {(0, 0)} with the group of rotations SO( 2) acting on it by

(x, y) T → R θ (x, y) T = (x cos θ -y sin θ, x sin θ + y cos θ) T (4.8)
where R θ ∈ SO( 2) is the rotation matrix by an angle θ ∈ [0, 2π). The group action is free because the only rotation having a fixed point in R 2 \ {(0, 0)} is the identity I = R 0 .

Let us consider the orbits of G in M. Given a point z ∈ M, its orbit is given by O z = {R θ • z|θ ∈ [0, 2π)} which are all the rotations of z around the origin (0, 0). The orbits are, therefore, all the circles centered at the origin. Moreover, since they are all circles they all have dimension dim O z = 1 for all z ∈ M. In Figure 4.9 in grey we have examples of the group orbits.

Consider the set K = {(x, 0) ∈ M|x > 0}, illustrated in Figure 4.2 in blue, given by the horizontal half-line from the origin to the positive x-axis. K also has dimension 1 and when a circle centered in the origin passes through K, its tangent is perpendicular to K, meaning that their tangent spaces are perpendicular at the intersection and generate a vector space of dimension two V ∼ = R 2 i.e. the tangent space of M. Therefore, K is a cross-section. Moreover, because for each positive real r > 0 there is only on value with that distance from the origin in K. Moreover, since for each positive real number r > 0 there is only one point in K at distance r from the origin, then each circle intersects K once, meaning that K is a regular cross-section. Given any z = (x, y) ∈ M, the rotation

R z = 1 x 2 + y 2 x y -y x ∈ SO(2) (4.9)
is the only one that brings z to the cross-section, i.e. R z • z ∈ K. Therefore, ρ : z ∈ M → R z is a moving frame.

Equivariant Function Operators from Invariant Manifold Operators

Now before we move on to obtain differential invariant operators, let us explain the interest of having invariant operators when our goal is to obtain equivariant neural network layers. We assume we are in the context where we compute the invariantization of operators F : M 1 → M 2 , with M 1 = X × Y and M 2 = Z, and the action G can be decomposed written as

g • (x, u) = (g • x, u), (4.10) (x 1 , x 2 ) ( x1 , 0) θ Figure 4
.2: Group orbits (grey) and cross-section K (blue) in M = R 2 \ {(0, 0)} under the action of SO (2).

for all g ∈ G, (x, u) ∈ X × Y . We define the actions on functions f : X → Y like in Chapter 1 (and analogously for functions f :

X → Z) as (g • f )(x) = f (g -1 • x). (4.11)
Now we want to show that for every G-invariant smooth operator ψ : X × Y → Z on M there is a corresponding operator ψ :

C ∞ (X, Y ) → C ∞ (X, Z) that is G-equivariant.
An equivariant operator in M 1 can be related to an equivariant one in the space of functions Y X . Suppose ψ :

M 1 = X × Y → Z is G-invariant, then take ψ : Y X → Z X to be ψ[f ](x) = ψ(x, f (x)), for all x ∈ X, f ∈ Y X .
Assuming that the action on Y is the identity g • u = u for u ∈ Y and the same for Z we have ψ

(g • f )(x) = ψ(x, (g • f )(x))) = ψ(x, f (g -1 • x)] = ψ(g -1 x, f (g -1 • x) = ψ(f )(g -1 • x) = [g • ψ(f )](x), (4.12) therefore g • ψ(f ) = ψ(g • f ) for all g ∈ G, f ∈ Y X .
In other words, an invariant operator in the Cartesian product X × Y to Z induces an equivariant operator taking functions in Y X to functions in Z X .

Jet-Space

In general, the action on M may not be free and regular. Indeed a necessary condition for this is that the dimension of the orbits O z of G be equal to the dimension of G, i.e. dim O z = dim G = r.

When the orbits are lower-dimensional, a solution might be to compute a moving frame in the higher dimensional jet space J n (M), for a large enough n, equipped with the prolonged group action [START_REF] Olver | Equivalence, invariants and symmetry[END_REF].

Bluntly speaking, a n-jet is the tuple of partial derivatives of order ≤ n of a section on a fibre bundle and the Jet-Bundle is a fibre bundle containing the spaces of n-jets as its fibres. In our case, we have to assume a fibre bundle structure on M.

We are mainly interested in jet-bundles of Euclidean spaces M = X × Y where

X = R d , Y = R. Given a multi-index I = (i 1 , • • • , i d ) ∈ N
(n) x the operator mapping C ∞ (X, Y ) to Y (n) d = R ( n+d n )
, and defined for any f ∈ C ∞ (X, Y ) by

J (n) x f = ∂ I f (x) I∈I d n = ∂ |I| ∂x i 1 1 ∂x i 2 2 • • • ∂x i d d f (x) I=(i 1 ,...i d )∈I d n . (4.13) Then given u (n) = u I I∈I d n ∈ Y (n) d and x ∈ X, the set C x,u (n) := (J (n) x ) -1 (u (n) ) is an equivalence class over C ∞ (X, Y ) for the equivalence relation f 1 ∼ f 2 ⇐⇒ J (n) x (f 1 ) = J (n)
x (f 2 ). This class C x,u (n) is represented in particular by the polynomial function defined, for any t ∈ X, by (4.14) with

u(t) = I=(i 1 ,...,i d )∈I d n u I I! (t 1 -x 1 ) i 1 . . . (t d -x d ) i d ,
I! = i 1 !i 2 ! . . . i d !.
It is the Taylor polynomial of order n at x of any function of the class.

The nth-order jet space of M, noted J n (M), is the union of all such equivalence classes, and can therefore be identified to X × Y

(n)

d . According to the above, for an element (x, u (n) ) = (x, (u I ) I∈I d n ), each u I is also a partial derivative of u evaluated in x, namely u I = ∂ I u(x). For example, if d = 3 and x = (x, y, z), u (0,0,0) = u(x), u (1,0,0) = ∂u ∂x (x) = u x (x), u (1,1,0) = ∂ 2 u ∂x∂y (x) = u xy (x) and so on. In practice we will often use these partial derivative notations to identify elements of the jet space, and omit the variable as it is explicit from the first component. For example in the case d = 2, n = 2, an element z ∈ M = J 0 (M) is identified by z = (x, u) = (x, y, z, u) and an element z (2) ∈ J 2 (M) by 2) ) = (x, y, z, u, u x , u y , u xx , u xy , u yy ) and analogously for higher orders. By prolonging the elements of M to the jet-space the goal is to find a new space where the group action acts freely. It is therefore necessary to know how exactly the group acts on the elements of J n (M). In the following we describe the prolonged group action which is the natural extension of the action π g on M to J n (M).

z (2) = (x, u (

Prolonged Group Action

The goal of looking into the jet-space is to have a higher dimensional space where the action of G is (locally) free and extends the manifold M. We have defined the jet-space, but it is left to know how G acts on it. This action is what we refer to prolongation of the group action to the jet-space.

An action on C ∞ (X, Y ) induces an action on the space of polynomials by restriction, (g • p)(t) = p(g -1 •t). Let (x, u (n) ) ∈ J n (M) and let u(t) be any function such that J (n)

x u = u (n) (for example the polynomial (4.14) associated to x and u (n) ). We define the prolongation of the action of G on M to the jet-space J n (M), as follows: for any g ∈ G and z

(n) = (x, u (n) ) ∈ J n (M), g • z (n) = g • (x, u (n) ) := (g • x, J (n) g•x (g • u)). (4.15)
The group transformation does not depend on the choice of the representative of u (n) , i.e., that for all

u, v ∈ C ∞ (X, Y ), if J (n) x u = J (n) x v then J (n) g•x g • u = J (n) g•x g • v.
This can be verified by checking that the expression above depends only on the values x and the derivatives J (n)

x u [START_REF] Olver | Applications of Lie groups to differential equations[END_REF]. We can prove that (4.15) is a left group action, i.e. for g 1 , g 2 ∈ G

g 1 • g 2 • (x, u (n) ) = g 1 • (g 2 • x, J (n) g 2 •x (g 2 • u)) = (g 1 • g 2 • x, J (n) g 1 •g 2 •x (g 1 • g 2 • u) = (g 1 • g 2 ) • (x, u (n) ). (4.16)
With the prolonged group action we can define a differential invariant.

Definition 13 (Differential invariant). Given manifolds M = X × Y and N , a differential invariant of order n is a smooth function

F : U → N , with U ⊆ J n (M) open, that is invariant under the (local) action of G, i.e. ∃V ⊆ G, ∀g ∈ V, ∀z (n) ∈ U, F (g • z (n) ) = F (z (n) ).
(4.17)

Since the jet-space of order n J n (M) is a manifold acted upon by G, if the action is locally free we can apply the method of moving frames to obtain the differential invariants of order n. It turns out that if n is large enough we can garantee the local freeness of the jet-space [START_REF] Olver | Generating differential invariants[END_REF][START_REF] Olver | Moving frames and singularities of prolonged group actions[END_REF]. Furthermore, a differential invariant defines an equivariant operator on functions just like an invariant in M defines an equivariant operator. Let ψ : J n (M 1 ) = X × Y (n) be a differential invariant and define ψ : (4.18) i.e. ψ • π g = π g • ψ.

C ∞ (X, Y ) → C ∞ (X, Z) as ψ[f ](x) = ψ(x, J (n) x f ). To see that ψ is G-equivariant, first notice that J (n) x (g • f ) = g • J (n) g -1 •x f . Now, we have ψ(g • f )(x) = ψ(x, J (n) x (g • f )) = ψ(x, g • J (n) g -1 •x f ] = ψ(g -1 x, J (n) g -1 •x f ) = ψ(f )(g -1 • x) = [g • ψ(f )](x),

Fundamental Invariants

Given a moving frame on a finite-dimensional manifold M there exists a set of generating invariants of order n from which all other invariants of order n can be obtained through functional combination.

First, let us consider an operator F : M → N invariant to an action of a Lie group G: ∀g ∈ G, ∀z ∈ M, F (g • z) = F (z). Then in particular, the existence of a moving frame ρ : M → G yields:

∀z ∈ M, F (z) = F (ρ(z) • z) = F (ı[z]) (4.19)
where we use the notation ı[z] = ı[id](z) = ρ(z) • z. Since we did not assume anything of I other than being an invariant, this applies to all invariant operators on M. Now recall from Section 4.3.

2 that J n (M) X × Y (n) d
where X = R d . Therefore an element

z (n) ∈ J (n) (M) is identified to the vector z (n) = (x, u (n) ) = x 1 , . . . , x d , (u I ) I∈I d n ,
where I d n is the set of d-dimensional multi-indices of modulus non-larger than n. We note proj i , 1 ≤ i ≤ d, the application that maps each z (n) ∈ J (n) (M) to its ith spatial coordinate x i , z (n) → x i , and proj I , I ∈ I d n , the one mapping z (n) to its coefficient u I of multi-index I, z (n) → u I . The fundamental invariants are the invariantized versions of these projectors.

Definition 14 (Fundamental invariants). The fundamental invariants of order n are the invariantizations (ı

[proj i ]) 1≤i≤d and (ı[proj I ]) I∈I d n of the d + n+d n coordinates projectors in the jet-space J (n) (M). By definition, ∀z (n) ∈ J (n) (M), ı[proj i ](z (n) ) = proj i (ı[z (n) ]) and ı[proj I ](z (n) ) = proj I (ı[z (n) ]) (4.20)
where, like earlier, ı[z n) . We will also note ı[

(n) ] = ρ(z (n) ) • z (
x i ] := ı[proj i ](z (n) ) and ı[u I ] := ı[proj I ](z (n) ).
From the above, we get that a G-invariant operator F defined on the jet-space verifies

∀z (n) ∈ J (n) (M), F (z (n) ) = F (ı[z (n) ]) = F proj 1 (ı[z (n) ]), . . . , proj d (ı[z (n) ]), (proj I (ı[z (n) ])) I∈I d n = F ı[proj 1 ](z (n) ), . . . , ı[proj d ](z (n) ), (ı[proj I ](z (n) )) I∈I d n = F ı[x 1 ], . . . , ı[x d ], (ı[u I ]) I∈I d n .
(4.21) In other words, any G-invariant operator defined on J (n) (M) is a function of the fundamental invariants of order n.

Differential Invariants of SE(2) on Images

Let us view a 2D image as a surface in M = R 3 given by the graph of a function u : Ω ⊂ R 2 → R with open Ω. We seek equivariance to the special Euclidean group, SE( 2), the group of planar rotations and translations. We denote g ∈ SE (2)

as g = (R, v) where R ∈ SO(2) is a planar rotation and v = (v 1 , v 2 ) ∈ R 2 is a translation. Its action π g , g ∈ G on a point (x, u) = ((x, y), u) ∈ M by π R,v (x, u) = (Rx + v, u) = ((x, ỹ), u) = ((x cos θ -y sin θ + v 1 , x sin θ + y cos θ + v 2 ), u) (4.22)
Before we can apply the method of moving frames we must assure that G acts freely on M. That is not the case, however. Take for example ((0, 0), u) and transform it by rotating it by any angle and we arrive at the same point, but the transformation is not the identity. Because the angle can be arbitrarily small this action is also not locally free. In order to be able to obtain a moving frame let us instead consider G acting on the n-th order jet-space J n (M) via the prolonged group action.

Given an element z (n) = (x, u (n) ) ∈ J n (M) we want to compute the prolonged action g •z (n) . In light of Section 4.3.2 we identify u(t) as the polynomial associated with u (n) . To obtain an expression for the prolonged action of G in the J n (M) we must compute the partial derivatives J (n) g•x (g • u) of the function g • u, according to (4.15).

∂ i+j ∂ xi ∂ ỹj (g • u) g•(x,y) = ∂ i+j ∂ xi ∂ ỹj u(g -1 • g • (x, y)) = ∂ i+j ∂ xi ∂ ỹj u(x, y) = ∂ i+j ∂ xi ∂ ỹj u((x -v 1 ) cos θ + (ỹ -v 2 ) sin θ, (v 1 -x) sin θ + (ỹ -v 2 ) cos θ). (4.23)
substituting the values t = g • x Keeping in mind the identification of the coordinates of the jet-space given by u (0,0) = u, u (1,0) = u x u (0,1) = u y and so on we can now obtain closed expressions for the prolonged action. In particular, the prolonged action on the first order jet-space J 1 (M) is π R,v (x, y, u, u x , u y ) = (x, ỹ, ũ, ũx , ũy ), where x, ỹ are the same as in (4.22), ũ = u and ũx = u x cos θ -u y sin θ ũy = u x sin θ + u y cos θ. By choosing an appropriate cross-section, we compute a moving frame ρ on J n (M) for n ≥ 1 from these five components only. Let K be the cross-section in J n (M) defined by

K = {z (n) ∈ J n |x = y = u y = 0, u x > 0}, near a point z (n) such that ∇u = (u x , u y ) = 0.
For every open set U ⊂ J n (M) such that ∀z (n) ∈ U, (u x , u y ) = (0, 0), K constitutes a regular cross-section, to see that first notice that it has 3 = dim SE(2) equations. Moreover when observing only the coordinates (u x , u y ) the orbits and cross-section reduce to those in Example 1 and there is a unique translation that maps (x, y) to (0, 0).

Writing the system of equations (4.7) gives us

x cos θ -y sin θ + v 1 = 0 x sin θ + y cos θ + v 2 = 0 u x sin θ + u y cos θ = 0. (4.26)
From the last constraint we can deduce

0 = ũy = u x sin θ + u y cos θ =⇒ θ = tan -1 - u y u x ,
hence θ is the opposite of the argument of ∇u. Moreover, the other parameters can be deduced from the other two constraints: v 1 = -x cos θ + y sin θ and v 2 = -x sin θ -y cos θ. The moving frame that is obtained from the cross-section K can thus be written in the form

ρ(z (n) ) = 1 ∇u u x u y -u y u x , -1 ∇u u x u y -u y u x x y . (4.27)
This completely determines ρ. Fundamental differential invariants can now be obtained by the invariantization of the partial derivatives of u.

Let us denote the fundamental invariants associated with the multi-index (i, j) as

I ij := ı[u i,j ]
. By the moving frame deduced from the cross-section K, we have ı[x] = ı[y] = 0. From (4.20), we know that given any differential operator F (z (n) ), its invariantization is given by

ı[F ](z (n) ) = ı[F ](x, y, (u i,j ) 0≤i+j≤n ) = F (0, 0, (I i,j ) 0≤i+j≤n ).
We show the fundamental invariants of the second order jet-space J 2 (M) in Table 4.1. They are computed by substituting (4.27) into (4.24) and (4.25).

ı[u] I 00 = u ı[u x ] I 10 = u 2 x + u 2 y ı[u xx ] I 20 = 1 ∇u 2 (u xx u 2 x + u xy u x u y + u yy u 2 y ) ı[u xy ] I 11 = 1 ∇u 2 (u x u y (u yy -u xx ) -u xy (u 2 y -u 2 x )) ı[u yy ] I 02 = 1 ∇u 2 (u xx u 2 y -u xy u x u y + u yy u 2 x )
Table 4.1: Invariantizations of the partial derivatives of u up to order 2, assuming ∇u = 0. These determine all other differential invariants of order two by functional combination, for example, the Laplacian ∆u = u xx + u yy = I 20 + I 02 .

Let us obtain a closed expression for the other invariants. First let us consider the polynomial u(t) associated to u (n) . If we expand the expression of the prolonged group action (4.23) we obtain

∂ n u ∂x i ∂y j (x, y) = ∂ n u(x,y) ∂ xi ∂ ỹj = ∂ n ∂ xi ∂ ỹj u(cos θ(x -v 1 ) + sin θ(ỹ -v 2 ), cos θ(ỹ -v 2 ) -sin θ(x -v 1 )) = ∂ ∂ x i ∂ ∂ ỹ j u(cos θ(x -v 1 ) + sin θ(ỹ -v 2 )), cos θ(ỹ -v 2 ) -sin θ(x -v 1 ))) = cos θ ∂ ∂x + sin θ ∂ ∂y i cos θ ∂ ∂y -sin θ ∂ ∂x j u(x, y).
(4.28) Substituting by the values of the moving frame ρ from (4.27), i.e., making the substitution cos θ = u x / ∇u and sin θ = u y / ∇u , and further expanding the expression results in

I ij = 1 ∇u n u x ∂ ∂x + u y ∂ ∂y i u x ∂ ∂x -u y ∂ ∂x j u = 1 ∇u n i k=0 i k u k x u i-k y ∂ i ∂x k ∂y i-k j l=0 j l (-1) l u j-l x u l y ∂ j ∂x l ∂y j-l u = 1 ∇u n i k=0 j l=0 i k j l (-1) l u k+j-l x u l+i-k y ∂ n ∂x k+l ∂y i+j-k-l u = 1 ∇u n n m=0 min{m,j} l=max{0,m-i} i m-l j l (-1) l u m-2l+j x u 2l-m+i y ∂ n ∂x m ∂y n-m u.

Invariants From a Recurrence Equation

In Sangalli et al. (2022a) we used a different method of computing the differential invariants of orders higher than two. There we based the computation of the invariants in the recurrence equations in [START_REF] Olver | Generating differential invariants[END_REF]. Those equations show that all differential invariants can be written as a functional combination of I 00 , I 02 and its invariant derivatives, which for SE (2) are (4.30) where D x and D y denote the total differentiation operators on the jet-space, which are the operators on the tangent bundle of the jet-space that given by D

D 1 = ı[D x ] := ∇u -1 (u x ∂ ∂x + u y ∂ ∂y ), D 2 = ı[D y ] := ∇u -1 (-u y ∂ ∂x x + u x ∂ ∂y ),
x = ∂ ∂x + 0≤i+j≤n u i+1,j ∂ ∂u i,j , and D y = ∂ ∂y + 0≤i+j≤n u i,j+1 ∂ ∂u i,j+1
. They are related to the partial derivatives in the sense that

D x F (x, u (n) ) = ∂ ∂x F (x, u(x)
) (and similarly for D y ) where u(x) is the polynomial associated to u (n) . The invariant derivatives (4.30) are equivalently the directional derivatives in the directions of the gradient and its perpendicular.

More specifically, the recurrence relation applied to SE(2) on J n (M) has as initial conditions I 00 and I 02 given in Table 4.1, and I 01 = 0, I 10 = D 1 I 00 , I 20 = D 1 I 10 , I 11 = D 2 I 10 and for i, j such that i + j = n ≥ 3:

I ij = -D 1 I i-1,j -P ij (F n-1 ) 1 I 10 , if i > 0 D 2 I 0,j-1 -P 0,j (F n-1 ) 1 I 10 , if i = 0. (4.31)
where the P ij are polynomials and F n are invariantizations of the derivatives up to order n, F n = (I i,j ) 0≤i+j≤n . The polynomials P ij can be determined by solving a system of equations, but since their purpose here is to show that the fundamental invariants of superior order are functional combination of the invariant derivatives of the fundamental invariant of order two, we do not need to know their explicit forms. The invariant derivatives do not commute, i.e. [D 1 , D 2 ] = 0 in general, but the following is valid

[D 1 , D 2 ] = D 1 • D 2 -D 2 • D 1 = J 1 D 1 + J 2 D 2 (4.32)
where J 1 and J 2 are some differential invariants or order ≤ 2. In particular this implies that all of the combinations of

D k 1 • • • D k K , k i ∈ {1, 2} and 0 ≤ K ≤ n generate all differential invariants of order n.
In the rest of the chapter we use the expressions (4.29) to compute the fundamental invariants, instead of using these recurrence relations as was done in Sangalli et al. (2022a).

Equivariant Neural Network Using Differential Invariants

Suppose we want to use differential invariants to obtain a SE(2)-equivariant neural network φ that takes signals on R 2 and outputs signals on R 2 . The following approach would work.

• Let an input be a function f ∈ C ∞ (X, Y ) = C ∞ (R 2 , R), we use a single-channel image
for simplicity here. The first step would be the computation of the differential invariants,

(ı[x], ı[y], (I 0 i,j ) 0≤i+j≤n ) = ρ(z 0 x ) • z 0 x = ρ(x, J (n) x f ) • (x, J (n) x f ).
where z 0 x := (x, J

(n)

x f ). From those invariants we can compute an arbitrary invariant of order n by applying a smooth function ψ 1 : J n (M) → R C . In a deep learning context such function can be represented/approximated by a Multi-Layer Perceptron (MLP). The output of the first layer is given by

f 1 (x) = φ 1 (f )(x) = ψ 1 (ı[x], ı[y], (I 0 i,j ) 0≤i+j≤n )
• Now from f 1 we compute the second layer in a similar manner. Let us denote

f 1 = (f 1 1 , . . . , f C 1 )
. The fundamental invariants are given by

(ı[x], ı[y], (I 1,k i,j ) 0≤i+j≤n ) = ρ(z 1,k x ) • z 1,k x = ρ(x, J (n) x f k 1 ) • (x, J (n) x f k 1 ).
where

z 1,k x := (x, J (n) x f k 1 ), k = 1, . . . C. Given a smooth function ψ 2 : (J n (M)) C → R C we can compute the second layer by φ 2 (f 1 )(x) = ψ 2 (ı[x], ı[y], (I 1,1 i,j ) 0≤i+j≤n , . . . ı[x], ı[y], (I 1,C i,j ) 0≤i+j≤n ).
The output from the second layer given f is given by

φ 2 = φ 2 • φ 1 .
The proccess above can be repeated L times to obtain a L-layer networks φ L = φ. Note that at each step the operators φ l are equivariant by the fact that they are functional combinations of differential invariants, and thus equivariant operators on functions by (4.18).

SE(2) Differential Invariants Networks

In this section we define the SE(2) Differential Invariants Networks (SE2DINNet). A SE2DINNet is made of equivariant blocks (SE2DIN blocks) consisting in computing the derivatives, followed by computing the differential invariants of SE( 2) up to a certain order, followed by a series of 1 × 1 convolutions, in the style of Network in Network (NiN) [START_REF] Lin | Network in network[END_REF]. We illustrate the SE2DIN block in Figure 4.5 and we will explain its components in this section.

Gaussian Derivative Layers

In order to compute the differential invariants we use Gaussian derivatives [START_REF] Koenderink | Representation of local geometry in the visual system[END_REF]. In classical computer vision Gaussian derivatives were combined with differential invariants to perform automatic scale selection [START_REF] Lindeberg | Edge detection and ridge detection with automatic scale selection[END_REF][START_REF] Lindeberg | Principles for automatic scale selection[END_REF]. Gaussian derivatives have also been used in neural networks to produce structured receptive fields in CNNs [START_REF] Jacobsen | Structured receptive fields in CNNs[END_REF][START_REF] Penaud-Polge | Fully trainable gaussian derivative convolutional layer[END_REF] and they provide a regularization effect to the CNN. In this work we combine Gaussian derivatives with differential invariants and apply it to build equivariant neural networks. Gaussian derivatives are used to compute the derivatives of a Gaussian filtered image, i.e. (4.33) where In [START_REF] Jacobsen | Structured receptive fields in CNNs[END_REF] this is used to define a basis of filters such that a CNN filter h : R 2 → R at scale σ can be written as

∂ i+j ∂x i ∂y j (u * G σ ) = u * ∂ i+j ∂x i ∂y j G σ = u * G i,j σ ,
G i,j σ = ∂ i+j Gσ ∂x i ∂y j . G 0,0 σ G 1,0 σ G 0,1 σ G 2,0 σ G 1,1 σ G 0,2 σ
h(x) = 0≤i+j≤n c i,j G i,j σ (x) (4.34)
so that the convolution by h becomes

(f * h)(x) = R 2 f (y)h(x -y)dy = R 2 f (y) i+j≤n c i,j G i,j σ (x -y)dy = i+j≤n c i,j R 2 f (y)G i,j σ (x -y)dy = i+j≤n c i,j (f * G i,j σ )(x), (4.35)
that is, the convolution by h can be obtained by linear combinations of the Gaussian derivatives of f . The basis of filters G i,j σ is shown to form a complete description of the local geometry of an image [START_REF] Koenderink | Representation of local geometry in the visual system[END_REF].

A one dimensional Gaussian derivative can be computed by means of the probabilist's Hermite polynomials. Let

H m = (-1) m e x 2 2 d m dx m e -x 2
2 , with m ∈ N be the Hermite polynomial of order m. The Gaussian derivative of order m in one variable, denoted G m σ can be computed as

G m σ (x) = (-1) m 1 √ σ m H m x σ √ 2 G σ (x). (4.36)
Furthermore, a 2-dimensional Gaussian derivative is a separable filter, i.e.

G i,j σ (x, y) = G i σ (x)G j σ (y) (4.37)
and can therefore be expressed with the one-dimensional Hermite polynomials. Moreover separability means that the discrete approximation of f * G i,j σ can be obtained by computing a convolution with a horizontal filter, with the values of G i σ (x) in the line y = 0 and a vertical one with the values of G j σ (y) in the line x = 0. Given a multi-channel function f : Z 2 → R C , a Gaussian derivative block is the concatenation of a layer computing Gaussian derivatives of each channel of f and a layer computing C linear combinations of those derivatives, which can be computed as 1 × 1 convolutions. A Gaussian derivative block is illustrated in Figure 4.4. The output of a Gaussian derivative block is a signal f :

Z 2 → R C given by, for o ∈ {1, . . . , C } f o = γ C i=1 f i * h i,o (4.38)
where for every i, o h i,o is a linear combination of Gaussian derivatives of scale σ like (4.34) and γ is a non-linearity function such as ReLU.

Gaussian filters are already SE(2)-equivariant, so their composition with a differential invariant yields a SE(2)-equivariant operator. Moreover the Gaussian filters have the property of smoothing a discrete signal. These properties motivate the use of Gaussian derivatives as the first part of each SE2DIN block.

SE2DIN Block

From the derivatives, we compute the invariantizations of the derivatives up to order n, (I i,j ) 0≤i+j≤n . As any differential invariant I of order n is a functional combination I = F ((I i,j ) 0≤i+j≤n ) [START_REF] Olver | Generating differential invariants[END_REF], a neural network with one hidden layer and taking as input (I i,j ) 0≤i+j≤n , can approximate any differential invariant of order n, according to the Theorem of Universal Approximation [START_REF] Leshno | Multilayer feedforward networks with a nonpolynomial activation function can approximate any function[END_REF]. In practice we compute a l-layer densely connected network, with l ≥ 2, applied at every spatial point (x, y) ∈ Ω which takes as input the vector (I i,j ) 0≤i+j≤n . Computationally this is equivalent to a series of l 1 × 1 convolutions with pointwise activation functions applied to the multi-channel image with values A(x, y) = (I i,j ) 0≤i+j≤n . The output of the first 1 × 1 convolution is equivalent to the application of an adaptive filter that is rotated according to the gradient at each point of the image, as illustrated in Figure 4.7. We visually compare the effect of a rotation on the input of a CNN vs a rotation on the input of a SE2DIN block in Figure 4.6.

As the moving frame was derived assuming ∇u = 0, I ij (x, y) is not defined for some i, j at (x, y) when ∇u(x, y) = 0. Therefore, we introduce the normalized invariants Īi,j (x, y) given by Īi,j (x, y)

= n m=0 min{m,j} l=max{0,m-i} i m-l j l (-1) l u m-2l+j x (x 0 , y 0 )u 2l-m+i y (x 0 , y 0 ) ∂ n ∂x m ∂y n-m u(x 0 , y 0 ) = lim (x 0 ,y 0 )→(x,y)
∇u(x 0 , y 0 ) n I i,j (x, y).

(4.39) They are still invariants as they are the limit of a function of invariants and are well defined when ∇u(x, y) = 0 as they are a polynomial expression in these values. Moreover the original invariants can be recovered with functional combinations, specifically, because Ī10 = ∇u 2 , then

I ij = 1 √ Īi+j 10
Īij therefore the normalized invariants form a generating set.

input image

H × W × C
Gaussian derivatives

H × W × C n+2 n differential invariants H × W × C n+2 n -1 • • • feature maps H × W × C • • • 1 × 1 convolutions 1 × 1 convolutions

Complexity Analysis

Assuming that the input feature maps have height H, width W and have C feature maps we can estimate the cost of computing the differential invariants up to order n by this method. Using either Gaussian derivatives or finite differences, each derivative can be computed as a separable filter, let us assume that all filters are of length K × K, then computing all of the Gaussian The cubic dependency on n may seem like a significant drawback but in practice the value of n is kept small as it was shown that a value as low as 4 of finite differences filters can represent any 3 × 3 convolutional filter [START_REF] Ruthotto | Deep neural networks motivated by partial differential equations[END_REF]. Overall the computation of the fundamental differential invariants takes O HW C n+2 n K + n 3 flops which is akin to a depthwise convolution if we assume n and K are fixed.

Rotation-Equivariant Pooling

Pooling is a standard layer in many network architectures and can be obtained by

P = D t • ϕ. (4.40)
where an operator ϕ is composed with a downsampling operator D t , t > 1, defined, for u :

Z d → R C
, where is a grid, by In order for a network containing a pooling layer to be rotation equivariant, the pooling layer must be rotation equivariant. A straightforward answer is to substitute the maximum in a square neighborhood of the max-pooling by the maximum in a circular neighborhood.

∀x ∈ Z d , D t (u)(x) = u(tx). (4.41)
In practice, however, images are defined on a grid Ω ⊆ Z d . In that case, the downsampling D t of (4.41), but with t ∈ N, can induce a large deviation from perfect equivariance. Indeed, using t = 2 and for images with an even dimension, even for 90 • rotations the downsampling D 2 is not rotation equivariant due to the grid shape of the images as illustrated in Figure 4.8. One way to achieve better results is to ensure that images have odd sizes in both dimensions, because in that case the downsampling by D 2 commutes with rotations by 90 • . An alternative that works for any size is to, given a function u and for every angle multiple of 90 • θ, to rotate u by θ, downsample and then rotate by -θ, and then compute an aggregation function of those operators, that is Dt

(u)(x) = Aggr(R i -π 2 • D t (R i π 2 • u)(x) i=0,1,2,3 ) (4.42)
where Aggr is some permutation invariant function, like the average or the max, and R θ denotes the rotation of angle θ. Dt is equivariant to rotations by

k π 2 , k = 0, 1, 2, 3 because the output of Dt given R k π 2 • u at R k π 2 •
x is the aggregation of the same four values as the output of Dt given u and x.

Alternatively, if we choose to compute a discrete approximation of R θ for angles not multiple • by a factor of 2. The position (0, 0) is assumed to be the pixel at the top-left corner. Of course in real scenarios images have a much bigger dimension and do not necessarily have such high frequency information, but the subsampling will still generate small errors to equivariance that can propagate to the next layers. In the experimental section it will be shown that the choice of pooling does have a significant impact in practice.

of π 2 , then we can use a smaller angle than π 2 in (4.42). In the following, we fix Aggr = max and we refer to (4.42) as P4Strides. In the case t = 2, D2 is equivalent to a simple subsampling for images with odd dimensions and to a 2 × 2 max-pooling for images with even dimension.

Architecture

We define the SE(2) Differential Invariants Network (SE2DINNet) as a network constructed by SE2DIN blocks applied with residual connections like the ResNet [START_REF] He | Deep residual learning for image recognition[END_REF]. In that way we compute a SE2DIN block from current feature map and add it to the feature map. The SE2DINNet is the repeated application of the previous step, illustrated in Figure 4.9. Given the interpretation of SE2DIN blocks as differential invariants, a SE2DINNet is viewed as a numerical scheme to solve a time-varying PDE obtained by applying an analogous to the Finite Difference method to solve

u in SE2DIN block SE2DIN block + SE2DIN block + SE2DIN block + • • • SE2DIN block + SE2DIN block φ Global MaxPooling u out
u t = F t ((I i,j ) 0≤i+j≤n ). (4.43)
Namely, given the input u 0 , we obtain u 1 , u 2 , . . . , u L , where u l is the output of the l -th layer and the solution of (4.43), at the time l∆t, with ∆t the temporal step. That is,

u l = u l-1 + ∆tF l ((I σ i,j (u l-1 )) 0≤i+j≤n )), (4.44)
where I σ i,j (u l-1 ) are the differential invariants computed from the Gaussian derivatives of u l-1 , i.e. the differential invariants of a Gaussian filtered u l-1 .

We restrict the temporal step to ∆t = 1. The network in Figure 4.9 before global maxpooling, φ is SE( 2

)-equivariant, i.e. φ((R θ , v) • u) = (R θ , v) • φ(u) for u : R 2 → R C 0 .

Experiments

Ablation Study

We begin by individually examining the effect of different components on the SE2DINNet. There are two components which are going to be examinated: (i) the use of Gaussian derivatives and (ii) the choice of pooling functions. In order to test the parameters we use MNIST (LeCun et al., 2010) and Fashion MNIST [START_REF] Xiao | Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms[END_REF]. Models are trained on the original training set of those datasets, but in order to evaluate the rotation invariance of the models we test at rotated test sets. The test sets are all rotations of the original test sets with angles given by i π 12 0 ≤ i ≤ 23. Let φ denote the features of the network, i.e. the result of the global max-pooling layer and let φ denote the output of the network, i.e. the prediction into one of the possible labels. Also let u 1 , . . . , u N be the images of the test set. As evaluation metrics we use • the overall accuracy, i.e. the proportion of correctly classified images for each angle

• the consistency i.e. the probability that an image and its rotation result in the same class,

Consistency(φ , θ) = 1 N N i=1 X (φ (u i ), φ (R θ • u i )) (4.45)
where X (A, B) = 1 if A = B and 0 otherwise.

• the Mean Square Error (MSE)

MSE(φ, θ) = 1 N N i=1 φ(u i ) -φ(R θ • u i ) 2 φ(u i ) 2 (4.46)
• the cosine similarity

CosineSimilarity(φ, θ) = 1 N N i=1 φ(u i ), φ(R θ • u i ) φ(u i ) 2 φ(R θ • u i ) 2 .
(4.47)

Gaussian Derivatives

Different methods of computing the derivative layers are tested in this section: finite difference method, Gaussian derivatives with fixed σ and Gaussian derivatives with increasing σ with respect to layer depth. In both cases where Gaussian derivatives are applied, σ starts at σ = 1

Cosine Similarity

Finite Differences Gaussian Derivatives (same sigma) Gaussian Derivatives (increasing sigma)

Figure 4.10: Evaluation metrics comparing methods on MNIST using Gaussian derivatives and finite differences to compute the derivatives. Models are trained on the MNIST training set and tested in rotated versions of the test set with angles (in degrees) varying in the interval [-180, 180).

but in the increasing case it is increased by a factor σ mult = √ 2 each layer. Keep in mind that the purpose here is not to find the best value of σ or its proportion, but to find the effect of the method of computing derivatives on the invariance. Figures 4.10 and 4.11 contain evaluation metrics for each of the tested methods. Firstly we notice that all models have a periodicity of 90 • on every metric. This is due to the fact that everything is built for a square grid. In that case, numerical derivatives are equivariant to rotations by 90 • of the image. When rotations deviate from that margin, however, finite difference methods are much more degraded compared to the Gaussian derivative. This is evidence of the Gaussian derivatives robustness to rotations.

The results of the finite difference models also show a drastic accuracy difference with respect to the methods using Gaussian derivatives when evaluated in the unrotated test sets. This is partially due to the fact that the Gaussian derivatives have a larger receptive field than

Cosine Similarity

Finite Differences Gaussian Derivatives (same sigma) Gaussian Derivatives (increasing sigma) the 3 × 3 finite difference stencils which, with only five layers and no pooling in between, is only a square of side 11. Figures 4.12 and 4.13 show the same metrics for networks applying different kinds of subsampling, effectively increasing the receptive field sizes of the networks.

There we can see that even with the presence of pooling, the finite difference method falls short of the Gaussian derivatives in terms of equivariance. MSE plots in those figures also suggest numerical instability of the finite difference method. We can also compare the filters when placed at regions with different gradient angles. Indeed, for each point in the domain, the output value of the composition of the invariants and the first 1 × 1 convolution in the SE2DIN block is given by the scalar product of the input image and a certain filter. The process is illustrated in Figure 4.7. In Figure 4.14 we perform an example where we take one of the rotated filters of the first layer of a Gaussian-derivativebased SE2DINNet and a finite-difference-based one and we compute the filter that would be Cosine Similarity strides, Gaussian derivatives P4Strides Gaussian derivatives strides, finite differences P4Strides, finite differences no subsampling, same sigma no subsampling, increasing sigma Looking at results from Figures 4.12 and 4.13 it is interesting to notice how much the choice of pooling affects equivariance. Even though the difference between strides and P4Strides is small in paper, it completely alters the behaviour of the network on unseen angles, even for datasets obtained by rotating the test set by multiples of 90 • . Interestingly, when the distance and similarity metrics are computed for untrained models, we do not obtain the same kinds of results. As can be observed in Figure 4.15, even naive strided SE2DIN blocks have a better invariance according to those metrics than all trained models and corrected strides have an almost perfect equivariance, suggesting that optimization (specifically on a single-orientation dataset) pulls the model farther from equivariance. Looking again at results from Figure 4.12 and 4.13, it is interesting to see that the model using Gaussian derivatives and P4Strides method compares well with the method without subsampling but with increasing σ, and it has the advantage of reducing computation times for subsequent layers. Based on that observation P4Strides are used when subsampling is needed in SE2DINNets.

MNIST-Rot

The MNIST-Rot dataset [START_REF] Larochelle | An empirical evaluation of deep architectures on problems with many factors of variation[END_REF] was constructed by randomly rotating by an angle in [π, -π) each image from the MNIST 12k dataset [START_REF] Larochelle | An empirical evaluation of deep architectures on problems with many factors of variation[END_REF], which itself is obtained by selecting 12000 images from the MNIST dataset for training/validation and 50000 for testing. The dataset is widely used for benchmarking of rotation invariant classification models. We divide the MNIST-Rot training set into 10000 images for training and 2000 for validation. Differently from the data of the previous section, in the MNIST-Rot dataset every image, training and test, is assigned a new, fixed, orientation.

The architecture used for this task is shown in Figure 4.16. As for the orders of differentiation, we use two, three and four. The architecture is based on ResNet18 [START_REF] He | Deep residual learning for image recognition[END_REF] but with the inverted bottlenecks from ConvNext [START_REF] Liu | A convnet for the 2020s[END_REF], in other words, the SE2DIN block we use is constituted of a layer computing the differential invariants (which would be analogous to the 3 × 3 convolution in the ResNet), followed by two 1 × 1 convolutions, the first having four times the output size of the second, as shown in Figure 4.16(b). In the normal ResNet, 3 × 3 layer is the middle one, but since we have to compute a functional approximation of differential invariants we put the fundamental invariants in the first layer of the block and the other two are used to compute the function approximation. Moreover, the layer with more feature maps is usually at the end of the block, but we put it in the second place in order to require less computations of the fundamental invariants layer, the one which is more P4Strides pooling strides pooling no pooling, increasing sigma untrained, P4Strides untrained, strides untrained, inc. sigma For training, we used the AdamW optimizer [START_REF] Loshchilov | Decoupled weight decay regularization[END_REF], i.e. the Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] optimizer with decoupled weight decay. We used a starting learning rate of 10 -3 and weight decay values of 10 -4 for the order two model and 2.5 • 10 -4 for the orders three and four models. The models were trained for 300 epochs with a batch size of 128 and the learning rate and weight decay are divided by 10 every 100 epochs.

Our results, as well as those of the state-of-the-art methods are shown in Table 4.2. We can see that the proposed method is still below the performance of the state-of-the-art but we can see that the method still achieves a good performance nonetheless, compared with the models that do not use data augmentation. Our model also has the advantage of being lightweight, as the number of floating point operations that it takes to compute the outputs is similar to that of a CNN with the same number of trainable parameters.

In the course of training the SE2DINNets, the main issue was ensuring numerical stability. Because the invariants form non-linear expressions which may cause the variance of the output to differ greatly from that of the input, it can in turn cause the gradients of the neural network computed by backpropagation to explode [START_REF] Glorot | Understanding the difficulty of training deep feedforward neural networks[END_REF]. In this experiment we used activity regularization (using norm l 2 and a value of 10 -5 ) in the computation of the invariants and found empirically that it helps dealing with these problems, but in Chapter 5 we propose a solution that is based on architectural design is much more effective to mitigate these issues.

Method

Test error(%) #params H-Net [START_REF] Worrall | Harmonic networks: Deep translation and rotation equivariance[END_REF] 1.69 33k P4CNN (Cohen and Welling, 2016a) 2.84 25k Z2CNN (Cohen and Welling, 2016a) 5.03 22k PDO-eConv [START_REF] Shen | Pdo-econvs: Partial differential operator based equivariant convolutions[END_REF] 1.87 26k PDO-eConv † [START_REF] Shen | Pdo-econvs: Partial differential operator based equivariant convolutions[END_REF] 0.709 650k RotEqNet [START_REF] Marcos | Rotation equivariant vector field networks[END_REF] 1.09 100k RotEqNet ‡ [START_REF] Marcos | Rotation equivariant vector field networks[END_REF] 1.01 100k E2CNN † [START_REF] Weiler | General E(2)-equivariant steerable CNNs[END_REF] 0.716 -E2CNN † [START_REF] Weiler | General E(2)-equivariant steerable CNNs[END_REF] 0.682 -SFCNN [START_REF] Weiler | Learning steerable filters for rotation equivariant CNNs[END_REF] 0.880 6.5M SFCNN † [START_REF] Weiler | Learning steerable filters for rotation equivariant CNNs[END_REF] 0.714 6.5M SE2DINNet, order two (Ours)

1.21 380k SE2DINNet, order three (Ours)

1.27 630k SE2DINNet, order four (Ours)

1.45 920k

Conclusions

In this chapter we reviewed the method of moving frames used to obtain differential invariants, and applied it to derive differential invariants of SE( 2) on two-dimensional signals. The computed differential invariants, called fundamental invariants, of a certain order n form a set that can generate all other invariants of order n through functional combination, and conversely every functional combination of these invariants is an invariant. These properties allow for the definition of a neural network layer, called SE2DIN block, that takes as input a stack of feature maps, computes their Gaussian derivatives, which are used to obtain fundamental differential invariants and combine the latter through a pixel-wise multi-layer perceptron , allowing us to approximate arbitrary differential invariant of order n.

The neural network, called SE2DINNet, obtained from the differential invariants was implemented for discrete images using Gaussian derivatives, as they are theoretically and empirically more adequate to this task than finite difference filters. This network is lightweight computation-wise, as computing the invariants has a computational complexity similar to that of a depthwise convolution.

The performance of these networks in a context where invariance matters was evaluated on the MNIST and FashionMNIST datasets by training on the normal training set and testing on rotated test sets. Moreover the models were tested on randomly rotated digits from the MNIST-Rot dataset, and achieved competitive results, even when compared to methods which use data augmentation.

Résumé en Français

Ce Chapitre aborde la méthode du repère mobile, qui peut être utilisée pour obtenir des invariants différentiels. Nous avons appliqué la méthode pour dériver des invariants de l'action de SE( 2), le groupe de Lie des rotations et des translations dans le plan, sur des signaux en deux dimensions. Les invariants différentiels calculés, appelés invariants fondamentaux, d'un certain degré n peuvent générer tous les autres invariants de degré n à partir de combinaisons fonctionnelles, et inversement toutes les combinaisons fonctionnelles de ces invariants sont invariants. Ces propriétés permettent la définition d'une couche de réseau de neurones, le bloc SE2DIN, qui calcule les dérivées Gaussiennes à partir des cartes de caractéristiques et utilise ces dérivées pour obtenir des invariants fondamentaux, qui sont à leur tour combinés par un Perceptron Multi-Couche (MLP) appliqué pixel par pixel. Cette formulation nous permet d'approximer un invariant différentiel arbitraire de degré n.

Le réseau de neurones, appelé SE2DINNet, obtenu à partir des invariants différentiels a été implémenté pour la classification d'images discrètes en utilisant les dérivées Gaussiennes, car elles sont en théorie et en pratique plus adaptées à cette tâche que les filtres de différences finies. Ce réseau peut être implémenté de manière efficace, car la complexité du calcul des invariants est similaire à la complexité d'une convolution depthwise.

Nous avons évalué la performance de cette architecture de réseaux de neurones dans un contexte où l'invariance est importante, plus spécifiquement dans les bases de données MNIST et FashionMNIST. Nous avons utilisé un protocole ou l'apprentissage est fait sur l'ensemble d'entraînement normal, mais le modèle est testé sur des images tournées. De plus, les modèles étaient testés sur la base de données de chiffres écrits à la main et tournés MNSIT-Rot [START_REF] Larochelle | An empirical evaluation of deep architectures on problems with many factors of variation[END_REF], et ils ont obtenu des résultats compétitifs, même quand comparés à des méthodes qui utilisent l'augmentation de données. Chapter 5

+ Global MaxPooling 128 u out (a) SE2DINNet Input H × W × q in n-th order Gaussian derivatives H × W × n+2 n q in n-th order differential invariants H × W × n+2 n -1 q in 1 × 1 convolution H × W × 4q out leaky ReLU H × W × 4q out 1 × 1 convolution H × W × q out (b) SE2DIN block subsampling + 1 × 1 convolution
Single-Moving-Frame Neural Networks

Introduction

The method based on differential invariants, presented in the previous chapter, defines an alternative to the group convolution to compute equivariant networks. In this chapter we build on this method and propose an alternative way to build equivariant neural networks based on the method of moving frames. Indeed, the ratios of polynomial functions involved in the computation of differential invariants can make it numerically unstable. In order to improve that, we define an equivariant network that only estimates one moving frame, instead of one per equivariant block, as before. This new approach is embedded in 3D CNNs, where one moving frame is computed from the Gaussian n-jets of the input image, and used to make each network block equivariant. This produces novel equivariant architectures for SE (3), that we call SE(3)-Moving Frames (SE3MovFNet), and which we apply to volumetric data. Empirically, we show that a SE3MovFNet improves the performance of competitive CNNs on a collection of datasets for 3D medical image classification.

The chapter is organized as follows. We discuss related work in the literature in Section 5.2. In Section 5.3 we introduce an alternate formulation of equivariant neural networks based on moving frames, in contrast to Chapter 4 we compute the moving frame only from the input function. In Section 5.4 we derive a moving frame in SE (3) and in Section 5.5 we introduce the SE3MovFNet based on the moving frames method. In Section 5.6 we validate de SE3MovFNet in a task of medical volume classification and overperform most of the benchmarks. We end the chapter with concluding remarks in Section 5.7.

Related Work

In the literature of group-equivariant networks there exist many approaches to plane rotationequivariant networks, for example, (Cohen and Welling, 2016a;[START_REF] Worrall | Harmonic networks: Deep translation and rotation equivariance[END_REF][START_REF] Weiler | Learning steerable filters for rotation equivariant CNNs[END_REF]. Also on 2D rotation-equivariant networks, some approaches are based on differential operators [START_REF] Shen | Pdo-econvs: Partial differential operator based equivariant convolutions[END_REF][START_REF] Jenner | Steerable partial differential operators for equivariant neural networks[END_REF]Sangalli et al., 2022a). In particular, the approach we present in this chapter is an extension of the moving frames-based SE(2)equivariant neural network introduced in the previous chapter, as well as in Sangalli et al. (2022a). In the domain of 3D CNNs, two of the possible data representations are point clouds (i.e. the domain is a finite set of points in R 3 ) and volumetric data (i.e. the domain is a grid Ω ∈ R 3 ). Many approaches that seek equivariance to space rotations are for CNNs that process point clouds [START_REF] Thomas | Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds[END_REF][START_REF] Chen | Equivariant point network for 3d point cloud analysis[END_REF][START_REF] Melnyk | Embed me if you can: A geometric perceptron[END_REF][START_REF] Thomas | Rotation-invariant point convolution with multiple equivariant alignments[END_REF].

Our work focuses on defining SE(3)-equivariant networks for data based on voxels, i.e., volumetric data. Some other approaches that aim to achieve this result are: [START_REF] Worrall | Cubenet: Equivariance to 3d rotation and translation[END_REF], which achieves equivariance to a discrete subgroup of SO(3); Weiler et al. (2018a), which uses a steerable filter basis based on spherical harmonics to learn general SE(3)equivariant filters, and [START_REF] Shen | PDO-s3DCNNs: Partial differential operator based steerable 3D CNNs[END_REF], which proceeds similarly using filters based on partial differential operators. Our approach relies on differential operators like [START_REF] Shen | PDO-s3DCNNs: Partial differential operator based steerable 3D CNNs[END_REF], but instead of using a steerable filter basis we apply a moving frame to invariantize the network. This consists in evaluating each neighborhood rotated by a matrix P ∈ SO (3), which depends on the neighborhood, and is computed in the first layer.

Fixed Moving Frame

In Chapter 4 differential invariants were used to build equivariant layers of a neural network. The computation of the differential invariants done in (4.29) involves computing the prolonged action (4.28) and substituting the values of the moving frame on θ. Therefore the computation of a fundamental invariant at one specific point x ∈ R 2 I ij (x) = I ij (x, f (n) (x)) can be divided into three steps: the computation of the derivatives of f at x, denoted J (n)

x f , the computation of the moving frame ρ(x, J

x f ) from those derivatives, and the application of the moving frame to the derivatives

I(x) = ρ(x, J (n) x f ) • (x, J (n)
x f ). A SE2DINNet, as defined in Chapter 4, is a serialization of these operators, as shown in Figure 5. 1(a).

In in the context of neural networks, the method above has a significant disadvantage: the expression ρ((x, f n (x)) • (x, f n (x)) forms ratio of polynomials in the values of f (n) of potentially high order. High order polynomials in neural networks are not a good thing because during forward propagation the variance of the outputs differs from the variance of the inputs. A similar effect occurs during backpropagation. That in turn can cause the gradients to explode or vanish [START_REF] Glorot | Understanding the difficulty of training deep feedforward neural networks[END_REF]. In our experiments in Chapter 4, we added an activity regularizer to the layer that computes the invariants, which avoided in practice the numerical issues with the gradients, but this strategy was no longer efficient in our attempts of generalization to SE(3)-equivariant 3D networks.

Therefore, in the present chapter we propose an alternative relying on one fixed moving frame, but in order to introduce it let us first recapitulate the approach of Chapter 4, specifically Section 4.4.2, in the case of a 3D network equivariant to SE (3). In here we assume that we know a SE(3) moving frame for volumetric data. The moving frame will be explicitly derived in Section 5.4 using the second order derivatives. A two-layer neural network can be obtained using invariants of order two as follows:

1. Let the image f ∈ C ∞ (X, Y ) = C ∞ (R 3 ,
R) be the input to the network. First we compute for all x,

J (2) x f = f (x), ∂f ∂x (x), ∂f ∂y (x), ∂ 2 f ∂x 2 (x), ∂ 2 f ∂x∂y (x), ∂ 2 f ∂y 2 (x) , input f : X → R n-jet z 0 x = J (n) x f moving frame ρ(z 0 x ) fundamental invariants ı[z 0 x ] = ρ(z 0 x ) • z 0 x feature maps f 1 (x) = ψ 1 (ρ(z 0 x ) • z 0 x ) n-jet z 1,j x = J (n) x f j 1 , 0 ≤ j ≤ C 1 moving frames (ρ(z 1,j x )) 1≤j≤C 1 fundamental invariants ı[z 1,j x ] = ρ(z 1,j x ) • z 1,j x , 0 ≤ j ≤ C 1 feature maps f 2 (x) = ψ 2 ((ρ(z 1,j x ) • z 1,j x ) 1≤j≤C 1 ) n-jet z 2,j x = J (n) x f j 2 , 0 ≤ j ≤ C 2 moving frames (ρ(z 2,j x )) 1≤j≤C 2 fundamental invariants ı[z 2,j x ] = ρ(z 2,j x ) • z 2,j x , 0 ≤ j ≤ C 2 feature maps f 3 (x) = ψ 3 ((ρ(z 2,j x ) • z 2,j x ) 1≤j≤C 2 )
. . .

(a) SE2DINNet input f : X → R n-jet z 0 x = J (n) x f moving frame ρ(z 0 x ) fundamental invariants ı[z 0 x ] = ρ(z 0 x ) • z 0 x feature maps f 1 (x) = ψ 1 (ρ(z 0 x ) • z 0 x ) n-jet z 1,j x = J (n) x f j 1 , 0 ≤ j ≤ C 1 invariants ρ(z 0 x ) • z 1,j x , 0 ≤ j ≤ C 1 feature maps f 2 (x) = ψ 2 ((ρ(z 0 x ) • z 1,j x ) 1≤j≤C 1 ) n-jet z 2,j x = J (n) x f j 2 , 0 ≤ j ≤ C 2 invariants ı[z 0 x ] = ρ(z 0 x ) • z 2,j x , 0 ≤ j ≤ C 2 feature maps f 3 (x) = ψ 3 ((ρ(z 0 x ) • z 2,j x ) 1≤j≤C 2 )
. . . followed by the computation of the fundamental invariants of order 2, which we will denote ı[z 0 x ] = ρ(x, J

x f ) • (x, J

x f ) = ρ(z 0 x ) • z 0 x where z 0 x = (x, J

x f ). Let ψ 1 : J 2 (M) → Y 1 , where Y 1 = R q , be a smooth map. In a deep learning context we assume ψ 1 to be a MLP with smooth activation functions. The first layer φ

1 : C ∞ (R 3 , Y ) → C ∞ (R 3 , Y 1 ), Y 1 = R C , is given by φ 1 [f ](x) = ψ 1 (ı[z 0 x ]) = ψ 1 (ρ(z 0 x ) • z 0 x ) = ı[ψ 1 ](z 0 x ) (5.1)
and it is SE(3)-equivariant, because ı[ψ 1 ] is an invariant and we apply it to functions like in (4.18).

2. We build the second layer analogously. The output of the first layer is a signal

f 1 = (f 1 1 , f 2 1 , . . . , f C ) = φ 1 (f ) : X → Y 1 .
We compute the derivatives J (2) x f 1 = (J (2) x (f 1 1 ), . . . , J (2) x (f C 1 ))

and the fundamental invariants

(ı[z 1,1 x ], • • • , ı[z 1,C
x ]) where z 1,j x = (x, J

x (f j 1 )) and ı[z 1,j x ] = ρ(z 1,j x ) • z 1,j x . Let ψ 2 : (J n (M)) C → Y 2 be a function given by a MLP. The second layer can be computed from the output of the first by

φ 2 [f 1 ](x) = ψ 2 ρ(z 1,1 x ) • z 1,1 x , • • • , ρ(z 1,C x ) • z 1,C x = ψ 2 (ı[z 1,1 x ], • • • , ı[z 1,C x ]).
(5.2)

Again this function is equivariant because it is a function of invariants. The second layer φ 2 = φ 2 • φ 1 is equivariant because it is a composition of equivariant operators. This process can be repeated to obtain L equivariant layers

φ l = φ l • φ l-1 , 0 ≤ l ≤ L. It is illustrated in Figure 5.1(a).
The expressions above require the computation of certain non-linear combinations of the values of J (n)

x f that were established to be numerically unstable in the last section. Here we will propose an alternative that uses only a single moving frame computed from the input image and renders the network much more stable.

The alternative we propose is the following. Instead of computing the differential invariants at each layer, involving the computation of the moving frame based on the previous layer's feature maps, we compute the moving frame based only on the network input signal and compute all subsequent layers based on this moving frame.

Keeping the same target as before, which is to build a two-layer equivariant network, we can compute f 1 exactly as in step one. Now, from f 1 we compute J

(2)

x f 1 for all x. Given some ψ 2 : (J 2 (M)) C → R C (which again should be regarded as a MLP) we can compute the output of the second layer. In contrast to (5.2), however, we transform z 1,j x applying ρ(z 0 x ) = ρ(x, J

x f ), and not ρ(z 1,j x ), which yields

φ 2 (f 1 )(x) = ψ 2 (ρ(z 0 x ) • z 1,1 x , . . . , ρ(z 0 x ) • z 1,C x ).
(5.

3)

The diagram of Figure 5.1(b) illustrates this method. The next result shows that repeated application of (5.3) defines a SE(n)-equivariant network.

Proposition 2. Let X = R d , Y = R C 0 = R, ρ : X × Y (n) d
→ SE(d) a moving frame, and L ∈ N * . For 1 ≤ l ≤ L,

• let Y l = R C l , C l ≥ 1 • assume that SE(d) acts on X × Y l as (x, u) → (Rx + v, u) • Let ψ l ∈ C ∞ (J n (M) C l-1 , Y l ) be L smooth maps.
Let φ 1 , . . . , φ L be the operators defined, for all f ∈ C ∞ (X, Y ), x ∈ X, by

φ 1 [f ](x) = ψ 1 ρ z 0 x • z 0 x = ψ 1 ı[z 0 x ] (5.4)
and, for 1

< l ≤ L either φ l [f ](x) = ψ l ρ z 0 x • z l-1 x (5.5) or (assuming C l = C l-1 ) φ l [f ](x) = φ l-1 [f ](x) + ψ l ρ z 0 x • z l-1 x (5.6)
where z 0 x = (x, J

x f ), z l x = (x, J

x φ l (f ) j ) 0≤j≤C l , 1 ≤ l ≤ L, and the action of G on J n (M) C l , is the action on J n (M) applied coordinate-wise. Then each φ l is SE(d)-equivariant, for 1 ≤ l ≤ L.

Proof. For l = 1 we have

φ 1 (g • f )(x) = ψ 1 ρ(x, J (n) x (g • f )) • (x, J (n) x (g • f )) .
Noting y = g -1 • x and recalling (4.15), g • (x, J

(n) x f ) = (g • x, J (n) (g•x) (g • f )), we can simplify (x, J (n) x (g • f )) = (g • y, J (n)
(g•y) (g • f )) = g • (y, J (n) y f ) = g • z 0 y .
(5.7)

Hence, (5.8)

φ 1 (g • f )(x) = ψ 1 ρ(g • z 0 y ) • (g • z 0 y ) = ψ 1 ı[g • z 0 y ] = ψ 1 ı[z 0 y ] by invariance of ı = φ 1 (f )(y) by (5.4) = φ 1 (f )(g -1 • x) = (g • φ 1 (f ))(x)
Therefore, φ 1 (g • f ) = g • φ 1 (f ).
Now, for l > 1, we have for the case (5.5),

φ l (g • f )(x) = ψ l ρ(x, J (n) x (g • f )) • (x, J (n) x φ l-1 (g • f ) j ) 1≤j≤C l .
(5.9)

As shown earlier, (x, J

x (g

• f )) = g • z 0 y with y = g -1 • x. Similarly, assuming that φ l-1 is equivariant, (x, J (n) x φ l-1 (g • f ) j ) = g • (y, J (n) y φ l-1 (f ) j )) = g • z l-1,j y , (5.10) so that φ l (g • f )(x) = ψ l ρ(g • z 0 y ) • (g • z l-1,j y ) 1≤j≤C l .
(5.11)

Since furthermore ρ(g • z 0 y ) = ρ(z 0 y ) • g -1 by definition of a moving frame, we finally get

φ l (g • f )(x) = ψ l ρ(z 0 y ) • g -1 • (g • z l-1,j y ) 1≤j≤C l = ψ l ρ(z 0 y ) • (z l-1,j y ) 1≤j≤C l = φ l (f )(y) = φ l (f )(g -1 • x) = (g • φ l (f ))(x).
(5.12) As for the case (5.6),

φ l (g • f )(x) = φ l-1 (g • f )(x) + ψ l ρ(x, J (n) x (g • f )) • J (n) x φ l-1 (g • f ) = φ l-1 (f )(y) + ψ l ρ(z 0 y ) • z l-1 y like above, with y = g -1 • x = φ l (f )(y) = (g • φ l (f ))(x).
(5.13) Therefore in all cases φ l (g • f ) = g • φ l (f ) provided this is true for φ l-1 , and the proposition follows by induction.

The proposition above shows that SE(d)-equivariant networks can be constructed this way using regular feedforward connection like in (5.5) and and using residual connections like in (5.6).

Moving Frames on SE(3)

The group SE( 3) is the direct product of Lie groups SO(3) and R 3 and since both are threedimensional, then SE( 3) is a 6-dimensional Lie group.

Here a volume refers to a signal on a 3-dimensional Euclidean domain, i.e. functions of the type f : R 3 → R C . A reasoning similar to the one used to obtain SE(2)-equivariant operators on images in Chapter 4 can be used to produce SE(3)-equivariant operators on such volumes, but this time we use a cross-section based on restrictions of the Hessian matrix, instead of the derivatives of order one.

As such we begin by representing volumes as submanifolds of M = R 3 × R C . Firstly we consider the case C = 1, but keeping in mind that for higher dimensions it is just a matter of channel-wise application. The action of SE(3) on M is then given by,

∀(R, v) ∈ SE(3), ∀(x, u) ∈ M π R,v (x, u) = (R • x + v, u).
(5.14)

If we proceed to extend M to the first-order jet space we will find that SE(3) does not act freely on J 1 (M). Indeed, the orbit of a point (x, u, u x , u y , u z ) is the Cartesian product of R 3 , {u} and a sphere with radius u 2

x + u 2 y + u 2 z , hence it has dimension 5 = dim SE(3) = 6. Therefore it is necessary to prolong the action to the second order jet-space in order to be able to obtain a moving frame. In this section we use a matrix notation for compactness: we denote ∇u = [u x , u y , u z ] T and

Hu =   u xx u xy u xz u xy u yy u yz u xz u yz u zz   .
(5.15)

In that way, the coordinates of the second order jet-space are identified by z (2) = (x, u (2) ) = (x, u, ∇u, Hu) and the prolonged action can be expressed as π R,v (z (2) ) = π R,v (x, u, ∇u, Hu) = (Rx + v, u, R T ∇u, RHuR T ).

(5.16)

We could use a cross-section like the previous case that depends on the gradient, but it would also have to depend on one of the entries of Hu. However, it is simpler to use the coordinate cross-section (5.17) i.e. x = 0 and Hu is diagonal. Since Hu is symmetric, we can find a P (x,u) ∈ SO(3)1 such that P (x,u) HuP T (x,u) is diagonal. However there is more than one way to do so. Indeed we can permute any two columns of P (x,u) and we can multiply any two columns of P (x,u) by -1 and still map (x, u (n) ) to K . In this way K is not regular. To tackle the first problem we add the inequality u xx > u yy > u zz to the cross-section. Now if the transformed coordinates P (x,u) HuP T (x,u) the inequality, and P is obtained from permutating the columns of P (x,u) , then P HuP T does not satisfy the inequality. Furthermore, to solve the second problem we demand the the first to columns of P (x,u) are positively aligned with ∇u, i.e. let v be one of those columns, we constrain v such that v T ∇u ≥ 0, which is equivalent to adding the inequalities u x ≥ 0 and u y ≥ 0 to the cross-section. Now we can define a regular cross-section

K = {(x, u (n) ) ∈ J n (M)|x = 0, u xy = u xz = u yz = 0},
K = {(x, u (n) ) ∈ J n (M)|x = 0, u xy = u xz = u yz = 0, u xx > u yy > u zz , u x ≥ 0, u y ≥ 0}. (5.18)
In particular the cross-section above is regular in an open set U ⊆ J n (M) where Hu has no repeated eigenvalues and ∇u = 0. Now K defines a moving frame ρ : (x, u) ∈ M → (P (x,u) , -P (x,u) x) ∈ G. Then the non-zero fundamental invariants of the jet-space of order two are u, v T i • ∇u and λ i , i = 1, 2, 3, where the v i s are the eigenvectors of Hu (columns of P (x,u) ) and the λ i s are the eigenvalues of Hu (diagonal coefficients of P (x,u) HuP T (x,u) ). The cross-section K is not necessarily regular, but we can check that ρ is indeed a moving frame 5.5 SE3MovFNet

Gaussian Derivatives on Volumes

Similarly to what is done with the SE2DINNets in Chapter 4, derivatives are computed using the Gaussian derivative operators. In the 3-dimensional case the Gaussian derivatives of a function f are computed by

∂ i+j+k ∂x i ∂y j ∂z k (f * G σ ) = f * ∂ i+j+k ∂x i ∂y j ∂z k G σ = f * G i,j,k σ .
(5.19) We refer to the Gaussian n-jet of a volume as the Gaussian derivatives of order ≤ n f

(n) σ := (f * G i,j,k σ ) 0≤i+j+k≤n .
We can also identify the Gaussian n-jet by tensor coordinates. In particular for n = 2 we write f

(n) σ = (f σ , ∇ σ f, H σ f ) where f σ = f * G σ , ∇ σ
is the Gaussian gradient i.e. derivatives of order one and H σ f is the Gaussian Hessian, i.e. derivatives of order two. Given an orthogonal matrix for each point P : Ω → SO(3) (e.g. the matrices defining a moving frame) we denote the local action P by (P • f

(n) σ )(x), e.g. for n = 2, (P • f (n) σ )(x) = (f σ (x), P (x) T ∇ σ f (x), P (x)H σ f (x)P (x) T
) (here we omit the translational part of the action). If f : Ω → R C is a multi-channel volume we can apply these operations channel-wise.

Architecture

Based on the exposition of Section 5.3, the general idea of our SE(3)-equivariant architecture, given an input signal f : Ω → R where Ω ⊆ R 3 is a 3-dimensional grid, is to first compute the matrices P (•,fσ) ∈ SO(3) of the moving frame for every x ∈ Ω. The matrices are computed by diagonalizing H σ f (x) for every x, i.e., find P (x,fσ) such that P (x,fσ) H σ f (x)P T (x,fσ) is diagonal. P (x,fσ) must also have the values in the diagonal in decreasing order and with columns that are positively aligned with the gradient. If it is not possible to find such a matrix we set P (x) = 0.

After computing P we compute blocks as shown in Figure 5.3b, which we call SE3MovF blocks, using the moving frame and the current features maps as input. The scale of each layer does not need to be necessarily the same, here we consider that a scale σ is used to compute the moving frames and a scale σ to compute the derivatives at each block.

input f : Ω → R Gaussian 2-jets f (2) σ : Ω → R 5 2 P SE3MovFBlock SE3MovFBlock + SE3MovFBlock + SE3MovFBlock + • • • SE3MovFBlock + GlobalMaxPooling uout (a) SE3MovFNet input feature maps f l : Ω → R C moving frame P (•,fσ ) : Ω → SO(3) Gaussian n-jets f (n) l,σ : Ω → R C n+3 n invariants P • f (n) l,σ : Ω → R C n+3 n output features maps f l+1 : Ω → R C
A simple form of a global architecture, which we call SE3MovFNet, is in Figure 5.3a. The feature maps of each block are summed like in residual networks, which mimics a numerical PDE scheme [START_REF] Ruthotto | Deep neural networks motivated by partial differential equations[END_REF]. The network in Figure 5.3 is specialized for a fixed number of channels, but by applying a 1 × 1 × 1 convolution between blocks we can increase the number of feature maps of the next layer. Pooling may also be performed by subsampling after a block. The global max-pooling at the end renders the equivariant architecture invariant [START_REF] Bronstein | Geometric deep learning: Grids, groups, graphs, geodesics, and gauges[END_REF], which is interesting for a classification problem.

Complexity Analysis

We can perform a complexity analysis of the network define above.

Let us assume that the input is given as a signal f :

Ω → R where Ω = {0, . . . , W -1} × {0, . . . , H -1} × {0, . . . , D -1} is a grid of size W × H × D, containing N = W • H • D voxels.
Moreover, let us assume that we compute the moving frame using Gaussian derivatives of scale σ and the derivatives at other layers using σ, and that the discrete Gaussian derivative filters all have dimension w × w × w , for the moving frame and w × w × w for the other layers.

The computation of the moving frame is done as follows :

• compute all Gaussian derivatives of order 1 and 2 of f . Gaussian derivatives are separable, thus each one can be obtained by three convolutions with a filter of size w, which have cost a cost of O(N w ) floating point operations (flops);

• compute the eigenvectors of the Hessian. Since the matrices have constant size 3 × 3 we consider this operation is done in constant time for each pixel and this step is done in O(N ) flops. From there on if we compute a layer with C input feature maps and C output feature maps:

• this layer computes n+3 n Gaussian derivatives for each input feature map, where n is the order of differentiation used, resulting in O n+3 n CwN flops; • to compute the prolonged group action, it can be verified that the equivariant group action can be expressed as polynomial in the partial derivatives, and thus it takes O n+3 n CN flops;

• the previous step is followed by a L-layer multi-layer perceptron at each voxel. Assuming that the output dimension at each layer of the MLP is at most q we have that this step takes O(CC + LC 2 ) flops.

The complexity of a layer of SE3MovF is the sum of the complexity of each step, i.e. it can be done in O n+3 n CN w + CC + LC 2 flops. In our experiments here we used n = 2 and L = 2 for all models, so the impact of those terms is very limited.

Experiments

Tetris

As a first test of the ability of the proposed layers to create equivariant/invariant networks, we apply them to the tetris pieces classification [START_REF] Thomas | Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds[END_REF]Weiler et al., 2018a). The training dataset contains the eight three-dimensional tetris pieces at a fixed orientation, like in Figure 5.4. The pieces are represented in a 40 × 40 × 40 voxel grid by four Gaussian functions with mean at center of the cubes and standard deviation of half of a voxel. We train a SE3MovFNet with four blocks like Figure 5.3b applied with residual connections. Each block has 40 feature maps and uses leaky ReLU activations, also, between the two layers of the MLP there is a 30% dropout. We also train a CNN where the blocks are replaced with a 3 × 3 convolution with a leaky ReLU activation and the same quantity of feature maps. The last layer is a global average pooling. We constructed a rotated dataset by, for each piece, rotating it by a random rotation in SO(3) and adding the rotated piece to the test set, 30 times for each piece.

Both models achieve 100% accuracy in the non-rotated dataset, but on the rotated dataset the SE3MovFNet achieves 97% ± 2% accuracy and the CNN achieves 19% ± 1%, as a result of 30 repetitions of the experiment. Overall this seems to confirm the rotational equivariance of the SE3MovFNet.

MedMNIST3D

The Moving Frame SE(3) networks are validated on the MedMNIST datasets (Yang et al., 2021a,b). MedMNIST refers to a collection of datasets for benchmarking of medical image analysis methods. They consist of 28 × 28 images and 28 × 28 × 28 volumes. The datasets of interest here are those based on volumetric data. Here is a brief description of the 3D datasets, some examples can be seem on Figure 5.5, a more detailed one can be obtained from (Yang et al., 2021a,b) • SynapseMNIST3D. Binary classification between excitatory and inhibitory synapses.

For each dataset we train a network with five SE3MovFr blocks with 16, 16, 32, 32, 64 filters, using a stride of two in the second block. Voxelwise MLPs are computed as two subsequent 1 × 1 × 1 convolutions followed by batch normalization (both) and leaky ReLU (only the first) and with the same number of neurons. We also train a CNN baseline with the same number of filters where each block consists of two 3 × 3 × 3 convolutions followed by batch normalization and leaky ReLU. Input volumes are resized to 29 × 29 × 29 so that subsampling by a factor of two is equivariant by rotations of 90 • around the coordinate-axes. For both the SE3MovFNet and the CNN we also trained models with rotation data augmentation. The augmentation is performed by applying a random rotation in SO (3) Overall results can be seen in Table 5.1. There we can see that the SE3MovFNet surpassed most of the benchmarks. Results of testing the models on rotated test sets are seen in Figures 5.6 and 5.7. Those results are obtained by taking each of the test sets of MedMNIST3D and rotating volumes around each of the coordinate axes by angles i π 12 for i = 0, . . . , 23. In those results we observe the SE3MovFNet has perfect invariance for 90 • rotations, evidenced by the periodicity of results, and a generally better equivariance than the CNN baseline with or without augmentation. It suffers, however, a significant loss of performance at orientations not multiple of 90 • . Moreover, while in some cases the data augmentation has a beneficial effect for the SE3MovFNet, it sometimes has the opposite effect of worsening rotation invariance. This could be a result of volumes being of very low resolution, as the discrete rotations can induce artifacts in rotated volumes.

Conclusions

In this chapter we proposed an alternate moving-frame approach to group-equivariant networks to the approach of Chapter 4. In the latter, an equivariant network is obtained by computing differential invariants from the feature maps of the previous layer, which involves implicitly computing a moving frame for each pixel of those feature maps. In the approach of the present chapter, we compute only the moving frames for the input image and apply it to the feature maps of all layers. We showed that it is possible to define an equivariant neural network from this new approach , which is more stable numerically because the computed invariants are linear with respect to the feature maps of the previous layer. We computed a moving frame for the action of SE( 3) on volumes and applied the method above to construct a SE(3)-equivariant neural network. We implemented the architecture, referred to as SE3MovFNet, using Gaussian derivatives. The performance of the network is overall positive, as it attained the best results in four out of the six evaluated datasets of MedMNIST, and maintains a reasonable accuracy when images are rotated. The performance when the test set is rotated by an angle that is not a multiple of 90 • around the coordinate axes, is much lower than otherwise. It might be interesting to identify the source of these problems, whether they come from artifacts from the discretization in low resolution or from the network itself.

Résumé en Français

Dans ce chapitre nous avons proposé une approche basée sur la méthode du repère mobile, pour obtenir un réseau équivariant par transformations de groupe. Cette approche représente une alternative à celle du Chapitre 4. Dans l'approche du chapitre précèdent, nous avons obtenu un réseau équivariant par le calcul des invariants différentiels à partir des cartes de caractéristiques de la couche précèdent, ce qui demande implicitement le calcul du repère mobile pour chaque pixel de ces cartes de caractéristiques. Dans ce chapitre, nous calculons le repère mobile seulement pour l'image d'entrée et nous l'appliquons aux cartes de caractéristiques de chaque couche. Nous avons montré qu'il est possible de définir un réseau équivariant à partir de cette nouvelle approche, qui est plus stable numériquement que l'approche du chapitre précédent, car les invariants sont linéaires par rapport aux cartes de caractéristiques de la couche précédente.

Nous avons calculé un repère mobile pour l'action de SE(3) (rotations et translations en 3 dimensions) sur les volumes et nous l'avons appliqué pour construire un réseau équivariant par l'action de SE (3). Nous avons implémenté la nouvelle architecture, qui a été appelée SE3MovFNet, avec les dérivées Gaussiennes.

Pour tester la performance et l'invariance de ces réseaux, nous avons utilisé MedMNIST3D [START_REF] Yang | Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification[END_REF], une collection de bases de données d'imagerie médicale en faible résolution. La performance du réseau est plutôt positive, et il a atteint les meilleurs résultats sur quatre de six bases de données de MedMNIST3D, et la performance est encore maintenue dans un niveau raisonnable même quand les images de test sont tournées. Toutefois, la performance quand les images sont tournées par un angle qui n'est pas multiple de 90 • autour des axes de coordonnées peut-être beaucoup plus faible que le cas contraire. Il serait intéressant de déterminer si ces Chapter 6

Perspectives for Moving Frames Based Networks

Introduction

In this thesis a new class of group equivariant neural networks, based on moving frames, has been proposed. The approach was applied to obtain rotations and translations equivariant neural networks in two and three dimensions. The method of moving frames, however, is general enough to work on other domains and Lie groups and can be applied to obtain invariants in a variety of contexts.

In the examples presented in earlier chapters, the action of the Lie group was on the the domain of the signal, i.e. a spatial transformation, but we can equally define moving frames for actions on the output the values, e.g. group transformations on the grey levels or color space. In Section 6.2 we explore the idea of a neural network invariant to an affine transformation on the grey levels of the input image, or a contrast change. The network is tested in a simple toy example of digit classification where the contrast in the images is artificially changed and is shown to perform well under drastic contrast changes, while the CNN sees a visible drop when exposed to these changes.

In Section 6.3 neural networks equivariant to similarity transforms (i.e. rotations, translations and scalings) is proposed. The networks are similar to the SE2DIN in Chapter 4, but with the addition that the the layers receive a multi-scale input and compute differential invariants using re-scaled Gaussian derivatives, similar to [START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF].

Finally, in Section 6.4 we propose a SE(3)-equivariant neural network for point clouds based on the method of moving frames. In that approach, instead of computing differential invariants, which is more challenging in point cloud data, we compute invariants on the spaces of k-tuples of points in R 3 . The approach we propose is equivalent to that of [START_REF] Thomas | Rotation-invariant point convolution with multiple equivariant alignments[END_REF], but the way we compute our moving frames is based on the covariance matrix of the points.

Contrast Change Invariant Networks

Let us consider a contrast change in an image. Again an image is a submanifold in M = R 2 ×R. We are specifically considering grey-scale images in here, but the same reasoning can be applied to multi-channel images, channel by channel. This time, the group of transformations we wish to have invariance to, is an affine transformation on the counter domain of our image (as well as translations on the input points). The group is given by S R 2 where S = {(α, β)|α > 0, β ∈ R} and S has composition law (α, β) • (α , β ) = (αα , αβ + β). It acts on M as (α, β, v) • (x, u) := (x + v, αu + β).

(6.1)

when applied to a function f :

R 2 → R [(α, β, v) • f ](x) = αf (x -v) + β. (6.2)
This action can be interpreted as a horizontal translation (translation of x) and a vertical affine transformations (affine transformation of f (x)).

In order for us to define a moving frame we derive the prolonged action in the jet-bundle. A simple computation reveals that for n = i + j > 0 and a smooth function

f : R 2 → R ∂ n (α, β, v) • f ∂x i 1 ∂x j 2 = ∂ n f ∂x i 1 ∂x j 2 = α ∂ n f ∂x i 1 ∂x j 2 . (6.3)
As such, the prolonged action of (α, β, v) on the jet space is given by (x, (u i,j ) 0≤|i+j|≤n ) → (x, (ũ i,j ) 0≤i+j≤n ) where x = x+v and ũ = ũ0,0 = αu+β are given by (6.1), and for 1 ≤ i+j ≤ n ũi,j = αu i,j .

From there we can define an appropriate cross-section in the jet bundle J n (M). For simplicity we compute a coordinate cross-section. In the translational part we choose the equation x = 0, resulting in v = -x. Since β only influences u we add the equation u = 0 to the cross-section, resulting in β = -αu. Now for determining α we can choose an equation of the form |u i,j | = 1 resulting in α = (|u i,j |) -1 . This results in the cross-section K i,j = {x = 0, u = 0, |u i,j | = 1}. The choice of i and j still remains, however.

We would like the invariant differential operators to have a consistent behaviour in all of the image, but choosing a single value for all of the domain would be problematic when ∂ i+j u ∂x i 1 ∂x j 2 = 0. Our method to choose a cross-section is the following. Suppose that for some i, j such that 1 ≤ i+j ≤ n we have that |u i,j | > |u k,l | for all k, l ∈ N such that 1 ≤ k +l ≤ n and (k, l) = (i, j), then that is true for an open set containing (x, u (n) ) and K i,j defines a moving frame

ρ(x, u (n) ) = (|u -1 i,j |, -|u -1 i,j |u, -x) =   1 max 1≤k+l≤n |u k,l | , - u max 1≤k+l≤n |u k,l | , -x   ∈ G. (6.4)
Note moving frame is not smooth everywhere. In particular, if u i,j = 0 for all 1 ≤ i + j ≤ n or the maximum is attained at least twice, then the expression above is not smooth, but in practice it does not cause any problems as it is well defined and can be used to invariantize signals.

We have constant invariants I 00 = ı[u] = 0, H = ı[x] = 0, I i * ,j * = 1 where i * , j * are the maximum values in (6.4) and for i + j ≥ 1 we have, for almost all (x, u (n) ) ∈ J n (M),

I i,j = u i,j max 1≤k+l≤n |u k,l | . (6.5)
For functions we implement the expression Fi,j (

x) = 1 max 1≤k+l≤n |∂ k,l f | + • ∂ n f (x) ∂ i x 1 ∂ j x 2 (6.6)
where is a small positive constant to prevent division by 0. A layer computing (6.6) is approximately invariant to change in contrast, depending on the value of . Therefore if we use the output of the invariants as the input to the next layer, the output of the next layer will also be (approximately) invariant. A contrast-invariant network can then be defined as a layer that computes (6.6) for example with Gaussian derivatives, followed by classical CNN layers.

Global Contrast Change Invariance Experiment. In order to test the efficacy of (6.5) in a neural network we prepare a benchmark task. Using the MNIST dataset we train a CNN and a contrast-invariant network obtained by first processing the data using (6.5) up to order two (where the derivatives are computed with Gaussian derivatives) and then using the same CNN architecture. Additionally we test a CNN where the input has been normalized following

Normalize(f )(x) = f (x) -inf x∈Ω f (x) sup x∈Ω f (x) -inf x∈Ω f (x) , (6.7)
where Ω is the domain of the discrete signal and f is assumed to be bounded. The expression above should result in perfect invariant to contrast changes, but we will see later that it performs badly when compared to local contrast changes.

After training the models are tested in the MNIST test set, but the images have been transformed according to (6.1) with α ∈ [.1, 10] and β ∈ [-5, 5]. Results are displayed in Figure 6.1. As we can see, outside a small range the CNN's performance suffer significantly. The model based on the moving frame, on the other hand, deals mostly well with most of the contrast changes. The CNN with normalized input performs the same for every transformation as expects, and thus surpasses the moving frames based method in this case.

Local Contrast Change Invariance Experiment. Now we test the efficacy of the models when local contrast changes are applied. Indeed consider that the transformation in the function f : R 2 → R is given by f → f where

f (x) = α(x)f (x) + β(x). (6.8) and α ∈ C ∞ (R 2 , R ≥0 ), β ∈ C ∞ (R 2 , R).
A transformation like that can more closely model lightning changes that can happen in natural images. A global normalization is not sufficient to attain invariance in this case. The moving frames approach, however, is local and should be approximately invariant to this transformation.

In order to test that, we use a test set obtained by taking each image from the MNIST test set and applying (6.8) where α(x) is given by α(x) = exp λ Ḡµ,σ (x) , β = 0, where Ḡµ,σ is a Gaussian function with mean µ and standard deviation σ, normalized to have values in (0, 1], and γ ∈ R. For each image we chose a random µ in a 12 × 12 square centered at the center of the image, σ uniformly chosen in the interval [5,8] and γ uniformly chosen in the interval [-log(15), log( 15 After testing we obtained accuracies 0.981 ± 0.001 for the network based on moving frames, 0.78 ± 0.01 for the CNN and 0.143 ± 0.004 for the CNN with normalized input, confirming our expectations.

It could be interesting to try a similar methodology to other group actions on gray values or color spaces, for instance, a power law transformation f → αf γ in the lightness component of a color image or general transformations in the color space of images. Moreover, a similar approach can be applied where instead of Gaussian derivatives we use convolutional or morphological filters.

Similarity Transformation Equivariant Network

The (special) similarity transforms group S(2) acting on the plane has elements of the form (rR θ , v), where r > 0 is a real number

, R θ ∈ SO(2) is the rotation of angle θ ∈ [0, 2π) and v ∈ R 2 . It acts in x ∈ R 2 by x → rR θ x + v. (6.9)
This group can be seen as a product of a scaling group, R >0 , and SE(2). In the vein of Lindeberg (2022), we can add scale equivariance to a network based on Gaussian derivatives. More specifically we first note a property of Gaussian derivative filters. Let π S t (u)(x) = u(tx), t > 0 denote the action of the scaling group R >0

(π S t (u) * G i,j σ )(x) = ∂ i+j ∂x i 1 ∂x j 2 (π S t (u) * G σ )(x) = ∂ i+j ∂x i 1 ∂x j 2 (u * G tσ )(tx) = π S t (u * G i,j tσ )(x).
(6.10)

We can see that the Gaussian derivatives have an interesting relationship with this scaling group, but it is not scale equivariance (at least not in a strict commutation of the same group action sense), as the scale of the Gaussian derivatives is changed when we change the order of operations. Now suppose that as input to the Gaussian derivatives we have a function u : R >0 × R 2 → R and that π S t (u)(s, x) = u(ts, tx). Let us define in that case the Gaussian derivatives as (6.11) in this case we have

(u * S G i,j σ )(s, x) = (u(s, •) * G i,j sσ )(x),
(π S t [u] * S G i,j σ )(s, x) = (π S t [u(s, •)] * G i,j sσ )(s, x) = π S t [u(s, •) * G i,j tsσ ](x) = (u(ts, •) * G i,j tsσ )(tx) = (u * S G i,j σ )(ts, tx) = π S t [u * S G i,j σ ](s, x),
(6.12) therefore these Gaussian derivatives are scale-equivariant.

Of course, in order to apply these filters to our desired input domain, we need to first map a function u : R 2 → R to a function Λu : R >0 × R 2 → R, similarly to what was done with the lifting in Chapter 2. We wish as well that the operator Λ respects an equivariance relationship, namely π S t [Λu] = Λπ S t [u] (noting that the actions on the left and right hand sides are in different domains). Like in Chapter 2 we call Λ a lifting. A network consisting of a lifting and a R >0 -equivariant operator φ satisfies φ

• Λ • π S t = π S t • φ • Λ.
The map Λu(s, x) := u(x) ∀s ∈ R >0 , x ∈ R 2 works well for this purpose.

Finally, we can use these scaled Gaussian derivatives in order to compute differential invariants of SE (2) and show that they are both SE(2)-equivariant and R >0 -equivariant, i.e. S(2)-equivariant. Firstly, let us define the action of SE(2) in images u : R >0 × R 2 → R as

π (R,v) [u](s, x) := u(s, R θ x + v), (6.13)
and a scaled n-th differential invariant as

I S [u](s, x) = I((G i,j σ * S u)(s, x), 0 ≤ i + j ≤ n) = I((G i,j sσ * u(s, •))(x), 0 ≤ i + j ≤ n), (6.14)
Given any SE(2) differential invariant I of the Jet-Space J n (M). We have

I S (π θ,v [u])(s, x) = I((G i,j sσ * π θ,v [u](s, •))(x), 0 ≤ i + j ≤ n) = I((G i,j sσ * π θ,v [u(s, •)])(x), 0 ≤ i + j ≤ n) = I(π θ,v [(G i,j sσ * u(s, •))](x), 0 ≤ i + j ≤ n) = I((G i,j sσ * u(s, •))(R θ x + v), 0 ≤ i + j ≤ n) = I S (u)(s, R θ x + v) = π θ,v [I S (u)](s, x), (6.15) and I S (π S t [u])(s, x) = I((G i,j σ * S π S t [u])(s, x), 0 ≤ i + j ≤ n) = I(π S t [G i,j σ * S u](s, x), 0 ≤ i + j ≤ n) = I((G i,j σ * S u)(ts, tx), 0 ≤ i + j ≤ n) = I S (u)(ts, tx) = π S t [I S (u)](s, x) (6.16)
In particular, this is true when I is a fundamental invariant or a functional combination of fundamental invariants. Furthermore, we can extend these scaled differential invariants to be able to use the multiscale information. We combine invariants of different scales by

I S (u)(s, x) = C i=1 ∞ -∞ I S i (t, x)h i (ts)dt (6.17)
for a compactly supported h : R >0 → R C . This can be computed as one-dimensional convolutions in parallel. This method seems like an interesting extension of the SE2DIN blocks that has both rotation and scaling equivariance and uses multi-scale information, but for now it is merely a theoretical proposition that has not yet been tested in practice.

Moving Kernel Point Convolutions Kernel Point Convolution

The Kernel Point Convolution (KPConv) [START_REF] Thomas | Kpconv: Flexible and deformable convolution for point clouds[END_REF] is a convolutional layer for point cloud based data. Here we review it before we discuss its equivariant counterpart. In this chapter, we represent a point cloud as a finite set {x 1 , x 2 , . . . , x N } = P ⊆ R 3 . A signal f : P → R C on the point cloud can be represented by a matrix F ∈ R N ×C where where its i-th row is given by f i = f (x i ). The convolution of a signal by a filter g : R 3 → R C×C is given by a signal (f * g) : P → R C (f * g)(x) =

x i ∈Nx g(x i -x)f i , (6.18)

where N x ⊆ P is the neighborhood of a point x ∈ P.

The most important part of (6.18) is the filter g. KPConv defines g in terms of kernel points. Firstly, the support of g is set to be the closed ball Br = {y ∈ R 3 | y ≤ r} for some r > 0. For a point y ∈ Br , g(y) is defined as (6.19) where y 1 , . . . , y K are the so-called kernel points, h : R 3 × R 3 → R is a weighting function and W k ∈ R C×C is a matrix. The weighting function h can be defined in different ways, in the KPConv paper we see some examples:

g(y) = K k=1 h(y, y k )W k ,
• Linear correlation: Similarly to the method in Chapter 5, we propose to compute the moving frame for one point cloud, the input point cloud.

The normal may not always be present in the data, and estimating the normal vector may sometimes result in a problem of orientation, as there are always two possibilities for normal vectors, facing opposite directions and the normal vectors may be inconsistent with each other. A solution is to always use both normal vectors and sum the result. When we do that it is impossible to distinguish between the result of a neighbordhood and its reflection, so this actually defines a E(3) equivariant operator when we applyto a signal on the ponit cloud.

Point Cloud Subsampling

In practice it is useful to subsample input point clouds to reduce the computations. Here below we mention two popular subsampling strategies, since we use the first one (Farthest Point Sampling) in our experiment in the next section, as an alternative to the second one (Grid Subsampling).

Farthest Point Sampling

Farthest point subsampling is a subsampling strategy that aims to obtain M < N points in a point cloud that maximize the total point distance. In practice, it is computed by a greedy algorithm as follows: This subsampling approach is also referred to as greedy clustering as the points in S can form the centers for clusters in P. An efficient implementation is detailed in Section 3.1 of [START_REF] Har-Peled | Fast construction of nets in low-dimensional metrics and their applications[END_REF].

Grid Subsampling

In this approach, adopted by [START_REF] Thomas | Kpconv: Flexible and deformable convolution for point clouds[END_REF], a cell size s is chosen and the space is divided into a cubic grid of voxels of side s. For each voxel V ijk , if there are points of P inside of V ijk compute the barycenter of those points and add it to the subsampled cloud S.

Experiment

ModelNet Firstly as a proof-of-concept we try out training an E(3)-equivariant KPCNN, and a regular KPCNN using only the classes 0 (airplane) and 1 (bed) of ModelNet [START_REF] Wu | 3D shapenets: A deep representation for volumetric shapes[END_REF]. The networks are constructed with an architecture similar to that of the KPCNN in [START_REF] Thomas | Kpconv: Flexible and deformable convolution for point clouds[END_REF], but the subsampling is given by the farthest point sampling. Afterwards we test both in the task of predicting the classes at different orientations. The rotated images are obtained by first selecting a random unit vector in R 3 for each point cloud and afterwards we rotate the points by different angles along those axes. We can see the results of the experiment in Figure 6.3. In this small experiment the E(3)-equivariant model improved the results of the non-equivariant one, for both the rotated and non-rotated test sets. The dataset, however, is very small and it is difficult to infer how the performance of the models scales with the size of the dataset, so additional experiments are necessary.

Conclusions

The aim of this chapter was to concretly show perspectives for equivariant neural networks based on the method of moving frames. More specifically, three possible applications of the method were shown.

The first one is a network invariant to affine contrast changes. A positive point of that network is that it is relatively straightforward to implement, as only the first layer needs to be based on differential invariants, and the following layers can be any CNN layers. It was shown in a simple experiment to overperform the CNN baseline when the constrast of the test set images is changed globally. It was also shown that it works well even when the contrast change is local.

The second of these approaches is a network equivariant to translations, rotations and rescalings. This approach is based on the Scale-Equivariant Gaussian derivative network of [START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF]) by computing the multi-scale Gaussian derivatives from the multi-scale feature maps of the previous layer.

In the last one we wrote in terms of moving frames the approach of (Thomas, 2020) to obtain SE(3)-equivariant neural network layer for point clouds. The particular moving frame we proposed is based on the covariance matrix of a neighborhood of points. Because SE(3)equivariance required the orientation of normals vectors we also proposed a E(3)-equivariant network which does not depend on the orientation of normal vectors. We tested the proposed model in a simple experiment and obtained results that show a much higher invariance to rotations than the non-rotation-equivariant KPConv [START_REF] Thomas | Kpconv: Flexible and deformable convolution for point clouds[END_REF].

This chapter shows the generality of the moving frames approach for deriving invariants and equivariant network layers, and that it can be applied to a variety of input domains and symmetry groups.

Résumé en Français

L'objective de ce chapitre est de montrer concrètement des perspectives pour les réseaux de neurones basés sur la méthode du repère mobile. Plus particulièrement, trois applications possibles de la méthode ont été montrées.

La première est un réseau invariant par changement de contraste affine. Ce réseau est obtenu à partir de l'application des invariants différentiels du changement de contraste à l'entrée du réseau. Un premier point positif de ce réseau est la simplicité de son implémentation, car seulement la première couche doit être basée sur des invariants différentiels, et les couches suivantes peuvent être n'importe quelle couche de CNN. Nous avons montré dans une expérience simple que ce réseau surpasse un CNN de base quand le contraste change globalement. De plus les invariants marchent mieux qu'une normalisation des niveaux de gris quand le changement de contraste est local.

La deuxième est un réseau équivariant par translation, rotation et changement d'échelle. Cette approche est basée sur le réseau équivariant par changement d'échelle proposé par [START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF] en calculant les dérivées Gaussiennes multi-échelle à partir des cartes des caractéristiques multi-échelle de la couche précédente.

Dans la dernière section nous avons réécrit, en termes de la méthode du repère mobile, l'approche proposée par Thomas (2020) pour obtenir une couche équivariante par SE(3) appliquée aux nuages de points. Le repère mobile que nous avons proposé est basé sur la matrice de covariance d'un voisinage des points. Notre méthode requiert la connaissance des orientations des vecteurs normaux. Pour traiter cette question nous avons aussi proposé un réseau équivariant par E(3) (groupe des rotations, translations et réflexions en 3D) qui ne dépend pas de l'orientation des vecteurs normaux. Nous avons testé les modèles proposés dans une expérience simple et nous avons obtenu des résultats qui montrent un taux d'invariance par rotation beaucoup plus élevé par rapport aux réseaux KPConv qui ne sont pas équivariants par rotation [START_REF] Thomas | Kpconv: Flexible and deformable convolution for point clouds[END_REF]. Ce chapitre montre la généralité de la méthode du repère mobile pour dériver des couches invariantes ou équivariantes. Il montre aussi qu'on peut appliquer cette méthode à une variété de domaines d'entrée et de groupes de symétrie.

Conclusions

The work done in this thesis focused on obtaining neural networks that are equivariant to transformations. There were two main topics that were explored with this end, the scaleequivariant networks based on scale-spaces and multi-scale operators in Chapters 2 and 3 and the group equivariant networks obtained from the method of moving frames from Chapters 4, 5 and 6. Here we discuss the main contributions and their perspectives for future applications.

• Scale-Equivariant Networks Based on Morphological Scale-Spaces. The extension of the semigroup scale-equivariant networks [START_REF] Worrall | Deep scale-spaces: Equivariance over scale[END_REF] presented in Chapter 2 allow for the use of morphological scale-spaces as the lifting operators in the network. It was shown that the semigroup scale-equivariant networks are indeed able to generalize better than CNNs when the scale of the training set is not representative to the scale of the test set, and even though data augmented CNNs have sometimes better generalization results than the scale-equivariant ones, the scale-equivariant network are more robust to changes in the scale of the training set. Moreover, in experiments where the only available information is the shape of the objects and the scale of the training set is not representative of the scale of the test set, it is expected that morphological scale-spaces surpasses the Gaussian ones due to the morphological operator' capacity of preserving the shape in the presence of downsampling and this was confirmed empirically in a synthetic shape segmentation experiment.

To further explore the advantages of scale-spaces other than the Gaussian one as the lifting layer, it would be interesting to experiment with trainable liftings that can parameterize many types of scale-spaces, such as the Lasry-Lions lifting we presented. Moreover, the semigroup scale-cross-correlation layers have a limiting factor of only dealing with integer scale discretizations, but other methods, e.g. [START_REF] Lindeberg | Scale-covariant and scale-invariant gaussian derivative networks[END_REF]; [START_REF] Sosnovik | Scale-equivariant steerable networks[END_REF] can deal with arbitrary scale discretizations, so it should be interesting to see if scale-space lifting layers can be combined with these methods to obtain improved results.

• Scale-Equivariant U-Net. The SEU-Net proposed in Chapter 3 is a segmentation network obtained through the application of the U-Net architecture, including its downsampling and upsampling layers, inside a scale-equivariant pipeline. We compare the SEU-Net, U-Net and a scale-equivariant network that does not perform upsampling in the equivariant pipeline, SResNet, in three segmentation experiments where we vary the scale of the test set but keep the scale of the training set: a strand segmentation task, the natural image segmentation task Oxford-IIIT Pet and the cell segmentation experiment DIC-C2DH-HeLa. In those experiments, it was shown that when the U-Net is applied to objects that do not reflect the scale distribution of the training set, it suffers a significant decrease in performance, while SEU-Net was shown a much lesser decrease in performance in the same situation. When the U-Net is trained with a scale jittering containing the scales of the test sets, its overall performance falls as a result and still generalizes worse than the SEU-Net. Moreover the SEU-Net benefits from scale-jittering by small scale factors. When compared to the SResNet, the SEU-Net is shown to have better results at both scale 1 and unseen scales, justifying not only the use of scale-equivariant layers, but the arrangement of those layers in a U-Net like architecture.

As is the case of scale-equivariant networks with morphological liftings, it would be interesting to generalize the SEU-Net to layers at non-integer scale discretizations, as the discretizations by powers of 2 may be too rough for most applications.

• Differential Invariant Blocks. In Chapter 4, the method of moving frames was applied together with Gaussian derivative layers in order to propose neural network blocks based on differential invariant operators. This methodology was applied to obtain neural network blocks for image data that are equivariant to rotations and translations in the plane.

The main block of the network, the SE2DIN block, is obtained by computing Gaussian derivatives from the input of the block, followed by the computation of fundamental differential invariants using the Gaussian derivatives and the functional combination of those invariant by a multi-layer perceptron to form new feature maps. The network constructed from these blocks, called SE2DINNet, is reasonably lightweight and the computation of its output takes a number of floating point operations similar to that of a CNN with the same number of parameters and layers. The method was evaluated in the task of rotated digit classification of MNIST-Rot and obtained competitive results when compared to similar models. The main difficulty of training the SE2DINNet was its numerical instability, an activity regularization was used to mitigate the problem but it did not address the cause of the issue.

It would be interesting to test the SE2DINNet in more complicated tasks, but the numerical issues limits its applications. The method of moving frames is general enough to be applied to many different domains and many different symmetry groups, as was discussed in Chapter 6. Following the leads of that chapter it would be interesting to apply the methodology used for the SE2DINNets to define networks invariant to contrast changes, or more generally networks invariant to group transformations in the color space. Furthermore, scale-equivariant and rotation equivariant networks can be combined by exploiting the scale-equivariance properties of the Gaussian derivatives. Other than the approaches discussed in Chapter 6, some other areas where networks with differential invariants could be applied are in physics informed networks equivariant to Lorentz transformations [START_REF] Bogatskiy | Lorentz group equivariant neural network for particle physics[END_REF] and networks in the Poincaré disk, equivariant to the isometries of the disk for fisheye images [START_REF] Lagrave | Hyperbolic equivariant convolutional neural networks for fish-eye image processing[END_REF].

• Single-Moving-Frame Networks. The approach of Chapter 5 was proposed to further mitigate the numerical problems of the differential invariants networks. In the approach of the SE2DIN blocks, the differential invariants are implicitly the result of computing a moving frame from the Gaussian derivatives and applying the moving frame to the derivatives through the prolonged group action, which creates non-linear functions that may cause numerical issues. The proposal of Chapter 5 was to use a single moving frame, computed from the Gaussian derivatives of the input image, to compute invariants. It was shown that this defines a rotation equivariant neural network and furthermore, the moving frame can be fed as input to the network and the application of the moving frame to the feature maps can be considered a linear operator with respect to the feature maps. A moving frame for the action of translations and rotations in volumes was applied with the methodology described above to obtain a SE(3)-equivariant network for volumes called SE3MovFNet. The networks were tested on datasets of low-resolution medical volumes classification. The SE3MovFNet achieved the state-of-the-art in four of the six tested datasets. It was shown to be perfectly equivariant to rotations by multiples of 90 • degrees around the coordinate axes, but in some tasks its results are degraded for other rotations, which may be caused by the rotation of low-resolution of the volumes.

It would be interesting to investigate how the methodology of a single moving frame compares with the SE2DINNets in 2D computer vision tasks. Moreover, the approach of a single moving frame can be used to obtain SE(3)-equivariant for point clouds, as was shown in Chapter 6. This approach can also be applied to the Lorentz transformationequivariant and the Poincaré disk isometriy-equivariant networks metioned above. Because the moving frames are computed before the rest of the network, a moving frame based on objects other than the jet-space can be investigated for these networks. In particular, the structure tensor [START_REF] Weickert | Anisotropic Diffusion in Image Processing[END_REF] may provide an alternative to the Hessian in order to find the rotation matrices, which would be interesting as it would depend only on first order derivatives and it also provides a description of the anisotropy of the images or volumes.

Overall these topics provided interesting results that motivate the study of equivariant networks, specially scale-equivariant networks and networks constructed from moving frames.

MOTS CL ÉS

Équivariance, invariance, r éseaux de neurones, vision par ordinateur, espaces-échelle.

R ÉSUM É

Dans le contexte des r éseaux de neurones, l' équivariance et l'invariance par des transformations peuvent induire une meilleure g én éeralisation à de nouvelles donn ées si ces derni ères contiennent les sym étries correspondantes. En particulier, dans le champs de la vision par ordinateur, la plupart des t âches doivent tenir compte de symmetries g éom étriques.

Ainsi, par exemple, la translation d'un objet dans une t âche de segmentation ne doit pas changer la classe de l'objet. L'objetif principal de cette th èse est d'explorer et d évelopper des r éseaux de neurones qui sont équivariants par rapport à certaines transformations. Les deux principales m éthodes qui ont ét é utilis ées sont les r éseaux équivariants par changement d' échelle bas és sur la correlation-crois ée sur le groupe des homoth éties, et les r éseaux équivariants par l'action d'un groupe de Lie bas és sur la m éthode des rep ères mobiles. La premi ère m éthode est bas ée sur la g én éralisation des r éseaux équivariants par un semi-groupe d' échelles qui ont ét é propos és r écemment, o ù les auteurs utilisent l'espace-échelle Gaussien pour transformer les images en des signaux sur un domaine des échelles et translations. La g én éralisation propos ée dans cette th èse permet d'utiliser un espace-échelle beaucoup plus g én éral. En particulier, les espaces-échelle morphologiques pr ésentent un avantage quand la seule information disponible sur l'objet d'int ér êt est sa g éom étrie. L' équivariance des op érateurs de sous-échantillonnage et sur échantillonnage est étudi ée et ceux-ci sont appliqu és avec les correlations-crois ées d' échelle pour obtenir le SEU-Net, une version de U-Net équivariante par changement d' échelle qui am éliore sa g én éralisation à des échelles non vues lors de l'entraînement. La m éthode du rep ère mobile est une approche classique pour obtenir des invariants diff érentiels par l'action d'un groupe de Lie sur une vari ét é. Dans ce travail de th èse, l'approche a ét é appliqu ée à la construction d'un r éseau de neurones équivariant par rotation et par translation. Le r éseau propos é a ét é test é sur une t âche de classification de chiffres manuscrits tourn és, et en d épit des certains probl èmes num ériques, le r éseau a obtenu de bons r ésultats par rapport à des r éseaux équivariants par rotation de taille similaire. Puis, pour éviter les probl èmes num ériques, un r éseau qui utilise un seul rep ère mobile pour calculer des invariants a ét é propos é et appliqu é pour cr éer des r éseaux équivariants par rotations et translations pour les volumes en 3D. Le r éseau a ét é test é sur un ensemble de bases de donn ées pour la classification de volumes m édicaux en faible r ésolution, et il a obtenu une performance à l' état-d'art dans la plupart des bases de donn ées test ées.

ABSTRACT

In the context of neural networks, equivariance or invariance to transformations can induce a better generalization to new data as soon as the data is symmetric to the relevant transformations. In particular, in the realm of computer vision most tasks have some kind of geometrical symmetry, for example, a translation of an object in segmentation tasks usually does not change the class of the object. The main objective of this thesis is exploring and developing neural networks that are equivariant to transformations. The two main frameworks which were used were the scale-equivariant networks based on scale-cross-correlation and the group-equivariant networks based on moving frames. The former is based on the generalization of the scale-semigroupequivariant networks recently proposed which used the Gaussian scale-space as a way to map images from their original domain to a domain of scales and translations. The generalization proposed in this thesis allows for a much more general class of scale-spaces to be used as liftings and it is shown that morphological liftings are beneficial when only the only information available is the shape of objects. Equivariance of the upsampling and downsampling operators is studied and applied with the scale-cross-correlation to create an architecture similar to the U-Net, the SEU-Net, which was shown to improve generalization to unseen scales of the U-Net. The method of moving frames is a classical approach to finding differential invariants in manifolds and in the present work it was applied to define neural network blocks that are equivariant to the action of a Lie group. In particular, it was applied to the definition of a neural network equivariant to rotations and translations of images. The proposed network was tested in the classification of rotated digits, and despite its numerical issues, it achieved results competitive with other rotation-equivariant models with similar size. Moreover, in order to deal with the numerical problems a solution which computes invariants from a single moving frame was proposed and applied to create a network equivariant to rotations and translations of 3D volumes. The 3D rotation-equivariant network was applied to tasks of low-resolution medical volume classification and achieved state-of-the-art results for most of the tested datasets.

KEYWORDS

Equivariance, invariance, neural networks, computer vision, scale-spaces.

  Scale-equivariant network with morphological lifting. (b) Prediction with Gaussian lifting (c) Prediction with dilation lifting

Figure 1 :

 1 Figure 1: An illustration of the morphological lifting(a) (especifically quadratic dilation lifting) proposed in Chapter 2 as an alternative to the Gaussian lifting. We shown the result of a shape classification experiment where the distribution of scales in the training set is not representative of the distribution of scales in the test set. The model with a dilation scale-space lifting (c) manages to improve the results of the model with a Gaussian scale-space lifting(b) in this case, as the morphological operators are better suited at preserving the shape of objects.

H/ 2 ×Figure 2 :

 22 Figure 2: Illustration of the SEU-Net architecture proposed in Chapter 3. Images are lifted to a scale-space representation with a lifting Λ, processed with a U-Net like architecture using scalecross-correlations and other equivariant or approximately equivariant operators and projected back into the shape.

Figure 3 :

 3 Figure 3: Results from Chapter 3 showing that the U-Net is not scale-equivariant empirically and that the SEU-Net greatly improves its equivariance. Both models were trained on the scale of the second biggest image.

Figure

  Figure4: Illustration of the idea behind our moving frame approach for equivariant networks (a) as well as the illustration of our implementation used in Chapter 4. The idea is to use the local structure of the image to obtain a rotation equivariant quantity (e.g. the direction of the gradient, shown in red) and apply filters (shown in blue) like in a convolution but turned with respect to that quantity. When the image is rotated, the red arrows are equal and so the filters. By computing a linear combination of differential invariants, as the first 1 × 1 convolution in (b) we can achieve an effect similar to the illustration in example (a).

  Invariants from a fixed moving frame

Figure 5 :

 5 Figure5: Illustration of the difference of the approach of Chapter 4, where the moving frame is computed at every SE2DIN block, and the approach used in Chapter 5, where the moving frame is computed in the first layer and used subsequent layers. Similar diagrams can be shown in the case the network uses residual connections.

  Réseau équivariant par changement d'échelle avec le dépliement Gaussien. (b) Prédiction avec le dépliement Gaussien (c) Prédiction avec le dépliement par dilatation

Figure 6 :

 6 Figure 6: Une illustration des dépliements morphologiques (a) (particulièrement les dépliements basés sur les dilatations quadratiques) proposés dans le Chapitre 2 comme une alternative au dépliement Gaussien. Nous montrons les résultats de l'expérience de classification de formes, où la distribution des échelles dans les données d'entraînement n'est pas la même que celle dans les données de teste. Le modèle avec un dépliement basé sur l'espace d'échelle des dilatations quadratiques (c) obtient de meilleurs résultats que les modèles avec le dépliement Gaussien (b). Dans ce cas, les opérateurs morphologiques sont plus efficaces pour préserver la forme des objets.

H/ 4 ×Figure 7 :

 47 Figure 7: Illustration de l'architecture SEU-Net proposée dans le Chapitre 3.

Figure 8 :

 8 Figure 8: Résultats du Chapitre 3 qui montrent que U-Net n'est pas empiriquement équivariant par changement d'échelle et que le SEU-Net améliore la performance sur la tâche. Tous les deux modèles étaient entraînés sur une base de données à l'échelle de la deuxième image la plus grande montrée.

Figure 9 :

 9 Figure 9: Illustration de l'idée principale derrière les réseaux équivariants basés sur les repères mobiles (a) ainsi qu'une illustration de l'implémentation de la méthode utilisée dans le Chapitre 4. L'idée est d'utiliser la structure locale de l'image pour obtenir une transformation (par exemple la rotation par la direction du gradient, en rouge) de façon équivariante et puis appliquer un filtre (en bleu) transformé par cette transformation. Si l'image est tournée, alors les flèches rouges et les filtres sont également tournés. En calculant une combinaison linéaire des invariants différentiels, comme la convolution 1 × 1 en (b), nous obtenons un effet similaire à l'illustration dans l'exemple (a).

  Figure 10: Illustration de la différence entre l'approche du Chapitre 4, où les repères mobiles sont calculés à chaque bloc SE2DIN, et celle du Chapitre 5, où le repère mobile est calculé dans la première couche et utilisé pour toutes les couches suivantes.

Figure 1 . 2 :

 12 Figure 1.2: Example of equivariance to transformations (similarity transformations in this case).The application of φ, which can be thought of as an equivariant neural network layer commutes with the transformation π g .

Figure 1 . 3 :

 13 Figure 1.3: Illustration of a manifold M and a chart (U, φ) mapping to Euclidean space R d .

Definition 2 (

 2 Chart, Atlas). Let M be a topological manifold of dimension d. A chart is a tuple (U, φ) where U ⊆ M is an open set and φ : U → R d is a homeomorphism i.e. a bijection such that φ and φ -1 are continuous. An atlas is a collection of charts (U α , φ α ) α∈A that covers M, i.e. α∈A U α = M. Definition 3 (Smooth manifold). If M is a topological manifold of dimension n, (U α , φ α ) α∈A is and atlas and for all α, β ∈ A the transition maps φ α • φ -1 β : R d → R d are smooth, then M is a smooth manifold.

  the derivative right-hand side of the equation denotes partial differentiation in R d . of T p M.

•

  the orthogonal group (i.e. rotations and reflections in the plane) O(d) = {A ∈ GL(d)|A T A = AA T = I n } where I d is the d × d identity matrix; • the special linear group SL(d) = {A ∈ GL(d)|det(A) = 1}; • and the special orthogonal group SO(d) = {A ∈ O(d)|det(A) = 1} i.e. rotations in the plane.

Figure 1 . 5 :

 15 Figure 1.5: Examples of group actions on images.

Figure 1

 1 Figure 1.6: A cylinder and a Möbius band are both examples of fiber bundles, with the same base spaces and fibers, however while the cylinder has the same topology as the Cartesian product, the band has a different topology.

Figure 2 . 2 :

 22 Figure 2.2: Illustration of the receptive field of a 4-layer CNN with 3 × 3 filters in each layer. Each convolution by a 3 × 3 filter increases the size of the receptive field by 2. In general each convolution by a K × K filter increases the receptive field size by K -1.

Figure 2 . 4 :

 24 Figure 2.4: An image f and the first five scales of its Gaussian scale-space representation, Λ G f (γ 0 , •), . . . Λ G f (γ 4 , •) using a base γ = 2.

1 2 ,

 2 +) scale-spaces. Here, to increase flexibility in the context of deep learning, we consider a parameter c > 0 learned by gradient descent with the rest of the parameters of the network. Examples of quadratic dilations and quadratic erosions scale-spaces applied to an image are shown in Figure2.5.

  (a) Dilation scale-space. (b) Erosion scale-space.

Figure 2 .

 2 Figure 2.5: The first five scales of quadratic dilations and erosions scale-spaces representation of an image f , Λf (γ 0 , •), . . . Λf (γ 4 , •) using base γ = 2.

  The MNIST Large Scale[START_REF] Jansson | Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges[END_REF] dataset is built upon the MNIST dataset[START_REF] Lecun | Mnist handwritten digit database[END_REF] and was introduced to evaluate the ability of CNNs to generalize to scales not seen in the training set. The dataset contains three training sets, tr1, tr2 and tr4, which consist of 50000 samples from the MNIST dataset upscaled by factors one, two and four respectively. Each training set is accompanied by 10000 samples in the same scale to be used as validation sets.

  Figure 2.6(a) -(c) shows examples of images on each of the training sets corresponding to the same MNIST image.As for the scales of the test set we have one for each of the scales 2 i 4 , i = -4, . . . , 12, amounting to 17 test sets.

  Figure 2.6 shows examples of the same image on different test sets.

  Test image on several different scales.

Figure 2 .

 2 Figure 2.6: Examples of training and testing MNIST Large Scale images.

Figure 2 .

 2 Figure 2.7: Accuracy of scale equivariant models in the Large Scale MNIST dataset, trained on each of the training sets tr1, tr2 and tr4 and tested on all test sets. Four different liftings were used: Identity, Gaussian, quadratic dilations and a convex combination of dilation and erosion.

Figure 2 .

 2 Figure 2.8: Accuracies with models training with scale dimensions sdim = 1 and sdim = 2. Both identity and quadratic dilations liftings were compared, with scale dropout values of 0% and 25%. Accuracies of scale equivariant models in the Large Scale MNIST dataset, trained on each of the training sets tr1, tr2 and tr4 and tested on all test sets.

Figure 2

 2 Figure 2.9: Example images from the (a) training and (d) test sets of the segmentation experiments, and segmentation results using equivariant models with the (b)(e) proposed dilation and (c)(f) Gaussian scale-spaces. Pixels in red are classified as disk, those in blue as star.

  Figure 2.10: The same image after being processed by the Gaussian (a) and quadratic dilations (b) scale-spaces and being subsampled by factors 2 i i = 0, 1, 2, 3, 4.

Figure 3 . 1 :

 31 Figure 3.1: Example where a U-Net trained on one scale distribution and is applied to predict the output on the training(a) and on an unseen(b) scale i.e. a scale not well represented in the training scale distribution. The image with the unseen scale represents the same object but the U-Net no longer segments it correctly.

Figure 3 . 2 :

 32 Figure 3.2: Diagrams representing a U-Net and a SEU-Net with depth 4 and C channels at the first layer given inputs of spatial dimension H × W . The SEU-Net blocks have the same dimension of their corresponding U-Net blocks, but with the additional scale dimension i.e.the first block has dimensions H × W × S × C, the second one H/2 × W/2 × S × 2C and so on. The lifting and projection layers are denoted respectively Λ and Π. In both diagrams, red arrows signify subsampling layers and blue arrows signify upsampling layers. Features maps on the same level (i.e. y coordinate in (a) and z coordinate in (b)) have the same spatial dimension. Subsequent feature maps not followed by an arrow are obtained by convolution / scale-cross-correlation layers followed by nonlinearities and batch normalization.

Figure 3 . 3 :

 33 Figure 3.3: Example illustrating the problem with upsampling in a scale-equivariant architecture. We have images f 1 and f 2 such that when both are downsampled and then upsampled they yield the same result, but if both are upsampled and then downsampled they yield different results.

Figure 3 . 4 :

 34 Figure 3.4: A slice from the 3D tomography and its corresponding target segmentation.

Figure 3 .

 3 Figure 3.5: Some images and ground truths from the Oxford Pet IIIT dataset.

Figure 3 . 6 .Figure 3 . 6 :

 3636 Figure 3.6: Overall results of the tomographic images dataset. The curves show the average MSE from three runs and the shaded regions show are between the minimum and maximum values attained. We omit the max-pooling for better visibility of the other methods. It can be seen in Appendix D. We see that in the cases of quadpool strides, equivariance brings a significant improvement.

  (a) Image and ground truth. (b) U-Net predictions. (c) SEU-Net (strides) predictions.

Figure 3 .Figure 3

 33 Figure 3.7: Predictions computed from a square crop of a test image in tomographic images dataset. Computed from a small region of a test image 3.7a. The first, second and third columns correspond to scales 0.5, 1 and 2, respectively.

Figure 3 .

 3 Figure 3.10: IoUs of the cell segmentation experiment with comparisons with U-Net, SResNet and data augmentation. U-Net (aug. 4) refers to the U-Net trained with scale jittering with range 4 and U-Net (aug. 1.5) refers to the U-Net trained with jittering with range 1.5. The same for SEU-Net (aug. 1.5).

Figure 3 .

 3 Figure 3.11: Sample test images at different scales and ground truth from the Oxford-IIIT Pet dataset, along with the U-Net and SEU-Net predictions. The scales present are 0.25, 0.5, 1 and 2 times the training scale.

Figure 3 .

 3 Figure 3.12: Predictions from DIC-HeLa at different scales, namely scales 0.5, 1 and 2.

Figure 4 . 1 :

 41 Figure 4.1: Example of the application of a kernel (shown as blue squares) where its orientation (shown as red arrows) depends on the local image geometry. In the previous example and in the method we are going to propose for rotation equivariant operators on images, an orientation for each point can be obtained as the direction of the gradient, for example.

  d let us denote by |I| = d k=1 i k its modulus and by I d n = {I ∈ N d , |I| ≤ n} the set of multi-indices in N d of modulus at most n ∈ N. For x ∈ X and n ∈ N, we introduce J

  the interest of later computing invariants, the action on the second-order jet-space maps u xx , u xy and u yy to ũxx , ũxy and ũyy , where ũxx = u xx (cos θ) 2 + 2u xy cos θ sin θ + u yy (sin θ) 2 , ũxy = (u yy -u xx ) cos θ sin θ -u xy ((sin θ) 2 -(cos θ) 2 ), ũyy = u xx (sin θ) 2 -2u xy cos θ sin θ + u yy (cos θ) 2 .(4.25)

Figure 4 . 3 :

 43 Figure 4.3: Gaussian derivative filters of order ≤ 2 computed in a discrete domain.

Figure 4 . 4 :

 44 Figure 4.4: Diagram illustrating how one can obtain filters as linear combinations of Gaussian derivatives.

Figure 4 . 5 :

 45 Figure 4.5: Illustration of a SE2DIN block applied to a MNIST-Rot image. Convolutions are followed by batch normalization and activation function. It takes as input an image of height H, width W and C channels and outputs an image with the same spatial dimensions and C channels.

Figure 4 . 6 :

 46 Figure 4.6: Example of the output of a CNN layer and a SE2DIN block given an input and its rotation.We can see that the SE2DIN block output rotates as the input is rotated. Images made using the models from Section 4.6.

Figure 4 .

 4 Figure 4.7: Illustration of how a single rotated filter is applied. The rotated filter is the output of the first 1 × 1 convolution in the SE2DIN block, i.e. a linear combination of the fundamental invariants. A linear combination of the the fundamental invariants can also be identified as the rotation of a linear combination of a local rotation of the Gaussian derivatives (where the rotation angle is determined by the gradient at each point), but instead of computing the rotations by interpolation they are computed by a non-linear expression of the convolutions of the image by the Gaussian derivatives. This examples shows the link between the differential invariants blocks and the initial goal of applying rotated filters illustrated in Figure 4.1.

2 Figure 4 . 8 :

 248 Figure 4.8: Subsampling of a 4 × 4 image and its rotation by 90• by a factor of 2. The position (0, 0) is assumed to be the pixel at the top-left corner. Of course in real scenarios images have a much bigger dimension and do not necessarily have such high frequency information, but the subsampling will still generate small errors to equivariance that can propagate to the next layers. In the experimental section it will be shown that the choice of pooling does have a significant impact in practice.

Figure 4 . 9 :

 49 Figure 4.9: Illustration of a basic SE2DINNet. Global maxpooling renders the architecture invariant. All blocks except the first and last have the same number of input channels C and filters C and follow the structure of a numerical scheme for solving a PDE.

Figure 4 .

 4 Figure 4.11: Evaluation metrics comparing methods on Fashion-MNIST using Gaussian derivatives and finite differences to compute the derivatives. Similarly to the results in Figure 4.10, models are trained on the Fashion-MNIST training set and tested in rotated versions of the test set.

Figure 4 .

 4 Figure 4.13: Evaluation metrics comparing methods using different kinds of subsampling schemes on MNIST.

  (a) Filters from finite difference stencils. (b) Filters from Gaussian derivative stencils.

Figure 4 .

 4 Figure 4.14: Some examples of filters obtained by linear combination of differential invariants in first SE2DIN layer of a network, by computing the rotation of the derivatives from equations (4.24) and (4.25). Rotation angles are equally spaced in the interval [0, π 2 ].

Figure 4 .

 4 Figure 4.15: Evaluation metrics of untrained and trained models on MNIST and its rotated versions. Because untrained models are being tested, only measure distances between features are computed, and not accuracy and consistency metrics.

Figure 4 .

 4 Figure 4.16: SE2DINNet used for MNIST-Rot.

Figure 5 . 1 :

 51 Figure5.1: Illustration of the difference of the approach of Chapter 4 and the approach we propose in this chapter where we compute a moving frame at the first layer and use it for subsequent layers. Similar diagrams can be shown in the case the network uses residual connections.

  by definition of the action on functions(4.11).

Figure 5 . 2

 52 Figure 5.2 shows examples of Gaussian derivatives up to order 2.We refer to the Gaussian n-jet of a volume as the Gaussian derivatives of order ≤ n f

Figure 5 . 2 :

 52 Figure 5.2: Examples of 3D Gaussian derivative filters of order ≤ 2.

Figure 5 . 3 :

 53 Figure 5.3: Example architecture of a SE3MovFNet, along with an example building block of the architecture.

Figure 5 . 4 :

 54 Figure 5.4: The eight tetris pieces in three dimensions.

Figure 5

 5 Figure 5.5: Some samples of different classes of each dataset.

Figure 5 . 7 :

 57 Figure 5.7: Predictions of models on half of the dataset of MedMNIST on the test set rotated by different angles around each of the coordinate axis.

  )]. Some example images are shown in Figure6.2.

Figure 6 . 2 :

 62 Figure 6.2: Original images and the transformed version we used to test local contrast change. Gray value scale is not the same for all images. White pixels are the largest grey value of the image and black the lowest.

  h(y, y k ) = exp -yy k 2 σ 2 ; (6.21)We propose the Moving KPConv (MKPConv) as(f * moving g)(x) = x i ∈Nx g(R x (x i -x))f i . (6.25)where R x is the matrix given by the moving frame at the neighborhood N x . Considering the functionF (N x ) = k x i ∈Nx g(x i )f i , we have that (f * moving g)(x) = ı[F ](N x ). (6.26)Moreover, F is invariant w.r.t. permutations of N x that fix x 1 , and therefore so is ı[F ].

1 .-p 3 .

 13 Start by selecting a random point p ∈ P and create S = {p}; 2. Find a new point q * : q * = arg max q∈P min p∈S q Add q * to S: S = S ∪ {q * }; 4. If |S| = M stop, otherwise got to step 2.

Figure 6 . 3 :

 63 Figure 6.3: Accuracies of the MKPCNN and the KPCNN.

Table 2 .

 2

	Scales Gaussian Lifting Dilation Lifting Closing Lifting	Id Lifting	U-Net	CNN
	1.						0006
	[ 1 2 , 2] [ 1 4 , 4]	0.92 ± 0.06 0.88 ± 0.07	0.97 ± 0.01 0.93 ± 0.02	0.89 ± 0.06 0.86 ± 0.05	0.91 ± 0.03 0.88 ± 0.03	0.86 ± 0.02 0.70 ± 0.04	0.68 ± 0.01 0.627 ± 0.008

0.9929 ± 0.0006 0.9929 ± 0.0006 0.9929 ± 0.0005 0.9927 ± 0.0008 0.9927 ± 0.0006 0.9083 ± 0.1: IoU between the ground truth images and the predictions obtained from equivariant models with different liftings, trained on images where objects only appear at scale one and tested on test sets where scales vary in the range [2 -i , 2 i ] for i = 0, 1, 2.

  • , 10 • ], linear contrast changes by values in the range [0.9, 1.1], random horizontal

		0.8								1.00
		0.7								0.95
										0.90
	IoU	0.3 0.4 0.5 0.6	0.2	0.4	0.5	0.7 UNet SEUNet (Quadratic) 1.0 1.4 scale SEUNet (Strides) SEUNet (MaxPool) SResNet SEUNet (Dropout, Quadratic) 2.0 SEUNet (Dropout, Strides) SEUNet (Dropout, MaxPool) SResNet (Dropout)	2.8	4.0	Consistency	0.65 0.70 0.75 0.80 0.85	0.2	0.4	0.5	0.7 UNet SEUNet (Quadratic) 1.0 1.4 scale SEUNet (Strides) SEUNet (MaxPool) SResNet SEUNet (Dropout, Quadratic) 2.0 SEUNet (Dropout, Strides) SEUNet (Dropout, MaxPool) SResNet (Dropout)	2.8	4.0
						(a) IoU

Table 4 . 2

 42

: Mean accuracies on the MNIST-Rot dataset. Legend: † -train time augmentation, ‡ -test time augmentation.

.

  • AdrenalMNIST3D. Scans of adrenal glands labeled into normal adrenal gland and adrenal mass. The numbers of training/validation/test volumes is 1118/98/298.

	• NoduleMNIST3D. Downsized thoratic CT scans. Contains two classes based on level
	of lung nodule malignancy. Has a 7:1:2 training/validation/test split.
	• OrganMNIST3D. Multiclass classification of CT scans into 11 different organs.
	• VesselMNIST3D. Entire brain vessels divided into healthy and aneurysm classes. 1694
	healthy and 215 aneurysm. Has a 7:1:2 training/validation/test split.

• FractureMNIST3D. Rib fractures classified into 3 different classes.

Table 5 .

 5 1: Accuracies on the MedMNIST dataset compared to the benchmarks.

				CNN	CNN augmented		SE3MovFNet	SE3MovFNet aug.
									0.65						0.65
		0.60						0.60						0.60
	accuracy	0.45 0.50 0.55	0	100	200 angle	300	accuracy	0.45 0.50 0.55	0	100	200 angle	300	accuracy	0.45 0.50 0.55	0	100	200 angle	300
	(a) FractureMNIST3D Z-axis	(b) FractureMNIST3D Y-axis	(c) FractureMNIST3D X-axis
			0.8							0.8							0.8
	accuracy	0.4 0.6					accuracy	0.4 0.6					accuracy	0.4 0.6
			0.2	0	100	200 angle	300			0.2	0	100	200 angle	300			0.2	0	100	200 angle	300
			(d) OrganMNIST3D Z-axis			(e) OrganMNIST3D Y-axis			(f) OrganMNIST3D X-axis
		0.88						0.88						0.88
		0.86						0.86						0.86
		0.84						0.84						0.84
	accuracy	0.78 0.80 0.82					accuracy	0.78 0.80 0.82					accuracy	0.78 0.80 0.82
		0.76						0.76						0.76
		0.74						0.74						0.74
				0	100	200 angle	300				0	100	200 angle	300				0	100	200 angle	300
	(g) NoduleMNIST3D Z-axis	(h) NoduleMNIST3D Y-axis	(i) NoduleMNIST3D X-axis
	Figure 5.6: Predictions of models on half of the dataset of MedMNIST on the test set rotated
	by different angles around each of the coordinate axis.

To avoid triviality, we assume that π X g is not the identity for all g ∈ G.

Note that S ∈ σ implies g • S ∈ σ, as the group multiplication is continuous and therefore the translation of a Borel set is a Borel set.

using the group operation • = + we have that x-y = x•y -1 = y -1 •x because the operation is commutative, we use the first one in the definition of group convolution

In the paper the network is also equivariant to reflections, but we examine a simpler case here.

https://www.robots.ox.ac.uk/ ~vgg/data/pets/, CC BY-SA 4.0 license

https://www.tensorflow.org/datasets/catalog/oxford_iiit_pet

if det P (x,u) = -1 we can multiply one of its rows by -1 so that the new matrix has determinant 1.

Acknowledgements

Introduction

Introduction en Français

Acknowledgements I am very grateful to my thesis supervisors, Jesús Angulo, Santiago Velasco-Forero and Samy

Cosine Similarity strides, Gaussian derivatives P4Strides Gaussian derivatives strides, finite differences P4Strides, finite differences no subsampling, same sigma no subsampling, increasing sigma applied in a zone where the gradient is a vector in the unit circle at angles i π 8 i = 0, 1, 2, 3, 4. The rotated finite difference filters in Figure 4.14(a) show very abrupt changes between consecutive angles while the Gaussian rotated filters in Figure 4.14(b) have a smooth transition, further justifying the choice of Gaussian derivatives as the method of approximating derivatives.

Pooling

In order to evaluate the contribution of choosing the right pooling functions we test the models with different pooling protocols: no subsampling, strided operators and P4Strides. In the nosubsampling case we also compare Gaussian derivatives with constant or increasing σ with the same values of σ as the previous subsection. In [START_REF] Thomas | Kpconv: Flexible and deformable convolution for point clouds[END_REF] the method was further generalized to work with deformable kernels, as opposed to the rigid kernels described above. In this section, however, we only study the case of rigid kernels for the KPConv and for its rotation-equivariant version described below.

SE(3)-Equivariant KPConv

Now we propose to make the KPConv SE(3)-equivariant with the method of moving frames. The same reasoning was applied by Thomas (2020) albeit not using moving frames explicitly.

Here we describe the method in terms of moving frames and propose a specific moving frame based on the covariance matrix of the points in a neighborhood.

In order to obtain equivariance we impose additional structure on the point cloud. We define an oriented point cloud as a point cloud P for which each point x i ∈ P is associated with a normal vector n i belonging to the unit sphere S 2 .

Let us write the neighborhood of a point x as a tuple N x = (x 1 , . . . , x M ) ∈ P M where x 1 = x, and let us note n 1 its corresponding normal vector. The space of possible neighborhoods of size M is given by M = (R 3 ) M . We can define the action of SE(3) on it by

x i and let X be the 3 × M matrix having (x i -µ)/M as columns, i.e.

is the empirical covariance matrix of the neighborhood. If we apply the action of (R, v) the transformed points become xi = Rx i and the the transformed normal vectors ni = Rn i , we obtain μ, X and a new covariance matrix Ā, given respectively by

We define a cross-section in this space by the constraints x 1 = 0 and A = D, where D is a diagonal matrix with values in the diagonal in descending order. This gives us six equations which must be solved, namely x 1i = 0, i = 1, 2, 3 and A 12 = A 13 = A 23 = 0 (note that A is symetrical and therefore this condition gives us A 21 = A 31 = A 32 = 0) which should be enough since SE(3) is 6-dimensional. We can easily deduce that in order to solve the equations, we must have v = -Rx 1 . To determine R, first we can make an eigen decomposition A = QDQ T , where D is sorted in descending order. If we set R = Q T we obtain Ā = QAQ T = QQ T DQQ T = D, therefore solving the moving frame equations. However, it turns out there are two solutions in this case. The simplest solution to this problem is to align the eigenvectors, i.e. the columns of Q, with the normal n 1 .