Louis Béthune

Paul Novello

Guillaume Coiffier

Thibaut Boissin

Mathieu Serrurier

Quentin Vincenot

Andres Troya-Galvis

Robust One-Class Classification with Signed Distance Function using 1-Lipschitz Neural Networks

We propose a new method, dubbed One Class Signed Distance Function (OCSDF), to perform One Class Classification (OCC) by provably learning the Signed Distance Function (SDF) to the boundary of the support of any distribution. The distance to the support can be interpreted as a normality score, and its approximation using 1-Lipschitz neural networks provides robustness bounds against l2 adversarial attacks, an underexplored weakness of deep learning-based OCC algorithms. As a result, OCSDF comes with a new metric, certified AUROC, that can be computed at the same cost as any classical AUROC. We show that OCSDF is competitive against concurrent methods on tabular and image data while being way more robust to adversarial attacks, illustrating its theoretical properties. Finally, as exploratory research perspectives, we theoretically and empirically show how OCSDF connects OCC with image generation and implicit neural surface parametrization.

Introduction

One class classification (OCC) is an instance of binary classification where all the points of the dataset at hand belong to the same (positive) class. The challenge of this task is to construct a decision boundary without using points from the other (negative) class. It has various safety-critical applications in anomaly detection, for instance to detect banking fraud, cyber-intrusion or industrial defect, in outof-distribution detection, to prevent wrong decisions of Machine Learning models, or in Open-Set-Recognition. How-ever, OCC algorithms suffer from limitations such as the lack of negative data, and robustness issues [START_REF] Azizmalayeri | Your out-of-distribution detection method is not robust![END_REF], the latter being an under-explored topic in the OCC spectrum. Even though some algorithms do not use negative examples, many work cope with the lack of negative data with Negative Sampling, either artificially [START_REF] Sipple | Interpretable, Multidimensional, Multimodal Anomaly Detection with Negative Sampling for Detection of Device Failure[END_REF] or using outlier exposure (Hendrycks and Dietterich, 2019;[START_REF] Fort | Exploring the limits of out-of-distribution detection[END_REF]. However, such samplings are often biased or heuristic. As for robustness, although some works design robust algorithms [START_REF] Goyal | DROCC: Deep Robust One-Class Classification[END_REF][START_REF] Shao-Yuan | Adversarially robust one-class novelty detection[END_REF], it is always only empirically demonstrated (Hendrycks and Dietterich, 2019).

In this paper, we introduce a new framework to perform OCC based on the Signed Distance Function (SDF), a function traditionally used in computer graphics. Assume the positive samples are independently and identically obtained from a distribution P X with compact support X ⊂ R d . Let ∂X = X / X be the boundary of the distribution. The Signed Distance Function is the function S : R d → R:

S(x) = d(x, ∂X ) if x ∈ X , -d(x, ∂X ) otherwise, (1)
where d(x, ∂X ) = inf z∈∂X ∥x -z∥ 2 . The idea of our algorithm, which we call One Class Signed Distance Function (OCSDF) is to learn the SDF to the boundary of the positive data distribution and use it as a normality score. We show that the Hinge Kantorovich-Rubinstein (HKR) loss introduced by [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF] allows provably learning the SDF with a 1-Lipschitz network.

SDF exhibits desirable properties. First, by implicitly parametrizing the domain X , it allows efficiently sampling points outside of X and performing principled Negative Sampling. Second, the SDF fulfils the Eikonal equation: ∥∇ x S(x)∥ = 1. In particular, S is 1-Lipschitz with respect to l2-nom : ∀x, z ∈ R d , ∥S(x) -S(z)∥ 2 ≤ ∥x -z∥ 2 . This property provides exact robustness certificates for OCSDF in the form of a certified AUROC that can be computed at the same cost as AUROC. This regularity translates into solid empirical robustness as compared to other OCC baselines. In other words, OCSDF alleviates the lack of negative data and the robustness issue. We go further and highlight We start with an uniform negative sampling, then we fit a 1-Lipschitz classifier f θ using the Hinge Kantorovich-Rubinstein loss. We apply the Adapted Newton Raphson algorithm 1 to attract the points towards the boundary of the domain ∂X thanks to the smoothness of f θ , which in addition allows providing robustness certificates.

interesting research perspectives regarding OCSDF. Indeed, we show that learning the SDF with a 1-Lipschitz network enables a generative procedure that allows visualizing points at the boundary of X . Moreover, It implicitly parametrizes the shape of X , which connects One-Class Classification with implicit surface parametrization, intensively used in computer graphics for shape reconstruction.

Our contributions are as follows.

(1) We introduce a new OCC framework based on the Signed Distance Function to the boundary of the data distribution. We theoretically demonstrate that the SDF can be learned with a 1-Lipschitz neural net using the Hinge Kantorovich-Rubinstein (HKR) loss and Negative Sampling;

(2) We evaluate the performances of OCSDF on several benchmarks and show its benefits for theoretical and empirical robustness; and (3) we demonstrate how OCSDF extends the applications of One Class Classification from traditional OOD detection to generative visualization and implicit surface parametrization for shape reconstruction from point clouds.

Related Work

One Class Classification (OCC) OCC is an instance of binary classification where all the points of the dataset at hand belong to the same (positive) class. The challenge of this task is to construct a decision boundary without using points from the other (negative) class. OCC amounts to finding a domain containing the support of the data distribution.

That is why OCC is mainly used in Out Of Distribution (OOD), anomaly or novelty detection, with positive samples considered In Distribution (ID) and negative ones as OOD, anomalies or novelties. This task dates back to [START_REF] Sager | An Iterative Method for Estimating a Multivariate Mode and Isopleth[END_REF][START_REF] Hartigan | Estimation of a Convex Density Contour in Two Dimensions[END_REF] and was popularized for anomaly detection with One-class Support Vector Machines (OC-SVM) [START_REF] Schölkopf | Support Vector Method for Novelty Detection[END_REF]. Since then, the field of OCC has flourished with many well-established algorithms such as Local Outlier Factors [START_REF] Breunig | Lof: Identifying density-based local outliers[END_REF], Isolation Forests [START_REF] Fei | Isolation forest[END_REF] and their variants (see [START_REF] Han | ADBench: Anomaly Detection Benchmark[END_REF] for a thorough benchmark). More recently, since Deep-SVDD [START_REF] Ruff | Deep one-class classification[END_REF] -followed by several works such as [START_REF] Bergman | Classification-based anomaly detection for general data[END_REF]Golan and El-Yaniv, 2018;[START_REF] Goyal | DROCC: Deep Robust One-Class Classification[END_REF][START_REF] Zenati | Adversarially learned anomaly detection[END_REF][START_REF] Sabokrou | Adversarially learned one-class classifier for novelty detection[END_REF] -Deep Learning has emerged as a relevant alternative to perform OCC due to its capacities to handle large dimensional data. However, methods of this field suffer from their lack of robustness and certifications, which makes them vulnerable to adversarial attacks. In addition, they always struggle to cope with the lack of OOD data.

In this paper, we tackle these problems with an OCC algorithm based on approximating the SDF using 1-Lipschitz neural nets. In addition, the SDF being intensively used in Computer Graphics, our algorithm establishes a new link between OCC and implicit surface parametrization.

SDF for neural implicit surfaces Historically, signed distance functions have been used in computer graphics to parametrize a surface as the level set of some function [START_REF] Novello | Exploring Differential Geometry in Neural Implicits[END_REF]. Given an incomplete or unstructured representation of a geometrical object (like a 3D point cloud or a triangle soup), recent methods aim at representing a smooth shape either as vectors in the latent space of a generative model [START_REF] Achlioptas | Learning Representations and Generative Models for 3D Point Clouds[END_REF][START_REF] Ben-Hamu | Multi-chart Generative Surface Modeling[END_REF][START_REF] Groueix | AtlasNet: A Papier-M\âch\'e Approach to Learning 3D Surface Generation[END_REF][START_REF] Chou | GenSDF: Two-Stage Learning of Generalizable Signed Distance Functions[END_REF] or directly as parameters of a neural net [START_REF] Park | DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation[END_REF][START_REF] Atzmon | SAL: Sign Agnostic Learning of Shapes from Raw Data[END_REF]. The first method allows for easy shape interpolation, while the latter proved to be a more robust approach [START_REF] Davies | On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapes[END_REF]. Those neural implicit surfaces alleviate both the problems related to memory requirements of voxel-based representations and the combinatorial nature of meshes, making them ideally suited for rendering using ray marching [START_REF] Hart | Sphere Tracing: A Geometric Method for the Antialiased Ray Tracing of Implicit Surfaces[END_REF] and constructive solid geometry. In those contexts, the constraint ∥∇ x f (x)∥ ≤ 1 is necessary to guarantee the validity of the geometrical query while having ∥∇ x f (x)∥ as close as possible to 1 allows for greedier queries and faster computation times. In practice, training an SDF requires a dataset (p, d) of points p ∈ R 3 with their corresponding signed distance d to the desired surface. Computing those distances requires the existence and availability of a ground truth, which is not always the case. Moreover, training tends to be unstable in general, and special care is needed for most computer graphics applications [START_REF] Sharp | Spelunking the deep: Guaranteed queries on general neural implicit surfaces via range analysis[END_REF]. Our method can instead be trained to approximate a surface without prior knowledge of the distances and is provably robust.

1-Lipschitz neural nets As noticed in [START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF][START_REF] Brau | Robust-by-design classification via unitary-gradient neural networks[END_REF] 1-Lipschitz neural nets (Stasiak and Yatsymirskyy, 2006;Li et al., 2019b;[START_REF] Su | Scaling-up diverse orthogonal convolutional networks by a paraunitary framework[END_REF] are naturally linked to the signed distance function. In particular, they are 1-Lipschitz, i.e. they fulfil ∥∇ x f (x)∥ ≤ 1 on the whole input space. They boast a rich literature, especially for convolutional neural nets (Gayer and Sheshkus, 2020; [START_REF] Wang | Orthogonal convolutional neural networks[END_REF][START_REF] Liu | Convolutional normalization: Improving deep convolutional network robustness and training[END_REF][START_REF] El Mehdi Achour | Existence, stability and scalability of orthogonal convolutional neural networks[END_REF]Li et al., 2019a;[START_REF] Trockman | Orthogonalizing convolutional layers with the cayley transform[END_REF][START_REF] Singla | Skew orthogonal convolutions[END_REF]. These networks benefit from several appealing properties: they are not subject to exploding nor vanishing gradients (Li et al., 2019a), they generalize well [START_REF] Peter L Bartlett | Nearly-tight vc-dimension and pseudodimension bounds for piecewise linear neural networks[END_REF][START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF], and they are elegantly connected to optimal transport theory [START_REF] Arjovsky | Wasserstein generative adversarial networks[END_REF][START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF]. 1-Lipschitz neural nets also benefit from certificates against l2-attacks (Li et al., 2019a;Tsuzuku et al., 2018); hence the approximation of S is robust against l2-adversarial attacks by design.

Robustness and certification While robustness comes with many aspects, this work focuses mainly on adversarial attacks [START_REF] Szegedy | Intriguing properties of neural networks[END_REF]. Extensive literature explores the construction of efficient attacks [START_REF] Goodfellow | Explaining and harnessing adversarial examples[END_REF]) [START_REF] Brendel | Decisionbased adversarial attacks: Reliable attacks against black-box machine learning models[END_REF] [START_REF] Carlini | Towards evaluating the robustness of neural networks[END_REF]. As nearly any deep learning architecture is vulnerable, defenses have also been developed with notably adversarial training [START_REF] Madry | Towards deep learning models resistant to adversarial attacks[END_REF] [START_REF] Zhang | Theoretically principled trade-off between robustness and accuracy[END_REF] [START_REF] Shafahi | Adversarial training for free! Advances in Neural Information Processing Systems[END_REF], or randomized smoothing [START_REF] Cohen | Certified adversarial robustness via randomized smoothing[END_REF][START_REF] Carlini | certified!!) adversarial robustness for free![END_REF]. Since early works pointed out the link between the Lipschitz constant of a network and its robustness, Lipschitz-constrained networks have also been studied [START_REF] Anil | Sorting out lipschitz function approximation[END_REF] [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF]. Similarly to classifiers, OCC Algorithms based on deep neural nets suffer from their natural weakness to adversarial attacks [START_REF] Azizmalayeri | Your out-of-distribution detection method is not robust![END_REF]. Although some works design robust algorithms [START_REF] Goyal | DROCC: Deep Robust One-Class Classification[END_REF][START_REF] Shao-Yuan | Adversarially robust one-class novelty detection[END_REF], the robustness achieved is always only empirically demonstrated (Hendrycks and Dietterich, 2019). Few works provide theoretical certifications (we only found (Bitterwolf et al., 2020) based on interval bounds propagation). In this work, we leverage the properties of 1-Lipschitz networks to provide certifications.

Tackling the lack of OOD data The previously mentioned OCC and OOD algorithms, as well as many others [START_REF] Hendrycks | A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks[END_REF][START_REF] Hsu | Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data[END_REF] are designed to avoid the need for OOD data. However, some works aim at falling back to classical binary classification by artificially generating negative samples. The idea of Negative Sampling is not recent and appeared in [START_REF] Forrest | Self-nonself discrimination in a computer[END_REF] for detecting computer viruses or to emulate the distinction made by antibodies between pathogens and body cells [START_REF] Gonzalez | Combining negative selection and classification techniques for anomaly detection[END_REF]. It has been introduced in anomaly detection by [START_REF] Ayara | Negative selection: How to generate detectors[END_REF] and studied by several works summarized in [START_REF] Jinyin | A study of detector generation algorithms based on artificial immune in intrusion detection system[END_REF], but has lost popularity due to its practical inefficiency (e.g. compared to One-Class Support Vector Machines (OCSVM) [START_REF] Stibor | Is negative selection appropriate for anomaly detection[END_REF]). Recently, some works revived the idea of using OOD data, either by artificial negative sampling [START_REF] Lee | Training confidence-calibrated classifiers for detecting out-ofdistribution samples[END_REF][START_REF] Sipple | Interpretable, Multidimensional, Multimodal Anomaly Detection with Negative Sampling for Detection of Device Failure[END_REF][START_REF] Goyal | DROCC: Deep Robust One-Class Classification[END_REF]Pourreza et al., 2021), or by using OOD data from other sources, a procedure called outlier exposure [START_REF] Fort | Exploring the limits of out-of-distribution detection[END_REF]Hendrycks et al., 2019). However, outlier exposure suffers from bias since OOD data does not come from the same data space. Therefore, we follow the first idea and sample negative data points close to the domain X , thanks to the orthogonal neural nets-based estimation of the SDF.

Method

The method aims to learn the Signed Distance Function (SDF) by reformulating the one-class classification of P X as a binary classification of P X against a carefully chosen distribution Q(P X ). We show that this formulation yields desirable properties, especially when the chosen classifier is a 1-Lipschitz neural net trained with the Hinge Kantorovich-Rubinstein (HKR) loss.

SDF learning formulated as binary classification

We formulate SDF learning as a binary classification that consists of classifying samples from P X against samples from a complementary distribution, as defined below.

Definition 1 ( B,ϵ ∼ Complementary Distribution (informal)) Let Q be a distribution of compact support included in B, with disjoint support from that of P X that "fills" the remaining space, with 2ϵ gap between X and supp Q. Then we write Q

B,ϵ ∼ P X .
A formal definition is given in Appendix A. For image space with pixel intensity in [0, 1], we take B = [0, 1] W ×H×C . For tabular data, a hypercube of side length ten times the standard deviation of the data along the axis. A data point falling outside B is trivially considered anomalous due to aberrant values. Binary classification between P X and any Q B,ϵ ∼ P X allows the construction of the optimal signed distance function, using the Kantorovich-Rubinstein (HKR) Hinge loss [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF], thanks to the following theorem.

Theorem 1 SDF Learning with HKR loss. Let L hkr m,λ (yf (x)) = λ max (0, m -yf (x)) -yf (x) be the Hinge Kantorovich Rubinstein loss, with margin m = ϵ, regularization λ > 0, prediction f (x) and label y ∈ {-1, 1}. Let Q be a probability distribution on B. Let E hkr (f ) be the population risk:

E hkr (f, P X , Q) :=E x∼P X [L hkr m,λ (f (x))] + E z∼Q [L hkr m,λ (-f (z))].
(2)

Let f * be the minimizer of population risk, whose existence is guaranteed with Arzelà-Ascoli theorem [START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF]:

f * ∈ arg inf f ∈Lip 1 (R d ,R) E hkr (f, P X , Q), (3)
where Lip

1 (R d , R) is the set of Lipschitz functions R d → R of constant 1. Assume that Q B,ϵ ∼ P X .
Then, f * approximates the signed distance function over B:

∀x ∈ X , S(x) = f * (x) -m, ∀z ∈ supp Q, S(z) = f * (z) -m. (4)
Moreover, for all x ∈ supp Q ∪ X : sign(f (x)) = sign(S(x)).

Note that if m = ϵ ≪ 1, then we have f * (x) ≈ S(x). In this work, we parametrize f as a 1-Lipschitz neural network, as defined below, because they fulfil f ∈ Lip 1 (R d , R) by construction.

Definition 2 (1-Lipschitz neural network (informal))

Neural network with Groupsort activation function and orthogonal transformation in affine layers parameterized like in [START_REF] Anil | Sorting out lipschitz function approximation[END_REF].

Details about the implementation can be found in Appendix D. Theorem 1 tells us that if we characterize the complementary distribution Q, we can approximate the SDF with a 1-Lipschitz neural classifier trained with HKR loss. We now need to find the complementary distribution Q.

Finding the complementary distribution by targeting the boundary

We propose to seek Q through an alternating optimization process: at every iteration t, a proposal distribution Q t is used to train a 1-Lipschitz neural net classifier f t against P X by minimizing empirical HKR loss. Then, the proposal distribution is updated in Q t+1 based on the loss induced by f t , and the procedure is repeated.

We suggest starting from the uniform distribution: Q 0 = U(B). Observe that in high dimension, due to the curse of dimensionality, a sample z ∼ Q 0 is unlikely to satisfy z ∈ X . Indeed the data lies on a low dimensional manifold X for which the Lebesgue measure is negligible compared to B. Hence, in the limit of small sample size n ≪ ∞, a sample Z n ∼ Q ⊗n 0 fulfills Z n B,ϵ ∼ P X . This phenomenon is called the Concentration Phenomenon and has already been leveraged in anomaly detection in [START_REF] Sipple | Interpretable, Multidimensional, Multimodal Anomaly Detection with Negative Sampling for Detection of Device Failure[END_REF]. However, the curse works both ways and yields a high variance in samples Z n . Consequently, the variance of the associated minimizers f 0 of equation 3 will also exhibit a high variance, which may impede the generalization and convergence speed. Instead, the distribution Q t must be chosen to produce higher density in the neighborhood of the boundary ∂X . The true boundary is unknown, but the level set L t = f -1 t ({-ϵ}) of the classifier can be used as a proxy to improve the initial proposal Q 0 . We start from z 0 ∼ Q 0 , and then look for a displacement δ ∈ R d such that z + δ ∈ L t . To this end, we take inspiration from the multidimensional Newton-Raphson method and consider a linearization of f t :

f t (z 0 + δ) ≈ f t (z 0 ) + ⟨∇ x f t (z 0 ), δ⟩. (5)
Since 1-Lipschitz neural nets with GroupSort activation function are piecewise affines [START_REF] Tanielian | Approximating lipschitz continuous functions with groupsort neural networks[END_REF], the linearization is locally exact, hence the following property.

Property 1. Let f t be a 1-Lipschitz neural net with Group-Sort activation function. Almost everywhere on z 0 ∈ R d , there exists δ 0 > 0 such that for every ∥δ∥ ≤ δ 0 , we have:

f t (z 0 + δ) = f t (z 0 ) + ⟨∇ x f t (z 0 ), δ⟩. (6) Since f t (z 0 + δ) ∈ L t translates into f t (z 0 + δ) = -ϵ, δ = - f t (z 0 ) + ϵ ∥∇ x f t (z 0 )∥ 2 ∇ x f t (z 0 ). (7)
Properties of L hkr m,λ ensure that the optimal displacement follows the direction of the gradient ∇ x f t (z 0 ), which coincides with the direction of an optimal transportation plan [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF]. The term ∥∇ x f t (z 0 )∥ enjoys an interpretation as a Local Lipschitz Constant (see [START_REF] Jordan | Exactly computing the local lipschitz constant of relu networks[END_REF]) of f t around z 0 , which we know fulfills ∥∇ x f t (z 0 )∥ ≤ 1 when parametrized with an 1-Lipschitz neural net. When f t is trained to perfection, the expression for δ simplifies to δ = -f t (z 0 )∇ x f t (z 0 ) thanks to Property 2.

Property 2 (Minimizers of L hkr m,λ are Gradient Norm Preserving (from [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF])). Let f * t be the solution of Equation 3. Then for almost every z ∈ B we have

∥∇ x f * t (z)∥ = 1.
In practice, the exact minimizer f * t is not always retrieved, but equation 7 still applies to imperfectly fitted classifiers. The final sample z ′ ∼ Q t is obtained by generating a sequence of T small steps to smooth the generation. The procedure is summarized in algorithm 1. In practice, T can be chosen very low (below 16) without significantly hurting the quality of generated samples. Finally, we pick a random "learning rate" η ∼ U([0, 1]) for each negative example in the batch to ensure they distribute evenly on the path toward the boundary. The procedure also benefits from Property 3, which ensures that the distribution Q t+1 obtained from Q t across several iterative applications of Algorithm 1 still

fulfils Q t+1 B,ϵ ∼ P X . The proof is given in Appendix B.
Property 3 (Complementary distributions are fix points).

Let Q t be such that Q t B,ϵ ∼ P X . Assume that Q t+1 is ob- tained with algorithm 2. Then we have Q t+1 B,ϵ ∼ P X .
Algorithm 1 Adapted Newton-Raphson for Complementary Distribution Generation

Input: 1-Lipschitz neural net f t Parameter: number of steps T Output: sample z ′ ∼ Q t (f ) 1: sample learning rate η ∼ U([0, 1]) 2: z 0 ∼ U(B) { Initial approximation.} 3: for each step t = 1 to T do 4: z t+1 ← z t -η T ∇xf (z t ) ∥∇xf (z t )∥ 2 2 (f (z t ) + ϵ){Refining.} 5: z t+1 ← Π B (z t+1 ){ Stay in feasible set.} 6: end for 7: return z T Remark. In high dimension d ≫ 1, when ∥∇ x f t (z)∥ = 1
and Vol(B) ≫ Vol(X ) the samples obtained with algorithm 1 are approximately uniformly distributed in the levels of f t . It implies that the density of Q increases exponentially fast (with factor d) with respect to the value of -|f t (•)|. This mitigates the adverse effects of the curse of dimensionality. This scheme of "generating samples by following gradient in input space" reminds diffusion models [START_REF] Ho | Denoising diffusion probabilistic models[END_REF], feature visualization tools [START_REF] Olah | Feature visualization[END_REF], or recent advances in VAE [START_REF] Kuzina | Alleviating adversarial attacks on variational autoencoders with mcmc[END_REF]. However, no elaborated scheme is required for the training of f t : 1-Lipschitz networks exhibit smooth and interpretable gradients [START_REF] Serrurier | When adversarial attacks become interpretable counterfactual explanations[END_REF] which allows sampling from X "for free" as illustrated in figure 4.

Remark.

A more precise characterization of Q t built with algorithm 1 can be sketched below. Our approach bares some similarities with the spirit of Metropolis-adjusted Langevin algorithm [START_REF] Grenander | Representations of knowledge in complex systems[END_REF]. In this method, the samples of p(x) are generated by taking the stationary distribution x t )∞ of a continuous Markov chain obtained from the stochastic gradient step iterates

x t+1 ← x t + ζ∇ x log p(x) + 2ζZ (8)
for some distribution p(x) and Z ∼ N (0, 1). By choosing the level set ϵ = 0, and p

(x) ∝ 1{f (x) ≤ 0} exp (-ηf 2 (x)) the score function ζ∇ x log p(x) is trans- formed into ∇ x f (x)|f (x)| with ζ = η T .
Therefore, we see that the density decreases exponentially faster with the squared distance to the boundary ∂X when there are enough steps T ≫ 1. In particular when X = {0} we recover p(x) as the pdf of a standard Gaussian N (0, 1). Although the similarity is not exact (e.g., the diffusion term

√ 2ηZ is lack- ing, T is low, η ∼ U([0, 1]) is a r. v.
), it provides interesting complementary insights on the algorithm.

Alternating minimization for SDF learning

Each classifier f t does not need to be trained from scratch. Instead, the same architecture is kept throughout training, and the algorithm produces a sequence of parameters θ t such that f t = f θt . Each set of parameters θ t is used as initialization for the next one θ t+1 . Moreover, we only perform a low fixed number of parameter updates for each t in a GAN fashion. The final procedure of OCSDF is shown in Figure 1 and detailed in algorithm 3 of Appendix B.

Properties

Certificates against adversarial attacks

The most prominent advantage of 1-Lipschitz neural nets is their ability to produce certificates against adversarial attacks [START_REF] Szegedy | Intriguing properties of neural networks[END_REF]. Indeed, by definition we have f (x + δ) ∈ [f (x) -∥δ∥, f (x) + ∥δ∥] for every example x ∈ X and every adversarial attack δ ∈ R d . This allows bounding the changes in AUROC score of the classifier for every possible radius ϵ > 0 of adversarial attacks.

Proposition 1 (certifiable AUROC). Let F 0 be the cumulative distribution function associated with the negative classifier's prediction (when f (x) ≤ 0), and p 1 the probability density function of the positive classifier's prediction (when f (x) > 0). Then, for any attack of radius ϵ > 0, the AUROC of the attacked classifier f ϵ can be bounded by

AUROC ϵ (f ) = ∞ -∞ F 0 (t)p 1 (t -2ϵ)dt. ( 9
)
The proof is left in Appendix C. The certified AUROC score can be computed analytically without performing the attacks empirically, solely from score predictions p 1 (t -2ϵ).

More importantly, the certificates hold against any adversarial attack whose l 2 -norm is bounded by ϵ, regardless of the algorithm used to perform such attacks. We emphasize that producing certificates is more challenging than traditional defence mechanisms (e.g, adversarial training, see [START_REF] Bai | Recent advances in adversarial training for adversarial robustness[END_REF] and references therein) since they do not target defence against a specific attack method. Note that MILP solvers and branch-and-bound approaches [START_REF] Tjeng | Evaluating robustness of neural networks with mixed integer programming[END_REF][START_REF] Wang | Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification[END_REF] can be used to improve the tightness of certificates, but at a higher cost.

Rank normal and anomalous examples

Beyond the raw certifiable AUROC score, l 2 -based Lipschitz robustness enjoys another desirable property: the farther from the boundary the adversary is, the more important the attack budget required to change the score. In practice, it means that an attacker can only dissimulate anomalies already close to being "normal". Aberrant and hugely anomalous examples will require a larger (possibly impracticable) attack budget. This ensures that the normality score is actually meaningful: it is high when the point is central in its local cloud, and low when it is far away.

Experiments

In this section, we evaluate the performances and properties of OCSDF on tabular and image data. All the experiments are conducted with tensorflow, and the 1-Lipschitz neural nets are implemented using the library Deel-Lip 1 . Our code is available at https://github.com/ Algue-Rythme/OneClassMetricLearning.

Toy examples from Scikit-Learn

We use two-dimensional toy examples from the Scikit-Learn library [START_REF] Pedregosa | Scikitlearn: Machine learning in python[END_REF]. Results are shown in figure 2. The contour of the decision function are plotted in resolution 300 × 300 pixels. The level sets of the classifier are compared against those of One Class SVM [START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF] and Isolation Forest [START_REF] Fei | Isolation forest[END_REF]. We also train a conventional network with Binary Cross Entropy against complementary distribution Q t , and we show it struggles to learn a meaningful decision boundary. Moreover, its Local Lipschitz Constant (Jordan and Dimakis, 2020) increases uncontrollably, as shown in table 1, which makes it prone to adversarial attacks. Finally, there is no natural interpretation of the prediction of the conventional network in terms of distance: the magnitude |f (•)| of the predictions quickly grows above 10 3 , whereas for 1-Lipschitz neural nets, it is approximately equal to the signed distance function S. We refer to Appendix E for visualizations.

Anomaly Detection on Tabular datasets

We tested our algorithm on some of the most prominent anomaly detection benchmarks of ODDS library [START_REF] Rayana | ODDS library[END_REF]. In this unsupervised setting (like ADBench [START_REF] Han | ADBench: Anomaly Detection Benchmark[END_REF] lies) are seen during training, but their true label is unknown.

To apply our method, the only hyperparameter needed is the margin m that we select in the range [0.01, 0.05, 0.2, 1.].

For each value, the results are averaged over 20 independent runs train/test splits. Following ADBench guidelines and best practices from the AD community, we only compute the AUROC, since this metric is symmetric under label flip.

We report the best average in table 2 along baselines from ADBench [START_REF] Han | ADBench: Anomaly Detection Benchmark[END_REF]. As observed by [START_REF] Han | ADBench: Anomaly Detection Benchmark[END_REF], none of the algorithms clearly dominates the others, because what is considered an anomaly (or not) depends on the context. Among 14 other methods tested, our algorithm ranks 7.1 ± 3.6/15, while the best (Isolation Forests) ranks 4.5 ± 3.2/15. The experiment shows that our algorithm is competitive with respect to other broadly used baselines. Nonetheless, it brings several additional advantages. First, our algorithm can be seen as a parametric version of kNN for the euclidean distance, which leverages deep learning to avoid the costly construction of structures like a KDTree [START_REF] Maneewongvatana | Analysis of approximate nearest neighbor searching with clustered point sets[END_REF] and the quadratic cost of nearest neighbor search, thereby enabling its application in high dimensions. Second, it provides robustness certificates. We illustrate those two advantages more thoroughly in the next section.

One Class learning on images

We evaluate the performances of OCSDF for OCC, where only samples of the normal class are supposed to be available. To emulate this setting, we train a classifier on each of the classes of MNIST and Cifar10, and evaluate it on an 39.0 ± 3.9 20.0 ± 7.0 75.9 ± 0.3 73.1 ± 1.2 58.9 ± 0.7

Table 3. AUROC score on the test set of MNIST and CIFAR10 in a one versus all fashion, averaged on 10 runs. We also report the AUROC of DeepSVDD [START_REF] Ruff | Deep one-class classification[END_REF] for completeness, along with the other AUROC scores of Isolation Forest (IF) and One-Class SVM (OC-SVM) reported in [START_REF] Ruff | Deep one-class classification[END_REF]. When the differences between some methods are not statistically significant, we highlight both. When the confidence intervals overlap, we highlight both. We also show the certifiable AUROC against l-2 attacks of norms ϵ ∈ {8/255, 16/255, 36/255}. Concurrent methods cannot provide certificates for ϵ > 0.

independent test set in a one-versus-all fashion. We compare our method against DeepSVDD [START_REF] Ruff | Deep one-class classification[END_REF], OCSVM [START_REF] Schölkopf | Support Vector Method for Novelty Detection[END_REF], and Isolation Forests [START_REF] Fei | Isolation forest[END_REF]. Details on the implementation of the baselines can be found in Appendix H. The mean AUROC score is reported in table 3 and averaged over 20 runs. It is computed between the 1, 000 test examples of the target class and the remaining 9, 000 examples from other classes of the test set (both unseen during training). The histograms of normality scores are given in Appendix H. OCSDF is competitive with respect to other baselines. In addition, it comes with several advantages described in the following.

CERTIFIABLE AND EMPIRICAL ROBUSTNESS

None of the concurrent methods can provide certificates against l 2 attacks: in the work of [START_REF] Goyal | DROCC: Deep Robust One-Class Classification[END_REF] the attacks are performed empirically (no certificates) with l ∞ radii. In table 3, we report our certifiable AUROC with various radii ϵ ∈ {0, 8/25, 16/255, 36/255, 72/255}. In figure 3 we report the empirical AUROC against l 2 -PGD attacks with three random restarts, using stepsize ζ = 0.025ϵ like Foolbox [START_REF] Rauber | Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in pytorch, tensorflow, and jax[END_REF]. These results illustrate our method's benefits: not only does it come with robustness certificates that are verified empirically, but the empirical robustness is also way better than DeepSVDD, especially for Cifar10. Note that for 1-Lipschitz network trained with L hkr m,λ loss, all the attacks tend to find the same adversaries [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF]) -hence PGD is also representative of the typical score that would have been obtained with other attack methods.

VISUALIZATION OF THE SUPPORT

OCSDF can be seen as a parametric version of kNN, which enables this approach in high dimensions. As a result, the decision boundary learned by the classifier can be materialized by generating adversarial examples with algorithm 1. The forward computation graph is a classifier based on optimal transport, and the backward computation graph is an image generator. Indeed, the back-propagation through a convolution is a transposed convolution, a popular layer in the generator of GANs. Overall, the algorithm behaves like a WGAN [START_REF] Arjovsky | Wasserstein generative adversarial networks[END_REF] with a single network fulfilling both roles. This unexpected feature opens a path to the explainability of the One Class classifier: the support learned can be visualized without complex feature visualization tools. In particular, it helps identify failure modes.

LIMITATIONS

We also tested our algorithm on the Cats versus Dogs dataset by training on Cats as the One Class and using Dogs as OOD examples. On this high-dimensional dataset, the AUROC barely exceeds 55.0%. This suggests that the SDF is relevant for tabular and simple image datasets (e.g MNIST) but fails to generalize in a meaningful way in higher dimensions. The distance used to define the learned SDF is the euclidian distance. It is well known that such distance has its limitations in pixel space. Using different distances that are more relevant to pixel space is a perspective left for future works. Our constructions rely on our ability to solve Eikonal equation ∥∇ x f ∥ = 1. This is easy to enforce on dense layers, but it is still an active research area for convolutions (see the recent works of [START_REF] El Mehdi Achour | Existence, stability and scalability of orthogonal convolutional neural networks[END_REF][START_REF] Singla | Improved techniques for deterministic l2 robustness[END_REF][START_REF] Xu | Lot: Layer-wise orthogonal training on improving l2 certified robustness[END_REF]).

In addition, scores reported in Table 3 use the same hyperparameters for all the classes of Cifar-10, while some concurrent approaches [START_REF] Goyal | DROCC: Deep Robust One-Class Classification[END_REF] tune them on a per-class basis. We observed significant improvements with per class tuning of the margin (see figure 9 in appendix), but we chose not to report them in Table 3, because this practice is criticized by recent contributions on which we are aligned [START_REF] Han | ADBench: Anomaly Detection Benchmark[END_REF][START_REF] Ding | Hyperparameter sensitivity in deep outlier detection: Analysis and a scalable hyper-ensemble solution[END_REF].

OCSDF for implicit shape parametrization

Our approach to learning the SDF contrasts with the computer graphics literature, where SDF is used to obtain the distance of a point to a surface (here defined as ∂X ). Indeed, SDFs are usually learned in a supervised fashion, requiring the ground truth of l 2 distance. This is classically achieved using Nearest-Neighbor algorithms, which can be cumbersome, especially when the number of points is high. Efficient data structures (e.g., using K-dtrees [START_REF] Maneewongvatana | Analysis of approximate nearest neighbor searching with clustered point sets[END_REF] or Octrees [START_REF] Donald | Octree encoding: A new technique for the representation, manipulation and display of arbitrary 3-d objects by computer[END_REF]) can mitigate this effect but do not scale well to high dimensions. Instead, OCSDF learns the SDF solely based on points contained in the support. While neural network training is not cheap by any mean, the network approach is advantageous at inference time. Indeed, with a dataset of size n, a single forward in the network costs O(1) (furthermore orthogonalization of matrices can be done once for all), while kNN costs O(n2 ) or O(n log n) (with trees).

Figure 5. Visualization of the SDF (3rd column) from sparse point clouds of size 2048 (2nd column) sampled from ground truth meshes (1st column) with Trimesh library, against the SSSR algorithm [START_REF] Boltcheva | Surface reconstruction by computing restricted voronoi cells in parallel[END_REF]) that attempts to reproduce the meshes solely from a point cloud (4th column). The SDF exhibits better extrapolation properties and provides smooth surfaces.

To illustrate this, we use models from Princeton's Model-Net10 dataset [START_REF] Wu | 3d shapenets: A deep representation for volumetric shapes[END_REF]. We sample n = 2048 within each mesh to obtain a 3d point cloud. We fit the SDF on the point cloud with the same hyperparameters as the tabular experiment. We use Lewiner marching algorithm [START_REF] Lewiner | Efficient implementation of marching cubes' cases with topological guarantees[END_REF] We chose the first percentile of E x∼P X [f (x)] as the level set of the iso-surface we plot. We compare our results visually against a baseline from (Boltcheva and Lévy, 2017) implemented in Graphite [START_REF] Levy | Graphite v3-1.8.2[END_REF] that rebuilds the mesh solely from the point cloud (without extrapolation). We highlight that n = 2048 is considered low resolution; hence many details are expected to be lost. Nonetheless, our method recovers the global aspect of the shape more faithfully, smoothly, and consistently than the other baseline.

Conclusion and future work

We proposed a new approach to One-Class Classification, OCSDF, based on the parametrization of the Signed Distance Function to the boundary of the known distribution using a 1-Lipschitz neural net. OCSDF comes with robustness guarantees and naturally tackles the lack of negative data inherent to OCC. Finally, this new method extends OCC beyond Out-of-Distribution detection and allows applying it to surface parametrization from point clouds and generative visualization.

OCSDF relies on the appropriate sampling of a complementary distribution. It can be improved by integrating proper priors (e.g steps in Fourrier space, Perlin noise prior or negative data augmentation), rather than random uniform, as long as Definition 1 is fulfilled. This perspective is very relevant and will be investigated in future works.

Contents A. Proofs and comments

A.1. Complementary distribution Definition 3 ( B,ϵ ∼ Complementary Distribution) Let P X a distribution with compact support X ⊂ B, with B ⊂ R d a bounded measurable set. Q is said to be (B, ϵ) disjoint from P X if (i) its support supp Q ⊂ B is compact (ii) d(supp Q, X ) ≥ 2ϵ (iii) for all measurable sets M ⊂ B such that d(M, X ) ≥ 2ϵ we have Q(M ) > 0. It defines a symmetric but irreflexive binary relation denoted Q B,ϵ ∼ P X .
The idea is to learn one class classifier by reformulating one class learning of P X as a binary classification of P X against a carefully chosen adversarial distribution Q(P X ). This simple idea had already occurred repeatedly in the related literature [START_REF] Sabokrou | Adversarially learned one-class classifier for novelty detection[END_REF]. Note that L hkr m,λ benefits from generalization guarantees as proved in [START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF]: the optimal classifier on the train set and on the test set are the same in the limit of big samples.

A.2. Learning the SDF with HKR loss

Theorem 1 SDF Learning with HKR loss. Let L hkr m,λ (yf (x)) = λ max (0, m -yf (x)) -yf (x) be the Hinge Kantorovich Rubinstein loss, with margin m = ϵ, regularization λ > 0, prediction f (x) and label y ∈ {-1, 1}. Let Q be a probability distribution on B. Let E hkr (f ) be the population risk:

E hkr (f, P X , Q) :=E x∼P X [L hkr m,λ (f (x))] + E z∼Q [L hkr m,λ (-f (z))].
(2)

Let f * be the minimizer of population risk, whose existence is guaranteed with Arzelà-Ascoli theorem [START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF]:

f * ∈ arg inf f ∈Lip 1 (R d ,R) E hkr (f, P X , Q), (3)
where Lip

1 (R d , R) is the set of Lipschitz functions R d → R of constant 1. Assume that Q B,ϵ ∼ P X . Then, f * approximates the signed distance function over B: ∀x ∈ X , S(x) = f * (x) -m, ∀z ∈ supp Q, S(z) = f * (z) -m. (4)
Moreover, for all x ∈ supp Q ∪ X : sign(f (x)) = sign(S(x)).

Proof. The results follow from the properties of L hkr m,λ loss given in Proposition 2 of [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF]. If Q B,ϵ ∼ P X , then by definition, the two datasets are 2ϵ separated. Consequently the hinge part of the loss is null: max (0, m -yf (x)) for all pairs (x, +1) and (z, -1) with x ∼ P X and z ∼ Q. We deduce that:

∀x ∈ X , f (x) ≥ m, ∀z ∈ supp Q, f (z) ≤ -m. ( 10
)
In the following we use:

F z = {arg min z0∈supp Q ∥x -z 0 ∥ 2 }, ∀x ∈ X and F x = {arg min x0∈X ∥x 0 -z∥ 2 }, ∀z ∈ (supp Q).
Since m = ϵ, we must have f (x) = m for all x ∈ F x , and f (z) = -m for all z ∈ F z , whereas S(x) = 0 and S(z) = -2m.

Thanks to the 1-Lipschitz property for every x ∈ X we have f (x) ≤ f (∂x) + ∥x -∂x∥ where ∂x = arg min x∈∂X ∥x -x∥ is the projection of x onto the boundary ∂X . Similarly f (z) ≥ f (∂z) -∥z -∂z∥. The -yf (x) term in the L hkr m,λ loss (Wasserstein regularization), incentives to maximize the amplitude |f (x)| so the inequalities are tight. Notice that S(x) = S(∂x) + ∥x -∂x∥ and S(z) = S(∂z) -∥z -∂z∥. This allows concluding:

∀x ∈ X , S(x)) = f * (x) -m, ∀z ∈ supp Q, S(z) = f * (z) -m. (11)

A.3. Finding the right complementary distribution

Finding the right distribution Q B,ϵ ∼ P X with small ϵ is also challenging.

We propose to seek Q through an alternating optimization process. Consider a sample z ∈ L t . If z is a false positive (i.e f t (z) > 0 and z / ∈ X ), training f t+1 on the pair (z, -1) will incentive f t+1 to fulfill f t+1 (z) < 0, which will reduce the volume of false positive associated to f t+1 . If z is a true negative (i.e f t (z) < 0 and z / ∈ X ) it already exhibits the wanted properties. The case of false negative (i.e f t (z) < 0 and z ∈ X ) is more tricky: the density of P X around z will play an important role to ensure that f t+1 (z) > 0.

Hence we assume that samples from the target P X are significantly more frequent than the ones obtained from pure randomness. It is a very reasonable assumption (especially for images, for example), and most distributions from real use cases fall under this setting.

Assumption 1 (P X samples are more frequent than pure randomness (informal)) For any measurable set M ⊂ X we have P X (M ) ≫ U(M ).

To ensure that property (iii) of definition 3 is fulfilled, we also introduce stochasticity in algorithm 1 by picking a random "learning rate" η ∼ U([0, 1]). To decorrelate samples, the learning rate is sampled independently for each example in the batch.

The final procedure depicted in algorithm 2 benefits from the mild guarantee of proposition 4. It guarantees that once the complementary distribution has been found, the algorithm will continue to produce a sequence of complementary distributions and a sequence of classifiers f t that approximates S.

B. Signed Distance Function learning framed as Adversarial Training

Property 4. Let Q t be such that Q t B,ϵ ∼ P X . Assume that Q t+1 is obtained with algorithm 2. Then we have Q t+1 B,ϵ ∼ P X .

Proof. The proof also follows from the properties of L hkr m,λ loss given in Proposition 2 of [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF]

Since Q t B,ϵ ∼ P X all examples z ∼ Q t generated fulfill (by definition) d(z, X ) ≥ 2ϵ ≥ 2m.
Indeed the 1-Lipschitz constraint (in property 2) guarantees that no example z t can "overshoot" the boundary. Hence for the associated minimizer f t+1 of L hkr m,λ loss, the hinge part of the loss is null. This guarantees that f t+1 (z) ≤ -m for z ∼ Q. We see that by applying algorithm 1 the property is preserved: for all z ∼ Q t+1 we must have f t+1 (z) ≤ -m = -ϵ. Finally notice that because z 0 ∼ U(B) and η ∼ U([0, 1]) the support supp Q covers the whole space B (except the points that are less than 2ϵ apart from X ). Hence we have Q t+1 B,ϵ ∼ P X as expected.

B.1. Lazy variant of algorithm 2

Algorithm 2 solves a MaxMin problem with alternating maximization-minimization. The inner minimization step (minimization of the loss over 1-Lipschitz function space) can be expensive. Instead, partial minimization can be performed by doing only a predefined number of gradient steps. This yields the lazy approach of algorithm 3. This provides a considerable speed up over the initial implementation. Moreover, this approach is frequently found in literature, for example, with GAN () or WGAN (). However, we lose some of the mild guarantees, such as the one of Proposition 4 or even Theorem 1. Crucially, it can introduce unwanted oscillations in the training phase that can impede performance and speed. Hence this trick should be used sparingly. The procedure of algorithm 3 bears numerous similarities with the adversarial training of Madry [START_REF] Madry | Towards deep learning models resistant to adversarial attacks[END_REF]

f t ← f θt 3: θ ← θ t 4: repeat 5:
Generate batch z ∼ Q t of negative samples with algorithm 1 6:

Sample batch x ∼ P X of positive samples 7:

Compute loss on batch L( θ) := E hkr (f θ , x, z)

8: Learning step θ ← θ + α∇ θ L( θ) 9:
until convergence of θ to θ t+1 . 10: until convergence of f t to limit f * .

Algorithm 3 One Class Signed Distance Function learning

Input: 1-Lipschitz neural net architecture f • , initial weights θ 0 , learning rate α, number of parameter update per time step

K 1: repeat 2: θ ← θ t 3:
Generate batch z ∼ Q t of negative samples with algorithm 1 4:

Sample batch x ∼ P X of positive samples 5:

for K updates do 6:

Compute loss on batch L(θ) := E hkr (f θ , x, z)

7: Learning step θ ← θ + α∇ θ L( θ) 8:
end for 9:

θ t+1 ← θ C. Certifiable AUROC (Proposition 1)
Proposition 2 (certifiable AUROC). Let F 0 be the cumulative distribution function associated with the negative classifier's prediction (when f (x) = 0), and p 1 the probability density function of the positive classifier's prediction (when f (x) = 0). Then, for any attack of radius ϵ > 0, the AUROC of the attacked classifier f ϵ can be bounded by

AUROC(f ϵ ) = ∞ -∞ F 0 (t)p 1 (t -2ϵ)dt. (12)
Proof. Let p 1 (resp. p -1 ) be the probability density function (PDF) associated with the classifier's positive (resp. negative) predictions. More precisely, p 1 (resp. p -1 ) is the PDF of f ♯ P X (resp. f ♯ Q) for some adversarial distribution Q, where f ♯ • denotes the pushforward measure operator [START_REF] Igorevich | Measure theory[END_REF] defined by the classifier. The operator f ♯ formalizes the shift between P X (resp. Q), the ground truth distributions, and p -1 (resp. p 1 ), the imperfectly distribution fitted by f . Let F -1 and F 1 be the associated cumulative distribution functions. For a given classification decision threshold τ , we can define the True Positive Rate (TPR), the True Negative Rate (TNR), and the False Positive Rate (FPR) in probabilistic terms:

• TPR: F -1 (τ )

• TNR: 1 -F 1 (τ )

• FPR: F 1 (τ )
The ROC curve is then the plot of F -1 (τ ) against F 1 (τ ). Hence, setting v = F 1 (t), we can define the AUROC as:

AUROC(f ) = 1 0 F -1 (F -1 1 (v))dv (13)
And with the change of variable dv = p 1 (t)dt

AUROC(f ) = ∞ -∞ F -1 (t)p 1 (t)dt. (14)
We consider a scenario with symmetric attacks: the attack decreases (resp. increases) the normality score of One Class (resp. Out Of Distribution samples) for decision threshold τ ∈ R. When the 1-Lipschitz classifier f is under attacks of radius at most ϵ > 0 we note f ϵ the perturbed classifier:

f ϵ (x) = min δ≤ϵ (21{f (x) ≥ τ } -1)f (x + δ). (15)
Note that f ϵ (x) ≤ f (x) + ϵ when f (x) < τ and f ϵ (x) ≥ f (x) -ϵ when f (x) ≥ τ thanks to the 1-Lipschitz property. This effectively translates the pdf of f ϵ by |ϵ| atmost.

We obtain a lower bound for the AUROC (i.e a certificate):

AUROC(f ϵ ) ≥ ∞ -∞ F -1 (t + ϵ)p 1 (t -ϵ)dt = ∞ -∞ F -1 (t)p 1 (t -2ϵ)dt. (16)
The certified AUROC score can be computed analytically without performing the attacks empirically, solely from score predictions f 1 (τ -2ϵ). More importantly, the certificates hold against any adversarial attack whose l2-norm is bounded by ϵ, regardless of the algorithm used to perform such attacks. We emphasize that producing certificates is more challenging than traditional defence mechanisms (e.g, adversarial training, see [START_REF] Bai | Recent advances in adversarial training for adversarial robustness[END_REF] and references therein) since they do not target defence against a specific attack method.

the number of steps T = 4 is small. We chose a margin m = 0.05 except for "blob and cloud" dataset where we used m = 0.1 instead. We take λ = 100. The networks are trained for a total of 10, 000 gradient steps.

E.2. Other benchmarks

For One Class SVM, we chose a parameter ν = 0.05, γ = 1 2 (which corresponds to the "scale" behavior of scikit-learn for features in 2D with unit variance) and the popular RBF kernel. For Isolation Forest, we chose a default contamination level of 0.05.

The conventional network (without orthogonality constraint) is trained with Binary Cross Entropy (also called log-loss) and Adam optimizer. It shares the same architecture with ReLU instead of GroupSort. It is also trained for a total of 10, 000 gradient steps for a fair comparison. It would not make sense to use L hkr m,λ loss for a conventional network since it diverges during training, as noticed in [START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF]. The Lipschitz constant of the conventional network grows uncontrollably during training even with Binary Cross-Entropy loss, which is also compliant with the results of [START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF].

F. Toy experiments on 3D point clouds

In figures 8 and 9, we provide additional examples of the use of SDF to reconstruct shapes from point clouds. We also compare OCSDF with Deep SDF (Park et al., 2019a), a standard baseline for neural implicit surface parametrization, to highlight the practical advantages of our method.

G. Tabular data

The optimizer is RMSprop with default hyper-parameters. We use the lazy variant, i.e algorithm 3. We chose λ = 100 and m ∈ {0.01, 0.05, 0.2, 1.}. The results are averaged over 20 runs, and we report the highest average among all values of m.

We use a batch size b = 128 a number of steps T = 4. The network is trained for a total of 40 epochs over the one class, using a warm start of 5 epoch with T = 0. All the experiments use a VGG-like architecture, depicted in table 6. Convolutions are parametrized using layers of Deel-Lip library [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF], which use an orthogonal kernel with a corrective constant on the upper bound of the Lipschitz constant of the kernel. Dense layers also use orthogonal kernels. We use l2-norm-pooling layer with windows of size 2 × 2 that operates as: (x 11 , x 12 , x 21 , x 22 ) → ∥[x 11 , x 12 , x 21 , x 22 ]∥.

We see in table 3 that it yields competitive results against other naive baselines such as Isolation Fortest (IF) or One Class SVM (OCSVM), and against the popular deep learning algorithm Deep SVDD [START_REF] Ruff | Deep one-class classification[END_REF].

Against DeepSVDD. We test 4 configurations for DeepSVDD.

1. The one from the original's paper [START_REF] Ruff | Deep one-class classification[END_REF] with LeNet architecture. 2. The Pytorch implementation from their official repository that combines LeNet and BatchNorm: https://github. com/lukasruff/Deep-SVDD-PyTorch.

3. The Tensorflow implementation from pyod [START_REF] Zhao | Pyod: A python toolbox for scalable outlier detection[END_REF] with dense layers and l2 activation regularization.

4. The Tensorflow implementation from pyod but with the same (unconstrained) architecture as Table 6 to ensure a fair comparison.

Each configuration is run 10 times, and the results are averaged. The best average among the four configurations is reported in the main paper, while we report in Table 7 the results of all the configurations. It appears that the method of the original paper is very sensitive to the network's architecture, and the results are hard to reproduce overall. Empirically, we also observe that the test/OOD AUROC is often smaller than train/OOD AUROC, which suggests that DeepSVDD memorizes the train set but do not generalize well outside of its train data. We also notice that those results are different from the ones originally reported by authors in their paper.

We report the histogram of predictions for the train set (One class), the set test (One class) and Out Of Distribution (OOD) examples (the classes of the test set) in figure 10.

H.2. Cifar-10

The Cifar-10 images are centedered-reduced per channel (the certificates and the attacks take into account the rescaling in the computation of the radius norm of an image. We take λ = 1000. We use a batch size b = 128, and a number of steps T = 32. We use the lazy variant, i.e algorithm 3. The network is trained for a total of 90 epochs over the one class (size of the support: ≈ 5000 examples), using a warm start of 10 epoch with T = 0. The learning rate follows a linear decay from 2.5e -4 to 2.5e -7 .

Note that the same hyper-parameters are used for all the runs and all the classes. This is different from some practices of litterature (see fig 11. in [START_REF] Goyal | DROCC: Deep Robust One-Class Classification[END_REF] ). Unfortunately this protocol is somewhat unfair since it explicitly leverages prior knowledge on the other classes -it can not be used in a real world scenario where the other classes are not available. Nonetheless, for completeness, we report in table 9 the clean AUROC on Cifar10 with per class tuning of the margin m.

H.3. Empirical robustness

We also report the complete results of the empirical robustness of OCSDF versus DeepSVDD for MNIST and Cifar10 in tables 10, 11, 12 and 13.

H.4. Image synthesis from One Class classifier

We perform a total of T = 64 steps with algorithm 1 by targeting the level set f -1 ({E P X [f (x)]}) (to ensure we stay inside the distribution). Moreover, during image synthesis, we follow a deterministic path by setting η = 1. The images generated were picked randomly (with respect to initial sample z 0 ∼ U(B)) without cherry-picking. Finally, note that our method is not tailored for example generation: this is merely a side effect of the training process of the classifier. There is no need a the encoder-decoder pair of VAE nor the discriminator-generator pair of a GAN. Moreover, no hyper-parameters other than m and T are required.

I. Known Limitations

Despite its performance and appealing properties, the method suffers from some important limitations we highlight below and that can serve as a basis for future work.

I.1. Gradient Norm Preserving networks (GNP) approximation power

The performance of the algorithm strongly depends on its capacity to properly learn the true minimizer f * of L hkr m,λ loss. Per Property 2 such minimizer must fulfill ∥∇ x f * (x)∥ 2 = 1 everywhere on the support of P X and Q t . Hence the performance of the algorithm (and the associated theoretical guarantees) depends on the capacity of the GNP network to fulfil this property. In the tabular case, it is easy to do using orthogonal matrices for affine layers and GroupSort (or FullSort [START_REF] Anil | Sorting out lipschitz function approximation[END_REF]) activation functions. However, in the image case, designing "orthogonal convolution" is still an active research area. Several solutions have been proposed, but they come with various drawbacks in terms of simplicity of implementation, computational cost, or tightness of the constraint. Hence the average gradient norm on image datasets struggles to exceed 0.3 in practice. Another limitation stems from low-rank adjoint operators (e.g the last layer of the network): during backpropagation they do not preserve gradient norm along all directions.

The Newton-Raphson trick that uses steps of size ∇xf (x) ∥∇xf (x)∥ 2 2 mitigates partially the issue. This suggests that the algorithm (in its current form) could benefit from further progress in Gradient Norm Preserving (GNP) architectures.

I.2. Limitations of the euclidean norm in image space

The algorithm provides metric guarantees in the construction of the Signed Distance Function (SDF) to the boundary. The l2-norm is not a crucial component of the construction: the proof of 1 and Proposition 2 of [START_REF] Serrurier | Achieving robustness in classification using optimal transport with hinge regularization[END_REF] can be applied to any norm. However, in every case, the Lipschitz constraint |f (x) -f (y)| ≤ ∥x -y∥ L on the network architecture must coincide with the norm ∥ • ∥ L used to build the signed distance function. Currently, only networks that are Lipschitz with respect to l ∞ and l 2 norms benefit from universal approximation properties [START_REF] Anil | Sorting out lipschitz function approximation[END_REF]. Those norms are often meaningful for tabular data, but not for images. Hence, metric guarantees are less useful in pixel space. The method still benefits from certificates against adversarial attacks, which is highly desirable for critical systems but lacks semantic interpretation otherwise.

Figure 1 .

 1 Figure1. Summary of One Class Signed Distance Function (OCSDF). We start with an uniform negative sampling, then we fit a 1-Lipschitz classifier f θ using the Hinge Kantorovich-Rubinstein loss. We apply the Adapted Newton Raphson algorithm 1 to attract the points towards the boundary of the domain ∂X thanks to the smoothness of f θ , which in addition allows providing robustness certificates.

  ) all the examples (normal examples and anoma-1 https://github.com/deel-ai/deel-lip OCSDF OCSVM (a) Two circles. (b) Two blobs. (c) Blob & cloud.

Figure 2 .

 2 Figure 2. Contour plots of our method with 1-Lipschitz (LIP) network and L hkr m,λ (HKR) loss on toy examples of Scikit-learn.

Figure 3 .

 3 Figure3. Empirical Mean AUROC on all classes against adversarial attacks of various radii in One Class setting, using default parameters of FoolBox[START_REF] Rauber | Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in pytorch, tensorflow, and jax[END_REF].

Figure 4 .

 4 Figure 4. Examples from algorithm 1 with T = 64 and η = 1.

  from scikit-image (Van der Walt et al., 2014), on a 200 × 200 × 200 voxelization of the input. We plot the mesh reconstructed with Trimesh (Dawson-Haggerty et al., 2019). The results are highlighted in figure 5.

  . In our case the adversarial examples are obtained by starting from noise U(B) and relabeled are negative examples. In their case, the adversarial examples are obtained by starting from P X itself and relabeled as positive examples. Algorithm 2 Alternating Minimization for Signed Distance Function learning Input: 1-Lipschitz neural network architecture f • Input: initial weights θ 0 , learning rate α 1: repeat 2:

  are normalized such that pixel intensity lies in [-1, 1] range. The set B is chosen to be the image space, i.e B = [-1, 1] 28×28 = [-1, 1] 784 . The optimizer is RMSprop with default hyperparameters. We chose m = 0.02 × (28 × 28 × (1 -(-1)) ≈ 0.79 : this corresponds to modification of 1% of the maximum norm of an image. We take λ = 200. We use a batch size b = 128, and a number of steps T = 16. We use the lazy variant, i.e algorithm 3. The network is trained for a total of 70 epochs over the one class (size of the support: ≈ 6000 examples), using a warm start of 10 epoch with T = 0. The learning rate follows a linear decay from 1e -3 to 1e -6 .

Figure 7 .

 7 Figure 7. Histograms of score functions for 1-Lipschitz network (left) and conventional neural network. The blue bars correspond to the distribution of f (x), x ∼ PX and the red bars to the distribution f (z), z ∼ U (B).

Figure 8 .

 8 Figure 8. SDF of 2048 points sampled from the mesh of a 3D CAD model.

Figure 9 .

 9 Figure 9. SDF of 2048 points sampled from the mesh of a 3D CAD model.

Figure 10 .

 10 Figure 10. Histogram of scores predicted by the classifier at the end of training for train examples (in blue), test examples (in red), and OOD Test examples (in green) on Mnist after one of the runs of the algorithm.

Figure 11 .

 11 Figure 11. Synthetic examples obtained by running algorithm 1 with T = 64 and η = 1.

Table 1 .

 1 Lower bound on the Local Lipschitz Constant (LLC) of conventional network after 10, 000 training steps for each toy example. It is the maximum of ∥∇x i f (xi)∥ over the train set.

	Local Lipschitz Constant	One Cloud 26.66 122.84 1421.41 53.90 258.73 Two Two Blob Two Clouds Blobs Cloud Moons

Table 2 .

 2 AUROC score for tabular data, averaged over 20 runs. The dimension of the dataset is denoted by d. In the Anomaly Detection protocol (AD) we use all the data (normal class and anomalies) for training, in an unsupervised fashion. The "#no.+an." column indicates part of normal (no.) and anomalous (an.) data used during training for each protocol. SOTA denominates the best score ever reported on the dataset, obtained by crawling relevant literature, or ADBench[START_REF] Han | ADBench: Anomaly Detection Benchmark[END_REF] results (table D4 page 37). We report the rank as (#rank) among 14 other methods.

	Dataset		d	#no.+an.	perc.		OCSDF (Ours)	Deep SVDD	OC SVM	IF	PCA	kNN	SOTA
			Average Rank			7.1 ± 3.6	11.2	7.5	4.5	5.7	7.8	4.5 ± 3.2 (IF)
	breastw		9	444+239	35%	(#10) 82.6 ± 5.9	65.7	80.3	98.3	95.1	97.0	99.7 (COPOD)
	cardio		21	1,655+176	9.6%	(#2) 95.0 ± 0.1	59.0	93.9	93.2	95.5	76.6	95.5 (PCA)
	glass		9	205+9	4.2%	(#7) 73.9 ± 4.1	47.5	35.4	77.1	66.3	82.3	82.9 (CBLOF)
	http (KDDCup99)	3	565,287+2,211	0.4%	(#11) 67.5 ± 37	69.0	99.6	99.96	99.7	03.4	99.96 (IF)
	Ionosphere		33	225+126	36%	(#7) 80.2 ± 0.1	50.9	75.9	84.5	79.2	88.3	90.7 (CBLOF)
	Lymphography	18	142+6	4.1%	(#8) 96.1 ± 4.9	32.3	99.5	99.8	99.8	55.9	99.8 (CBLOF)
	mammography	6	10,923+260	2.32%	(#6) 86.0 ± 2.5	57.0	84.9	86.4	88.7	84.5	90.7 (ECOD)
	musk		166	2,965+97	3.2%	(#8) 92.6 ± 20.	43.4	80.6	99.99 100.0 69.9	100.0 (PCA)
	Optdigits		64	5,066+150	3%	(#12) 51.0 ± 0.9	38.9	54.0	70.9	51.7	41.7	87.5 (CBLOF)
	Pima		8	500+268	35%	(#12) 60.7 ± 1.0	51.0	66.9	72.9	70.8	73.4	73.4 (kNN)
	satimage-2		36	5,732+71	1.2%	(#3) 97.9 ± 0.4	53.1	97.3	99.2	97.6	92.6	99.8 (CBLOF)
	Shuttle		9	45,586+3,511	7%	(#4) 99.1 ± 0.3	52.1	97.4	99.6	98.6	69.6	99.6 (IF)
	smtp (KDDCup99)	3	95,126+30	0.03%	(#4) 87.1 ± 3.5	78.2	80.7	89.7	88.4	89.6	89.7 (IF)
	speech		400	3,625+61	1.65% (#15) 46.0 ± 0.2	53.4	50.2	50.7	50.8	51.0	56.0 (COF)
	thyroid		6	3,679+93	2.5%	(#5) 95.9 ± 0.0	49.6	87.9	98.3	96.3	95.9	98.3 (IF)
	vertebral		6	210+30	12.5%	(#4) 48.6 ± 2.6	36.7	38.0	36.7	37.0	33.8 53.2 (DAGMM)
	vowels		12	1,406+50	3.4%	(#2) 94.7 ± 0.7	52.5	61.6	73.9	65.3	97.3	97.3 (kNN)
	WBC		30	357+21	5.6%	(#10) 93.6 ± 0.1	55.5	99.0	99.0	98.2	90.6	99.5 (CBLOF)
	Wine		13	119+10	7.7%	(#5) 81.5 ± 0.9	59.5	73.1	80.4	84.4	45.0	91.4 (HBOS)
	MNIST	OCSDF	OCSDF	OCSDF	OCSDF	OCSDF	OC SVM		Deep SVDD	IF
	Certificates		ϵ = 0	ϵ = 8/255 ϵ = 16/255 ϵ = 36/255 ϵ = 72/255	ϵ = 0		ϵ = 0	ϵ = 0
	mAUROC	95.5 ± 0.4 93.2 ± 2.1	89.9 ± 3.5	78.4 ± 6.4	57.5 ± 7.5	91.3 ± 0.0 94.8 ± 0.9 92.3 ± 0.5
	digit 0	99.7 ± 0.1 99.6 ± 0.2	99.5 ± 0.2	99.0 ± 0.6	96.2 ± 3.0	98.6 ± 0.0	98.0 ± 0.7	98.0 ± 0.3
	digit 1	99.8 ± 0.0 99.7 ± 0.0	99.6 ± 0.1	99.2 ± 0.3	96.2 ± 1.6	99.5 ± 0.0 99.7 ± 0.1 97.3 ± 0.4
	digit 2	90.6 ± 2.0 85.3 ± 1.9	78.2 ± 2.3	53.1 ± 5.2	14.1 ± 4.6	82.5 ± 0.1 91.7 ± 0.1 88.6 ± 0.5
	digit 3	93.4 ± 1.2 90.0 ± 1.7	85.0 ± 2.3	66.2 ± 4.6	26.9 ± 5.0	88.1 ± 0.0	91.9 ± 1.5	89.9 ± 0.4
	digit 4	96.5 ± 0.9 95.3 ± 1.2	93.9 ± 1.7	89.4 ± 3.6	76.2 ± 9.8	94.9 ± 0.0 94.9 ± 0.8 92.7 ± 0.6
	digit 5	93.9 ± 2.2 89.0 ± 3.2	81.6 ± 4.7	54.0 ± 8.7	15.6 ± 6.9	77.1 ± 0.0	88.5 ± 0.9	85.5 ± 0.8
	digit 6	98.7 ± 0.6 98.1 ± 0.7	97.2 ± 0.9	93.1 ± 2.6	74.9 ± 10.4 96.5 ± 0.0 98.3 ± 0.5 95.6 ± 0.3
	digit 7	97.1 ± 0.6 96.5 ± 0.5	95.6 ± 0.6	92.2 ± 0.8	81.2 ± 1.7	93.7 ± 0.0	94.6 ± 0.9	92.0 ± 0.4
	digit 8	89.4 ± 2.6	83.3 ± 5.1	74.7 ± 9.0	50.3 ± 15.9 24.4 ± 14.0 88.9 ± 0.0 93.9 ± 1.6 89.9 ± 0.4
	digit 9	96.4 ± 0.3 95.3 ± 0.9	93.8 ± 1.3	87.8 ± 3.1	68.9 ± 7.6	93.1 ± 0.0 96.5 ± 0.3 93.5 ± 0.3
	CIFAR10	OCSDF	OCSDF	OCSDF	OCSDF	OCSDF	OC SVM		Deep SVDD	IF
	Certificates		ϵ = 0	ϵ = 8/255 ϵ = 16/255 ϵ = 36/255 ϵ = 72/255	ϵ = 0		ϵ = 0	ϵ = 0
	mAUROC	57.4 ± 2.1	53.1 ± 2.1	48.8 ± 2.1	38.4 ± 1.9	22.5 ± 1.4	64.8 ± 8.0	64.8 ± 6.8	55.4 ± 8.0
	Airplane	68.2 ± 4.5 64.3 ± 3.9	60.1 ± 3.2	49.4 ± 1.1	31.2 ± 3.6	61.6 ± 0.9	61.7 ± 4.1	60.1 ± 0.7
	Automobile	57.3 ± 1.7	52.5 ± 3.0	47.6 ± 4.2	36.1 ± 6.8	19.8 ± 7.8	63.8 ± 0.6 65.9 ± 2.1	50.8 ± 0.6
	Bird	51.8 ± 2.7 47.5 ± 1.8	43.2 ± 1.6	33.4 ± 3.4	19.5 ± 5.7	50.0 ± 0.5 50.8 ± 0.8 49.2 ± 0.4
	Cat	58.8 ± 1.2 54.6 ± 0.8	50.3 ± 0.8	40.0 ± 1.5	24.4 ± 2.3	55.9 ± 1.3	59.1 ± 1.4	55.1 ± 0.4
	Deer	49.4 ± 2.4	45.3 ± 2.1	41.4 ± 1.9	32.2 ± 1.5	18.8 ± 1.4	66.0 ± 0.7	60.9 ± 1.1	49.8 ± 0.4
	Dog	56.3 ± 0.6	51.9 ± 1.0	47.5 ± 1.6	36.7 ± 2.9	20.6 ± 4.0	62.4 ± 0.8	65.7 ± 2.5	58.4 ± 0.5
	Frog	52.6 ± 1.8	48.7 ± 1.7	44.9 ± 1.6	35.8 ± 1.4	22.4 ± 1.1	74.7 ± 0.3	67.7 ± 2.6	42.9 ± 0.6
	Horse	49.5 ± 0.9	45.5 ± 1.0	41.5 ± 1.2	32.5 ± 1.5	18.8 ± 1.6	62.6 ± 0.6	67.3 ± 0.9	55.1 ± 0.7
	Ship	68.6 ± 1.8	64.6 ± 1.4	60.4 ± 1.3	49.3 ± 2.4	29.8 ± 4.9	74.9 ± 0.4 75.9 ± 1.2 74.2 ± 0.6
	Truck	61.3 ± 3.4	56.5 ± 2.1	51.5 ± 1.1

Table 4 .

 4 Comparison of our approach against DeepSDF.

	Methods	OCSDF (ours)	DeepSDF (Park et al., 2019a)
	Target	Generate Q	B,ϵ ∼ P	Compute S(x) from P
	Cost	Backward pass	Nearest Neighbor search
	Loss	HKR L hkr m,λ	∥f (x) -S(x)∥ 2 2
	Guarantees	f is 1-Lipschitz	None

  One Class (OC) protocol : in this setting the normal examples are split in train and test set (approximately 50% of data each). The network is trained on normal train examples. The AUROC score is computed between normal test examples and anomalies. We report the results in table 5. Anomaly Detection (AD) protocol : in this setting the normal examples and anomalies are part of the train set. The network is trained on all examples. The AUROC score is computed between normal examples and anomalies. We report the results in table 2.

Table 5 .

 5 AUROC score on test set + anomalies with tabular data, averaged over 20 runs. The dimension of the dataset is denoted by d. In the One Class protocol (OC), the classifier is trained on a subset of normal data, and test on unseen normal data and anomalous data. The "#train" column indicates train set size.

		Dataset	d	#train	OCSDF (Ours)
		Arrhythmia	274	193	80.0 ± 1.3
		Thyroid	6	1,839	98.3 ± 0.1
		Mammography	6	5,591	88.0 ± 1.4
		Vowels	12	703	96.1 ± 0.9
		Satimage-2	36	2,866	97.8 ± 0.0
		smtp	3	47,553	83.8 ± 3.5
	Datasets	Toy2D & tabular& Implicit Shape	Mnist & Cifar-10
	# parameters		792K		2,103K
	layer 1	dense-512 (fullsort)	conv-3x3-64 (groupsort)
	layer 2	dense-512 (fullsort)	conv-3x3-64 (groupsort)
	layer 3	dense-512 (fullsort)	conv-3x3-64 (groupsort)
	layer 4	dense-512 (fullsort)	conv-3x3-64 (groupsort)
	layer 5	dense-1 (linear)	l2 norm pooling -2x2
	layer 6				conv-3x3-128 (groupsort)
	layer 7				conv-3x3-128 (groupsort)
	layer 8				conv-3x3-128 (groupsort)
	layer 9				conv-3x3-128 (groupsort)
	layer 10				l2 norm pooling -2x2
	layer 11				conv-3x3-256 (groupsort)
	layer 12				conv-3x3-256 (groupsort)
	layer 13				conv-3x3-256 (groupsort)
	layer 14				l2 norm pooling -2x2
	layer 15				global l2 norm pooling
	layer 16				dense-1 (linear)

Table 6 .

 6 Architectures of the 1-Lipschitz networks used for the experiments, using layers of Deel-Lip library.

Table 7 .

 7 Average AUROC over 10 runs for each digit of Mnist. We re-code the method in Tensorflow using various configurations, due to the discrepancies of protocols found in the literature. In overall, DeepSVDD is very sensitive to the network architecture and can dramatically overfit.

	Dataset	One cloud Two circles Two blobs Blob and cloud Two moons
	LLC conventional	26.66	122.84	1421.41	53.90	258.73

Table 8 .

 8 Lower bound on the Local Lipschitz Constant (LLC) of the conventional network after 10, 000 training steps. It is obtained by computing the maximum of ∥∇x i f (xi)∥ over the train set. The LLC of the conventional network grows uncontrollably during training and fails to provide metric guarantees.

Table 9 .

 9 Interestingly, we see that the algorithm recovers some in-distribution examples successfully. The examples for which the image is visually deceptive are somewhat correlated with low AUC scores. Those failure cases are also shared by concurrent methods, which suggests that some classes are harder to distinguish. Notice that sometimes Out Of Distribution (OOD) Mnist digits, from other classes not seen during training, are sometimes generated. This suggests that most Mnist digits can be built from a small set of elementary features that are combined during the generation of Q t . Visualizations of generated digits are presented in Figure11. Best AUROC of OCSDF on Cifar-10 with per class tuning of the margin m.

	Airplane Automobile Bird	Cat	Deer Dog Frog Horse Ship Truck	Mean
	79.0	65.2	55.4 60.4 52.5 56.9 57.7	55.0	71.9	67.0	62.1 ± 8.0

Table 10 .

 10 Empirical AUROC against l2-PGD adversarial attacks on Mnist for DeepSVDD. PGD ϵ = 8/255 ϵ = 16/255 ϵ = 36/255 ϵ = 72/255 ϵ = 144/255 ϵ = 255/255

	Mnist	OCSDF	OCSDF	OCSDF	OCSDF	OCSDF	OCSDF
	l2-mean	93.2 ± 5.0	92.8 ± 5.3	91.8 ± 6.0	89.8 ± 7.4	83.1 ± 11.9	68.2 ± 19.5
	digit 0	99.4	99.4	99.3	99.1	98.2	94.9
	digit 1	99.1	99.0	98.8	98.4	96.7	90.9
	digit 2	89.7	89.1	87.5	84.4	74.0	51.0
	digit 3	91.1	90.5	89.2	86.4	77.3	56.6
	digit 4	95.2	94.9	94.1	92.6	87.1	73.0
	digit 5	84.6	83.7	81.2	76.3	60.8	33.3
	digit 6	98.0	97.9	97.6	96.9	94.4	86.9
	digit 7	95.3	95.0	94.4	93.0	88.5	77.2
	digit 8	85.8	85.1	83.3	79.7	68.7	46.9
	digit 9	93.9	93.6	92.7	91.0	85.2	70.9

Table 11 .

 11 Empirical AUROC against l2-PGD adversarial attacks on Mnist for one run of OCSDF in One Class learning.

	CIFAR10	DeepSVDD DeepSVDD DeepSVDD DeepSVDD	DeepSVDD	DeepSVDD
	l2-PGD	ϵ = 8/255	ϵ = 16/255 ϵ = 36/255 ϵ = 72/255 ϵ = 144/255 ϵ = 255/255
	mean	54.4 ± 8.3	46.5 ± 8.1	28.4 ± 7.0	09.2 ± 3.7	0.7 ± 0.6	0.1 ± 0.0
	Airplane	62.82	55.09	36.21	12.57	0.76	0.01
	Automobile	43.02	36.94	21.98	8.35	1.24	0.09
	Bird	58.50	49.13	27.66	6.47	0.16	0.01
	Cat	45.80	36.82	18.76	3.91	0.12	0.00
	Deer	63.02	53.55	30.82	7.41	0.23	0.00
	Dog	50.33	43.32	27.95	10.88	1.43	0.09
	Frog	63.44	55.31	35.33	11.12	0.63	0.01
	Horse	44.67	36.55	20.09	5.37	0.26	0.00
	Ship	64.38	57.67	40.89	17.47	1.87	0.04
	Truck	48.39	40.85	24.79	8.46	0.81	0.03

Table 12 .

 12 Empirical AUROC against l2-PGD adversarial attacks on Cifar10 for DeepSVDD.

	CIFAR10	OCSDF	OCSDF	OCSDF	OCSDF	OCSDF	OCSDF
	l2-PGD	ϵ = 8/255 ϵ = 16/255 ϵ = 36/255 ϵ = 72/255 ϵ = 144/255 ϵ = 255/255
	mean	59.0 ± 9.3	58.2 ± 9.2	56.7 ± 9.6	53.9 ± 10.3	48.4 ± 10.3	39.8 ± 13.8
	Airplane

https://www.deel.ai/

10: until convergence of f t .

Acknowledgements

This work has benefited from the AI Interdisciplinary Institute ANITI, which is funded by the French "Investing for the Future -PIA3" program under the Grant agreement ANR-19-P3IA-0004. The authors gratefully acknowledge the support of the DEEL project. 2

D. 1-Lipschitz Neural Networks

We ensure that the kernel remains 1-Lipschitz by re-parametrizing them: Θ i = Π(W i ) where W i is a set of unconstrained weights and Θ i the orthogonal projection of W i on the Stiefel manifold -i.e the set of orthogonal matrices. The projection Π is made with Björck algorithm [START_REF] Björck | An iterative algorithm for computing the best estimate of an orthogonal matrix[END_REF], which is differentiable and be included in the computation graph during forward pass. Unconstrained optimization is performed on W i directly.

E. Toy experiment in 2D

All datasets are normalized to have zero mean and unit variance across all dimensions. The domain B is chosen to be the ball of radius 5. This guarantees that X ⊂ B for all datasets. The plots of figure2 are squares of sizes [-5, 5] for (a), [-3, 3] for (b)(c)(e) and[-4, 4] for (d

I.3. Tuning of margin

The algorithm is not quite agnostic to the data: the margin m > 0 used in L hkr m,λ loss is an important parameter that serves as prior on the typical distance that separates the One Class support from the anomalies. This hyper-parameter can be guessed from the final application at hand (very much like the "scale" parameter of radial basis function kernels), manually with grid search algorithms or more extensive procedures. Theorem 1 suggests that a small margin m works best. However, the VC dimension associated with the corresponding set of classifiers increases polynomially with 1 m (see Proposition 6 of [START_REF] Béthune | Pay attention to your loss: understanding misconceptions about 1lipschitz neural networks[END_REF]). Hence, the algorithm benefits from faster convergence and more stability during training when m is big. Fortunately, this tradeoff present in most deep learning-based algorithms is solely controlled by this one-dimensional parameter in our case. Any heuristic estimation from data or with a one-dimensional line search is feasible, even with a limited computational budget.

J. Hardware

The hardware consists of a workstation with NVIDIA 1080 GPU with 8GB memory and a machine with 32GB RAM. Hence, while being based on deep learning, and obviously benefiting from faster hardware, the requirements are affordable to most practitioners. The typical duration of an epoch on most challenging datasets was under a minute.