Martin Bauw

Santiago Velasco-Forero

Jesus Angulo

Claude Adnet

Olivier Airiau

NEAR OUT-OF-DISTRIBUTION DETECTION FOR LOW-RESOLUTION RADAR MICRO-DOPPLER SIGNATURES HAL PREPRINT

Keywords: Anomaly detection, Out-of-distribution detection, Micro-Doppler, Radar target discrimination, Deep learning, Self-supervised learning

Near out-of-distribution detection (OODD) aims at discriminating semantically similar data points without the supervision required for classification. This paper puts forward an OODD use case for radar targets detection extensible to other kinds of sensors and detection scenarios. We emphasize the relevance of OODD and its specific supervision requirements for the detection of a multimodal, diverse targets class among other similar radar targets and clutter in real-life critical systems. We propose a comparison of deep and non-deep OODD methods on simulated low-resolution pulse radar micro-Doppler signatures, considering both a spectral and a covariance matrix input representation. The covariance representation aims at estimating whether dedicated second-order processing is appropriate to discriminate signatures. The potential contributions of labeled anomalies in training, self-supervised learning, contrastive learning insights and innovative training losses are discussed, and the impact of training set contamination caused by mislabelling is investigated.

Introduction

Near out-of-distribution detection (OODD) aims at distinguishing one or several data classes from semantically similar data points. For instance, identifying samples from one class of CIFAR10 among samples of the other classes of the same dataset solves a near OODD task. On the other hand, separating CIFAR10 samples from MNIST samples is a far OODD task: there is no strong semantic proximity between the data points being separated. OODD defines a kind of anomaly detection (AD) since OODD can be seen as separating a normal class from infinitely diverse anomalies, with a training set only or mostly composed of normal samples, and anomalies being possibly semantically close to normal samples [START_REF] Ren | A simple fix to Mahalanobis distance for improving near-ood detection[END_REF]. This training paradigm relies on lower supervision requirements compared to supervised classification, for which each class calls for a representative set of samples in the training data. This work considers both unsupervised and semi-supervised AD (SAD). Unsupervised AD trains the model with a representative set of normal data samples, while semi-supervised AD also benefits from labeled anomalies [START_REF] Hendrycks | Deep anomaly detection with outlier exposure[END_REF][START_REF] Ruff | Deep semi-supervised anomaly detection[END_REF] that can not be representative, since anomalies are by definition infinitely diverse. A distinction can however be observed between benefiting from far and near anomalies, in analogy with far and near OODD, to refine the discrimination training. The contribution of self-supervision will be taken into account through the supply of far artificial anomalies for additional supervision during training.

Near OODD constitutes an ideal mean to achieve radar targets discrimination, where an operator wants an alarm to be raised everytime specific targets of interest are detected. This implies discriminating between different kinds of planes, or ships, sometimes being quite similar from a radar perspective and without numerous labeled samples available. For example, two ships can have close hull and superstructure sizes, implying close radar cross-sections, even though their purpose and equipment on deck are completely different. Analogous observations could be made for helicopters, planes and drones. In an aerial radar context, whereas separating aerial vehicles would constitute a near OODD task, spotting weather-related clutter would define a far OODD. Such an OODD-based detection setup is directly applicable to other sensors.

The motivation behind the application of OODD methods to low-resolution pulse Doppler radar (PDR) signatures stems from the constraints of some air surveillance radars. Air surveillance PDRs with rotating antennas are required to produce very regular updates of the operational situation and to detect targets located at substantial ranges. The regular updates dictate the rotation rate and limit the number of pulses, and thus the number of Doppler spectrum bins, over which to integrate and refine a target characterization. The minimum effective range restricts the pulse repetition frequency (PRF), which in turns diminishes the range of velocities covered by the Doppler bins combined. The operating frequencies of air surveillance radars are such that they can not make up for this Doppler resolution loss [START_REF] Levanon | Radar signals[END_REF]. This work aims at exploring the potential of machine learning to discriminate targets within these air surveillance radars limitations, using the targets Doppler spectrums. Refining radar targets discrimination with limited supervision is critical to enable the effective detection of targets usually hidden in cluttered domains, such as small and slow targets.

The AD methods examined will take a series of target Doppler spectrums as an input sample. This series is converted into a second-order representation through the computation of a covariance matrix to include an AD method adapted to process symmetric positive definite (SPD) inputs in our comparison. Radar Doppler signatures with sufficient resolution to reveal micro-Doppler spectrum modulations is a common way to achieve targets classification in the radar literature, notably when it comes to detecting drones hidden in clutter [START_REF] Daniel | Temporal deep learning for drone micro-doppler classification[END_REF][START_REF] Björklund | Target detection and classification of small drones by deep learning on radar micro-doppler[END_REF][START_REF] Gérard | Micro-doppler signal representation for drone classification by deep learning[END_REF]. The processing of second-order representations is inspired by their recent use in the machine learning literature [START_REF] Huang | A riemannian network for spd matrix learning[END_REF][START_REF] Yu | Second-order convolutional neural networks[END_REF], including in radar processing [START_REF] Brooks | A Hermitian positive definite neural network for micro-doppler complex covariance processing[END_REF][START_REF] Arnaudon | Riemannian medians and means with applications to radar signal processing[END_REF], and is part of the much larger and very active research on machine learning on Riemannian manifolds [START_REF] Brooks | Riemannian batch normalization for spd neural networks[END_REF][START_REF] Michael M Bronstein | Geometric deep learning: going beyond euclidean data[END_REF]. This paper first details the simulation setup which generates the micro-Doppler dataset, then describes the OODD methods compared. Finally, an experimental section compares quantitatively various supervision scenarios involving SAD and self-supervision. The code for both the data generation and the OODD experiments is available 2 . The code made available does not restrict itself to the experiments put forward in the current document, pieces of less successful experiments being kept for openness and in case they help the community experiment on the data with similar approaches.

Micro-Doppler dataset

A PDR is a radar system that transmits bursts of modulated pulses, and after each pulse transmission waits for the pulse returns. The pulse returns are sampled and separated into range bins depending on the amount of time observed between transmission and reception. The spectral content of the sampled pulses is evaluated individually in each range bin, as depicted on Fig. 1. This content translates into the Doppler information which amounts to a velocity descriptor: the mean Doppler shift reveals the target bulk speed, and the spectrum modulation its rotating blades. These Doppler features are available for each burst, under the assumption that the velocities detected in a given range bin change negligibly during a burst. The number of pulses in a burst, equating the number of samples available to compute a spectrum, determines the resolution of the Fourier bins or Doppler bins. The PRF sampling frequency defines the range of speeds covered by the spectrum. PDR signatures are generated by a MATLAB [START_REF]MATLAB. version 9.11.0[END_REF] simulation. The Doppler signatures are a series of periodograms, i.e. the evolution of spectral density over several bursts, one periodogram being computed per burst. Both periodograms and the series of periodograms can be called spectrum in the remainder of this paper. The samples on which the discrete Fourier transform is computed are sampled at the PRF frequency, i.e. one sample is available per pulse return for each range bin.

The main parameters of the simulation are close to realistic radar and target characteristics. A carrier frequency of 5 GHz was selected, with a PRF of 50 KHz. An input sample is a Doppler signature extracted from 64 bursts of 64 pulses, i.e. 64 spectrums of 64 points, ensuring the full rank of the covariance matrix computed over non-normalized Doppler, i.e. Fourier, bins. The only simulation parameter changing across the classes of helicopter-like targets is the number of rotating blades: Doppler signatures are associated with either one, two, four or six rotating blades, as can be found on drones and radio-controlled helicopters. The quality of the dataset is visually verified: a non-expert human is easily able to distinguish the four target classes, confirming the discrimination task is feasible. The classes intrinsic diversity is ensured by receiver noise, blade size and revolutions-per-minute (RPM) respectively uniformly sampled in [4.5, 7] and [450, 650], and a bulk speed uniformly sampled so that the signature central frequency changes while staying approximately centered. The possible bulk speeds and rotor speeds are chosen in order for the main Doppler shift and Each pulse leads to one complex-valued I/Q sample per range bin, while each burst is composed of several pulses. Each range bin is thus associated with a complex-valued discrete signal with as many samples as there are pulses. Air surveillance radars with rotating antennas are required to provide regular situation updates in every direction, severely constraining the number of pulses per burst acceptable. the associated modulations to remain in the unambiguous speeds covered by the Doppler signatures [START_REF] Levanon | Radar signals[END_REF]. Example signatures and their covariance representations are depicted for each class on Fig. 2. For each class, 3000 samples are simulated, thus creating a 12000-samples dataset. While small for the deep learning community, possessing thousands of relevant and labeled real radar detections would not be trivial in the radar industry, making larger simulated datasets less realistic for this use case.

OODD methods

This work compares deep and non-deep OODD methods, called shallow, including second-order methods harnessing the SPD representations provided by the covariance matrix of the signatures. The extension of the deep learning architectures discussed to SAD and self-supervised learning (SSL) is part of the comparison. The use of SSL here consists in the exploitation of a rotated version of every training signature belonging to the normal class in addition to its non-rotated version, whereas SAD amounts to the use of a small minority of actual anomalies taken in one of the other classes of the dataset. In the first case one creates artificial anomalous samples from the already available samples of a single normal class, whereas in the second case labeled anomalies stemming from real target classes are made available. To avoid confusion, one should note that this single normal class can be made of one or several target classes, which end up being considered as a unique normality. No SSL or SAD experiments were conducted on the SPD representations, since the SSL and SAD extensions of the deep methods are achieved through training loss modifications, and the SPD representations were confined to shallow baselines. In the previously described setup, SSL is nothing more than SAD with artificial data points provided by SSL transformations.

Non-deep methods

Common non-deep anomaly detection methods constitute our baselines: one-class support vector machines (OC-SVM) [START_REF] Schölkopf | Estimating the support of a high-dimensional distribution[END_REF], isolation forests (IF) [START_REF] Fei | Isolation forest[END_REF], local outlier factor (LOF) [START_REF] Breunig | Lof: identifying density-based local outliers[END_REF] and random projections outlyingness (RPO) [START_REF] David L Donoho | Breakdown properties of location estimates based on halfspace depth and projected outlyingness[END_REF]. The three first methods are selected for their widespread use [START_REF] Chandola | Anomaly detection: A survey[END_REF][START_REF] Ruff | Deep one-class classification[END_REF][START_REF] Bauw | From unsupervised to semi-supervised anomaly detection methods for HRRP targets[END_REF], and the diversity of the underlying algorithms. OC-SVM projects data points in a feature space where a hyperplane separates data points from the origin, thus creating a halfspace containing most samples. Samples whose representation lies outside of this halfspace are then considered to be anomalies. IF evaluates how easy it is to isolate data points in the feature space by recursively partitioning the representation space. The more partitions are required to isolate a data point, the more difficult it is to separate this point from other samples, and the less anomalous this point is. LOF uses the comparison of local densities in the feature space to determine whether a point is anomalous or not. Points that have local densities similar to the densities of their nearest neighbors are likely to be inliers, whereas an outlier will have a much different local density than its neighbors. RPO combines numerous normalized outlyingness measures over 1D projections with a max estimator in order to produce a unique and robust multivariate outlyingness measure, which translates into the following quantity:

O(x; p, X) = max u∈U |u T x -M ED(u T X)| M AD(u T X) (1)
where x is the data point we want to compute the outlyingness for, p the number of random projections (RP) u of unit norm gathered in the set U, and X the training data matrix. M ED stands for median, a location estimator, and M AD for median absolute deviation, a spread estimator. The max implies retaining only the worst outlyingness measure available among all the 1D projections, i.e. the worst normalized deviation from the projected median. In [START_REF] Velasco | Robust rx anomaly detector without covariance matrix estimation[END_REF], the asymptotic equivalence between Eq. 1 for a large number of RPs and the Mahalanobis distance (up to a constant factor) is established, with the motivation of obtaining an equivalent of the latter without computing a covariance matrix. This equivalence with the Mahalanobis distance indicates that RPO with enough RPs, after the max integration over RPs, describes a normality ellipsoid in the input space, i.e. the representation space of x (see Appendix A). The RPO outlyingness actually leads to the definition of a statistical depth approximation [START_REF] David L Donoho | Breakdown properties of location estimates based on halfspace depth and projected outlyingness[END_REF][START_REF] Peter | Projection pursuit[END_REF], another quantity that orders data points from a given set from most to least normal.

Deep methods

The deep AD methods experimented on in this work are inspired by the deep support vector data description (SVDD) original paper [START_REF] Ruff | Deep one-class classification[END_REF]. Deep SVDD achieves one-class classification by concentrating latent space representations around a normality centroid with a neural network trained to minimize the distance of projected data samples to the centroid. The centroid is defined by the average of the initial forward pass of the training data, composed of normal samples. The intuition behind the use of Deep SVDD for AD is similar to the way one can detect anomalies with generative models: whereas generative models detect outliers because they are not as well reconstructed as normal samples, deep SVDD projects outliers further away from the normality centroid in the latent space. The AD score is thus the latent distance to the normality centroid after projection by the trained deep SVDD network. One can note that Deep SVDD is a deep learning adaptation of SVDD [START_REF] David | Support vector data description[END_REF], which can be equivalent to the OC-SVM method in our comparison if one uses a Gaussian kernel. The training loss of Deep SVDD for a sample of size n with a neural network Φ with weights W distributed over L layers is as follows:

min W 1 n n i=1 ||Φ(x i ; W ) -c|| 2 + λ 2 L l=1 ||W l || 2 (2)
where c is the fixed normality centroid. The second term is a weights regularization controlled by λ. Deep SVDD naturally calls for a latent multisphere extension. An example of such an extension is Deep multi-sphere SVDD (MSVDD) [START_REF] Ghafoori | Deep multi-sphere support vector data description[END_REF], which is part of our comparison. Deep MSVDD initializes numerous latent normality hyperspheres using k-means and progressively discards the irrelevant centroids during training. The relevance of latent hyperspheres is determined thanks to the cardinality of the latent cluster they encompass. The deep MSVDD training loss is:

min W,r1...r K 1 K K k=1 r 2 k + 1 νn n i=1 max(0, ||Φ(x i ; W ) -c j || 2 -r 2 j ) + λ 2 L l=1 ||W l || 2 (3)
The first term minimizes the volume of hyperspheres of radius r k , while the second is controlled by ν ∈ [0, 1] and penalizes points lying outside of their assigned hypersphere, training samples being assigned to the nearest hypersphere of center c j . A second Deep SVDD variant considered here is Deep RPO [START_REF] Bauw | Deep random projection outlyingness for unsupervised anomaly detection[END_REF], which replaces the latent Euclidean distance to the normality centroid with a RPs-based outlyingness measure adapted from Eq. 1 in the latent space, leading to the following loss:

min W 1 n n i=1 mean u∈U |u T Φ(x i ; W ) -M ED(u T Φ(X; W ))| M AD(u T Φ(X; W )) + λ 2 L l=1 ||W l || 2 (4)
This training loss uses the outlyingness defined in Eq. 1 after the neural network encoding, with a max estimator transformed into a mean as suggested in [START_REF] Bauw | Deep random projection outlyingness for unsupervised anomaly detection[END_REF] for better integration with the deep learning setup. The mean estimator computes a mean over the set of RPs available to compute the latent outlyingness, while the 1 n computes a mean over the sample of size n. The use of a mean instead of a max removes the convergence to the Mahalanobis distance-inferred ellipsoid (up to a constant factor) already mentioned in the RPO definition for a large set of RPs. This RPO variant however remains affine invariant (see Appendix B). The loss nonetheless still combines 1D outlyingness measures individually centered by their median and normalized by their median absolute deviation, but with no ellipsoid-like score distribution guarantee in the input space once integrated (see Appendix A). No square was applied to the first loss term, in accordance with the quantity put forward in [START_REF] David L Donoho | Breakdown properties of location estimates based on halfspace depth and projected outlyingness[END_REF]. Whereas Deep SVDD uses the Euclidean distance to the latent normality centroid as a test score, Deep MSVDD replaces this score with the Euclidean distance to the nearest latent centroid remaining after training, from which the associated radius is subtracted. Very often in our experiments, even with multimodal normality during training, only one latent sphere remains at the end of Deep MSVDD training. Deep RPO replaces the Euclidean distance score with an RPO computed in the encoding neural network output space.

SAD is achieved through outlier exposure [START_REF] Hendrycks | Deep anomaly detection with outlier exposure[END_REF][START_REF] Ruff | Deep semi-supervised anomaly detection[END_REF], which adds supervision to the training of the model thanks to the availability of few and non representative labeled anomalies. To take into account anomalies during training, Deep SAD [START_REF] Ruff | Deep semi-supervised anomaly detection[END_REF] repels the outliers from the normality centroid by replacing the minimization of the distance to the centroid with the minimization of its inverse in the training loss. With m labeled anomalies x in a sample the Deep SVDD loss thus becomes, with a training objective balancing parameter η:

min W 1 n + m n i=1 ||Φ(x i ; W ) -c|| 2 + η n + m m j=1 (||Φ( x j ; W ) -c|| 2 ) -1 + λ 2 L l=1 ||W l || 2 (5)
Labeled anomalies in the training set need to be distinguished from potential unlabeled anomalies that are considered to be normal samples, which confuse the AD by contaminating the training set instead of providing supervision. This adaptation can be repeated for both Deep RPO and Deep MSVDD, although for Deep MSVDD the multiplicity of normality centers calls for an additional consideration on how to choose from which centroid the labeled anomalies should be repelled as long as several centroids are kept active. The experiments implementing Deep MSVDD adapted to SAD with an additional loss term for labeled anomalies were inconclusive, such an adaptation will therefore not be part of the presented results. The additional loss term either minimized the latent distance between anomalies and dedicated centroids, or maximized the latent distance between anomalies and normality centroids. Once adapted to SAD, the Deep RPO loss becomes, similarly to the transformation that led to Eq. 5:

min W 1 n + m n i=1 mean u∈U |u T Φ(x i ; W ) -M ED(u T Φ(X; W ))| M AD(u T Φ(X; W )) + η n + m m j=1 mean u∈U |u T Φ( x j ; W ) -M ED(u T Φ(X; W ))| M AD(u T Φ(X; W )) -1 + λ 2 L l=1 ||W l || 2 (6)
Arbitrary sets of outliers could not be completely gathered around a reference point since they are not necessarily concentrated in a common mode [START_REF] Ruff | Deep semi-supervised anomaly detection[END_REF][START_REF] Steinwart | A classification framework for anomaly detection[END_REF]. However, this does not forbid the concentration of identified modes among labeled anomalies close to dedicated centroids to provide additional supervision during training, a case which is part of the experiments presented. The possibly arbitrary distribution of normal and anomalous centroids and the relative distance between the centroids adds a way to use prior information regarding the proximity between the training samples. Such a setup can seem close to classification with rejection [START_REF] Hendrycks | Deep anomaly detection with outlier exposure[END_REF][START_REF] Peter | Classification with a reject option using a hinge loss[END_REF], since the concentration of data points around dedicated normal and anomalous centroids can be interpreted as classification while the data points attached to no centroid and thus supposedly repelled from all centroids by the trained network constitutes a rejection. This parallel with classification with rejection is not necessarily relevant since the availability of labeled anomalies to train AD methods is usually very limited if not nonexistent. In contrast, supervised classification of identified data modes would imply rich, representative and relatively balanced datasets for each latent mode. The limited availability of labeled anomalies applies to actual anomalies and not to artificial anomalies provided by the transformation of existing training samples i.e. through self-supervision. With proper transformations self-supervision can produce as many labeled anomalies for training as there are normal samples, or even more if each normal sample is transformed multiple times. However this does not overcome the lack of representativeness of labeled anomalies. This is also made difficult since the choice of transformations requires expert knowledge.

The reunion of normal latent representations achieved through the deep one-class classification methods mentioned is analogous to the alignment principle put forward in [START_REF] Wang | Understanding contrastive representation learning through alignment and uniformity on the hypersphere[END_REF], which also argued for a latent uniformity. Whereas the alignment principle compels similar samples to be assigned similar representations, the uniformity principle demands the preservation of maximal information. One way to achieve that according to [START_REF] Wang | Understanding contrastive representation learning through alignment and uniformity on the hypersphere[END_REF] is to push all features away from each other on the unit hypersphere to intuitively facilitate a uniform distribution. The extension of the Deep SVDD loss to encourage a form of latent uniformity using the pairwise distance between normal samples during training was investigated without ever improving the baselines. The experiments conducted to evaluate the contribution of a pairwise distance of normal samples latent representations loss term revolved around the following training loss format, where the term tasked with enforcing latent uniformity is weighted using β and was expected to be judiciously balanced with the overall latent concentration:

min W 1 n n i Φ(x i ; W ) -c 2 + β n n i =j x i -x j 2 -1 + λ 2 L l=1 ||W l || 2 (7)
The failure to make a loss term enforce a form of latent uniformity could signal the necessity of associating such a constraint with latent representations confined to a relevant manifold.

Riemannian methods for covariance matrices

Two SPD-specific AD approaches were considered. The first approach consists in replacing the principal component analysis (PCA) dimensionality reduction preceding shallow AD with an SPD manifold-aware tangent PCA (tPCA). The tPCA projects SPD points on the tangent space of the Fréchet mean, a Riemannian mean which allows to compute an SPD mean, keeping the computed centroid on the Riemannian manifold naturally occupied by the data. Using tPCA offers the advantage of being sensible to the manifold on which the input samples lie, but implies that input data is centered around the Riemannian mean and not too scattered. This makes tPCA a questionable choice when the objective set is AD with multimodal normality [START_REF] Pennec | Barycentric subspace analysis on manifolds[END_REF], something that is part of the experiments put forward in this work. In other words, although the linear approximation of the SPD inputs around its Riemannian mean provides us with a manifold-aware dimensionality reduction, the distribution of the inputs in the SPD matrices manifold may lead this approximation to be excessively inaccurate, leading to irrelevant reduced representations. This is due to the linear approximation not preserving the Riemannian distances between points [START_REF] Sommer | Manifold valued statistics, exact principal geodesic analysis and the effect of linear approximations[END_REF]. Nonetheless, the Euclidean PCA being a common tool in the shallow AD literature, tPCA remains a relevant candidate for this study since it enables us to take a step back with respect to non-deep dimensionality reduction.

The second SPD-specific approach defines a Riemannian equivalent to Deep SVDD: inspired by recent work on SPD neural networks [START_REF] Lou | Differentiating through the fréchet mean[END_REF][START_REF] Brooks | Riemannian batch normalization for spd neural networks[END_REF][START_REF] Huang | A riemannian network for spd matrix learning[END_REF][START_REF] Yu | Second-order convolutional neural networks[END_REF], which learn intermediate representations while keeping them on the SPD matrices manifold, a Deep SVDD SPD would transform input covariance matrices and project the latter into a latent space comprised within the SPD manifold. Taking into account SAD and SSL labeled anomalies during training was expected to be done as for the semi and self-supervised adaptations of Deep SVDD described earlier, where labeled anomalies are pushed away from the latent normality centroid thanks to an inverse distance term in the loss. For Deep SVDD SPD, the distance would be a geometry-aware distance such as the Log-Euclidean distance. Despite diverse attempts to make such a Deep SVDD SPD model work, with and without geometry-aware non-linearities in the neural network architecture, no effective learning was achieved on our dataset. This second approach will therefore be missing from the reported experimental results. Since this approach defined the ReEig [START_REF] Huang | A riemannian network for spd matrix learning[END_REF] non-linearity rectifying small eigenvalues of SPD representations, the related shallow AD approach using the norm of the last PCA components as an anomaly score was also considered. This negated PCA is motivated by the possibility that, in one-class classification where fitting occurs on normal data only, the first principal components responsible for most of the variance in normal data are not the most discriminating ones when it comes to distinguishing normal samples from anomalies [START_REF] Miolane | ICLR 2021 challenge for computational geometry & topology: Design and results[END_REF][START_REF] Rippel | Modeling the distribution of normal data in pre-trained deep features for anomaly detection[END_REF]. This approach was applied to both spectral and covariance representations, with the PCA and tPCA last components respectively, but was discarded as well due to poor performances. The latter indicate that anomalous samples are close enough to the normal ones for their information to be carried in similar components, emphasizing the near OODD nature of the discrimination pursued. Preprocessing This work is inspired by [START_REF] Ruff | Deep one-class classification[END_REF], which experimented on MNIST, a dataset in which samples are images of objects without background or irrelevant patterns. In order to guarantee a relevant neural architecture choice, this kind of input format is deliberately reproduced. The idea of creating MNIST-like benchmarks has been of interest in different scientific communities such as biomedical images [START_REF] Yang | Medmnist classification decathlon: A lightweight automl benchmark for medical image analysis[END_REF] and 3D modeling [START_REF] Jimenez Rezende | Unsupervised learning of 3d structure from images[END_REF]. The series of periodograms, i.e. non-SPD representations are therefore preprocessed such that only the columns with top 15% values in them are kept, this operation being done after a switch to logarithmic scale. This results in periodograms where only the active Doppler bins, portraying target bulk speed and micro-Doppler modulations, have non-zero value. Only a grayscale region of interest (ROI) remains in the input matrix with various Doppler shifts and modulation widths, examples of which are shown on Figure 3. This preprocessing leads to the "(SP)" input format as indicated in the results tables, and is complementary to the covariance representation. Covariance matrices are computed without such preprocessing, except for the switch to logarithmic scale which precedes the covariance computation. Comparing covariance-based OODD to OODD on spectral representations is fair since both representations stem from the same inputs, the covariance only implying an additional transformation of the input before training the AD. All input data is min-max normalized except for the covariance matrices used by tPCA.

Deep learning experiments

The test AUC score of the best validation epoch in terms of AUC is retained, in line with [START_REF] Chong | Simple and effective prevention of mode collapse in deep one-class classification[END_REF]. All experiments were conducted with large 1000 samples batches, which stabilizes the evolution of the train, validation and test AUCs during training. The training is conducted during 300 epochs, the last 100 epochs being fine-tuning epochs with a reduced learning rate, a setup close to the one in [START_REF] Ruff | Deep one-class classification[END_REF]. As was suggested in [START_REF] Chong | Simple and effective prevention of mode collapse in deep one-class classification[END_REF] a relatively small learning rate of 10 -4 is chosen to help avoid the latent normality hypersphere collapse, i.e. the convergence to a constant projection point in the latent space in the non-SAD and non-SSL cases, with λ = 10 -6 . Such a latent normality collapse is made impossible when SAD or SSL samples are concentrated around dedicated centroids or scattered away from normality centroids, since the network is then trained to disperse representations. Loss terms integrating labeled anomalies for extra training supervision are balanced with η = 1, and for Deep MSVDD ν = 0.1.

Hyperparameters are kept constant across all experiments conducted, in order to ensure fair comparisons. In the results tables, the second and third columns indicate whether SAD and SSL samples were used for additional supervision during training, and describe how such samples affected the training loss if present. When the SAD or SSL loss term is defined by a centroid, it means that the distance to the mentioned centroid is minimized during training, whereas "away" implies the projection of the SAD or SSL samples are repelled from the normality centroid thanks to an inverse distance as described previously. For example, the first line of the second part of Table 2 describes an experiment where SAD samples are concentrated around the SAD samples latent centroid, and SSL samples concentrated around the SSL samples latent centroid. Centroids are computed, as for the normal training samples, with the averaging of an initial forward pass, therefore yielding the average latent representation.

Non-deep learning experiments Shallow AD conducted on the covariance representation after a common PCA uses the upper triangular part of the min-max normalized input as a starting point, avoiding redundant values. This contrasts with the Riemannian approach replacing PCA with the tPCA, the latter requiring the raw SPD representation. Furthermore, shallow approaches were also tried on the periodograms individually, where each row of an input signature, i.e. one vector of Doppler bins described for one burst, was given a score, the complete signature being then given the mean score of all its periodograms. This ensemble method did not yield relevant results and is therefore missing from our comparison. Such an approach ignores the order of periodograms in signatures.

Neural network architecture While the MNIST-like input format is thus replicated, the 2D features remain specific to radar signal processing and may therefore benefit from a different neural network architecture. Several neural networks architectures were considered, including architectures beginning with wider square and rectangular convolutions extended along the (vertical) bursts input axis, with none of the investigated architectures scoring systematically higher than the Fashion-MNIST architecture from the original Deep SAD work [START_REF] Ruff | Deep semi-supervised anomaly detection[END_REF], which was only modified in order to handle the larger input size. The latter was consequently selected to produce the presented results. This architecture projects data with two convolutional layers followed by two dense layers, each layer being separated from the next one by a batch normalization and a leaky ReLU activation. The outputs of the two convolutional layers are additionally passed through a 2D max-pooling layer.

Riemannian AD The tPCA was computed thanks to the dedicated Geomstats3 [START_REF] Miolane | Geomstats: A python package for Riemannian geometry in machine learning[END_REF] function, while experiments implementing a Riemannian equivalent of Deep SVDD were conducted using the SPD neural networks library torchspdnet4 [START_REF] Brooks | Second-order networks in pytorch[END_REF]. The AD experiments based on a SPD neural network ending up inconclusive, they are not part of the results tables.

Potential contribution of SAD and SSL

The contribution of additional supervision during training through the introduction of SAD samples and SSL samples is examined in Table 2. Regarding SAD experiments, labeled anomalies will be taken from a single anomalous class for simplicity, and because only four classes are being separated, this avoids unrealistic experiments where labeled anomalies from every anomalous class are seen during training. When SAD samples are used during training, labeled anomalies represent one percent of the original training set size. This respects the spirit of SAD, for which labeled anomalies can only be a minority of training samples, which is not representative of anomalies. This is especially realistic in the radar processing setup initially described where labeled detections would rarely be available. SSL samples are generated thanks to a rotation of the spectral input format, rendering the latter absurd but encouraging better features extraction since the network is asked to separate similar patterns with different orientations. SSL samples are as numerous as normal training samples, implying they don't define a minority of labeled anomalies for training as SAD samples do, when they are taken into account.

Individually, SAD samples lead to better performances than SSL ones, but the best results are obtained when combining the two sets of samples for maximal training supervision. Deep SVDD appears to be substantially better at taking advantage of the additional supervision provided by SAD and SSL samples. Quite surprisingly for a radar operator, the best test AUC is obtained when SSL samples are concentrated around a specialized centroid while SAD samples are repelled from the normality centroid. Indeed, SSL samples being the only absurd samples considered in our experiments radarwise, it could seem more intuitive to project SAD samples, which remain valid targets, next to a dedicated centroid while repelling SSL samples. Likewise, on an ideal outlyingness scale, SSL samples should be further away from normality than SAD samples. This counter-intuitive performance could stem from the test set which only evaluates the separation of targets in a near OODD context. No invalid target representation, like the SSL samples are, is present in the test set, only valid representation from the four target classes make up the latter. This is consistent with the application put forward in this study: use OODD to discriminate between various kinds of radar detections.

Training with a contaminated training set

Unsupervised AD refers to the experiments of Table 1 where only training samples assumed to be normal supervise the training of the neural network. Real-life datasets, labeled by algorithms or experts, are unlikely to respect that assumption and will suffer from contamination of normal samples with unlabeled anomalies. The results in SAD samples were repelled from the normality centroid or concentrated next to their dedicated latent reference point, here they will be processed as normal samples. SSL samples again appear to better contribute to improving AD when concentrated next to a specialized centroid, while the performance drop due to contamination does not seem to be particularly stronger for one of the approaches considered.

Conclusion

The near OODD performances of various deep and non-deep, unsupervised and semi-supervised AD methods were compared on a radar Doppler signatures simulated dataset. Deep AD approaches were evaluated in various supervision setups, which revealed the relevance of combining a minority of labeled anomalies with transformed normal training samples to improve semi-supervised near OODD performances, and avoid latent normality distribution collapse. Among the limitations of our study, one can note the lack of OODD experiments on a multimodal normal training set with unbalanced normal classes, which would make the OODD task more realistic. Additionally, further reducing the Doppler resolution and the number of training samples available could make the targets discrimination task closer to reality. Taking into account a varying Doppler resolution, i.e. a varying number of pulses per burst, could also improve the relevance of the proposed discrimination. The benefits of deep learning clearly showed, and while not leading to the best overall performances, geometry-aware processing with tangent PCA proved to be the source of a substantial improvement for non-deep AD.

=⇒

(u T (x -µ X )) 2 u T Σu ≤ (x -µ X ) T Σ -1 (x -µ X ) (10)
where u T Σu ≥ 0 since Σ is positive definite. As suggested in [START_REF] Arnold | Applied multivariate statistical analysis[END_REF] if one takes u = αΣ -1 (xµ X ), with α ∈ R * , one gets:

=⇒ ((αΣ -1 (x -µ X )) T (x -µ X )) 2 (αΣ -1 (x -µ X )) T Σ(αΣ -1 (x -µ X )) ≤ (x -µ X ) T Σ -1 (x -µ X ) (11)
=⇒ ((αΣ -1 (x -µ X )) T (x -µ X )) 2 α 2 (Σ -1 (x -µ X )) T I d (x -µ X ) ≤ (x -µ X ) T Σ -1 (x -µ X ) (12)
the latter leading to the equality case, showing us the upper bound is attainable. Since the upper bound is reachable for u = αΣ -1 (xµ X ), and that such format allows u to be a RP of unit norm as required by the definition of RPO for Eq. 1, one can intuitively conclude that using a maximum estimator over numerous RPs the bound is approached or reached in Eq.10 by the left term, i.e. for U a large set of unit norm RPs:

max u∈U (u T (x -µ X )) 2 u T Σu = (x -µ X ) T Σ -1 (x -µ X ) (13)
This result is actually well-known and called the Maximization Lemma in [START_REF] Arnold | Applied multivariate statistical analysis[END_REF], where instead of u ∈ U, u is an arbitrary nonzero vector, i.e. max u∈U is replaced by max

x =0

. The proof for this lemma is actually the previous example case where u = αΣ -1 (xµ X ) is shown to make the equality case happen. Note that using the equality case format for u implies knowing the covariance matrix Σ -1 . This emphasizes the relevance of RPO to compute a multivariate outlyingness without requiring a covariance matrix computation. From Eq. 13, one can deduce:

max u∈U (u T x -u T µ X ) 2 σ 2 (u T x) = (x -µ X ) T Σ -1 (x -µ X ) (14)
this expression makes the projected sample u T x and the projected mean u T µ X appear. Note that for the denominator the transformation is as follows:

σ 2 (u T x) = E (u T (x -µ X ))(u T (x -µ X )) T = E u T (x -µ X )(x -µ X ) T u = u T Σu (15)
Eq. 14 now only differs from RPO due to the square and the estimators of first and second order statistics: RPO replaces the mean and the variance of the projected inputs x ∈ X with the robust estimators median and median absolute deviation respectively, which in turn adds a constant factor with respect to the Mahalanobis distance. Regarding the relevance of these robust estimators choice and the additional constant factor see [START_REF] Velasco | Robust rx anomaly detector without covariance matrix estimation[END_REF].

We thus get back to the conclusion of [START_REF] Velasco | Robust rx anomaly detector without covariance matrix estimation[END_REF] indicating the equivalence (up to a constant factor) between the Mahalanobis distance and RPO computed with an infinite number of RPs under the multivariate elliptical distribution hypothesis. Whereas RPO as presented in Table 1 used the max estimator to integrate over RPs as defined in [START_REF] Velasco | Robust rx anomaly detector without covariance matrix estimation[END_REF], Deep RPO replaces the max with a mean estimator in Eq. 1 to measure outlyingness in the latent representation space provided by a neural network. This replacement is motivated by empirical results and an interpretation provided in [START_REF] Bauw | Deep random projection outlyingness for unsupervised anomaly detection[END_REF]. The drawback of this change is that the Mahalanobis equivalence guarantee is lost, since the Maximization Lemma leading to Eq. 13 can not be used with a mean. Working with an ellipsoid instead of a latent hypersphere as in [START_REF] Ruff | Deep one-class classification[END_REF] supposedly made the latent normality boundary used by the training objective more flexible and tailored to the data, and was the original motivation of [START_REF] Bauw | Deep random projection outlyingness for unsupervised anomaly detection[END_REF]. Recall that this normality boundary is fitted to training data before training and frozen, the boundary being defined by a single location estimator in the case of Deep SVDD, and by as many location and spread estimators as there are RPs in the case of Deep RPO.

Intuitively, the mean will pull the quantity integrated over the large set of RPs away from the upper bound. Even though the score is based on projected 1D outlyingnesses each normalized by their respective location and spread estimators, there is no assurance that once integrated with a mean into one final outlyingness these quantities generate a normality ellipsoid similar to the Mahalanobis one. Actually, nothing indicates the integrated outlyingness describes any kind of ellipsoid in the input vector representation space. However, since the mean integrates over the dimensions created by the set of RPs, and since along these dimensions each 1D coordinate is centered and normalized using its

• M AD(u T X) is the median absolute deviation of the same scalars, i.e. M AD(u T X) = M ed(|u T x -M ed(u T X)|).

AX + b is a permissive notation defining the affine transformation of every column features vector with A and b, i.e. the affine transformation of the whole data distribution on which the location and scatter estimators, respectively the M ed and the M AD, are applied. We also want to prove the affine invariance of that same quantity where the sup is replaced by a mean estimator:

O RP O-M EAN (x, X) = mean u =1 |u T x -M ed(u T X)| M AD(u T X) (18)
This RPO-MEAN is the RPO variant introduced in [START_REF] Bauw | Deep random projection outlyingness for unsupervised anomaly detection[END_REF] and used in the encoding neural network output space for our Deep RPO experiments. The Eq. 16 stems from [START_REF] David L Donoho | Breakdown properties of location estimates based on halfspace depth and projected outlyingness[END_REF][START_REF] Peter | Projection pursuit[END_REF] and defines the outlyingness used to generate the statistical depth [START_REF] Zuo | General notions of statistical depth function[END_REF] called random projection depth (RPD), the latter corresponding to the following expression: RP D(x, X) = 1 1+O RP O (x,X) . Note that this depth, and the associated outlyingness of Eq. 16, are defined with an infinite number of random projections u ∈ R d , thus the two quantities (RPO and RPD) can only be implemented with a stochastic approximation, i.e. with a large but finite number of random projections. For instance, in the experiments described in this paper, the sup and mean over all RPs of unit norm are approximated with a max and mean over 1000 RPs respectively. For other experiments on RPO with diverse quantities of RPs, see [START_REF] Bauw | Deep random projection outlyingness for unsupervised anomaly detection[END_REF].

N

  I/Q points ∈ C sampled at P RF representing the content of a single range bin

Figure 1 :

 1 Figure1: Each pulse leads to one complex-valued I/Q sample per range bin, while each burst is composed of several pulses. Each range bin is thus associated with a complex-valued discrete signal with as many samples as there are pulses. Air surveillance radars with rotating antennas are required to provide regular situation updates in every direction, severely constraining the number of pulses per burst acceptable.

Figure 2 :

 2 Figure 2: One sample of each target class: the varying number of rotating blades defines the classes, the modulation pattern being easily singled out. The first line of images shows Doppler signatures, i.e. the time-varying periodogram of targets over 64 bursts of 64 pulses. On those images, each row is the periodogram computed over one burst, and each column a Fourier i.e. a Doppler bin. The second line contains the covariance SPD representation of the first line samples. The width of the Doppler modulations around the bulk speed on the periodograms varies within each class, as well as the bulk speed, the latter being portrayed by the central vertical illumination of the signature.

Figure 3 :

 3 Figure 3: Random samples of the fourth class after the preprocessing erasing the irrelevant background, which makes the dataset closer to the MNIST data format.One can notice the varying modulation width of the target spectrum and central Doppler shift. The fourth class has the highest number of rotating blades on the helicopter-like target, hence the higher complexity of the pattern. These samples illustrate the input format of the various AD methods compared in this work, and are min-max normalized so that their values belong to [0, 1]. Only the inputs of the Riemannian setup where shallow learning AD is used on SPD inputs after tPCA uses a different input format, where the input covariance is not normalized.

Figure 4 :

 4 Figure 4: Left -Training metrics of a successful run where normal samples are concentrated around their average initial projection, and SAD and SSL samples are pushed away thanks to a loss term using the inverse of the distance with respect to the normality latent centroid. This is one of the most successful setups in Table 2, and one of the easiest AD experiments since the two classes defining normality here are class 3 (four blades are responsible for the modulation pattern around the central Doppler shift) and class 4 (six blades are responsible for the modulation pattern around the central Doppler shift), meaning the separation with the other classes deemed anomalous is actually a binary modulation complexity threshold. One of the contributions of the SAD and SSL supervisions can be observed on the evolution of AUCs during training: no AUC collapse can be seen during training, discarding the possibility of a latent distribution collapse during training. Experiments showed that large training batches contributed to stable AUCs growth. Spikes in the training loss match the drops in AUCs. Right -Latent distribution of the training samples visualized in 2D using t-SNE after projection by the untrained (top) and the trained neural network (bottom). One can notice that normal training samples from both normal classes are completely mixed up with the minority of SAD labeled anomalies from class 1 in red (one blade), semantically similar, whereas SSL samples which are rotated normal training samples are already gathered in their own latent subclusters. SAD labeled anomalies end up well separated after training.

  AD experiments are conducted for two setups: a first setup where normality is made of one target class, and a second setup where normality is made of two target classes. When a bimodal normality is experimented on, the normal classes are balanced. Moreover, the number of normal modes is not given in any way to the AD methods, making the experiments closer to the arbitrary and, to a certain extent, unspecified one-class classification useful to a radar operator. Within the simulated dataset, 90% of the samples are used to create the training set, while the rest is equally divided to create the validation and test sets. All non-deep AD methods are applied after a preliminary dimensionality reduction, which is either PCA or tPCA. The number of RPs used to compute the outlyingness score with RPO and Deep RPO is the same and set to 1000, even though the estimator used differs between the shallow and the deep approach. All deep experiments were run on a single GPU, which was either a NVIDIA Tesla P100 or a NVIDIA RTX 2080. In both cases running one deep AD setup for 10 seeds took approximately one hour. Non-deep experiments were CPU-intensive and also required around one hour for 10 seeds on a high-end multi-core CPU.

	4 Experiments

Table 1 :

 1 4.1 Unsupervised OODD with shallow and deep learningUnsupervised AD results, for which the training is only supervised by normal training samples, are presented in Table1. These results indicate the superiority of deep learning for the OODD task considered, while demonstrating the substantial contribution of geometry-aware dimensionality reduction through the use of tPCA for non-deep AD. RPO is kept in Table1even though it does not achieve useful discrimination because it is the shallow equivalent of Deep RPO, one of the highlighted deep AD methods, deprived of the neural network encoder and with a max estimator instead of a mean, as was previously justified. Deep MSVDD does not lead to the best performances, and is as effective as Deep SVDD and Deep RPO, which could seem surprising at least when normality is made of two target classes. Indeed, since Deep MSVDD has the possibility to use several disjointed hyperspheres to capture the latent normality distribution, one could expect it to better model more complex, e.g. multimodal, normality. Unsupervised AD experiments results (average test AUCs in % ± StdDevs over ten seeds). These machine learning methods are trained on fully normal training sets, without labeled anomalies for SAD or self-supervision transformations. The four last methods are our deep AD baselines, trained on normalized spectral representations only. Deep MSVDD "mean best" indicates the neural network was trained using a simpler loss, analogous to the Deep SVDD loss, where only the distance to the best latent normality centroid is minimized, thus discarding the radius loss term. PCA and tPCA indicate that the AD model is trained after an initial dimensionality reduction, which is either PCA or tangent PCA.

	AD method (input format)	SAD loss SSL loss Mean test AUC (1 mode) Mean test AUC (2 modes)
	OC-SVM (SP-PCA) OC-SVM (SPD-PCA) OC-SVM (SPD-tPCA) IF (SP-PCA) IF (SPD-PCA) IF (SPD-tPCA) LOF (SP-PCA) LOF (SPD-PCA) LOF (SPD-tPCA) RPO (SP-PCA) RPO (SPD-PCA) RPO (SPD-tPCA) Deep SVDD (SP) Deep MSVDD (SP) Deep MSVDD "mean best" (SP) no SAD / / / / / / / / / / / / no SAD no SAD Deep RPO (SP) no SAD	/ / / / / / / / / / / / no SSL no SSL no SSL no SSL	49.16 ± 26.69 64.68 ± 9.10 57.59 ± 3.91 50.96 ± 17.37 52.36 ± 22.47 66.91 ± 9.65 56.80 ± 2.38 66.44 ± 21.37 78.38 ± 8.86 49.61 ± 6.89 51.08 ± 19.66 33.97 ± 7.36 83.03 ± 6.83 82.27 ± 9.67 82.29 ± 7.20 83.60 ± 5.35	45.48 ± 27.53 58.23 ± 15.12 55.33 ± 9.48 48.50 ± 18.76 47.50 ± 20.32 61.23 ± 12.65 61.55 ± 10.29 65.83 ± 19.52 73.56 ± 10.09 50.43 ± 7.13 54.95 ± 17.58 38.08 ± 14.58 78.29 ± 6.68 78.30 ± 8.28 78.02 ± 6.80 78.13 ± 6.02

Table 2 :

 2 Table 3 depict how sensible the deep AD methods previously introduced are to training set contamination. The contamination is carried out using the one percent SAD samples already used for SAD experiments. While in the SAD experiments AD method (input format) SAD loss SSL loss Mean test AUC (1 mode) Mean test AUC (2 modes) Experiments with additional supervision provided by SAD and/or SSL labeled samples during training (average test AUCs in % ± StdDevs over ten seeds). When available, SAD samples are the equivalent of one percent of the normal training samples in quantity. The first half of the Table reports performances where only one of the two kinds of additional supervision is leveraged, while the second half describes the performances for setups where both SAD and SSL labeled samples contribute to the model training. Each couple of lines compares Deep SVDD and Deep RPO in a shared AD supervision setup, thus allowing a direct comparison. c. stands for centroid.

	Deep SVDD (SP) Deep RPO (SP) Deep SVDD (SP) Deep RPO (SP) Deep SVDD (SP) Deep RPO (SP) Deep SVDD (SP) Deep RPO (SP) Deep SVDD (SP) Deep RPO (SP) Deep SVDD (SP) Deep RPO (SP) Deep SVDD (SP) Deep RPO (SP) Deep SVDD (SP) Deep RPO (SP)	no SAD no SAD no SAD no SAD SAD c. SAD c. away away SAD c. SAD c. SAD c. SAD c. away away away away	SSL c. SSL c. away away no SSL no SSL no SSL no SSL SSL c. SSL c. away away SSL c. SSL c. away away	86.79 ± 6.54 88.70 ± 5.10 81.43 ± 8.62 80.21 ± 9.06 86.79 ± 8.94 81.38 ± 6.09 93.93 ± 4.82 84.19 ± 5.32 91.00 ± 6.45 87.79 ± 5.81 89.98 ± 7.79 78.86 ± 9.10 95.06 ± 4.20 89.82 ± 5.21 94.63 ± 4.31 90.91 ± 5.94	83.91 ± 7.92 84.59 ± 8.54 77.01 ± 8.20 78.93 ± 9.39 87.65 ± 6.44 76.45 ± 6.30 93.50 ± 7.61 80.37 ± 7.22 90.51 ± 7.38 82.69 ± 8.51 91.03 ± 6.71 79.11 ± 9.64 93.91 ± 7.31 87.17 ± 8.17 94.02 ± 7.30 92.69 ± 7.98
	AD method (input format)	SAD loss SSL loss Mean test AUC (1 mode) Mean test AUC (2 modes)
	Deep SVDD (SP) Deep MSVDD (SP) Deep MSVDD "mean best" (SP) no SAD no SAD no SAD Deep RPO (SP) no SAD Deep SVDD (SP) no SAD Deep RPO (SP) no SAD Deep SVDD (SP) no SAD Deep RPO (SP) no SAD	no SSL no SSL no SSL no SSL SSL c. SSL c. away away	80.76 ± 7.11 78.31 ± 11.18 79.84 ± 7.82 81.29 ± 5.92 85.34 ± 6.85 86.66 ± 6.41 79.62 ± 9.02 76.16 ± 9.87	76.02 ± 6.66 74.49 ± 9.13 74.89 ± 7.01 74.82 ± 5.89 81.36 ± 7.47 82.78 ± 8.25 75.38 ± 8.28 76.56 ± 8.69

Table 3 :

 3 Contamination experiments results (average test AUCs in % ± StdDevs over ten seeds): the SAD labeled anomalies are integrated within the training samples and taken into account as normal samples during training, thus no SAD loss term is used for SAD samples. The contamination rate is one percent, i.e. the equivalent of one percent of the normal training samples in labeled anomalies is added to confuse the AD. c. stands for centroid.

This preprint is an expanded version of a paper accepted at ECML PKDD 2022 and soon to appear in the conference proceedings edited by Springer Nature.

https://github.com/Blupblupblup/Doppler-Signatures-Generation https://github.com/Blupblupblup/Near-OOD-Doppler-Signatures

https://geomstats.github.io/

https://gitlab.lip6.fr/schwander/torchspdnet

An operator is permutation invariant if any permutation of the inputs does not change the output: g(x1, ... , xt) = g(x π(1) , ... , x π(t) )for any permutation π.

Acknowledgements

This work was supported by the French Defense Innovation Agency (Cifre-Défense 001/2019/AID). The authors additionally thank Maxime Bombar and Lev-Arcady Sellem for their constructive comments regarding the mathematical properties of RPO-MEAN.

Appendix A About the relation between RPO, Deep RPO and the Mahalanobis distance

To elaborate on the relation between RPO and the Mahalanobis distance, the latter describing an ellipsoid in the data points representation space, let us remind the results presented in [START_REF] Velasco | Robust rx anomaly detector without covariance matrix estimation[END_REF]. Let us consider a data point x ∈ R d belonging to a data matrix X d×n following an Elliptically Symmetric Distribution (ESD) containing n samples for which we want to compute an outlyingness score O(x). The ESD hypothesis guarantees that the sample covariance matrix Σ is positive definite. Using Σ, one can define an outlyingness score using the Mahalanobis distance:

where µ X is the data points mean, i.e. the mean column of X. According to the extended Cauchy-Schwarz inequality [START_REF] Arnold | Applied multivariate statistical analysis[END_REF], for any nonzero vector u ∈ R d :

own location and spread estimators, the mean still relates to an ellipsoid in the high-dimensional representation space generated by the RPs.

In order to ensure the relevance of such replacement for our application, which as previously explained in this appendix removes the equivalence with a Mahalanobis distance when numerous RPs are used to estimate RPO, Deep RPO with mean and Deep RPO with max were compared for every deep experimental setup presented in the paper. The resulting performances confirmed the superiority of Deep RPO with mean. A few of these performances are presented in Table 4 and Table 5. Since these experiments rely on a deep neural network, one way to see the difference between Deep RPO with mean and Deep RPO with max is to consider that while with max the gradient will be based on a single projection, the mean produces a gradient stemming from all projections simultaneously.

AD

B Affine invariance of RPO with max and mean estimators

We want to prove the affine invariance of the following quantity, called the random projection outlyingness (RPO):

that is, we want to prove the following equality:

where:

• x ∈ R d×1 is the data point for which we want to compute an outlyingness measure; • X ∈ R d×n is the data matrix containing n features vectors in R d (i.e. the data distribution, including x);

• u ∈ R d×1 is a random projection vector of unit norm, i.e. u ∈ S d-1 where S d-1 = {x ∈ R d : x 2 = 1};

• A ∈ R d×d is a non-singular matrix (for the affine transformation); • b ∈ R d×1 is a constant vector (for the affine transformation);

• M ed(u T X) is the median of the scalars generated by the 1D projection of all x in X by u;

Proof: Let us start by noticing that both the upper and lower parts of the RPO ratio are invariant to the bias term b of the affine transformation:

M AD(u

This indicates both RPO and RPO-MEAN are translation invariant, a partial requirement to achieve affine invariance.

Let us factor the upper and lower parts of the RPO ratio to make a unit norm vector u T A u T A appear:

This enables us to rewrite the RPO ratio as follows:

Thus for any u, x and X, let f (u) be:

if one defines φ(u

, for g the mean or sup operator applied to every existing random projection u:

Combining the two last equalities provides us with the invariance to the linear transformation defined by A:

In other words, since φ is a bijection and all existing random projections of unit norm are considered during the integration over S d-1 , the operator g is not affected whether the RPs are transformed by φ beforehand or not. The intuition behind this invariance is that we are looking over the same infinite set of random projections, the transformation matrix A at most reshuffling the RPs in the infinite set the estimator integrates over. More generally, that is true for any permutation invariant 5 operator g.

The previous translation invariance with respect to b and the linear transformation invariance with respect to A prove the affine invariance of both RPO and RPO-MEAN.


Hervé Mathez
email: mathez@ipnl.in2p3.fr

Patrice Russo

Guo-Neng Lu

Patrick Pittet

Laurent Quiquerez

Jacques Lecoq

Gérard Bohner

A Charge-Sensitive Amplifier Associated with APD or PMT for Positron Emission Tomography Scanners

Keywords:

We present a Charge-Sensitive Amplifier (CSA) to be coupled with a 511-KeV 2-photon detector for positron emission tomography scanners. The circuit has been designed to be associated with an Avalanche Photodiode (APD) or Photo-Multiplier Tube (PMT) with large capacitance. It is a two-stage structure. The input stage consists of a foldedcascode fully-differential part and a common-mode feedback (CMFB) circuit. The output stage employs complementary source followers. The amplifier has been designed in a 0.35µm BiCMOS process with optimization of noise and speed performances to meet specific constraints. Its main characteristics evaluated by post-layout simulations are: 70-dB DC gain, 4.6-GHz GBW, 20-ns peaking time for pulsed stimulus, 3900-electron equivalent input noise charge (ENC), 135-mW power consumption at 3.5 V supply.

I. INTRODUCTION

Positron Emission Tomography (PET) scanners have been recognized as very powerful and sensitive instruments for biomedical purposes such as brain studies, cardiac imaging, early cancer diagnosis and therapy. They operate by indirect detection of radioisotope's positron emission, which annihilates with an electron to produce a pair of 511-KeV (gamma) photons emitting in opposite direction. Each escaped photon may hit a scintillator to generate a pulsed light that can be detected using a photomultiplier tube (PMT) or an avalanche photodiode (APD). Via sensitive and rapid detection of the 511-KeV photon pair, the positron annihilation event can be localized on a straight line of coincidence (line of response, or LOR).

PET scanners should make use of low-noise and rapid electronics associated with PMT or APD. The associated electronics may include successive charge-sensitive amplification, analog filtering, A-to-D conversion and digital signal processing, as shown in Fig 1a.

For developing such systems, there has been demand for custom design of detector-associated electronics to meet specific requirements, especially on speed and sensitivity performances. One challenge of the design task is the detector-coupled charge-sensitive amplifier, which is the first stage of the associated electronics.

The performance requirements for the amplifier include: about 5-GHz gain-bandwidth product (GBW), minimum detectable charge of 4000 electrons, 40dB dynamic range. On the other hand, it should be mentioned that the detector capacitance is large: 10pF for the PMT and 50pF for the APD respectively.

We present in this paper the design of an amplifier for this application. The proposed circuit is based on a fully differential amplifier (Fig. 1b), which allows input adaptation to either PMT or APD (having reversed output), and output adaptation to the following differential-mode shaper stage. Its symmetrical structure is also advantageous for reduction of electromagnetic noise.

II. Circuit description

The charge-sensitive amplifier consists of two stages. The input stage is a fully differential structure including a common-mode feedback (CMFB) block. The output stage is a source-follower performing impedance adaptation.

A. Input stage

The input stage for differential operation is a symmetrical structure. Its differential operation can be described using a half circuit because of its symmetrical structure. The half circuit is shown in Fig. 2. This half circuit is a folded-cascode amplifier employing an input p-type MOSFET (M1) and an npn transistor (Q7). The input MOSFET with very low leakage gate current has minimized noise effect of the related shot-noise current source. The use of bipolar component in the cascode structure allows improvements of gain and frequency performances. It also occupies a much smaller surface area compared to a large-size MOSFET.

Fig. 2. Half circuit of the amplifier for differential operation

As the photodetector to be coupled has large capacitance, large-size MOSFETs are employed, especially for the input transistor M1 (W/L = 5000µm/0.35µm). By setting a bias current up to 10mA for M1, its transconductance g m may reach 100mA/V to enhance gain and gain-bandwidth product (GBW), and to reduce its channel thermal noise.

The required gain for the amplifier is about 70dB. It can be achieved using M8 as active charge with a bias current of 0.35mA. This simple active charge allows improved output dynamic swings, which is an aspect to be considered as the supply voltage is only 3.5V for the circuit.

The DC gain of the half circuit is given by: ( )

4 1 7 8 4 1 7 1 0 1 - - + - = ds m ds ds m m r g g r g g A (1)
where is only a few hundred ohms because of large sizes and large bias currents. However, thanks to large transconductance of Q7, and the gain can approximately be estimated by:

8 1 0 ds m g g A - ≈ (2)
The bandwidth is determined by the time constant of the output node, given by:

out ds c C g f π 2 8 = (3)
To preserve bandwidth while ensuring close-loop stability, a feedforward compensation technique [START_REF] Sansen | Feedforward Compensation techniques for High Frequency CMOS amplifier[END_REF] is implemented by adding passive components R c and C c . Proper choice of the added components allows second-pole cancelation by a negative zero. R c is chosen to satisfy the following condition:

7 1 1 m c ds g R g ≤ ≤ (4)
and C c is optimized for a good phase margin.

The fully differential part of the input stage (not including the feedforward compensation network) is shown in Fig. 3a. An additional transistor M5 sets the bias currents of the input pair (M1, M2). Its gate bias not only controls I ds1 and I ds2 , but also affects the bias currents of the differential output branches (i.e. branch of M9, Q6 and that of M8, Q7 respectively). Thus it can be connected to the output of the CMFB circuit to adjust the common-mode output bias voltage V out,cm (= (V out1 + V out2 )/2). The CMFB circuit with connections to the fully differential part is presented in Fig. 3b. It is similar to the one proposed by [START_REF] Gray | Analysis and Design of Analog Integrated Circuit[END_REF]. The common-mode output bias voltage of the differential part V out,cm is sensed by a transistor pair (M21, M24). An increase of V out,cm will raise the sum of their drain currents (I ds21 + I ds24 ). Assuming constant bias currents for M26 and M27, the rise of (I ds21 + I ds24 ) will lead to the decrease of the sum of another pair's drain currents (I ds22 + I ds23 ), which is equal to the drain current of M25 I ds 25 . The resulting reduction of the gatesource voltage |V gs25 | is feedback to M5 of the differential part, allowing final lowering of V out,cm . Thanks to this feedback control, a very low common-mode gain of the input stage (e.g. A cm << 1) can be achieved.

It should be noted that the node at the drain of M25 has a related non-dominant pole expressed as:

>> ds m r g

It may cause instability when choosing too low bias current for M25. To avoid this problem, the minimum current ratio I ds25 /I ds5 should be about 0.5.

B. Source-follower output stage

The output stage of the amplifier consists of two source followers driven respectively by the differential outputs of the input stage. In order to optimize output swings, the source follower driven by V out2 is composed of n-type transistors, while the other source follower for V out1 employs p-type components (shown in Fig. 4). It is noted that the input signal of the amplifier is always a positive one. This means that the non-inverting output of the input stage V out2 has a upward swing range from V out,cm to (V dd -V ds9,sat ) while its inverting output V out1 has a downward swing from V out,cm to (V ds4,sat + V ce7,sat ). The proposed complementary source followers as output stage operate for these swing ranges.

However, there is some loss of symmetry for the differential outputs of the amplifier; in particular the output bias voltages may be significantly different. This disadvantage will not raise serious problems for the application case, because the amplifier coupling with the following shaper stage is capacitive. Other dissymmetrical problems such as gain differences may be minimized through careful design.

C. Frequency and pulsed stimulus response of the amplifier

As the following output stage is a source-follower circuit with a gain close to unity, the GBW of the amplifier is approximately given by:

out m c C g f A GBW π 2 1 0 = ≈ (6)
For a pulsed charge stimulus, the rise time of the amplifier's response is determined by the time constant:

f d csa C GBW C ⋅ ⋅ = π τ

III. Noise analysis

The equivalent circuit of the CSA half circuit for noise analysis is shown on Fig. 5. Noise sources from the detector and its biasing network do not appear in Fig. 5 and are not considered here (Detailed analysis of such noise sources can be found in [START_REF] O'connor | Prospects for charge sensitive amplifiers in scaled CMOS[END_REF] and [START_REF] Gryboś | Design of low noise charge amplifier in sub-micron technology for fast shaping time[END_REF]).

C f C D i na ² e nr ² R f -G CRRC Shaper V s C in e na ²

Fig. 5. Equivalent circuit for noise calculation

The parallel and serial input referred noise sources of the CSA are denoted respectively as i na ² and e na ². A voltage noise source, denoted as e nr ², is inserted to take into account the thermal noise of the feedback resistor R f . C D is the detector capacitance and C in is the input transistor gate capacitance.

The parallel input noise i na ² corresponds to the shot noise generated by the input transistor's gate leakage current. For CMOS technology, this input current is negligible and so its related noise can be neglected.

The feedback resistor's thermal noise e nr ² is given by:

f nr kTR df e d 4 2 = ( 8
)
where k is the Boltzmann constant and T the temperature in Kelvin.

The bandwidth of this white noise source is limited by the time constant of the feedback network (τ f = R f C f ) and by the peaking time τ s of the CRRC shaper. The contribution of this noise evaluated at the shaper output is:

( ) f s f s f s f nr C kT C kT v τ τ τ τ τ ≈ + = 2 (9)
Since the peaking time τ s is much lower than the feedback network characteristic constant time τ f , the resistor feedback thermal noise contribution at the shaper output can be neglected.

The serial noise e na ² is mainly contributed by the noise from the input transistor.

Its power spectral density is given by: where g m1 is the transconductance of transistor M1, C ox is the gate oxide capacitance per area, K f is the process dependant flicker noise parameter.

The first term in (10) takes into account the thermal noise of the M1 channel.

The second term in (10) describes the flicker noise resulting mainly from the charge trapping in the transistor channel.

The serial noise is firstly amplified by the CSA and then filtered by the CRRC shaper. Its contribution at the shaper output is given by [START_REF] Moser | Review: Silicon detector systems in high energy physics[END_REF]:

        +         + ≈ WL C K g kT C C C v ox f m s f d in na 2 1 2 2 2 3 2 57 . 1 πτ (11)
The equivalent input noise charge (ENC) is given by: ( )

WL C K g kT e C C ENC ox f m s d in na 2 1 2 3 2 57 . 1 + + ≈ πτ (12)
where e is the Neper number.

Because of the short peaking time of the shaper, ENC is dominated by the input transistor's thermal noise contribution.

Equation ( 12) shows that a low noise CSA design should implements input transistors with large transconductance while maintaining their input capacitance much lower than the detector capacitance. Such a tradeoff implies the implementation of large input transistor with minimum gate length L (0.35µm) to achieve high W/L ratio (for large g m ) with the minimum WL product (for reducing the input capacitance).

IV. CHARACTERISTICS

The proposed amplifier has been designed in a 0.35µm BiCMOS process (AMS). The floor plan of the layout is shown in Fig. 6a. Layout of the high-gain input stage is generated with application of symmetry and commoncentroid techniques. For the large-size input differential pair (M1, M2), its layout is an array of 10x10 commoncentroid cells in parallel connections (Fig. 6b). Post-layout simulations have been performed to evaluate the characteristics of the designed amplifier. Fig. 7 shows the frequency response of the amplifier. The estimated GBW is 4.6GHz with a phase margin of 79°. The phase evolution with increasing the frequency is not monotonic, because practical implementation of feedforward compensation has difficulties in precise zeropole cancelation. It is however necessary to minimize phase oscillation for stability consideration. This can be done by adjusting the values of the compensation components (R c and C c ). pF capacitive coupling, the pulsed stimulus is applied to the amplifier input. The corresponding peak-to-peak differential output voltage is slightly over 1V with a peaking time of 25ns. Fig. 8b shows simulated response to an APD-like output signal: the stimulus has a rise time of 5ns and an exponential decay time of 40ns. In this case, the peak-to-peak differential output voltage is 0.9V and the peaking time is 103 ns. Post-layout simulations with extracted parasitic parameters show significant noise performance degradation at 150µV RMS (ENC ~ 3900 electrons) compared to 130µV RMS by schematic simulations. This increase of noise is mainly due to serial resistance of the poly-Si gate of the input transistor pair. This extracted resistance is about 70Ω compared to the g m1 -1 (∼ 10Ω).

Symmetry axis

Table 1 summarizes the characteristics of the amplifier, obtained by post-layout simulations. The fabrication of the circuit is underway. We expect to undertake the testing work as soon as it is fabricated.

V. CONCLUSION

We have designed a charge-sensitive amplifier for development of PET scanners. Its speed and noise performances are optimized for this application. By postlayout simulations, we have evaluated its main characteristics: 70-dB gain, 4.6-GHz GBW, 20-ns peaking time for pulsed stimulus, 3900-electron noise (ENC), and 135-mW power dissipation at 3.5-V supply.

  Fig. 1a) Architecture of the detector-associated electronics; b) charge-sensitive amplifier using a fully differential amplifier

Fig. 3 .

 3 Fig. 3. Input stage consisting of a fully differential part (a) and a CMFB circuit (b)

Fig. 4 .

 4 Fig. 4. Output stage consisting of two complementary source followers

  a) Floor plan of the amplifier layout; b) Layout showing an array of 10x10 common-centroid cells, which is the large-size input differential transistor pair (M1, M2)

Fig 7 .

 7 Fig 7. Frequency response of the amplifier

Fig. 8 .

 8 Temporal response of the amplifier to pulsed stimulus (a) and APD-like input signal Area 0.195 mm

Table 1 .

 1 Main characteristics of the amplifier, obtained by postlayout simulations

	2

APD or PMT

[START_REF] Gryboś | Design of low noise charge amplifier in sub-micron technology for fast shaping time[END_REF] where C d is the detector capacitance, and C f is the feedback capacitor of the amplifier. C f should properly be chosen. On the one hand, according to the above expression, it should be increased to reduce the time constant, but on the other hand, it should be small enough to minimize effect of the input-related pole on the stability.

ACKNOWLEDGEMENT

This work has been done in the context of INNOTEP collaboration devoted to novel imaging system for in-vivo monitoring. We thank this collaboration for the funding.

We would also like to thanks Mr. E. Bechetoille for his technical help in layout optimization.