Tomasz Skalski
email: tomasz.skalski@pwr.edu.pl

Piotr Graczyk
email: graczyk@univ-angers.fr

Bartosz Ko

Maciej Wilczyński
email: maciej.wilczynski@pwr.edu.pl

Ma Lgorzata Bogdan

Patrick Tardivel

Pattern recovery and signal denoising by SLOPE when the design matrix is orthogonal *

Sorted ℓ1 Penalized Estimator (SLOPE) is a relatively new convex regularization method for fitting high-dimensional regression models. SLOPE allows to reduce the model dimension by nullifying some of the regression coefficients and by equalizing some of nonzero coefficients. This allows to identify situations where some of true regression coefficients are equal. In this article we will introduce the SLOPE pattern, i.e., the set of relations between the true regression coefficients, which can be identified by SLOPE. We will also present new results on the strong consistency of SLOPE estimators and on the strong consistency of pattern recovery by SLOPE when the design matrix is orthogonal and illustrate advantages of the SLOPE clustering in the context of high frequency signal denoising.

1 Introduction

Introduction and motivations

The Linear Multiple Regression concerns the model Y = Xβ+ε, where Y ∈ R n is an output vector, X ∈ R n×p is a fixed design matrix, β ∈ R p is an unknown vector of predictors and ε ∈ R n is a noise vector. The primary goal is to estimate β. In the low-dimensional setting, i.e., when the number of predictors p is not larger than the number of explanatory variables n and X is of full rank, the ordinary least squares estimator βOLS has an exact formula βOLS = (X ′ X) -1 X ′ Y . For practical reasons there is an urge to avoid the highdimensionality curse, therefore we want the estimate to be sparse, i.e., to be descriptible by a smaller number of parameters. Several solutions were proposed to deal with such problem. One of them, the Least Absolute Shrinkage and Selection Operator (LASSO [START_REF] Sh | Basis pursuit[END_REF][START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF]) involves penalizing the residual sum of squares ∥Y -X β∥ 2 2 with an ℓ 1 norm of β multiplied by a tuning parameter λ:

βLASSO := arg min b∈R p 1 2 ∥Y -Xb∥ 2 2 + λ∥b∥ 1 .
The LASSO estimator is not unbiased, but is a shrinkage estimator which reduces some of the coefficients of β to zero, resulting in a sparser estimate. In the case of X being an orthogonal matrix, i.e. X ′ X = I p , the exact formula for βLASSO introduced by Tibshirani [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF] is based Another approach to reduce the dimensionality is the Sorted ℓ 1 Penalized Estimator (SLOPE [START_REF] Bogdan | Statistical estimation and testing via the sorted L1 norm[END_REF][START_REF] Bogdan | SLOPE -Adaptive Variable Selection Via Convex Optimization[END_REF][START_REF] Zeng | Decreasing Weighted Sorted l 1 Regularization[END_REF]), which not only generalizes the LASSO method, but also allows to clusterize the similar coefficients of β. In SLOPE, ℓ 1 -norm is replaced by its sorted version J Λ , which depends on the tuning vector Λ = (λ 1 , . . . , λ p ) ∈ R p , where λ 1 ≥ . . . ≥ λ p ≥ 0:

J Λ (β) := p i=1 |β| (i) λ i ,
where {|β| (i) } p i=1 is a decreasing permutation of absolute values of β i , namely

βSLOP E := arg min b∈R p 1 2 ∥Y -Xb∥ 2 2 + γJ Λ (b) .
The case of Λ being an arithmetic sequence was studied by Bondell and Reich [START_REF] Bondell | Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with OSCAR[END_REF] and called the Octagonal Shrinkage and Clustering Algorithm for Regression (OSCAR). The special case of SLOPE with λ 1 = λ 2 = . . . = λ p > 0 is LASSO. For Λ = (0, . . . , 0) we obtain the OLS estimator.

Clustering the predictors allows for additional dimension reduction by identifying variables with the same values of regression coefficients. One may recently observe the rise of interest in methods, which cluster highly correlated predictors [START_REF] Bondell | Simultaneous factor selection and collapsing levels in ANOVA[END_REF][START_REF] Gertheiss | Sparse modeling of categorial explanatory variables[END_REF][START_REF] Maj-Kańska | Delete or merge regressors for linear model selection[END_REF][START_REF] Sz | Group Lasso merger for sparse prediction with high-dimensional categorical data[END_REF][START_REF] Oelker | Regularization and model selection with categorical predictors and effect modifiers in generalized linear models[END_REF][START_REF] Stokell | Modelling High-Dimensional Categorical Data Using Nonconvex Fusion Penalties[END_REF]. SLOPE is ideal for this task, since it is capable to identify the low-dimensional structure, which is called the SLOPE pattern, defined by Schneider and Tardivel with the subdifferential of the SLOPE norm J Λ [START_REF] Schneider | The geometry of uniqueness, sparsity and clustering in penalized estimation[END_REF]. For the convention of this article we let sign(0) = 0. As k we will denote the number of clusters of patt(b) = (m 1 , . . . , m p ) ′ i.e., the number of nonzero components of |b|.

Definition 1.0.1 (SLOPE pattern [START_REF] Schneider | The geometry of uniqueness, sparsity and clustering in penalized estimation[END_REF]). The SLOPE pattern is a function patt : R p → Z p such that patt(b

) i = sign(b i )rank(|b i |),
where rank

(|b i |) ∈ {1, 2, . . . ,
|b i | = |b j | ⇒ |patt(b) i | = |patt(b) j | (clustering preservation), (d) |b i | > |b j | ⇒ |patt(b) i | > |patt(b) j | (hierarchy preservation). Example 1.0.1. patt(4, 0, -1.5, 1.5, -4) = (2, 0, -1, 1, -2).
Remark 1.0.1 (Subdifferential description of the SLOPE pattern [START_REF] Schneider | The geometry of uniqueness, sparsity and clustering in penalized estimation[END_REF]).

Let Λ = (λ 1 , . . . , λ p ) satisfy λ 1 > . . . > λ p > 0. Then patt(b 1 ) = patt(b 2 ) ⇐⇒ ∂ J Λ (b 1 ) = ∂ J Λ (b 2 ),
where ∂ f (b) is a subdifferential of the function f : R p → R in b, i.e.:

∂ f (b) = {v ∈ R p : f (z) ≥ f (b) + v ′ (z -b) ∀z ∈ R p }.
The subdifferential approach may be applied to a wider class of penalizers being polyhedral gauges, cf. [START_REF] Tardivel | The Geometry of Model Recovery by Penalized and Thresholded Estimators[END_REF].

Definition 1.0.2 (Pattern recovery by SLOPE). We say that the SLOPE estimator β

SLOP E

recovers the pattern of β when

patt β SLOP E = patt(β).
The clustering properties of SLOPE have been studied before, cf. [START_REF] Bondell | Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with OSCAR[END_REF][START_REF] Figueiredo | Ordered weighted l1 regularized regression with strongly correlated covariates: Theoretical aspects[END_REF], but the researchers consider strongly correlated predictors, which are being used in financial mathematics to group the assets with respect to their partial correlation with the hedge fund return times series [START_REF] Kremer | Sparse index clones via the sorted ℓ 1 -norm[END_REF]. In our article we present the clusterization by SLOPE when the predictors are orthogonal, i.e. X ′ X = I p . Such class of matrices is being widely used in signal analysis, [START_REF] Rao | Orthogonal transforms for digital signal processing[END_REF][START_REF] Sh | Atomic decomposition by basis pursuit[END_REF]. For general X the problem is considered in our parallel article [START_REF] Skalski | Pattern recovery by SLOPE[END_REF].

To study the properties of SLOPE we often use the unit ball in the dual norm of J Λ , which was studied e.g. by Zeng and Figueiredo [25]. This dual ball is also known as a signed permutahedron, see e.g. [START_REF] Negrinho | Orbit Regularization[END_REF][START_REF] Schneider | The geometry of uniqueness, sparsity and clustering in penalized estimation[END_REF].

C Λ = {π = (π 1 , π 2 , . . . , π p ) ∈ R p : j≤i |π| (j) ≤ j≤i λ j : i = 1, 2, . . . , p}. (1)
In this article we prove novel results on the strong consistency of SLOPE both in estimation and in pattern recovery. We also introduce a new, based on minimaxity, approach to relations between βSLOP E and βOLS .

Outline of the paper

In Section 2 we derive the connections between βSLOP E and βOLS in the orthogonal design. We use the minimax theorem of Sion, cf. [START_REF] Aubin | Mathematical Methods of Game and Economic Theory[END_REF]. In Section 3 we focus on the properties of βSLOP E . We use the geometric interpretation of SLOPE to explain its ability to identify the SLOPE pattern and provide new theoretical results on the support recovery and clustering properties using a representation of SLOPE as a function of the ordinary least squares (OLS) estimator. Similar approach for LASSO was used by Ewald and Schneider, cf. [START_REF] Ewald | Uniformly Valid Confidence Sets Based on the Lasso[END_REF] In Section 4 we discuss asymptotic properties of the SLOPE estimators in the low-dimensional regression model in which p is fixed and the sample size n tends to infinity. To be more precise, for each n ≥ 1 we consider a linear regression model

Y (n) = X (n) β + ε (n)
with the vector of observations Y (n) ∈ R n and the design matrix

X (n) ∈ R n×p . Here ε (n) = (ε (n) 1 , ε (n) 2 , . . . , ε (n) ) ′ ∈ R n
is a noise term, which has the normal distribution N (0, σ 2 I n ). We make no assumptions about the relations between ε (n) and ε (m) for n ̸ = m. We consider the low-dimensional setup when p is fixed and n goes to infinity and provide the conditions under which the SLOPE estimator is strongly consistent. Additionally, in case when for each n the design matrix is orthogonal, we provide the conditions on the sequence of tuning parameters such that SLOPE is strongly consistent in the pattern recovery. Compared to similar results in [START_REF] Skalski | Pattern recovery by SLOPE[END_REF], dealing with the general design matrices, our results are obtained using substantially simpler techniques and impose weaker restrictions on the sequence of tuning parameters required for the pattern recovery. In Section 5 we show the applications of the SLOPE clustering in terms of high frequency signal denoising and illustrate them with simulations. The Appendix covers the proofs of technical results.

Approach by minimax theorem 2.1 Technical results

Let r SLOP E denote the minimum value of the SLOPE criterion, attained by βSLOP E , i.e.

r SLOP E := min b∈R p 1 2 ∥Y -Xb∥ 2 2 + J Λ (b) = 1 2 ∥Y -X β SLOP E ∥ 2 2 + J Λ ( β SLOP E
).

Since

∥ β SLOP E ∥ 2 ≤ √ p∥ β SLOP E ∥ ∞ and λ 1 ∥ β SLOP E ∥ ∞ ≤ J Λ ( β SLOP E ) ≤ r SLOP E , it follows that λ 1 β SLOP E 2 ≤ √ p r SLOP E ≤ √ p 1 2 ∥Y -X0∥ 2 2 + J Λ (0) = √ p 2 ∥Y ∥ 2 2 .
We immediately get the following result.

Corollary 2.0.1.

β SLOP E 2 2
≤ M 0 , where

M 0 = p ∥Y ∥ 4 2 4λ 2 1 .
From this corollary it is seen that we can clearly limit our search to vectors β from the compact set M ⊂ R p defined by

M := b ∈ R p : ∥b∥ 2 2 ≤ M 0 .
Therefore, we can equivalently define a SLOPE solution by This statement follows from the definition of C Λ (1) and from the reflexivity of finitedimensional spaces.

β SLOP E = arg min b∈M 1 2 ∥Y -Xb∥ 2 2 + J Λ (b) . (2)

Saddle point

Let the function r :

M × C Λ → R be defined by r(b, π) := 1 2 ∥Y -Xb∥ 2 2 + π ′ b.
As an immediate consequence of (( 2)) and Proposition 2.0.1 we obtain

r SLOP E = min b∈R p 1 2 ∥Y -Xb∥ 2 2 + J Λ (b) = min b∈M 1 2 ∥Y -Xb∥ 2 2 + J Λ (b) = min b∈M max π∈C Λ 1 2 ∥Y -Xb∥ 2 2 + π ′ b = min b∈M max π∈C Λ r(b, π).
It turns out that the order of the maximization over π ∈ C Λ and the minimization over b ∈ M can be switched without affecting the result. To see this, note that both C Λ and M are convex and compact. Moreover, for each fixed π ∈ C Λ , r(b, π) is a convex continuous function with respect to b ∈ M and, for each fixed b ∈ M, r(b, π) is concave with respect to π ∈ C Λ (in fact, it is linear). Therefore, all assumptions of the Sion's minimax theorem are fulfilled (see [1, p. 218]) and thus there exists a saddle point (β

* , π * ) ∈ M × C Λ such that max π∈C Λ min b∈M r(b, π) = min b∈M r(b, π * ) = r(β * , π * ) = max π∈C Λ r(β * , π) = min b∈M max π∈C Λ r(b, π) = r SLOP E .
In the next section we shall see that the first coordinate of any saddle point (β * , π * ) is the SLOPE estimator.

SLOPE solution in the orthogonal design

Since for each fixed π ∈ C Λ , the function r(b, π) is convex with respect to b ∈ M, any point

b π ∈ M, at which the gradient ∂r(b, π) ∂b is zero, is a global minimum. If we rewrite r(b, π) as r(b, π) = 1 2 Y ′ Y -Y ′ Xb + 1 2 b ′ b + π ′ b
and differentiate with respect to b, we obtain

∂r(b, π) ∂b = -X ′ (Y -Xb) + π.
Equating this gradient to 0 gives the following equation for the optimum point b π :

b π = X ′ Y -π = βOLS -π.
Substituting this into the equation for r(b π , π), we find that

r(b π , π) = 1 2 Y ′ Y -b ′ π X ′ Y + 1 2 b ′ π b π + π ′ b π = 1 2 Y ′ Y -b ′ π X ′ Y + b ′ π b π + b ′ π π - 1 2 b ′ π X ′ Xb π = 1 2 Y ′ Y - 1 2 b ′ π b π = 1 2 Y ′ Y - 1 2 b ′ π b π = 1 2 Y ′ Y - 1 2 (X ′ Y -π) ′ (X ′ Y -π). Let p j = |{i : |m i | = k + 1 -j}| be the number of elements of the j th cluster of β, P j = i≤j p i and P k+1 = p. Lemma 2.1. Let π * = (π * 1 , . . . , π * p ) ′ ∈ C Λ be any solution of π * = arg min π∈C Λ ( βOLS -π) ′ ( βOLS -π)
and let β * = (β * 1 , . . . , β * p ) ′ be the corresponding point from M given by

β * = βOLS -π * .
Then, (π -π * ) ′ β * ≤ 0, for all π ∈ C Λ and hence

(a) sign (β * i ) • sign (π * i ) ≥ 0, i = 1,
) i | = k + 1 -j, then |π * | i ∈ |π * | (Pj-1+1) , . . . , |π * | (Pj ) , (c) if there is a k ∈ {2, . . . , p} such that k-1 i=1 π * σ(i) < k-1 i=1 λ i and π * σ(k) > 0, then β * σ(k-1) = β * σ(k) .
The proof is given in the Appendix. An immediate consequence of the Lemma is the following result.

Lemma 2.2. The point (β * , π * ) defined as in Lemma 2.1 is the saddle point of the function r(b, π).

The proof is given in the Appendix. We use the last lemma to prove the main result of this section.

Theorem 2.3. Let the point β * be defined as in Lemma 2.1. Then β * is the SLOPE estimator of β.

Proof. Using the fact that max

π∈C Λ r(β * , π) = min b∈M max π∈C Λ r(b, π) (see previous lemma) we have 1 2 ∥Y -Xβ * ∥ 2 2 + J Λ (β * ) = max π∈C Λ 1 2 ∥Y -Xβ * ∥ 2 2 + λπ ′ β * = max π∈C Λ r(β * , π) = min b∈M max π∈C Λ r(b, π) = min b∈R p 1 2 ∥Y -Xb∥ 2 2 + J Λ (b) . Corollary 2.3.1. [14] βOLS - βSLOP E = π * = arg min π∈C Λ βOLS -π 2 2 is the proximal projection of βOLS onto C Λ . Remark 2.3.1. For each π ∈ C Λ , the point βSLOP E = b π = βOLS -π belongs to M := b ∈ R p : ∥b∥ 2 2 ≤ M . Therefore, the constant M is chosen so that M > max{M 0 , M 1 }, where M 1 := max π∈C Λ ∥ βOLS -π∥ 2 2 ≤ M.
3 Properties of SLOPE in the orthogonal design have equal absolute value (clusterization), but their signs are opposite. For βOLS being the red point located on the area labelled by (1, 2) both components of βSLOP E are positive and the first component is smaller than the second one. The blue polytope is the dual SLOPE unit ball C Λ and labels M 2 = {(0, 0), (±1, 0), (0, ±1), (±1, ±1), (±2, ±1), (±1, ±2)} associated to the areas of this figure correspond to all SLOPE patterns for n ≥ p = 2. In the orthogonal design, one may also explicitly compute the SLOPE estimator. Indeed, by the Corollary 2.3.1, βSLOP E is the image of βOLS by the proximal operator of the SLOPE norm. Therefore, this operator has a closed form formula [START_REF] Bogdan | SLOPE -Adaptive Variable Selection Via Convex Optimization[END_REF][START_REF] Tardivel | Simple expression of the LASSO and SLOPE estimators in low-dimension[END_REF][START_REF] Dupuis | Proximal operator for the sorted l 1 norm: Application to testing procedures based on SLOPE[END_REF]. This explicit expression gives an analytical way to learn that SLOPE solution is sparse and built of clusters.

The next theorem gives a sufficient condition for the clustering effect of the SLOPE estimator in the orthogonal design.

Theorem 3.2. Consider a linear model with orthogonal design X ′ X = I p . Let π be a permutation of (1, 2, . . . , p) such that

β OLS π(1) ≥ β OLS π(2) ≥ . . . ≥ β OLS π(p) . For i ∈ {1, 2, . . . , p -1}, if β OLS π(i) -β OLS π(i+1) ≤ λ i -λ i+1 , then β SLOPE π(i) = β SLOPE π(i+1) .
Proof. By Lemma 3.1, in the orthogonal design, the calculation of SLOPE reduces to the application of the proximal algorithm of SLOPE to the βOLS . The result may be inferred from [START_REF] Bogdan | SLOPE -Adaptive Variable Selection Via Convex Optimization[END_REF]Lemma 2.3].

β 1 β 2 π * SLOPE OLS π * SLOPE OLS π * SLOPE OLS (-2,1) (-1,2) (1,2) (2,1) (2,-1) (1,-2)
(-1,-2) (-2,-1) (-1,0) (-1,1) (0,1) (1,1)
(1,0)

(1,-1) (0,-1) (-1,-1) (0,0) In the following theorem we derive necessary and sufficient conditions under which SLOPE in the orthogonal design recovers the support of the vector β = (β 1 , . . . , β p ) ′ , i.e.

β SLOPE i = 0 ⇐⇒ β i = 0.
Theorem 3.3. Under orthogonal design, let π be a permutation of (1, 2, . . . , p) satisfy-

ing β OLS π(1) ≥ | β OLS π(2) | ≥ . . . ≥ | β OLS π(p) |.
Without loss of generality suppose that supp(β) = {1, 2, . . . , p 0 } with p 0 < p. The necessary and sufficient condition for SLOPE to identify the set of relevant covariables is the following:

(a) min 1≤i≤p0 β OLS i > max p0+1≤i≤p β OLS i , (b) p0 i=k β OLS π(i) > p0 i=k λ i , for k = 1, 2, . . . , p 0 , (c) k i=p0+1 β OLS π(i) ≤ k i=p0+1 λ i , for k = p 0 + 1, p 0 + 2, . . . , p.
Proof. The result may be inferred from the properties of the proximal SLOPE [3, Lemma 2.3 and Lemma 2.4] and from Lemma 3.1.

Asymptotic properties of SLOPE

In this section we discuss several asymptotic properties of SLOPE estimators in the lowdimensional regression model in which p is fixed and the sample size n tends to infinity. For each n ≥ 1 we consider a linear model

Y (n) = X (n) β + ε (n) , (4)
where

Y (n) = (y 1 , y 2 , . . . , y n ) ′ ∈ R n is a vector of observations, X (n) ∈ R n×p is a determin- istic design matrix with rank(X (n) ) = p, β = (β 1 , β 2 , . . . , β p ) ′ ∈ R p is a vector of unknown regression coefficients and ε (n) = (ε (n) 1 , ε (n) 2 , . . . , ε (n)
n ) ′ ∈ R n is a noise term, which has the normal distribution N (0, σ 2 I n ). We make no assumptions about the dependence between ε (n) and ε (m) for n ̸ = m. In particular, ε (n) does not need to be a subsequence of ε (m) .

When defining the sequence ( β

SLOP E n

) of SLOPE estimators, we assume that the tuning vector varies with n. More precisely, for each n ≥ 1 its coefficients λ

(n) 1 ≥ λ (n) 2 ≥ . . . ≥ λ (n) p ≥ 0 are fixed and λ (n) 1 > 0. By β SLOP E n
we denote the SLOPE estimator corresponding to the tuning vector Λ (n) = (λ

(n) 1 , . . . , λ (n) p ) ′ : β SLOP E n = arg min b∈R p 1 2 Y (n) -X (n) b 2 2 + J Λ (n) (b) . (5)

Strong consistency of the SLOPE estimator

Below we discuss consistency of the sequence ( β

SLOP E n

) of SLOPE estimators, defined by (5).

Theorem 4.1. Consider the linear regression model ( (4)) and assume that

lim n n -1 X (n) ′ X (n) = C,
where C is a positive definite matrix. Let β

SLOP E n

, n ≥ 1, be the SLOPE estimator corresponding to the tuning vector

Λ (n) = (λ (n) 1 , λ (n) 2 , . . . , λ (n) p ) ′ . (a) If lim n→∞ λ (n) 1 n = 0, then β SLOP E n a.s. -→ β. (b) If lim n→∞ λ (n) 1 n = λ 0 > 0, then β SLOP E n
is not strongly consistent for β.

Before proving the above theorems we start with stating a simple technical lemma. It follows quickly from the Borel-Cantelli Lemma and the well-known tail inequality: If Z ∼ N(0, 1), then P(Z > t) ≤ t -1 e -t 2 /2 / √ 2π, t > 0.

Lemma 4.2. Assume that (Q n ) n∈N is a sequence of Gaussian random variables, defined on the same probability space, which converges in distribution to N(0, σ 2 ) for some σ ∈ (0, ∞). Then, for any δ > 0,

lim n→∞ Q n (log(n)) 1/2+δ = 0 a.s.
Our proof of the strong consistency of SLOPE is based on the strong consistency of the OLS estimator. The latter result is a folklore and we prove it in our setting. If

lim n n -1 (X (n) ) ′ X (n) = C, where C is positive definite, then β OLS n a.s.
-→ β.

Proof. We have

β OLS n -β = ((X (n) ) ′ X (n) ) -1 (X (n) ) ′ Y (n) -β = ((X (n) ) ′ X (n) ) -1 (X (n) ) ′ ε (n) . Then √ n β OLS n -β has the normal distribution N (0, (n -1 (X (n) ) ′ X (n) ) -1
) and its components satisfy the assumptions of Lemma 4.2. Since log(n)

∈ C (Λ (n) ) such that β SLOP E n = ((X (n) ) ′ X (n) ) -1 ((X (n) ) ′ Y (n) -π * n ). Since π * n takes values in C Λ (n) , it follows that ∥π * n ∥ ∞ ≤ λ (n) 1 . Hence, π * n n a.s. -→ 0, (6)
because π * n n ∞ ≤ λ (n) 1 n → 0.
The assumption that rank(X (n) ) = p implies that the matrix (X (n) ) ′ X (n) is invertible and hence the least squares estimator of β is unique and has the form

β OLS n = ((X (n) ) ′ X (n) ) -1 (X (n) ) ′ Y (n) .
Combining with (( 6)) the fact that β OLS n a.s.

-→ β, we conclude that

β SLOP E n = ((X (n) ) ′ X (n) ) -1 ((X (n) ) ′ Y (n) -π * n ) = β OLS n -((X (n) ) ′ X (n) ) -1 π * n = β OLS n - (X (n) ) ′ X (n) n -1 π * n n a.s. -→ β -C -1 0 = β. (b) Since β SLOP E n minimizes over b ∈ R p the function l(b) := 1 2 ∥Y (n) -X (n) b∥ 2 2 + J Λ (n) (b)
and since λ

(n) 1 ∥b∥ ∞ ≤ J Λ (n) (b), it follows that 0 ≤ l(0) -l( β SLOP E n ) = ( β SLOP E n ) ′ (X (n) ) ′ Y (n) - 1 2 ( β SLOP E n ) ′ (X (n) ) ′ X (n) β SLOP E n -J Λ (n) ( β SLOP E n ) ≤ ( β SLOP E n ) ′ (X (n) ) ′ Y (n) - 1 2 ( β SLOP E n ) ′ (X (n) ) ′ X (n) β SLOP E n -λ (n) 1 ∥ β SLOP E n ∥ ∞ = ( β SLOP E n ) ′ (X (n) ) ′ X (n) β OLS n - 1 2 ( β SLOP E n ) ′ (X (n) ) ′ X (n) β SLOP E n -λ (n) 1 ∥ β SLOP E n ∥ ∞ .
Suppose to the contrary that β SLOP E n a.s.

-→ β. Then, using the facts that β OLS n a.s.

-→ β and that lim

n n -1 (X (n) ) ′ X (n) = C, we have 0 ≤ l(0) -l( β SLOP E n ) n a.s. -→ β ′ Cβ - 1 2 β ′ Cβ -λ 0 ∥β∥ ∞ = 1 2 β ′ Cβ -λ 0 ∥β∥ ∞ .
For λ 0 > 0 this provides a contradiction since the inequality λ 0 ∥β∥ ∞ ≤ 1 2 β ′ Cβ does not hold when the value of β is sufficiently close to 0.

Asymptotical pattern recovery in the orthogonal design

We again consider a sequence of linear models (( 4)) but this time we assume that for each n the deterministic design matrix

X (n) of size n × p satisfies (X (n) ) ′ X (n) = nI p . (7)
As usual, we assume Gaussian errors, ε (n) ∼ N(0, σ 2 I n ).

Let

β SLOP E n = β SLOPE 1 (n), . . . , β SLOPE p (n)
′ be the SLOPE estimator defined by [START_REF] Bondell | Simultaneous factor selection and collapsing levels in ANOVA[END_REF]. With the above notation we present the main result of this section.

Theorem 4.3. Assume that lim n→∞ λ (n) 1 n = 0
and that there exists δ > 0 such that

lim inf n→∞ λ (n) i -λ (n) i+1 √ n (log(n)) 1/2+δ = m > 0 for i = 1, . . . , p -1. (8)
Then we have patt(

→ patt(β).

Proof. Without loss of generality we may assume that β = (β 1 , . . . , β p ) ′ and β 1 ≥ β 2 ≥ . . . ≥ β p ≥ 0. Indeed, we can always achieve such condition by permuting the columns of X (n) and changing their signs. Since the space of models is discrete, we have to show that for large n, patt( βSLOPE n ) = patt(β) a.s. We divide the proof into the following four parts:

(a)

β i = β j > 0 =⇒ β SLOPE i (n) = β SLOPE j (n) a.s. for large n, (b) β i > β i+1 =⇒ β SLOPE i (n) > β SLOPE i+1 (n) a.s. for large n, (c)
β i = 0 =⇒ β SLOPE i (n) = 0 a.s. for large n, (d) β i > 0 =⇒ β SLOPE i (n) > 0 a.s. for large n.
The points (b) and (d) follow quickly by the strong consistency of βSLOPE (n). To prove (a) and (c) we reduce the problem to the orthogonal design case. We have

arg min b∈R p [ 1 2 ∥Y (n) -X (n) b∥ 2 2 + J Λ (n) (b) ] = arg min b∈R p [ 1 2 ∥( Y (n) ) ′ -( X (n) ) ′ b∥ 2 2 + J Λ (n) (b) ],
where

Y (n) = Y n / √ n, X (n) = X (n) / √ n and Λ (n) = Λ (n) /n. Clearly, (7) implies that ( X (n)
) ′ X

(n)

= I p , which allows to use results from the orthogonal design. However, we note that the OLS estimators β

OLS n = ( β OLS 1 (n), . . . , β OLS p (n)
(n) = X (n) β + ε (n) / √ n. Let π n be a permutation of (1, 2, . . . , p) satisfying | β OLS πn(1) (n)| ≥ | β OLS πn(2) (n)| ≥ . . . ≥ | β OLS πn(p) (n)|.
By the strong consistency of the OLS estimator, taking n sufficiently large, we may ensure that the clusters of β do not interlace in β OLS n in the sense that if

β i > β j , then β OLS i (n) > β OLS j
(n) a.s. for n sufficiently large. Let us now consider point (i). Let S i denote the cluster containing β i > 0, that is, the set S i = {j ∈ {1, . . . , p} : β j = β i }. In view of the ordering of β, there exists k i ∈ {1, . . . , p} such that S i = {π n (j) : j ∈ {k i , k i + 1, . . . , k i + #S i -1}} .

We will show that if π n (k), π n (k + 1) ∈ S i , then for large n

β SLOPE πn(k) (n) = β SLOPE πn(k+1) (n) a.s., (9)
which implies that

β SLOPE j (n) = β SLOPE k
(n) for j, k ∈ S i and finishes the proof of (a). Now assume that π n (k), π n (k + 1) ∈ S i . Then, by Theorem 3.2, the condition ( 9) is satisfied if

β OLS πn(k) (n) -β OLS πn(k+1) (n) ≤ Λ (n) k -Λ (n) k+1 = 1 n λ (n) k -λ (n) k+1 (10)
holds for large n and both β OLS πn(k) (n) and β OLS πn(k) (n) have the same sign. The latter is ensured by the strong consistency of the OLS estimator and the fact that β i > 0.

If π n (k), π n (k + 1) ∈ S i , then we have the following bound

β OLS πn(k) (n) -β OLS πn(k+1) (n) ≤ j∈Si β OLS j (n) -β OLS i (n) . (11)
Take any j ∈ S i . Since both β OLS j (n) and β OLS i (n) have the normal distribution with the same mean, by Lemma 4.2, we have

lim n→∞ √ n β OLS j (n) -β OLS i (n) (log(n)) 1/2+δ = 0 a.s.
In view of ( 11) and ( 8), this implies that (10) holds true for large n. Hence, (a) follows.

It remains to establish (c). Assume that β p0 > 0 = β p0+1 = . . . = β p . Clearly, condition (a) from Theorem 3.3 is satisfied thanks to the strong consistency of the OLS estimator. For (b), we have for k = 1, 2, . . . , p 0 ,

p0 i=k Λ (n) i = 1 n p0 i=k λ (n) i ≤ p 0 λ (n) 1 n ,
which converges to 0. On the other hand, the left-hand side of (b) converges a.s. to p0 i=k β i , which is positive. Thus, condition (b) from Theorem 3.3 holds for large n. Condition (c) from Theorem 3.3 follows from Lemma 4.2. Indeed, we have for δ > 0 and k = p 0 + 1, . . . , p,

lim n→∞ √ n (log(n)) 1/2+δ k i=p0+1 | β OLS πn(i) (n)| = k i=p0+1 lim n→∞ | √ n β OLS πn(i) (n)| (log(n)) 1/2+δ = 0 a.s., while lim n→∞ √ n (log(n)) 1/2+δ k i=p0+1 Λ (n) i ≥ k i=p0+1 lim n→∞ λ (n) i -λ (n) i+1 √ n(log(n)) 1/2+δ = m > 0
Thus, all assumptions of Theorem 3.3 are verified and the proof is complete.

Applications and simulations

Below we present an application of SLOPE in signal denoising. In our example X ∈ R 300×100 is an orthogonal system of trigonometric functions, i.e. X i,(2 * j-1) = sin(2πij/n) and X i,(2 * j) = cos(2πij/n) for i = 1, . . . , 100 and j = 1, . . . , 150. Here β ∈ R p is a vector consisting of two clusters: 20 coordinates with absolute value 100 and 20 coordinates with absolute value 80. To avoid large bias caused by the shrinkage nature of LASSO and SLOPE, we debias them by combining with the Ordinary Least Squares method in the following way. Firstly we recover the support and clusters of a true vector β and secondly we apply the OLS method to the regression model Y = XU β + ε, where the rows of U ∈ R n×2 are defined as

U i,• =      [sign( βSLOP E i ), 0], 1 ≤ i ≤ k, [0, sign( βSLOP E i )], k + 1 ≤ i ≤ 2k, [0, 0],
else. We compare the Mean Square Error and the signal denoising of the classical OLS estimation, the LASSO with the tuning parameter λ cv minimizing the cross-validated error, the denoised version of LASSO with λ = 5λ cv and the denoised version of SLOPE with the tuning vector Λ chosen with respect to the scaled arithmetic sequence (λ i = 3.5(p -1)). We also compare the Mean Square Error of the SLOPE estimator with others.

A Appendix

Proof of Lemma 2.1. It follows that the function g : This implies (π -π * ) ′ β * ≤ 0, for all π ∈ C Λ , because

C Λ → [0, ∞) defined by g(π) := ( βOLS -π) ′ ( βOLS -π)
∇g(π * ) = -2( βOLS -π * ) = -2β * .
In the proof of parts (a), (b) and (c) we use the fact that π * maximizes π ′ β * over π ∈ C Λ To prove part (a) suppose that sign(β * i ) • sign(π * i ) < 0 for some i and define

π = (π * 1 , . . . , π * i-1 , -π * i , π * i+1 , . . . , π * p ) ′ .
Then we have (π

* ) ′ β * < π ′ β * , which is impossible since π ∈ C Λ .
To prove part (b), consider a permutation τ of ( Proof of Lemma 3.1. Let H = X(X ′ X) -1 X ′ be a projection matrix on Col(X) -the linear subspace spanned by the columns of X. Then ∥Y -Xb∥ Since the difference ∥Y -Xb∥ 2 2 -∥ β OLS -b∥ 2 2 does not depend on b, it follows that the equality ((3)) holds.

  k} is a rank of |b i | in a vector of distinct nonzero values among {|b 1 |, . . . , |b p |}. We adopt the convention that rank(0) = 0. As M p we denote the set of all possible SLOPE patterns of b ∈ R p . Fact 1.0.1 (Basic properties of SLOPE pattern). (a) for every 1 ≤ l ≤ ∥patt(b)∥ ∞ there exists j such that |patt(b) j | = l, (b) sign(patt(b)) = sign(b) (sign preservation),(c)

Proposition 2 .0. 1 . [ 3 ,

 213 Proposition 1.1] Let C Λ be the unit ball in the dual SLOPE norm. Then, for each b ∈ R p , J Λ (b) = max π∈C Λ π ′ b.

Lemma 3 . 1 . 2 +

 312 [START_REF] Bogdan | Statistical estimation and testing via the sorted L1 norm[END_REF] Equation (1.14)] In the linear model with orthogonal design X ′ X = I p we have J Λ (b) .

Figure 1 :

 1 Figure 1: The dual unit ball C Λ for Λ = (4, 2) ′ and examples of βSLOP E and βOLS in the orthogonal design for n ≥ p = 2. The labels of each colored set refer to the pattern of βSLOP E for βOLS lying in this set. The arrows point from ( βOLS -βSLOP E ) to βOLS .

Proposition 4 . 2 . 1 .

 421 Consider the linear regression model ( (4)).

  ) are the same in the original model and its scaled version Y

Figure 3 :

 3 Figure 3: Comparison of signal denoising by OLS, LASSO, debiased LASSO and debiased SLOPE (respective images from left) on the coordinates [120, 125] of the regression model Y = Xβ +ε. The black lines correspond to the true values of Xβ. The red lines correspond to the estimators Y = X β + ε.

Figure 4 :

 4 Figure 4: Signal denoising by debiased SLOPE on all coordinates of the regression model Y = Xβ + ε. The (almost overlapping) black line and the red line correspond respectively to the true values of Xβ and to Y = X βSLOP E + ε.

Figure 5 :

 5 Figure 5: Pattern recovery by LASSO (left image) and by debiased SLOPE (right image) in the same setting as above. The horizontal lines correspond to the true values of β. As one may observe, in the presented setting LASSO does not recover the true support, while debiased SLOPE perfectly recovers support, sign and clusters.

  If |β * σ(k-1) | > |β * σ(k) | then π ′ β * = (π * ) ′ β * + δ(|β * σ(k-1) | -|β * σ(k) |) > (π * ) ′ β * ,which is impossible.Proof of Lemma 2.2. At first we note that for allπ ∈ C Λ r(β * , π) = 1 2 ∥Y -Xβ * ∥ 2 2 + π ′ β * = 1 2 ∥Y -Xβ * ∥ 2 2 + (π * ) ′ β * +(π -π * ) ′ β * = r(β * , π * ) + (π -π * ) ′ β * ≤ r(β * , π * ),where the last inequality follows from the fact that (π -π * ) ′ β * ≤ 0, for all π ∈ C Λ , see the proof of 2.1. Therefore, max π∈C Λ r(β * , π) = r(β * , π * ). Moreover, from the definition of the point β * it is seen that r(β * , π * ) = min β∈M r(β, π * ). These two facts imply that min β∈M max π∈C Λ r(β, π) ≤ max π∈C Λ r(β * , π) = r(β * , π * ) , π) (by the max-min inequality), we have the equality throughout. This completes the proof.

2 2 = 2 = 2 = 2 = 2 ,

 22222 ∥(I p -H)Y + HY -Xb∥ 2 2 = ∥(I p -H)Y + H(Y -Xb)∥ 2 ∥(I p -H)Y ∥ 2 2 + 2 [(I p -H)Y ] ′ H(Y -Xb) + ∥H(Y -Xb)∥ 2 ∥(I p -H)Y ∥ 2 2 + 2Y ′ (I p -H)H(Y -Xb) + ∥H(Y -Xb)∥ 2 ∥(I p -H)Y ∥ 2 2 + ∥H(Y -Xb)∥ 2because H = H ′ , H(I p -H) = 0 and HX = X. In the orthogonal design we haveH = XX ′ and β OLS = X ′ Y , which yields ∥H(Y -Xb)∥ 2 2 = ∥X ′ Y -X ′ Xb∥ 2 2 = ∥ β OLS -b∥ 2 2 . Therefore ∥Y -Xb∥ 2 2 = ∥(I p -H)Y ∥ 2 2 + ∥ β OLS -b∥ 2 2 .

  Proof of Theorem 4.1. (a) It follows from Theorem 2.1 that there exists a vector π * n

	1/2+δ = o(	√	n), we get the
	assertion by Lemma 4.2.

Table 1 :

 1 Comparison of MSE between different regression methods.

	OLS	LASSO-CV LASSO-LS SLOPE-LS
	M SE(β, •) 613.6797	417.0071	171.7957	20.74967

  1, 2, . . . , n) such that (|π * τ (1) |, . . . , |π * τ (p) |) and (|β * 1 |, . . . , |β * p |) are similarly sorted. Define the point π = (s 1 • π * τ (1) , s 2 • π * τ (2) , . . . , s p • π * τ (p) ), where s i = sign(β * i ), for i = 1, 2, . . . , p. If (|π * τ (1) |, . . . , |π * τ (p) |) ̸ = (|π * 1 |, . . . , |π * p |), then, by the Hardy-Littlewood-Pólya rearrangement inequality,π ′ β * = , and that |π * σ(k) | > 0.In this case there is a sufficiently small δ > 0, such thatπ = (π * 1 , . . . , π * i-2 , π * i-1 + δs i-1 , π * i -δs i , π * i+1 , . . . , π * p ) ′ ∈ C Λ .

	p	p
	i=1	|π * τ (i) ||β * i | > i | ≥ (π k-1 i=1 |π * i ||β * i=1 |π * σ(i) | < k-1 i=1 λ i

* ) ′ β * , which is impossible since π ∈ C Λ .

Finally, to prove part (c), suppose that

Author was supported by a French Government Scholarship.


Marc Foletto

Vincent Puech

Jérôme Fontane

Laurent Joly

Léanne Pitchford

Evidence of the influence of plasma jets on a helium flow into open air

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Open Archive TOULOUSE Archive Ouverte (OATAO)

OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible. This is an author-deposited version published in : http://oatao.univ-toulouse. T HE experiments reported here were conducted using a microdischarge configuration, in which a helium jet impinging on open air at atmospheric pressure is generated by flowing helium at several liters per minute through a dielectric tube with an inner diameter of 2.35 mm. A plasma is initiated by applying a positive voltage (5.5 kV, 20 kHz, ∼100 ns rise time, and ∼250 ns pulse width) to a hollow electrode (1.6-mm inner diameter) centered inside the dielectric. The dielectric extends 6 mm beyond the end of the electrode. The photograph in the far right in Fig. 1 shows a typical plasma jet. The plasma appears continuous when viewed by eye, but rapid imaging techniques reveal that the plasma is reignited at each voltage pulse. Most of the light emitted by the plasma jet is produced by an ionization wave propagating in the helium-rich region past the exit of the tube. The air surrounding the helium flow is much more difficult to ionize, and so the propagation of the ionization wave is confined to the volume of the helium channel. The length of the plasma jet is determined by the length of the helium channel and the applied voltage waveform. The plasma so generated has been referred to variously as a plasma bullet, a plasma jet, and as a guided streamer [START_REF] Lu | On atmospheric-pressure nonequilibrium plasma jets and plasma bullets[END_REF], [START_REF] Boeuf | Dynamics of a guided streamer ('plasma bullet') in a helium jet in air at atmospheric pressure[END_REF].

Because of the significant difference in mass densities of helium and air (a factor of about 7), Schlieren imaging can be used to easily visualize the helium flow into open air. This is an optical technique that converts variations of refractive index caused by spatial gradients in concentration profiles into spatial variations in the intensity of a collimated light source traversing the medium.

Schleiren images of the helium channel are shown in Fig. 1 without (upper row) and with (lower row) ignition of plasma jets for different Reynolds number, Re, in the conditions defined above. Emission from the plasma is not observable. The Reynolds number of the flow at the exit plane of the dielectric tube is defined as Re = V∅ int /ν, where V is the mean helium flow velocity, ν is the helium kinematic viscosity, and Ø int is the inner diameter of the dielectric. From left to right, the Reynolds number increases from 150 (2 L/min flow rate) to 550 (7.3 L/min flow rate). The helium flow is laminar in the tube for these values of Re and remains laminar for a certain distance past the tube exit. (The presence of the electrode inside the tube could influence the laminarity of the helium flow past the tube exit, but for our geometry, this is not the case.) In the laminar zone, there is little mixing of helium with air, and the radial components of the helium flow velocity are almost zero, as evidenced by the small opening angle of the helium column. The flow becomes turbulent past some transition point that is indicated by the red symbols in the panels in Fig. 1, and thereafter mixing is rapid. The one shot images in the right on each row are photographs taken with an exposure time of 10 ms, and these show the presence of vortices not distinguishable in the images that have been averaged over 100 shots (for a more precise identification of the transition point). As expected, the transition point moves closer to the tube with increasing Re. For low Re (<360), the generation of the plasma jet causes a significant decrease in the length of the laminar zone, a phenomenon first reported in [START_REF] Oh | Imaging gas and plasma interactions in the surface-chemical modification of polymers using micro-plasma jets[END_REF] and [START_REF] Jiang | Interplay of discharge and gas flow in atmospheric pressure plasma jets[END_REF]. For higher Re, where the transition occurs naturally closer to the dielectric, the influence of the plasma is less. Other effects of the plasma ignition on the helium flow have been discussed recently in [START_REF] Robert | Rare gas flow structuration in plasma jet experiments[END_REF].

One possible reason for the influence of the plasma on the helium column is via the momentum imparted by the ions to the neutrals in the advancing sheath region near the surface inside the dielectric tube, where the power deposition is highest. This effect of the ion wind is well known from studies of surface discharges in air for flow control [START_REF] Unfer | Modeling and comparison of sinusoidal and nanosecond pulsed surface dielectric barrier discharges for flow control[END_REF].

  fr/ Eprints ID : 11926 To link to this article : DOI:10.1109/TPS.2014.2331393 URL : http://dx.doi.org/10.1109/TPS.2014.2331393 To cite this version : Foletto, Marc and Puech, Vincent and Fontane, Jérôme and Joly, Laurent and Pitchford, Leanne Evidence of the influence of plasma jets on a helium flow into open air. (2014) IEEE Transactions on Plasma Science . ISSN 0093-3813 Any correspondance concerning this service should be sent to the repository administrator: staff-oatao@listes-diff.inp-toulouse.fr Evidence of the Influence of Plasma Jets on a Helium Flow into Open Air Marc Foletto, Vincent Puech, Jérôme Fontane, Laurent Joly, and Leanne C. Pitchford Microplasma jets propagating in a helium flow surrounded by open air at ambient temperature have attracted the attention of many researchers for their putative ability to deliver high fluxes of reactive oxygen and nitrogen species to a target situated some centimeters away. This is of particular interest in biomedical applications. In this paper, we use Schlieren images to show that ignition of the plasma jet causes a reduction in the length of the laminar zone of the helium flow. Atmospheric-pressure plasmas, hydrodynamics.

Fig. 1 .

 1 Fig. 1. Schlieren visualization of a helium flow exiting a dielectric tube (2.35-mm inner diameter) and impinging on open air, as a function of helium flow rate, expressed in terms of the Reynolds number, Re, in the tube. The tube is positioned vertically and can be seen in the shadow in the lower part of each image. The images in the upper row correspond to cases without plasma ignition, whereas the lower panels correspond to cases where a plasma jet is present. The laminar-to-turbulent flow transition point is indicated with red symbols (the transition point is outside the field of view for the two images in the upper left).