V Viallon
email: viallon@math.univ-lyon1.fr

S Lambert-Lacroix

H Hoefling
email: hhoeflin@gmail.com

F Picard

Adaptive Generalized Fused-Lasso: Asymptotic Properties and Applications

. Finally we present an original application of the Generalized Fused-Lasso to the Joint Modeling framework where the design itself suggests the graph to be used in the penalty; an illustration is provided on road safety data.

Introduction

This paper deals with the general framework of regression that aims at unraveling relationships between a response variable Y and a vector of p covariates or features x ∈ R p . From a practical standpoint, a good comprehension of this relationship is notably useful for prediction matters but also to understand the dynamics of variable Y itself that may be driven by some influential components of x. In Statistics, a natural way to study the relationship between Y and x is to first assume that this relationship can be correctly approximated by some simple model like Generalized Linear Models [START_REF] Mccullagh | Generalized Linear Models[END_REF]) and then estimate the vector of parameters β of this model. Estimation is generally performed by maximizing the log-likelihood of the model, yielding Maximum Likelihood Estimators (MLEs). Under mild conditions, these estimators, and in turn, prediction based on these estimators, enjoy good statistical properties (see, e.g., [START_REF] Fahrmeir | Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models[END_REF]). However, their variance are increasing functions of the number of parameters, which scales as p for the models mentioned above. In situations where only a fraction of the components of x are expected to be relevant, model selection then becomes desirable: in addition to yield more interpretable models, it reduces the effective number of parameters to be estimated, hence the variance of both estimates and predictions. Another popular approach is to maximize a penalized version of the log-likelihood. Penalties based on ℓ q norms of parameter β, q ≥ 0, lead to shrunk estimates: absolute values of the estimates are biased towards zero. By sacrificing some bias, the variance of the predicted values can be decreased and shrinkage may improve the overall prediction accuracy. Moreover, penalties based on ℓ q norms of the parameters for q ≤ 1 encourage sparsity in the vector of parameters, hence model selection. They are methods of choice to return interpretable models that can enjoy good prediction properties.

The most famous example of such a method is the "Least Absolute Shrinkage and Selection Operator" (Lasso) which was originally based on a penalization of the leastsquares criterion using the ℓ 1 -norm of the parameters [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF]. If considering a vector of parameters β = (β 0 , . . . , β p ), with β 0 the intercept term, the original Lasso penalty is pen (β) = λ p j=1 |β j |, with λ some tuning parameter. Since then, it has been extended to Generalized Linear Models ( [START_REF] Van De Geer | High-dimensional generalized linear models and the lasso[END_REF][START_REF] Friedman | Regularization paths for generalized linear models via coordinate descent[END_REF]. The statistical properties of the Lasso have been extensively investigated, both in the lowdimensional (fixed p) and high-dimensional (diverging p or even p ≥ n) cases [START_REF] Knight | Asymptotics for Lasso-type estimators[END_REF][START_REF] Fan | Variable selection via nonconcave penalized likelihood and its oracle properties[END_REF][START_REF] Leng | A note on lasso and related procedures in model selection[END_REF][START_REF] Wang | Robust regression shrinkage and consistent variable selection via the LAD-LASSO[END_REF][START_REF] Zhao | On model selection consistency of lasso[END_REF][START_REF] Bunea | Sparsity oracle inequalities for the lasso[END_REF][START_REF] Bickel | Simultaneous analysis of lasso and dantzig selector[END_REF]. For instance, under some conditions, the Lasso enjoys sparsistency, i.e. model selection consistency: it selects the right components of vector x with high probability. However, even in the low-dimensional case, [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF] stated that non trivial conditions on the Gram matrix were necessary to ensure the Lasso sparsistency. This condition is closely related to the irrepresentable condition introduced in [START_REF] Zhao | On model selection consistency of lasso[END_REF], or the restricted eigenvalue condition of [START_REF] Bickel | Simultaneous analysis of lasso and dantzig selector[END_REF]. We also refer the reader to Section 6.13 in [START_REF] Bühlmann | Statistics for High-Dimensional Data: Methods, Theory and Applications[END_REF] for more details about these conditions. This has motivated some (inner) modifications of the Lasso itself. [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF] proposed the Adaptive Lasso with penalty pen (β, w) = λ p j=1 |β j |/w j which uses adaptive weights w j for a differential penalization in the ℓ 1 norm. In the fixed p setting, [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF] further showed that the Adaptive Lasso enjoys an asymptotic oracle property with no particular assumption on the design matrix: as n grows to infinity, it identifies the correct subset model with probability tending to one and estimates of non-zero components perform as well as if the true underlying model were given in advance.

Another way to improve the Lasso is to consider additional terms in the penalty. For instance the Elastic-Net approach proposed by [START_REF] Zou | Regularization and variable selection via the elastic net[END_REF] consists in adding a quadratic penalty term to the initial ℓ 1 penalty such that pen(β) = λ 1

p j=1 |β j | + λ 2 p j=1 β 2 j .
The Elastic-Net is particularly desirable in the presence of highly correlated features. Extensions of the Elastic-Net have been introduced to handle structured features. For instance, the Smooth Lasso (S-Lasso) relies on an ℓ 2 -fusion penalty such that pen (β) = λ 1 j |β j | + λ 2 j>1 (β jβ j-1 ) 2 [START_REF] Land | Variable fusion: a new method of adaptive signal regression[END_REF] (this is an extension of the Elastic-Net because it can be shown that j>1 (β jβ j-1 ) 2 = β T J T Jβ =: β 2 J , for some matrix J; see [START_REF] Hebiri | The smooth-lasso and other ℓ 1 + ℓ 2 penalized methods[END_REF]). It is appealing when features can be ordered and their coefficients vary smoothly. Adaptive versions of the Elastic-Net [START_REF] Ghosh | Adaptive elastic net: An improvement of elastic net to achieve oracle properties[END_REF][START_REF] Zou | On the adaptive elastic net with a diverging number of parameters[END_REF] and of the S-Lasso have also been proposed (El [START_REF] Anbari | On the adaptive grill estimator with diverging number of parameters[END_REF]. From a theoretical standpoint, these methods return estimates that enjoy sparsistency under weaker conditions than the ones required by the Lasso; see [START_REF] Yuan | On the non-negative garrotte estimator[END_REF]; [START_REF] Jia | On model consistency of the elastic net when p ≫ n[END_REF]; [START_REF] Hebiri | The smooth-lasso and other ℓ 1 + ℓ 2 penalized methods[END_REF]; El [START_REF] Anbari | On the adaptive grill estimator with diverging number of parameters[END_REF].

In fact, the S-Lasso was inspired by the Fused-Lasso of [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF] that has been proposed to enforce similarity between the effects of successive features by using penalty pen (β) = λ 1 j |β j | + λ 2 j |β jβ j-1 |. Theoretical properties of Fused-Lasso estimates were studied in the Gaussian sequence model (which can be reformulated as a linear regression model with the identity matrix as the design matrix, so that p = n for this model) by [START_REF] Rinaldo | Properties and refinements of the fused lasso[END_REF] and [START_REF] Qian | On pattern recovery of the fused lasso[END_REF]. [START_REF] Rinaldo | Properties and refinements of the fused lasso[END_REF] worked under the assumption that the variance of the noise vanishes as n grows to infinity and established oracle prediction inequalities for a modified version of the Lasso estimate (he called this version adaptive Fused Lasso but no adaptive weight are invoked). In [START_REF] Qian | On pattern recovery of the fused lasso[END_REF], the authors work under the assumption that the variance of the noise is constant and show that Fused-Lasso estimates are generally not able to recover the signal pattern. In the more general linear regression setting, [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF] established some asymptotic properties for the Fused-Lasso estimates, in the simpler case of fixed p. More recently, [START_REF] Vaiter | Robust sparse analysis regularization[END_REF] established the sparsistency of the method in the high-dimensional case under a bounded-noise assumption (and of course some assumption on the Gram matrix).

The Fused-Lasso has been generalized by [START_REF] Höfling | A coordinate-wise optimization algorithm for the Fused Lasso[END_REF] to handle complex structure among features effects. The motivation of the Generalized Fused-Lasso is that when available, prior information regarding the structure of features effects should be used to increase the selection and prediction performance. The procedure consists in using an external graph G defined by a set of p vertices V that stands for the p components of vector β * , and a set of edges E, such that two connected coefficients are supposed to vary smoothly. This leads to the Generalized Fused-Lasso penalty pen(β, G) = λ 1 j |β j | + λ 2 (j,ℓ)∈E |β jβ ℓ |. The algorithm proposed by [START_REF] Höfling | A coordinate-wise optimization algorithm for the Fused Lasso[END_REF] enables to introduce weights in the penalty term following the idea of the Adaptive Lasso proposed by [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF]. They give fast algorithms for solving the Adaptive Generalized Fused Lasso which are based on coordinate-wise optimization but do not present any theoretical results.

In this work we propose to investigate the theoretical as well as the empirical properties of the Adaptive Generalized Fused-Lasso. In Section 3 we first prove that the resulting estimator enjoys asymptotic oracle properties, as n goes to infinity and p is fixed, in the context of both linear and logistic regression models. Our asymptotic results constitute a milestone for future extension to the case of diverging p. Using simulation studies we show the empirical benefits of using a ℓ 1 -based fusion penalty on support recovery and prediction, compared with other strategies (Section 4). We also show the benefits of using adaptive weights and/or relaxation on Generalized Fused-Lasso estimates. Finally we propose in Section 5 an original application of the Generalized Fused-Lasso to the framework of Joint Modeling, where the design of the study provides the graph to be used in the penalty, and we apply this approach on road-safety data.

The Adaptive Generalized Fused-Lasso in GLMs

Model and loss functions

In this paper, we focus on two particular cases of generalized linear models [START_REF] Mccullagh | Generalized Linear Models[END_REF]: the usual linear regression model and the logistic regression model. We mention that theoretical results for other generalized linear models would follow from similar arguments. We decided to focus only on linear and logistic regression models because they are the only ones handled in the algorithms of [START_REF] Höfling | A coordinate-wise optimization algorithm for the Fused Lasso[END_REF], and also for the sake of clarity, since notations can become cumbersome when dealing with the whole class of generalized linear models. For i = 1, . . . , n, let Y i and x i = (x i1 , . . . , x ip ) T be the response variable and a p-dimensional vector of features (or covariates) respectively. We consider the case of a fixed design, i.e., Y i is a random variable but vectors of features x i are assumed to be fixed (non-random). Without loss of generality, we further assume that n i=1 x ij = 0. Finally define z i = (1, x T i ) T . The linear regression model is very standard, notably when the response variable is continuous. It writes

Y i = z T i β * + σǫ i
, where ǫ i denotes some random noise, and σ > 0 is fixed and unknown. The vector of coefficients β * ∈ R p+1 is unknown and has to be estimated. More precisely, β * = (β * 0 , β * 1 , . . . , β * p ) T where β * 0 denotes the intercept parameter and β * \0 = (β * 1 , . . . , β * p ) corresponds to the regression coefficients pertaining to the covariates. Under this model, estimation is generally performed by ordinary-least squares, which consists in minimizing the squared error loss J sq defined by

J sq (β) = n i=1 J sq (Y i , z T i β) = 1 2 n i=1 Y i -z T i β 2 , (1)
where function

J sq : (u, v) ∈ R 2 → (u -v) 2 /2 is introduced for future use.
When the response variable is binary (Y i ∈ {0, 1} is often called label or class), the logistic model is the most standard one. Introducing the logit function, logit(x) = log(x/(1x)), the class probability, or equivalently the expectation of Y i , is

π i = P(Y i = 1) = logit -1 (z T i β * ) = 1/(1 + exp(-z T i β * )))
, where logit -1 is the reciprocal of the logit function. Again, the vector β * ∈ R p+1 is unknown and has to be estimated. Estimation is usually performed by maximizing the log-likelihood of the model, which is equivalent to minimizing the logistic loss function J lo , given by

J lo (β) = n i=1 J lo (Y i , z T i β) = - n i=1 Y i z T i β -log 1 + exp(z T i β) , (2)
where function

J lo : (u, v) ∈ R 2 → -uv + log(1 + exp(v)
) is introduced for future use.

The Adaptive Generalized Fused-Lasso penalty

We focus on ℓ 1 -based Fused penalties inspired from the Fused-Lasso of [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF]. In this framework features (x 1 , . . . , x p ) correspond to p successive positions, and to account for sparsity in terms of successive differences, the original Fused penalty is defined by:

pen(β) = λ (1) n p j=1 |β j | + λ (2) n p j=2 |β j -β j-1 |,
and depends on two tuning parameters λ

(1) n and λ

(2)

n . Then this Fused framework has been generalized by [START_REF] Höfling | A coordinate-wise optimization algorithm for the Fused Lasso[END_REF] to the case of networks of features. This Generalized Fused penalty is of great interest when connected features along some network may have similar effects on the response. Consider a graph G = (V, E), with node set V that corresponds to the indices of coefficients in β \0 (i.e. V = {1, ...p}), and edge set E that corresponds to pairs of connected coefficients indices (j, ℓ) with j > ℓ. The graph G that is used in the penalty corresponds to some prior knowledge given by an expert, and hence is fixed. The Generalized Fused-Lasso penalty consists in penalizing all coefficient differences for which an edge exists in G:

pen(β; G) = λ (1) n j∈V |β j | + λ (2) n (j,ℓ)∈E |β j -β ℓ |. (3)
The Fused-Lasso penalty of [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF] corresponds to a Generalized Fused-Lasso penality based on a chain graph, where (j, ℓ) ∈ E if and only if ℓ = j -1 (see Figure 1 for a simple illustration).

Adaptive weights can further be introduced following the idea of the Adaptive Lasso proposed by [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF]. This results in the Adaptive Generalized Fused-Lasso penalty:

pen Ada (β; G, w) = λ (1)
n j∈V w

(1)

j |β j | + λ (2) n (j,ℓ)∈E w (2) jℓ |β j -β ℓ |.
where w

(1) j and w

(2)

jℓ are weights vectors. In this paper (where p is fixed) they are based on initial Maximum-Likelihood (ML) estimates β of β * . More precisely, they are set to w

(1) j = | β j | -γ and w (2) jℓ = | β j -β ℓ | -γ
, where γ is some fixed positive constant. The rationale is to penalize more heavily coefficients (or differences of coefficients) when their ML estimates are small: the higher γ, the more trust is put into the ML estimates. A typical value for γ is 1 (value that is used in our simulations and applications). The Adaptive Generalized Fused criterion Q is then simply defined, for given graph G and weights w, as

Q(β) = J(β) + pen Ada (β; G, w), (4)
where the loss function is J = J lo or J = J sq depending on the working model (see equations ( 1) and (2) above).

Application of the Generalized Fused penalty to Joint Modeling

Interestingly, another source of knowledge concerning features structure can be provided by the design of the study itself. In this work, we propose an original application of the Generalized Fused-Lasso framework to the case of joint estimation of multiple sparse regression models. The joint modeling framework described here has some connections with seemingly unrelated regression problems and multi-task learning (see [START_REF] Huang | A selective review of group selection in high dimensional models[END_REF] and the references therein). More specifically we consider the very common case of data collected from distinct strata, which often arises in epidemiology where each stratum can be defined by crossing gender, age and ethnicity for instance. The design is structured according to a known (and fixed) categorical vector (C 1 , . . . , C n ) taking values in {1, . . . , C}, with C ≥ 1 the total number of strata. Let n c be the number of observations falling into stratum c (so that n = c n c ). Without loss of generality, we further assume that the n observations are ordered so that the first n 1 observations correspond to stratum 1 (that is C i = 1 for i = 1, . . . , n 1 ), the next n 2 observations correspond to stratum 2, and so forth.

In the case of a linear regression model, we would have

Y i = z T i β * C i + σǫ i for i = 1, . . . , n,
where β * c , c = 1, . . . , C, denotes the vector of parameter for stratum c. The purpose of the analysis is to determine whether the distribution of the response varies across strata,

Chain Graph

Clique Graph

Star Graph

β 1 β 2 β 3 β 4 β 5 β 6 j = 0 j = 1 j = 2 j = 0 j = 1 j = 2
Figure 1: Typical examples of graphs used in the Generalized Fused-Lasso penalty. (Left)

The chain graph is typically used when covariates are naturally ordered as is the case for CGH array. This example illustrates the situation where p = 6. (Middle) The clique graph is typically used in the joint modeling context (p = 2 and C = 4 in this example).

(Right) The star graph is typically used in the joint modeling context, when one stratum serves as the reference (p = 2 and C = 4 in this example).

i.e. to detect which components of β * c do vary with c. Constructing independent (possibly sparse) models for each stratum would not take advantage of the common structure, while constructing a single model for the whole data set would mask the differences. Alternatively, the Generalized Fused-Lasso can be used to couple estimations obtained from each stratum, encouraging them to share a common structure. More precisely, we propose the following penalty:

C c=1 λ (1) n p j=1 w (1) j |β c,j | + λ (2) n p+1 j=0 c 1 >c 2 w (2) c 1 ,c 2 ,j |β c 1 ,j -β c 2 ,j |,
where w

(1) j and w

(2) c 1 ,c 2 ,j are appropriate adaptive weights. Parameter λ

(2) n governs the amount of shrinkage for differences between strata: if null, this penalty resumes to C independent Lasso penalties. If positive, the Fused part of the penalty encourages coefficients β c 1 ,j and β c 2 ,j to be at least close to each other (that is, the j th coefficient in strata c 1 and c 2 respectively). In Appendix we give more details on particular aspects of Joint modeling: we show that the general context described here induces a clique graph in the penalty, while the case of designs with a control stratum induces a star graph instead (see Figure 1 for a simple illustration).

Theoretical results

In this section, we study the asymptotic properties of the Adaptive Generalized Fused-Lasso estimator in both the Gaussian and logistic regression contexts. More precisely, we assume that p is fixed and let n grow to infinity. The case where the dimension p may grow as the sample size n increases is much more tricky and is left for future research.

As mentioned above, our results readily extend to other generalized linear models, under the same assumptions as AL1 and AL2 below, and using the same arguments as the ones used in our proofs (see Sections 7.2 and 7.4 in the Appendix). Before stating our results some notations and assumptions need to be introduced. Let A = {1 ≤ j ≤ p, β * j = 0} be the support of β * \0 and p 0 = |A| its cardinality (i.e., the true vector of regression parameters β * \0 is assumed to have p 0 non-zero elements). Further consider the set B = {(j, ℓ) ∈ E, β * j = 0 and β * j = β * ℓ }. For future use, note that B ⊂ A × A. For any integer k, we denote by 0 k , 1 k and I k the vector of 0's and 1's in R k and the identity matrix of size k. For any subset S ⊆ {1, . . . , p}, we denote by S its complement in {1, . . . , p}, and for any vector β ∈ R p+1 we further denote by β S the vector given by the coordinates of β the index of which are in {0} ∪ S. Moreover, we denote by • the usual Euclidian norm. For any x ∈ R, we define the function sign(x) which equals +1 if x > 0, -1 if x < 0 and 0 if x = 0. We also introduce

Z = [z 1 , . . . , z n ] T , X = [x 1 , . . . , x n ] T and Y = [Y 1 , . . . , Y n ] T .
Finally, for any (possibly random) event Ω, we denote by I(Ω) the indicator function which equals 1 if Ω is true and 0 otherwise. In the Gaussian context, we work under the following assumptions.

• A1 Variables ǫ 1 , . . . , ǫ n are independent identically distributed random variables with mean 0 and variance 1;

• A2 Matrix Z T Z/(nσ 2 ) converges to C
where C is a positive definite matrix.

In the logistic regression context we denote by I(β) the empirical Fisher's matrix of size (p + 1) × (p + 1). For future use, observe that I(β * ) = Z T DZ, where D denotes the n × n diagonal matrix with i-th diagonal element given by π i (1π i ). For any δ ≥ 0, we further denote by N n (δ) the neighborhood of β * defined by

N n (δ) = β ∈ G/ I(β) -1 2 T (β -β * ) ≤ δ .
We will work under the following conditions:

• AL1 I(β * )/n converges to C where C is a positive definite (p + 1) × (p + 1) matrix;

• AL2 As n goes to ∞, max β∈Nn(δ) I(β) -1 2 I(β * ) I(β) -1 2 T -I p+1 → 0.
Assumptions AL1 and AL2 are standard when working under generalized linear models [START_REF] Mccullagh | Generalized Linear Models[END_REF]. Assumption AL1 is similar to A2, which was used by, e.g., [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF] in their study of the Fused-Lasso in the Gaussian context. Let us remark that under AL1 or A2, criterion (4) corresponds, for n large enough, to a strong convex optimization problem, and thus is not concerned by the issue of multiple local minima. We first state an asymptotic result for logistic regression models in the non adaptive case, which is similar to Theorem 1 of [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF], established in the Gaussian case for the Generalized Fused-Lasso based on chain-graphs. The proof is given in the Appendix (Section 7.2).

Theorem 1 Let β be the minimizer of criterion Q defined in (4) with J = J lo or J = J sq in the non adaptive case (that is with w

(1) j = 1 and w (2) jℓ = 1 for all j, ℓ). If λ (m) n / √ n → λ (m) 0
≥ 0 (m = 1, 2), then under assumptions A1-2 for the Gaussian case and AL1-2 for the logistic case,

√ n β -β * → d arg min(V),
where V is the function defined, for u = (u 0 , . . . , u p ) ∈ R p+1 , as

V(u) = u T W + a 2 u T Cu + λ (1) 0 p j=1 {u j sign(β * j )I(β * j = 0) + |u j |I(β * j = 0)} +λ (2) 0 (j,ℓ)∈E {(u j -u ℓ )sign(β * j -β * ℓ )I(β * j = β * ℓ ) + |u j -u ℓ |I(β * j = β * ℓ )}.
Above, W has an N (0 p+1 , C) distribution and a = 1 in the logistic case and a = σ 2 in the linear case.

This result establishes the root-n consistency of non-adaptive Generalized Fused-Lasso estimates. However, this result also implies that when λ

(m) n = O( √ n), for m = 1, 2, the
support of β * can not be recovered with high probability by non-adaptive Fused-Lasso estimates, as stated in the following proposition whose proof is given in the Appendix (see Section 7.3).

Proposition 2 Let β be the minimizer of criterion Q defined in ( 4) with J = J lo or J = J sq in the non adaptive case (that is with w

(1) j = 1 and w

(2)

jℓ = 1 for all j, ℓ). Further set A n = {1 ≤ j ≤ p, β j = 0}. If λ (m) n / √ n → λ (m) 0 ≥ 0 (m = 1, 2)
, then under assumptions A1-2 for the Gaussian case and AL1-2 for the logistic case,

lim sup n P( A n = A) ≤ c < 1,
where c is a constant depending on the true model.

We now show that for appropriate choices of λ

(m) n = O( √ n) for m = 1, 2, the Adaptive
Generalized Fused-Lasso estimator β ad , defined as the minimizer of criterion Q in (4), enjoys asymptotic oracle properties in both the Gaussian and logistic regression contexts, contrasting with its non-adaptive counterpart. We introduce A n = {1 ≤ j ≤ p, β ad j = 0} and B n = {(j, ℓ) ∈ E, β ad j = 0 and β ad j = β ad ℓ }. Some more notations are needed before stating our result: in particular, the number s 0 of distinct non-zero values in β * \0 "supported" by G needs to be precisely defined. The edge set E can be decomposed as

E = E A ∪ E Ā where E A = {(j, ℓ) ∈ E : β * j β * ℓ = 0} and E Ā = {(j, ℓ) ∈ E : β * j β * ℓ = 0}
correspond to the set of edges made of vertices in A only and the set of edges made of at least one vertex in Ā respectively. Subset E A can further be decomposed as

E A = E = A (β * ) ∪ E = A (β * ) with E = A (β * ) = {(j, ℓ) ∈ E A : β * j = β * ℓ } = B and E = A (β * ) = {(j, ℓ) ∈ E A : β * j = β * ℓ }.
Then consider the graph G B = (A, B) and denote by s 0 the number of its connected components (e.g., in the particular case where G is a chain graph, s 0 is the number of segments consisting of non-zero and equal coefficients). Observe that d 0 ≤ s 0 ≤ p 0 , where p 0 = |A| is the number of non-zero components in β * \0 and d 0 is the number of distinct non-zero values in β * \0 . We actually have

s 0 = p 0 if and only if (β * j = β * ℓ = 0 ⇒ (j, ℓ) / ∈ E).
On the other hand, s 0 = d 0 if and only if for all (j, ℓ) such that β * j = β * ℓ , j and ℓ belong to the same connected component of G B . Now denote by A 1 , . . . A s 0 the sets of vertices of each connected components of G B , with A = s 0 s=1 A s , and set j s = min{A s } for s = 1, . . . , s 0 . Now we can define

β * B = (β * 0 , β * j 1 , . . . , β * js 0
) T , which is composed by the intercept and the s 0 distinct non-zero values of β * \0 supported by G; we further set β ad B = ( βad 0 , βad j 1 , . . . , βad js 0

) T its estimate. Now denote by X B the matrix of size n × s 0 , whose s-th column, 1 ≤ s ≤ s 0 , is X Bs = j∈As X j , where X j is the j-th column of X. Finally set Z B = (1 n , X B ) and denote by C B the (s 0 + 1) × (s 0 + 1) positive definite matrix that is defined as the limit, as n → ∞, of (i) (Z B T Z B )/(nσ 2 ) in the linear case and (ii) I(β * B )/n in the logistic case, where I(β * B ) denotes the Information matrix of the model induced by B, that is

I(β * B ) = Z T B DZ B .
We have now all the ingredients to state our main result.

Theorem 3 If λ (m) n / √ n → 0 and λ (m)
n n (γ-1)/2 → ∞, (m = 1, 2), then, under assumptions A1-2 for the Gaussian case and AL1-2 for the logistic case, the Adaptive Generalized Fused-Lasso estimator satisfies the following properties:

Consistency in variable selection: P

[A n = A] → 1 and P [B n = B] → 1 as n → +∞.

Asymptotic normality:

√ n

β ad B -β * B -→ d N 0 s 0 +1 , C B -1 .
The proof of Theorem 3 is provided in the Appendix (see Section 7.4).

Simulation study

We perform an extensive simulation study to explore the performance of the Adaptive Generalized Fused-Lasso penalties in Logistic Regression. Our objectives are:

1. to illustrate our theoretical results using designs inspired from [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF] with p = 4 on Adaptive Fused penalties based on chain graphs, 2. to study the influence of the network provided to the Generalized Fused penalty on the performance in terms of selection, estimation and prediction, 3. to show the interest of using the Generalized Fused-Lasso in the context of joint modeling.

We mention that simulations were also performed in the context of linear regression, which lead to similar conclusions (not shown).

Simulations Setting

Setting (Y, X). Our theoretical results being stated in the fixed design setting, we first define N as the maximal sample size considered in a given scenario (for instance if samples of size n = 24, 120, 240 and 1200 were considered then N = 1200), and we generate N i.i.d.

predictors x i ∈ R p , i = 1, . . . , N from a N (0 p , C \0\0 ) distribution
, where C \0\0 is some given covariance matrix of size p × p. Then, given X = (x 1 , . . . , x n ) T and β * , the vector of true coefficients (see the paragraph below for the choice of β * ), the n-vector of labels is generated according to a Bernoulli distribution such that Y i ∼ B(π i ), with π i defined as in Section 2. Fifty such vectors of labels are generated and results presented below correspond to averages over these 50 replicates (confidence intervals based on the central limit theorem also appear on our figures).

Choosing β * . In all our experiments, the true intercept term β * 0 is set to 0. Then the remaining components of vector β * are chosen to achieve a given Signal to Noise Ratio (SNR), which is defined as SNR 2 (Z, β * ) = Zβ * 2 /(nσ 2 ) in linear regression. In Generalized Linear Models the link function should be accounted for in the definition of the covariates effects, which motivates a likelihood-based generalization of the SNR such that:

SNR(Z, β * ) = n i=1 E J lo (Y i , β) -J lo (Y i , z T i β * ) n i=1 EJ lo (Y i , β)
, with J lo (Y i , β) the log-likelihood of the null model. This generalization reduces to a R 2 in linear regression [START_REF] Heinzl | Pseudo R-square measures for Poisson regression models with over or underdispersion[END_REF].

Choosing G = (V, E). For a given true vector of coefficients β * the best graph is the one whose edges correspond to equal coefficients only ((j, ℓ) ∈ E, with j > ℓ, if and only if β * j = β * ℓ ), and the worst graph is the one whose edges correspond to coefficients

{β * j = β * ℓ } {β * j = β * ℓ } (j, ℓ) ∈ E TN differences FN differences (j, ℓ) / ∈ E FP differences TP differences Criteria Spe G = TN/(TN + FP) Sens G = TP/(TP + FN)
Table 1: Measures of the suitability of a particular graph G = (V, E), for a given true vector of coefficients β * : sensitivity-like and specificity-like criteria. FP for False Positive, TP for True Positive, FN for False Negative and TN for True Negative.

of different values only. In the latter case, the graph in the penalty provides a false information regarding pairs of equal coefficients. Between these two extreme cases, there exists a continuum of discrepancies between informations supported by E and the true differences in β * : when choosing a particular graph, the statistician actually tries to guess the true structure of β * and, of course, the better the guess the better the expected results. The suitability of graph G = (V, E), i.e., the accuracy of the guess, can be measured by sensitivity-like and specificity-like criteria. We denote by Sens G , the proportion of null differences that are supported by the graph among true null differences, and Spe G the proportion of non-null differences that are not supported by the graph among true nonnull differences (see Table 1). The highest these criteria the most accurate the guess, and the most suitable is the graph for the problem at hand. In some experiments below, we make this suitability vary to investigate its impact on selection, estimation and prediction performance of Adaptive Generalized Fused-Lasso estimates.

Implementation

The Adaptive Generalized Fused Lasso was solved with the algorithms described in [START_REF] Höfling | A coordinate-wise optimization algorithm for the Fused Lasso[END_REF]; the corresponding R package will be made available soon.

The Relaxed Adaptive Generalized Fused-Lasso. The Relaxed Lasso of [START_REF] Meinshausen | Relaxed lasso[END_REF] has been shown to lead to less biased estimates than the crude Lasso: we therefore implemented Relaxed versions for the Adaptive Generalized Fused-Lasso (as well as for each of the competing method presented below), setting coefficient φ of Meinshausen (2007) to 0. These methods return unshrunk (unbiased) estimates of non-null coefficients that are obtained by a 2-step approach. The principle of the Relaxed Adaptive Generalized Fused-Lasso is as follows. For any given pair (λ 1 , λ 2 ), we first run the Adaptive Generalized Fused-Lasso method, and obtain a vector β ad λ 1 ,λ 2 . Unshrunk estimates β λ 1 ,λ 2 are then obtained by standard (i.e., unpenalized) maximum likelihood computed under a constraint induced by the sparsity and structure of β ad λ 1 ,λ 2 . More precisely, adopting notations similar to those used in [START_REF] Rinaldo | Properties and refinements of the fused lasso[END_REF], there exists a (possibly trivial) partition {P 1 , . . . , P Ĵ } of {1, . . . , p} such that β ad λ 1 ,λ 2 = Ĵ j=1 νj I P j , where I P j is the p-dimensional vector whose kth coordinate is 1 if k ∈ P j and 0 otherwise. Further introduce ŝ0 = the number of distinct non-null coefficients of β ad λ 1 ,λ 2 . Now, denoting by X k the k-th column of matrix X, we set Z = (1 n , X) where X is the [n × ŝ0 ] matrix with column j defined by X j = k∈P j X k . A vector of unshrunk estimates for the distinct non-null coefficients of β ad λ 1 ,λ 2 can then be obtained as

β λ 1 ,λ 2 = arg max β n i=1 {Y i z i β -log (1 + exp( z i β))} ,
where z i is the i-th line of matrix Z. The vector β λ 1 ,λ 2 ∈ R p+1 is then easily derived from β λ 1 ,λ 2 . This vector is the one returned by our Relaxed Adaptive Generalized Fused-Lasso.

Selection of the tuning parameters. The Fused-Lasso methods (as well as the competing methods presented below) involve some tuning parameters. Given a (possibly 2-d) grid of potential values for these tuning parameters, two main approaches exist to select the optimal one(s): cross-validation and Bayesian Information Criterion (BIC). In the context considered in this paper (large n and small p), both approaches generally lead to comparable results, the one based on the BIC being generally faster. Therefore, we only consider this latter approach here. For instance, parameters (λ 1 , λ 2 ) of the Relaxed Adaptive Generalized Fused-Lasso method are selected as the minimizers, on a predefined 2-d grid, of the following criterion

BIC(λ 1 , λ 2 ) = 2J lo ( β λ 1 ,λ 2 ) + log(n)df λ 1 ,λ 2 ,
where df λ 1 ,λ 2 is set to the number of distinct non-null coefficients of β λ 1 ,λ 2 .

Competing methods

We compete the Adaptive Generalized Fused-Lasso with the Lasso and two other methods: a version of the Group Lasso with an extra ℓ 1 -norm penalty (see Section 7.5 in the Appendix for more details) and a recent method proposed by [START_REF] Sun | Penalized logistic regression for high-dimensional DNA methylation data with case-control studies[END_REF] called the Generalized Elastic Net. Given a graph G = (V, E), this method relies on the following penalty

pen(β, G) = λ   α β 1 + (1 -α) (j,ℓ)∈E (β j -β ℓ ) 2   ,
and is implemented in the pclogit R-package. The authors interpret this penalty as a graph-based Elastic Net penalty. This penalty has not been extended to handle adaptive weights yet and does not propose two independent tuning parameters.

An important result of our numerous experiments is that the performance of non-fused based penalties (i.e. the Lasso, the Generalized Elastic-Net, the Group Lasso and their variants) are all comparable to the performance of the Relaxed Adaptive Lasso (in most cases they are actually indistinguishable). Concerning the Generalized Elastic-Net this illustrates the poor influence of the (graph-based) ℓ 2 norm part of the penalty. This could be explained by i) the lack of selection property of the ℓ 2 norm that does not encourage differences to be zero, ii) the current implementation of the method: the pclogit package uses dependent tuning parameters which may lack of flexibility. It is worth mentioning that the GE-Net is closely related to the Smooth Lasso of [START_REF] Hebiri | The smooth-lasso and other ℓ 1 + ℓ 2 penalized methods[END_REF], whose implentation is available in the linear case only (hence not used in the following simulations that focus on the Logistic case). However we report that when compared to the Fused Lasso in the linear case (not shown), the Smooth Lasso shows better performance than the Lasso in ultra-correlated cases only. As for the Group Lasso, its penalty is composed by the Lasso penalty plus the Group Lasso penalty: our results suggest that this latter penalty may not be appropriate when considering networks of features since it does not improve upon the "pure" Lasso. Consequently results pertaining to the Generalized Elastic-Net and to the Group Lasso are in general not displayed in the sequel for a better legibility of our results (they are displayed in our first experiments for illustration only).

Thus the simulation study focuses on ℓ 1 -based Fused penalties only, and we explore the properties of "raw" Generalized Fused-Lasso (referred to as Fused on our figures), Adaptive Generalized Fused-Lasso (Ada-Fused), Relaxed Generalized Fused-Lasso (Relaxed-Fused), and Relaxed Adaptive Generalized Fused-Lasso (Relaxed-Ada-Fused). The comparisons are made using the Relaxed Adaptive Lasso (Relaxed-Ada-Lasso) as a reference that does not account for any network structure among features.

Evaluation criteria

Accuracy on support recovery. When comparing model selection algorithms, it is common to use criteria originally devoted to the evaluation of binary classification algorithms. For the evaluation of support recovery for instance, each component β * j is either zero or non-zero, as is its estimate. Set as before A = {1 ≤ j ≤ p, β * j = 0} and, for any estimate β of β * , A n = {1 ≤ j ≤ p, βj = 0}. A true positive is then defined as a node that belongs to A ∩ A n , (i.e. the set of true positives is {1 ≤ j ≤ p, β * j = 0, β j = 0}). Similarly, a true negative is defined as a node that belongs to Ā ∩ Ān , (i.e. the set of true negatives is {1 ≤ j ≤ p, β * j = 0, β j = 0}). The accuracy on support recovery, denoted by Acc.A, is then obtained by adding these two terms and dividing the result by p.

Accuracy on pairs of coefficients. In the Fused framework, we also need to evaluate the performance regarding the classification of pairs of coefficients. As an illustration, in the particular case of the chain-graph penalty (corresponding to consecutive coefficients), this classification of pairs reduces to the classification of zero and non-zero elements of the vectors of successive differences. Its evaluation enables to assess the capacity of the method to detect equal consecutive coefficients. In the context of Joint Modeling this evaluation enables to assess the method capacity to detect heterogeneity and homogeneity across strata.

Indicator support recovery equality among connected coefficients TPR

|A ∩ A n |/|A| |E = (β * ) ∩ E = ( β)|/|E = (β * )| FPR | Ā ∩ A n |/| Ā| |E = (β * ) ∩ E = ( β)|/|E = (β * )| Acc (| Ā ∩ Ān | + |A ∩ A n |)/p (|E = (β * ) ∩ E = ( β)| + |E = (β * ) ∩ E = ( β)|)/|E|
Introduce for any vector β ∈ R p+1 and any subset of edges E ⊆ E, the subsets E = (β) := {(j, ℓ) ∈ E, β j = β ℓ } and E = (β) := {(j, ℓ) ∈ E, β j = β ℓ } of edges in E corresponding to pairs of equal and non-equal components in vector β respectively. Considering pairs, a true positive is an edge that belongs to E = (β * ) ∩ E = ( β); definitions of true negatives and accuracy for pairs of components follows similarly (see Table 2). Regarding the choice of E, we consider the particular choices E = E (no restriction), E = E A = {(j, ℓ) ∈ E : β * j β * ℓ = 0} (restriction to edges in the graph corresponding to pairs of non-zero true coefficients) and

E = E Ā := {(j, ℓ) ∈ E : β * j β * ℓ = 0}
(restriction to edges in the graph corresponding to pairs consisting of at least one zero coefficient). In the sequel, we denote by Acc.E, Acc.E.A and Acc.E.Abar the accuracy associated with E = E, E = E A and E = E Ā respectively.

Estimation and Prediction Performance. We also evaluate estimation consistency by computing the mean squared error (MSE) of the various estimators β based on empirical versions of E( β *β 2 ). We finally evaluate Prediction accuracy on an external (test) sample. More precisely, after estimating β * by β on a dataset of size n, we generate an independent test sample of N observations (z

(0) i , Y (0)
i ) (keep in mind that N is the largest sample size considered in a given scenario). Then, for every observation i of the test sample, we compute the predicted label Y (0) i = I(logit -1 (z (0)T i β) > 0.5), and the prediction accuracy is given by Acc

.Pred= (1/n) n i=1 I(Y (0) i = Y (0) i ).
4.5 Illustration of our asymptotic results based on Zou's example Sampling Covariates. Our first example is adapted from [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF] with p = 4 and p 0 = 3. Introduce ρ 1 = -0.39 and ρ 2 = 0.23, C 11 = 1ρ 1 I 3 + ρ 1 J 3 where J 3 is the 3 × 3 matrix of 1's, and C 12 = ρ 2 1 3 . The covariance matrix C of the covariates is then defined as [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF] used this example to illustrate situations where the Adaptive Lasso dramatically outperforms the crude Lasso (especially in terms of support recovery).

C = C 11 C 12 C T 12 1 .
Setting β * . We set β * ∈ {log(2.5), log(2.5), log(2.5), 0}.

Network generation. We consider a Generalized Fused-Lasso penalty that is based on a chain graph so that we have Spe G = Sens G = 2/3.

Setting p/n. To illustrate our asymptotic results we evaluate the methods for increasing sample sizes, namely n = 30, 60, 120, 300, 600, 1200 and 2000.

Results. Results are presented in Figure 2. First these results illustrate that Adaptive Generalized Fused-Lasso estimates are consistent (their MSE converge to 0 as n grows), and this consistency is slightly faster for Relaxed Adaptive Generalized Fused-Lasso estimates.

They also tend to share the same support as β * (Acc.A), and their successive differences also tend to share the same support as successive differences of β * (Acc.E.A and Acc.E.Abar), when n grows. On the contrary non-Adaptive Generalized Fused-Lasso estimates do not enjoy these asymptotic oracle properties: they do not share the same support as β * and are much slower to reach consistency (especially for the non-relaxed versions). Note that as expected the Relaxed Adaptive Lasso never shrinks successive differences of non-zero estimates to 0 so that this method is not efficient for the support recovery of successive differences (Acc.E.A is "artificially" high for low SNR values because estimates are all null, and so is their difference). More importantly, both consistency and sparsistency are slower for the Relaxed Adaptive Lasso, compared to (Relaxed) Adaptive Generalized Fused-Lasso.

As for prediction accuracy, all the methods perform similarly for n ≥ 300. For n < 300 however, Relaxed Adaptive Lasso is outperformed by all the versions of the Fused-Lasso, especially the ones using adaptive weights. As mentioned in Section 4, results obtained with the Adaptive Group Lasso are almost the same as those obtained with the Adaptive Lasso. As for GE-Net, results are a little worse because of the absence of adaptive weights. This simple example confirms that Fused-like estimates outperform Lasso estimates in some situations, in terms of both support recovery and prediction accuracy.

4.6 Assessing the performance of Adaptive Generalized Fused-Lasso estimates Setting β * . in order to explore difficult/easy configurations in terms of SNR, we make non-null elements of β * take values in {log(1.1), log(2), log(4), log(8), log(12)}.

Sampling covariates. We sample x i such that x i ∼ N p (0, C), C AR(1) = 1/16 ρ |i-j| j=1,p i=1,p , with ρ = -0.39 as proposed in [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF].

Setting p/n and p 0 /p. We fix p = 24. As our theoretical results are asymptotic in n, we consider cases where n/p ∈ {1, 5, 10, 50} to explore the performance of the methods in asymptotic and non-asymptotic settings. Moreover, we explore different values of ratio Relaxed-Ada-Lasso Fused Ada-Fused Relaxed-Fused Relaxed-Ada-Fused Relaxed-Ada-Group-Lasso Relaxed G-ENet Networks Generation for graph-based penalties. To study the influence of the graph in the Fused penalty, we generate networks of features with different topologies. To do so, we use a simplified version of the Stochastic Block Model [START_REF] Airoldi | Mixed-membership stochastic blockmodels[END_REF], often called the affiliation graph model. In its simplest form this model considers two categories of nodes (null and non-null coefficients) whose connectivity is governed by parameter θ such that P{j ∼ k|j ∈ A, k ∈ Ā} = 1θ, P{j ∼ k|j ∈ A, k ∈ A} = θ, and P{j ∼ k|j ∈ Ā, k ∈ Ā} = θ, where j ∼ k for j > k stands for (j, k) ∈ E. When θ increases there are fewer edges between null and non-null coefficients, the easiest configuration being when θ = 1, the most difficult when θ = 0. Parameter θ can also be interpreted in terms of Specificity and Sensitivity of the graph provided to the penalty since

E(Spe G (θ)) = E(Sens G (θ)) = θ.
Examples of configurations considered here are given in Figure 3. Lastly, we mention that edges of the graph are fixed across replicates.

Result 1: ℓ 1 -based Fused penalties show a cooperative effect on support recovery. When graph G is highly specific (θ = 1; 0.8), ℓ 1 -based fused penalties show better performance on support recovery compared with the Relaxed Adaptive Lasso (Figure 4). When applied to the difference of coefficients, the ℓ 1 -norm helps in the identification of true zeros and true non-zeros based on informative edges. Of course, when the graph pro-θ = 1 θ = 0.8 θ = 0.4 θ = 0 vides an erroneous information (edges between null and non-null coefficients only -θ = 0), the Relaxed Adaptive Lasso should be preferred. Note that the "cooperative effect" of the ℓ 1 -based fused penalty is higher when the proportion of non-null coefficients (p 0 /p) is high, maybe because the proportion of informative edges increases with p 0 /p.

Result 2: The "raw" Fused-Lasso is not robust to mis-specification of the graph in the penalty. The "raw" Fused-Lasso penalty (i.e. non-adaptive and without relaxation) shows poor performance when the graph is mis-specified. The Accuracy on support recovery is worst than the Relaxed Adaptive Lasso as soon as θ < 1 (Fig. 4), and estimated parameters show an important bias even when n/p = 50 with a high SNR (Figure 6). Moreover, accuracies on pairs of coefficients (Figure 4) reflect the absence of selection, as the majority of coefficients are not shrunk to zero when using the Fused penalty only (Figure 6).

Result 3: Adaptive weights ? relaxation ? or both ? A general result is that using adaptive weights and/or relaxation is always preferable than the "raw" Fused-Lasso: i) Accuracies are higher (Acc.A, Acc.E.A, Acc.E.Abar), and ii) the modified versions of the Fused-Lasso are more robust to mis-specification of the graph. Conclusions are similar concerning the comparison with the Relaxed Adaptive Lasso that should be preferred when the specificity of the penalty is low. Then considering the Adaptive version of the Fused-Lasso (without relaxation), increases the accuracy support recovery that is higher than the Fused-Lasso, and increases also the accuracy on differences involving null coefficients (Acc.E.Abar). However, when focusing on non-null coefficients, some bias remains (even if reduced, Fig. 6), and the relaxation strategy appears to be the most effective regarding bias reduction. Lastly, we determine if considering both relaxation and adaptive weights can improve the performance of the Generalized Fused-Lasso. Table 3 provides a global view of variations of the Accuracies for support recovery according to n/p which allows us to draw a global conclusion (across specificities and SNRs): i) when n/p is low (∼ 1), neither the adaptive nor the relaxed Fused-Lasso outperform the "raw" Fused-Lasso, which suffices maybe because there is not enough available information to improve the selection procedure.

ii) when n/p is moderate (∼ 5, 10),using separate strategies is equivalent (relaxation without weights or weights without relaxation)

iii) when n/p is high (∼ 50), the adaptive version is marginally more performant as more information is available for an accurate estimation of weights, and using relaxation does not change the performance.

iv) whatever n/p, all ℓ 1 -based Fused penalties are more accurate than the Lasso for prediction (even when the graph provided is not specific, Fig. 5b), and there is no difference among them. Figure 5: Influence of the n/p ratio on average accuracies for the detection of non-null coefficients (support recovery, Acc.A, 5a), and on prediction accuracies, 5b). Rows correspond to configurations (θ = 1, 0.8, 0.4, 0). Results correspond to p 0 = p/2.

β * = log(2)
Relaxed-Ada-Lasso Fused Ada-Fused Relaxed-Fused Relaxed-Ada-Fused

β * = log(4) β * = log(8) β * = log(12) θ = 1 θ = 0.8 θ = 0.4 θ = 0
(a) Estimated coefficients βj with n/p = 10. In this last simulation study we illustrate the interest of using Generalized Fused-Lasso estimates in the context of joint modeling, as described in Section 2.3. We start by setting the number of strata to C = 4 and we set n c = 200 so that n = c n c = 800. Those parameters remain fixed in the study.

β * = log(2) Relaxed-Ada-Lasso Fused Ada-Fused Relaxed-Fused Relaxed-Ada-Fused β * = log(4) β * = log(8) β * = log(12) θ = 1 θ = 0.8 θ = 0.4 θ = 0 (
Setting β * . Then we choose p = 20 and p 0 = 6 for each stratum, and non-null values of β * c take values in {log(1.1), log(1.5), log(2), log(2.5), log(3), log(5), log(7), log(9)}. Sampling covariates. Covariates are generated for each stratum using a centered Gaussian distribution with covariance matrix C supposed to be the same for each stratum. Covariance matrix C is set to C AR(1) (as in the previous Section) with ρ = 0.5.

Varying Homogeneity between strata. The graph provided in the penalty is made of p + 1 cliques of size C, each clique being used to connect coefficients {β * j1 , . . . , β * jC }, for j = 0, . . . , p. Then we consider 3 different configurations that differ in the repartition of null and non-null coefficients within cliques, as illustrated in Figure 7. The first configuration corresponds to the case where nodes in the graph either correspond to 4 null or non-null theoretical coefficients (as in Fig. 7a). It is the most favorable configuration for Fused-like estimates (Spe G = 0.07, Sens G = 1.00). In the two other configurations true vectors β * c do vary across strata. The second design considers three types of cliques (Fig. 7b, Spe G = 0.05 and Sens G = 0.97), one being made of null coefficients only, and the other two mixing null and non-null in the same proportion (3/4). The third configuration is the most difficult one with cliques connecting half null and non-null coefficients (Fig. 7c, Spe G = 0.04 and Sens G = 0.96).

Competing methods. For the joint modeling problem, the Relaxed Adaptive Lasso (which corresponds to criterion (4) with λ

(2) n = 0) is very similar to computing Independent Relaxed Adaptive Lasso estimates on each stratum. This latter option is more flexible because the sparsity parameter has not to be equal for each stratum. We therefore include it in our analysis instead of the Relaxed Adaptive Lasso: it is referred to as Independent Relaxed Adaptive Lassos (and denoted by Relaxed-Ada-Lasso-Indep on our figures). We also consider another version of the Lasso which consists in solving one Relaxed Adaptive Lasso on the whole data set (obtained by putting all the strata together), after selecting a reference stratum, and adding interaction terms between the remaining strata and the covariates. This method is described in more details in Section 7.5 of the Appendix. It is referred to as Interaction Relaxed Adaptive Lasso hereafter (and denoted by Relaxed-Ada-Lasso-Inter on our figures).

Results. Figures 8 presents the results. To make this figure more legible, only Adaptive versions of the Fused-Lasso estimates are presented: non-Adaptive versions achieved slightly worse performances in this simulation study. Our empirical findings confirm that configuration 1 is easier than configuration 2 which is itself easier than configuration 3, for Fused-type estimates. For instance, regarding Fused-type estimates only, best [resp. worst] support recovery (Acc.A) and prediction (Acc.pred) accuracies are obtained for configuration 1 [resp. configuration 3]. As expected, Independent Relaxed Adaptive Lassos are are not sensitive to the level of heterogeneity across strata. But, for easy to moderately difficult configurations (configurations 1 and 2 here), Fused-type estimates (especially the Relaxed Adaptive Generalized Fused-Lasso ones) clearly outperform Independent Relaxed Adaptive Lasso estimates, in terms of overall support recovery, prediction accuracy and, of course, detection of heterogeneity across the strata (Acc.E.A and Acc.E.Abar). Under the most difficult configuration, there is no gain in using Fused-type estimates in terms of overall support recovery (for high SNR values, we even observe that Independent Relaxed Adaptive Lassos could achieve slightly better results). However, Relaxed Adaptive Generalized Fused-Lassos are at least comparable to Independent Relaxed Adaptive Lassos in terms of overall support recovery and still clearly better in terms of detection of heterogeneity across strata. Interestingly, they are also slightly better in terms of prediction accuracy (this result is consistent with what was also observed in Section 4.6 even in situations where the graph was poorly suited). Another interesting result is that the method relying on interaction terms is always outperformed by Fused-type estimates. Lastly, and overall, Relaxed Adaptive Generalized Fused-Lasso is slightly better [resp. worse] than its non-relaxed counterpart for support recovery, measured by Acc.A [resp. detection of heterogeneity across strata, measured by Acc.E.A].

Joint modeling to analyze road-safety data

Road safety is a major (political) concern in the West. Driving under the influence of alcohol (DUI) is an established risk factor of car accidents. Interestingly, several studies also suggest that DUI increases the risk of dying in an accident. But this result remains controversial: biological evidence supporting this assumption is still lacking and the observed effect of DUI on the risk of dying in an accident could be due to confounding variables Ada-Fused Relaxed-Ada-Fused Relaxed-Ada-Lasso-Indep Relaxed-Ada-Lasso-Inter

Figure 8: Comparisons of the methods in the joint modeling problem with dependent features (ρ = 0.5), according to various criteria. Each column corresponds to a given configuration, which is itself related to homogeneity between strata: from the most homogeneous (Configuration 1) to the most heterogeneous (Configuration 3).

only. In the following we propose a joint modeling approach to handle the effects of both DUI and confounding variables on the risk of dying in an accident.

Our dataset consists of n = 21, 064 drivers involved in single-car personal injury crashes. The data were obtained from the systematic reports on road traffic injury crashes made by the police between [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF]2009 in France (metropolitan). Current data show 33 covariates including the characteristics of the crash: type of road, urban/rural location, lighting conditions (night with or without street lighting, dawn, daylight), and meteorological conditions (rain, wind, fog, fine weather) as well as the year of the crash. Data also include characteristics of the drivers such as gender, age, seat belt use and vital status (killed, injured or uninjured). Data on the crash-involved vehicles were also available, with the location of the main impact and the vehicle first registration year for instance. In the following we focus on the vital status of the driver only since every vehicle has one driver but a variable number of passengers. Lastly, we mention that crashes in which no driver was injured nor killed were excluded from the study, as well as crashes in which the only individuals to be injured or killed were passengers.

We define 4 strata based on gender and DUI: strata 1-2 for Males and Females not driving under the influence when the accident occured (10, 031 and 6, 385 individuals respectively), and 3-4 for Males and Females driving under the influence when the accident occurred (4, 093 and 555 individuals respectively). We use joint modeling to couple the estimation of the four logistic models relating the probability of dying in a car accident to various risk factors (one for each stratum). Most risk factors are indeed expected to share the same effect on the probability of dying across the four strata. In particular, we pay a particular attention to intercept parameters: they should be homogeneous across strata if neither gender nor alcohol directly modified the risk of dying in a car accident (gender and/or alcohol could have an indirect effect if other coefficients were observed to vary across strata). We compare different penalization strategies (Relaxed Adaptive Fused-Lasso, Independent Relaxed Adaptive Lassos, and Interactions Relaxed Adaptive Lasso) and we also present unpenalized estimates derived from standard logistic regression models independently built on each stratum. Each method returns a R 34×4 matrix, where the four columns correspond to the four vectors of parameters βc ∈ R 34 , including the intercept term, for strata c = 1, . . . , 4 (Fig. 9).

Overall, there is a high concordance between supports of vectors βc , c = 1, . . . , 4, returned by penalized methods, especially Relaxed Adaptive Generalized Fused-Lasso and Independent Relaxed Adaptive Lassos. For instance, penalized methods all agree on the absence of significant influence of most recorded variables on the risk of dying in an accident, given an accident occurred (the driver's professional activity, the reason the journey -professional or not-the period of the year -Q1, Q2 and Q3-i.e. the first, second and third thirds of the year, the period of the day, the vehicle age, the year of crash, atmospheric conditions, the adherence condition, the flatness or the curvature of the road). On the other hand, the risk appears to be higher for older drivers and for drivers under drugs, as well as if the accident occurred on a shoulder, if the car hit a wall or a tree and if the main impact was on the left side of the car. Factors that reduce the risk of dying are the use of a seat-belt, non-bidirectional roads (i.e., roads with a median strip) and city roads (where speed is generally lower). Interestingly the Relaxed Adaptive Fused-Lasso and Independent Relaxed Adaptive Lassos return slightly different estimates for the influence of city roads. This can be explained by aliasing between covariate "City Roads" (that equals 1 if the road section is managed by a city, and not by a county, a region nor the country), and covariate "City" that equals one if the crash occurred in a city (but not necessarily on a road section managed by a city). These two covariates are correlated since covariate "City'" is most often 1 when "City Roads" is 1. It would make sense to exclude one of them but we keep both for illustration purposes. According to the Relaxed Adaptive Fused, both covariates have homogeneous effects across strata, which suggests no interaction between these covariates and gender and DUI. This is also suggested by the Interactions Relaxed Adaptive Lasso. As for independent Lasso estimates, they suggest differences across strata, which are hard to interpret in this case. There are a few other differences between the three penalized methods, but when Relaxed Adaptive Fused-Lasso and Interactions Relaxed Adaptive Lasso estimates disagree, the Relaxed Adaptive Fused-Lasso most often agrees with Independent Relaxed Adaptive Lassos. This is consistent with our observations from the simulation study where these latter two methods performed the best. Let us finally mention that the structure of the matrix obtained with the Adaptive Fused Lasso which was the best for the detection of heterogeneity across strata on our simulation study, was 100% consistent with the one obtained with the Relaxed Adaptive Fused-Lasso.

Finally, the most important result is that Relaxed Adaptive Fused-Lasso indicates that intercepts do vary across strata, suggesting an effect of both gender and DUI on the risk of dying in a single-crash accident. More precisely, sober females are at a higher risk than sober males, and, to a lesser extent, females under the influence are at a higher risk than males under the influence. Moreover, irrespective on gender, drivers under the influence are at a higher risk than sober drivers. However this result should be tempered by potential confounding due to speed, which was not available here. For instance, drivers under the influence are likely to drive faster than sober drivers. The effects of speed may be partly captured by other covariates, but not entirely. Consequently residual "speed effects" could be responsible for detected heterogeneities between intercepts.

Discussion

In this work we established the asymptotic oracle properties of the Adaptive Generalized Fused-Lasso estimates in the case of linear and logistic regression models, for fixed p. To our knowledge these results are the first established for Fused-Lasso estimates in the setting of GLMs, and are also the first established for the Generalized Fused penalty based on a graph. Of course these results should be extended especially to the the high-dimensional case where p may grow with n (or even p ≥ n). As mentioned in the Introduction, most published papers dealing with Fused-Lasso estimates in high-dimension focused on the chain-based Fused penalty in the Gaussian sequence model (i.e. the original Fused Lasso of [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF] that supposes an identity design matrix X). [START_REF] Vaiter | Robust sparse analysis regularization[END_REF] recently established some results for a class of penalized least-squares based on a class of ℓ 1 penalties that encompasses Generalized Fused Lassos. However the extension of such results to GLMs with Adaptive weights would not be straightforward and thus would be an interesting lead. Above all, non-asymptotic oracle prediction inequalities still need to be established for Fused-like estimates: under the Linear Model for instance, and setting β the Fused-Lasso estimate for some appropriate values of the tuning parameters, it is easy to obtain inequalities of the form

X( β -β * ) 2 ≤ κ
corresponding components in a graph. However, if studying lung cancer, they would not share similar coefficients at all, and such an edge should not be drawn.

A particular situation where the graph is suggested by the design of the study itself is what was called Joint Modeling here, where data come from various strata. In the most general case, the graph consists of p+1 cliques, whose common size is the number of strata. When the main question of interest is the detection of heterogeneity across the strata, we believe that this type of graph is very natural. Indeed, it encourages coefficients to be homogeneous across the strata, so that detected heterogeneities come from the data. It has some connections with the statistical tests theory where tests are generally performed under the null hypothesis (absence of heterogeneity in this case), and data need to be far from this assumption in order to reject the null hypothesis. But even in the Joint Modeling context, other graphs may be considered, under some particular circumstances: for instance, stars may replace cliques if one stratum can serve as a reference.

Appendix

Application to the joint modeling of sparse regression models

Our objective here is to show how to rewrite the joint modeling problem as a particular case of the Generalized Fused-Lasso problem introduced in (4). With the same notations as in Section 2.3, the criterion considered in the joint modeling context can be written,

C c=1 i:C i =c J (Y i , z T i β c ) + λ (1) n p j=1 w (1) j |β c,j | + λ (2) n p+1 j=0 c 1 >c 2 w (2) c 1 ,c 2 ,j |β c 1 ,j -β c 2 ,j |, (5)
where J can typically be set to either J sq or J lo depending on whether linear or logistic models are considered (see (1) and (2) above). In the absence of the λ

(2) n -penalty term, optimizing this criterion reduces to independently solve C Lasso problems, one for each stratum. The λ (2) n -penalty term encourages coefficients β c 1 ,j and β c 2 ,j (that is, the coefficient of the j-th covariate in the c 1 -th and c 2 -th stratum respectively) to be equal or at least close to each other: in this sense, it couples the Lasso estimates that would be obtained without this λ

(2) n -penalty. In (5) above, intercept terms are not directly penalized (the sum involved in the λ

(1) n -penalty starts at j = 1) but are encouraged to be equal (the sum involved in the λ

(1) n -penalty starts at j = 0). Of course, other options for the intercept terms can be considered.

We now show how to rewrite criterion (5) as a version of (4). Towards this aim, set p+1) and denote by Z = diag(Z 1 , . . . , Z C ), the [n×C(p+1)] blockdiagonal matrix, whose blocks are of size [n c × (p + 1)] and defined as Z c = [1 X c ]. Here X c stands for the [n c × p] sub-matrix of X with rows corresponding to observations falling in stratum c (by assumption, the vector of response variables is given by Y = (Y T 1 , . . . , Y T C ) T with Y c := {Y i , C i = c} denoting the vector of length n c containing the response variables corresponding to observations of the c-th stratum). Letting, for any two integers n 1 , n 2 , n 1 %n 2 be the rest of the division of n 1 by n 2 , criterion (5) then writes

β = (β T 1 , . . . , β T C ) T ∈ R C(
where the edge set E along with the node set V = {1, . . . , C(p + 1)} define a graph G = (V, E) which consists of p + 1 cliques of size C (see Figure 1). Edges (j, ℓ) present in graph G are those for which j > ℓ and ℓ%(p + 1) = j%(p + 1). Criterion ( 6) is exactly criterion (4) with Z instead of Z, and some terms, corresponding to intercept parameters, that are not penalized in the λ

(1)

n -penalty. In some situations, one stratum can be regarded as the reference: for instance, when strata correspond to various treatments, the control treatment can serve as the reference. Without loss of generality, we can assume that the reference stratum is the first one, in which case an adapted version of penalty ( 5) is The graph induced by this penalty consist of p + 1 star graphs (one for each covariate plus one for the intercept) with the reference at its center: more precisely, in the j-th star graph, the only present connections are between coefficient β 1,j and the C -1 remaining coefficients β c,j , for c = 1. An illustration is presented in the right panel of Figure 1, in the case p = 2 and C = 4.

Proof of Theorem 1

This proof is an adaptation to the logistic case of the proof given by [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF]. The major difference concerns the treatment of the loss function. Let us define V n (u) = Q(β * + u/ √ n) -Q(β * ) with u = (u 0 , . . . , u p ) T , and Q defined as in criterion (4) with J = J lo . Obviously V n (u) is minimized at √ n( ββ * ). Similarly to [START_REF] Tibshirani | Sparsity and smoothness via the fused lasso[END_REF], we obtain:

V n (u) = J lo β * + u √ n -J lo (β * ) + λ (1) n p j=1 β * j + u j √ n -|β * j | + λ (2) n (j,ℓ)∈E β * j -β * ℓ + (u j -u ℓ ) √ n -|β * j -β * ℓ | .
For any fixed u, the last two terms of the right-hand side converge to the last two terms in expression (5) of V(u) as n goes to ∞. As for the first two terms, a Taylor expansion Above, r j = sign(u * j ) if u * j = 0 and r j is some real number in [-1, 1] otherwise. Similarly, t jℓ = sign(u * ju * ℓ ) if u * j = u * ℓ and t jℓ is some real number in [-1, 1] otherwise. For any index j ∈ A there is some s = s(j) such that j ∈ A s(j) , where A s still denotes the set of vertices of the s-th connected component of G B (see the paragraph before the statement of Theorem 3 for the definitions of these objects). Then summing up the KKT optimality conditions over the set A s(j) , we have Similarly, setting B = {(j, ℓ) ∈ E ∩ Ā × Ā} and denoting by G 0 = ( Ā, B), the set Ā can be decomposed as Ā = ∪ s 1 s=1 Ās , where 1 ≤ s 1 ≤ pp 0 and Ās is the subset of vertices constituting the s-th connected component of G 0 . Then, for any j / ∈ A, there exists some s = s(j) such that j ∈ Ās(j) and summing up the KKT optimality conditions over Ās(j) , we have k∈ Ās(j)

W k + (Cu * ) k + λ (1) 0 r k + λ (2) 0 ℓ∈E = k (β * ) sign(β * k -β * ℓ ) = 0, (10)
with |s k | ≤ 1. If u * j = 0 for all j / ∈ A, equations ( 9) along with equation (8) form a system of s 0 + 1 equations with p 0 + 1 ≥ s 0 + 1 variables, that can be written as

W B + M 1 u * {0}∪A + λ (1) 0 r B + λ (2) 0 t B = 0,
where M 1 is the (s 0 +1)×(p 0 +1) matrix whose (s, j) element is m s,j = k∈A s-1 C k,j (with C k,j the (k, j) element of C and A 0 = {0}), and W B , r B and t B are vectors in R s 0 +1 whose s-th elements are k∈A s-1 W k , k∈A s-1 sign(β * k ) and k∈A s-1 ℓ∈E = k (β * ) sign(β * kβ * ℓ ) respectively. Now, denoting by M † 1 the pseudo-inverse of M 1 , there exists some vector ω ∈ R p 0 +1 such that

u * {0}∪A = (I p 0 +1 -M † 1 M 1 )ω -M † 1 (W B + λ (1) 0 r B + λ (2) 0 t B ). (11)
On the other hand, if u * j = 0 for all j / ∈ A, equations (10) form a system of s 1 equations that can be written, in view of (11), (2) 0 t B ) ≤ λ

W B + λ (2) 0 t B + M 2 (I p 0 -M † 1 M 1 )ω -M † 1 (W B + λ (1) 0 r B + λ (2) 0 t B ) ≤ λ (
(1) 0 r B < 1.

Since L ⊂ A, the first sum converges to 0 in probability. Moreover, the second sum also converges to 0 in probability, while the third sum tends to -∞, which contradicts M n (ℓ) = O P (1) for all ℓ = 1, . . . , p. We therefore conclude that P((j, ℓ) ∈ B c n ) → 0, for all (j, ℓ) ∈ B, which completes the proof of Theorem 3.

Competing methods

The (Adaptive) Group Lasso (see [START_REF] Huang | A selective review of group selection in high dimensional models[END_REF] for a recent review) is particularly appealing when features can be naturally divided into K nonoverlapping groups G k of dimension d k , such that k d k = p. Any vector β ∈ R p can then be written β = (β G 1 , . . . , β G K ) T , with β G k = (β G k 1 , . . . , β G k d k ) T ∈ R d k , and the Group Lasso penalty writes pen(β, G)

= λ 1 β 1 + λ 2 J j=1 β G j ,
The "vanilla" version of the Group Lasso uses only the λ 2 -penalty term above: selection is then performed at the group level which means that if a group is selected, all the variables it is made of are selected. The added ℓ 1 -norm penalty allows the Group Lasso to exclude some variables within a group, which is willing in the situations considered in our experiments. We use the spams package (originally accompanying [START_REF] Mairal | Online Learning for Matrix Factorization and Sparse Coding[END_REF]) to compute this method, with adaptive weights derived from initial ML estimates of the parameters (same weights as the w

(1) j used for Adaptive Generalized Fused estimates).

In the joint modeling context, we also consider the Lasso with interaction. More precisely, and with the notations of Sections 2.3, we first select the first stratum as the reference stratum. Then, for every c > 1, we can write

β * c = β * 1 + δ * c ,
where δ * c ∈ R p+1 measures the heterogeneity between β * c and β * 1 . This is equivalent to working under the overall model (ruling every observation 1 ≤ i ≤ n, with n = c n c ):

Y i = β * T 1 Z i + c>1 δ * T c Z i I(C i = c) + ǫ i .
The Adaptive Lasso can then be used to select non-zero elements in estimates of both β * We used the relaxed version of this method.

Figure 2 :

 2 Figure 2: Comparison of the methods according to various criteria evaluated on Zou's example, for increasing sample sizes.

Figure 3 :

 3 Figure 3: Networks generation for graph-based penalties with p = 24 and p 0 = 6. Two classes of nodes are present: rectangle white nodes correspond to null coefficients in β * while colored circle nodes correspond to non-null coefficients. Parameter θ governs the intra/inter-group connectivity.

  Accuracies (support recovery and prediction) for ℓ1-based Fused penalties (n/p = 10, p0 = p/2). Accuracies (support recovery and prediction) for ℓ1-based Fused penalties (n/p = 10, p0 = p/4).

Figure 4 :

 4 Figure4: Average accuracies for the detection of non-null coefficients (support recovery, Acc.A), and for the detection of null differences of coefficients without null coefficients (Acc.E.A) and with null-coefficients (Acc.E.Abar). The last row corresponds to the prediction accuracy (Acc.pred). Columns correspond to configurations (θ = 1, 0.8, 0.4, 0). Results correspond to p 0 = p/2 (4a) and p 0 = p/4 (4b).

  Accuracies for support recovery (Acc.A) for Fused penalties with p = 24, and n/p ∈ {1, 5, 10, 50}, p0 = p/2. accuracies for Fused penalties with p = 24, and n/p ∈ {1, 5, 10, 50}, p0 = p/2.

  b) Estimated coefficients βj with n/p = 50.

Figure 6 :

 6 Figure 6: Estimated coefficients with respect to different configurations. The x-axis corresponds to p = 24 positions of coefficients β. Rows correspond to configurations (θ = 1, 0.8, 0.4, 0). Results correspond to p 0 = p/2.

Figure 7 :

 7 Figure 7: Graphical representation of the three configurations considered in the simulation study for the joint modeling problem. Keep in mind that the graph on which the Fused penalty is based for this particular problem is composed by p+1 = 21 cliques of size C = 4. The figure presents every possible type of cliques for each configuration. Nodes correspond to coefficients, the theoretical value of which being either zero (white rectangles) or β * > 0 (blue circles).

Figure 9 :

 9 Figure9: Estimates of the logistic regression models for studying the risk a dying in a car accident with strata defined by combining gender and alcohol consumption.

  jℓ |β jβ ℓ |,

  jβ 1,j |.

  where W B , r B and t B are the vectors in R s 1 whose s-th elements are given by k∈ Ās W k , | Ās | and k∈ Ās ℓ∈E = k (β * ) sign(β * kβ * ℓ ) respectively. We can now conclude by observing thatc ≤ P W B + λ (2) 0 t B + M 2 (I p 0 -M † 1 M 1 )ω -M † 1 (W B + λ

1

  and δ * c , c > 1, by minimizing, for an appropriate λ > 0

Table 2 :

 2 Evaluation criteria for model selection.

Table 3 :

 3 Average Accuracies on support recovery (Acc.A) and on Prediction (Acc.Pred) according to the use of adaptive and/or relaxed Generalized Fused-Lasso estimates. Averages are computed across different configurations and SNRs.

		Acc.A		Acc.Pred
		non-relaxed relaxed non-relaxed relaxed
			n/p = 1
	non-adaptive	0.62	0.57	0.58	0.58
	adaptive	0.54	0.55	0.58	0.58
			n/p = 5
	non-adaptive	0.67	0.73	0.61	0.62
	adaptive	0.71	0.69	0.61	0.61
			n/p = 10
	non-adaptive	0.71	0.78	0.62	0.63
	adaptive	0.78	0.76	0.63	0.63
			n/p = 50
	non-adaptive	0.79	0.88	0.65	0.65
	adaptive	0.89	0.88	0.65	0.65

⋄ The views and opinions expressed herein are those of the author and do not necessarily reflect the views of Novartis.

p 0 log(p) n with high probability, for some positive constant κ, but the main question is about getting a similar inequality with p 0 replaced by the quantity s 0 introduced in Section 3.[START_REF] Rinaldo | Properties and refinements of the fused lasso[END_REF] established such an inequality in the Gaussian sequence model, and only for a modification of the Fused-Lasso (that was called Adaptive Fused-Lasso even though it does not rely on adaptive weights). We believe that our simulation study suggests that such an inequality should hold for Adaptive Generalized Fused-Lasso estimates, even if it was conducted in the low-dimensional case. Indeed, we observed very good prediction performance for these estimates under the Logistic Model. In particular, when the graph is appropriately chosen, predictive performance of Fused-like estimates are much higher than those of the Lasso for moderate sample sizes.From the modeling point of view, using a graph that provides a correct information in the penalty is of course crucial, and we demonstrated that the performance of Adaptive Generalized Fused-Lasso estimates is deeply related to the suitability of the chosen graph, especially for support recovery. This graph constitutes a formal description of some prior knowledge on the problem that is investigated. We may stress that this graph does not describe correlations among features but similarity between the effects of these features under some model. Correlated features may share similar coefficients but this is not always the case. For instance in epidemiology, smoking and alcohol consumption are generally highly correlated. They further may share similar effects under a logistic model when studying cardiovascular diseases so that it might make sense to draw an edge between the

Acknowledgements

Part of this work was supported by the Interuniversity Attraction Pole (IAP) research network in Statistics P5/24, by the MSTIC project of the Joseph-Fourier University, and by the ABS4NGS ANR project (ANR-11-BINF-0001-06).

yields

But, under AL1, we have 1 2

Moreover, AL1 implies that the minimum eigenvalue of I(β * ) goes to ∞ and, under AL2, it is well known [START_REF] Gourieroux | Asymptotic properties of the maximum likelihood estimator in dichotomous logit models[END_REF]) that

where W has an N (0 p+1 , C) distribution. By Slutsky's theorem, we therefore have V n (u) → d V(u). Since V n is convex, the epi-convergence results of [START_REF] Geyer | On the Asymptotics of Constrained M-Estimation[END_REF] can finally be used to complete the proof of Theorem 1.

Proof of Proposition 2

For the sake of brevity, the proof is only given in the logistic case; the Gaussian case follows from the exact same lines. If λ

(2) 0 = 0, the proof is the same as in the "pure" Lasso case (see [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF]). If λ

(2) 0 = 0, the result follows from an adaptation of Zou's proof. Below, we assume that λ

(1) 0 = 0; the case where λ

(1) 0 = 0 is slightly easier and omitted. First observe that P( A n = A) ≤ P(

√ n βj = 0 ∀j / ∈ A). Moreover, in virtue of Theorem 1, we have lim sup n P( √ n βj = 0 ∀j / ∈ A) ≤ P(u * j = 0 ∀j / ∈ A). Therefore, we only need to show that c = P(u * j = 0 ∀j / ∈ A) < 1. For any j ∈ {1, . . . , p}, introduce

, where the sets of pairs of vertices E = (β * ) and E = (β * ) were introduced in Section 4.4. Let u * = argmin(V). Setting W = (W 0 , . . . , W p ) T and Cu * = ((Cu * ) 0 , . . . , (Cu * ) p ) T , we have, by the Karush-Kuhn-Tucker (KKT) optimality condition,

and for all j ∈ A,

and for all j / ∈ A,

(-1) I(j<ℓ) t jℓ = 0.

Proof of Theorem 3

For the sake of brevity, the proof is solely detailed in the Gaussian case. The proof in the logistic case mostly follows from similar arguments along with others used to derive Theorem 1. We will briefly describe how to proceed when the Gaussian case differs from the logistic one.

The following proof is an adaptation of the proof given by [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF]. The main difference concerns the treatment of the penalty term. Let us define

We have the two following behaviors :

Denote by C A the (p 0 + 1) × (p 0 + 1) sub-matrix of C constituted of rows and columns associated with indexes in {0}∪A and by W A a random Gaussian vector N 0 p 0 +1 , σ 4 C A . Then V n (u) → d V(u) for every u, with V defined for u = (u 0 , . . . , u p ) ∈ R p+1 , by

Recall the notations introduced just before stating Theorem 3. Any vector u ∈ R p+1 such that u j = 0 for all j / ∈ A and u j = u ℓ for all (j, ℓ) ∈ B has s 0 + 1 distinct non-zero values. Denoting by u 0 , u j 1 , . . . , u js 0 these values, and setting u B = (u 0 , u j 1 , . . . , u js 0 ) T , we have

where W B is a random Gaussian vector N 0 s 0 +1 , σ 4 C B . Clearly, V has a unique minimum for u ∈ R p+1 such that u j = 0 for all j / ∈ A and u j = u ℓ for all (j, ℓ) ∈ B and u B = -C -1 B W B /σ 2 . Since V n is convex we can proceed as in [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF] by using the epi-convergence results of [START_REF] Geyer | On the Asymptotics of Constrained M-Estimation[END_REF] to prove the asymptotic normality part.

Similar arguments can be used to get the asymptotic normality part for the logistic case since, as we have shown in the proof of Lemma 1, the limit distribution of

Let us now turn our attention to the variable selection consistency. Namely, we have to show that ∀j ∈ A, P(j ∈ A n ) → 1 and that ∀j / ∈ A, P(j ∈ A n ) → 0. The first claim is an easy consequence of the previous asymptotic result (see [START_REF] Zou | The Adaptive Lasso and Its Oracle Properties[END_REF]). To prove the second claim, consider an index j / ∈ A and denote by C j the subset of vertices constituting the connected component of G to which j belongs. Let C 0 j = {ℓ ∈ C j , β * ℓ = 0}; clearly, j ∈ C 0 j . Our aim is to prove that P(ℓ ∈ A n ) → 0, for all ℓ ∈ C 0 j . Towards this aim, observe that the subgradient equations enable to write, for k = 1, . . . , p:

∈ E}, and setting t kℓ = -t ℓk for (ℓ, k) ∈ E, we have the following more compact form for the subgradient equations:

where, in particular, t kℓ = sign( β ad kβ ad ℓ ) for any (k, ℓ) ∈ Ẽ such that β ad k = β ad ℓ . Next, the quantity M n (k) := x T k (y -Z β ad )/ √ n can be decomposed (see the proof of Theorem 2

given by Zou ( 2006)) as the sum of two variables which both converge in distribution to some normal distribution, so that M n (k) = O P (1) as n → ∞, for ℓ = 1, . . . , p. Let us note that in the logistic case, property (7) along with assumption AL1 enables to show that each component of the gradient ∇J( β ad ) is a O P (1) as well.

Let us now suppose that there exist some ℓ ∈ C 0 j such that β ad ℓ = 0. In this case, either the set S neg = {ℓ ∈ C 0 j : β ad ℓ < 0} or the set S pos = {ℓ ∈ C 0 j : β ad ℓ > 0} is not empty (or both sets are not empty). If S neg = ∅, let b min = min k∈Sneg β ad k . Further denote by L the subset of S neg of connnected indices ℓ such that β ad ℓ = b min . Since S neg = ∅, L has at least one element. Then, summing up the optimality conditions over the indices in L, we obtain

Since L ⊂ S neg , r k = -1, for all k ∈ L, and by definition of L, t kℓ = -1 for all ℓ such that β ad ℓ = b min . Moreover when β * ℓ = 0 then β * ℓ = β * k , and

which contradicts M n (ℓ) = O P (1) for all ℓ = 1, . . . , p. That leads to P(S neg = ∅) → 1.

If S neg = ∅, then S pos = ∅, and similar arguments can be used (with maxima instead of minima) to get a contradiction. Putting all this together, we conclude that for all ℓ ∈ C 0 j , P(ℓ ∈ A n ) → 0.

It remains to show the consistency for the set B n . As for A n , we need to prove that ∀(j, ℓ) / ∈ B, P((j, ℓ) ∈ B c n ) → 1 and that ∀(j, ℓ) ∈ B, P((j, ℓ) ∈ B c n ) → 0. Let us prove the first claim. If (j, ℓ) / ∈ B either (β * j = 0 and/or β * ℓ = 0), or (β * j = 0, β * ℓ = 0 and β * j = β * ℓ ). In the first case, when j ∈ A c , we have proved previously that P(j ∈ A c n ) → 1, so P((j, ℓ) ∈ B c n ) → 1. In the second case, if j, ℓ ∈ A, and (j, ℓ) / ∈ B, the asymptotic normality result indicates that β ad jβ ad ℓ → P β * jβ * ℓ = 0; thus P((j, ℓ) ∈ B c n ) → 1. Now let us prove the second claim, using as previously the subgradient equations. Let j be an index of A such that for some ℓ ∈ A we have (j, ℓ) ∈ B. Then, for some 1 ≤ s(j) ≤ s 0 , j ∈ A s(j) , where A s still denotes the set of vertices of the s-th connected component of G B . Suppose that there exists some ℓ ∈ A s(j) such that β ad ℓ = β ad j . As previously we define b min = min k∈A s(j) β ad k and L the subset of A s(j) of connected indices ℓ such that β ad ℓ = b min . Then, summing up the optimality conditions over the indices in L, we obtain