Member, Rémi Cogranne
email: remi.cogranne@utt.fr

Dr Rémi

Prof Jessica Fridrich
email: fridrich@binghamton.edu

Modeling and Extending the Ensemble Classifier for Steganalysis of Digital Images Using Hypothesis Testing Theory

Keywords: Hypothesis testing theory, information hiding, optimal detection, multi-class classification, ensemble classifier. I

The machine learning paradigm currently predominantly used for steganalysis of digital images works on the principle of fusing the decisions of many weak base learners. In this paper, we employ a statistical model of such an ensemble and replace the majority voting rule with a likelihood ratio test. This allows us to train the ensemble to guarantee desired statistical properties, such as the false-alarm probability and the detection power while preserving the high detection accuracy of original ensemble classifier. It also turns out the proposed test is linear. Moreover, by replacing the conventional total probability of error with an alternative criterion of optimality, the ensemble can be extended to detect messages of an unknown length to address composite hypotheses. Finally, the proposed well-founded statistical formulation allows us to extend the ensemble to multiclass classification with an appropriate criterion of optimality and an optimal associated decision rule. This is useful when a digital image is tested for presence of secret data hidden by more than one steganographic method. Numerical results on real images show the sharpness of the theoretically established results and the relevance of the proposed methodology.

thus a stego-object that can be sent over an insecure channel without raising any suspicion. Steganography that hides messages in digital images has received great attention since mid 1990's.

A. Modern Steganalysis

While steganography has been advanced to hide data more efficiently and more securely, see e.g. [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF], the related field of steganography detection, steganalysis, has also been developing at a fast rate. Currently, there exist two main trends in steganalysis:

1) Optimal detectors, as referred to in [START_REF] Ker | Moving steganography and steganalysis from the laboratory into the real world[END_REF], find an optimal statistical test with respect to a given criterion based on a statistical model of cover objects. One of the first optimal detectors was proposed for Least Significant Bit (LSB) replacement in [START_REF] Dabeer | Detection of hiding in the least significant bit[END_REF] using a simple statistical model of pixels. The detection has been further improved by considering more sophisticated models [START_REF] Cogranne | A cover image model for reliable steganalysis[END_REF]- [START_REF] Thai | Statistical detection of data hidden in least significant bits of clipped images[END_REF]. The methodology of optimal detection has also been applied to LSB matching in the spatial domain [START_REF] Cogranne | An asymptotically uniformly most powerful test for LSB matching detection[END_REF] and to the Jsteg algorithm in the JPEG domain [START_REF] Zitzmann | Hidden information detection based on quantized Laplacian distribution[END_REF], [START_REF] Thai | Statistical model of quantized DCT coefficients : Application in the steganalysis of jsteg algorithm[END_REF]. 2) A large portion of steganalysis methods today are implemented using machine learning. First, a feature representation of cover objects, that could reveal steganography, is selected. Then, a classifier is trained to distinguish between cover and stego features. One of the first feature based steganalysis can be found in [START_REF] Farid | Detecting hidden messages using higher-order statistical models[END_REF], where the Fisher Linear Discriminant (FLD) was used for machine learning. Soon, more efficient machine learning techniques were proposed, such as the Support Vector Machines (SVM) [START_REF] Pevný | Steganalysis by subtractive pixel adjacency matrix[END_REF], [START_REF] Pevný | From blind to quantitative steganalysis[END_REF], combined with a plethora of representations of digital images. Recently, the FLDbased ensemble classifier [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF] has been successfully introduced as a scalable alternative to SVMs for features of a high dimension. Machine learning based steganalysis methods are usually much more powerful than optimal detectors derived from simple models. They can also be extended to multi-class detection [START_REF] Pevný | Multiclass detector of current steganographic methods for jpeg format[END_REF] and payload estimation [START_REF] Pevný | From blind to quantitative steganalysis[END_REF]. Their drawback, however, is that the theoretical statistical properties of such detectors are generally unknown: the false-alarm and correctdetection probabilities are always evaluated empirically from a large set of digital images. While optimal detectors derived from a cover model perform much worse in practice, since modeling such complex objects as digital images is challenging, these detectors offer certain indisputable advantages. First, one can theoretically find an explicit formula for the statistical performance of the optimal test and guarantee a false-alarm rate, which is crucial in practical application when searching for hidden data in large data sets. Second, such framework provides valuable insight into the problem of how the properties of digital images impact statistical detectability, see [START_REF] Sedighi | Content-Adaptive Steganography by Minimizing Statistical Detectability[END_REF], [START_REF] Sedighi | Content-adaptive pentary steganography using the multivariate generalized gaussian cover model[END_REF] for an example of steganography designed to minimize the impact on a statistical image model.

B. Contribution and Organization of this Paper

The present paper leverages the advantages of both approaches. By employing an accurate statistical model for the base learners' projections in an ensemble classifier [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF], it is possible to replace the majority voting detector with a likelihood ratio test. This allows us to (1) establish the statistical properties of the proposed optimal test, (2) use alternative criteria for the detector design (e.g., the Neyman-Pearson criterion instead of the frequently used Bayesian criterion), and (3) extend the classifier to multi-class detection while using optimality criteria established within the detection theory. The new framework, when used with the Bayesian criterion, preserves the high detection performance of the original ensemble classifier. To the best of our knowledge, this approach, which casts the ensemble base learners' within hypothesis testing theory, has never been studied before.

Although the present paper focuses on steganalysis with ensemble classifiers whose base learners are FLDs, the proposed approach is applicable to other instances of ensembles whose base learners are linear (e.g., linear SVMs). We now highlight the main contributions of this paper with respect to the original formulation of the ensemble classifier for steganalysis [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF]:

1) The statistical model of base learners' projections, whose accuracy is verified on real data sets, allows us to formally establish the statistical properties of the proposed test. For instance, one can compute the threshold that guarantees a desired false-alarm rate together with the highest power one can expect. This extension of the ensemble also allows using other measures of performance than the usual minimal total probability P E and to draw the receiver operating characteristic (ROC) curve. This part has already been partially published in [START_REF] Cogranne | Theoretical model of the FLD ensemble classifier based on hypothesis testing theory[END_REF]. 2) Moreover, with the proposed approach, we show that a statistical test can be designed for the ensemble so that it remains optimal (under a mild assumption) when the embedding payload / hidden message length is unknown. 3) Finally, the ensemble is extended to a multi-hypotheses detection/classification problem. This is possible because we casted the ensemble classifier within the hypothesis testing theory based on the proposed statistical model. A novel criterion of optimality is proposed (and the associated optimal statistical test is presented) to preserve the constraint on the false-alarm probability that, up to our knowledge, has never been studied in steganalysis.

This paper is an extension of our prior work [START_REF] Cogranne | Theoretical model of the FLD ensemble classifier based on hypothesis testing theory[END_REF]. It provides three majors advancements with respect to this prior art: 1) a detailed description of the implementation, which has been modified to improve the accuracy; 2) extension to the case of an unknown payload together with a proof of optimality of the proposed test; and 3) extension to the multi-hypotheses (multiclass steganalysis) with a novel criterion of optimality.

Though the present paper focuses on steganalysis, the proposed methodology is general enough to be used as a supervised learning method in a broader context, such as for content retrieval [START_REF] Jiang | Similarity-based online feature selection in content-based image retrieval[END_REF], face recognition [START_REF] Su | Hierarchical ensemble of global and local classifiers for face recognition[END_REF], and automatic annotation [START_REF] Fan | Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation[END_REF], which also employ ensemble classifiers.

The paper is organized as follows. Section II provides a brief summary of ensemble classifiers for steganalysis using high-dimensional feature-spaces. Then, Section III states the steganalysis problem within the framework of the statistical hypothesis testing theory. This is followed by a description of the proposed optimal Likelihood Ratio Test (LRT) including the study of its statistical properties. Section IV extends the proposed optimal detection approach to the case of multiclass classification through an original criterion of optimality never used in steganalysis. Numerical results on a large image database for both spatial and JPEG domain steganographic methods presented in Section V show the sharpness of the theoretical results and the relevance of the proposed methodology. Finally, Section VI summarizes the present work and concludes the paper.

II. FLD ENSEMBLE CLASSIFIER (BACKGROUND)

In the whole paper, matrices are represented with capital bold letters X, vectors are denoted with lower case bold letters x, scalars with lower case letters x, and sets and probability distributions with calligraphic capital letters X .

With the rapidly increasing dimension of feature spaces for steganalysis, ensemble classifiers have received a great interest because their computational complexity scales favorably with respect to the feature dimensionality. However, their theoretical performance remains largely unstudied. The present paper focuses on the ensemble classifier as originally designed for the BOSS competition [START_REF] Bas | Break our steganographic system -the ins and outs of organizing boss[END_REF] and further developed in [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF]. Each base learner is a Fisher Linear Discriminant classifier trained on a uniformly randomly selected subset of features.

Formally, let f ∈ R d be a (column) vector of d features extracted from one image. It is assumed that the features extracted from N pairs of cover and corresponding stego images, respectively denoted C = (c 1 , . . . , c N ) and S = (s 1 , . . . , s N ), are available together with their class labels. This set is divided into two disjoint subsets of N trn training and N tst testing samples, N trn + N tst = N . During the training phase, each base learner is given a subset of features on which an FLD classifier is trained. 1 Let the training sets of cover and stego image features be matrices of size d × N trn denoted C trn = (c trn 1 , . . . , c trn N trn ) and S trn = (s trn 1 , . . . , s trn N trn ). The FLD assumes that, within these two classes, the features are i.i.d. with means µ c and µ s , of size d × 1, and covariance matrices Σ c and Σ s of size d × d. Among all linear decision rules defined by:2

C : H 0 if w T f < b H 1 if w T f > b (1)
where f is a feature vector to be classified, the FLD finds the weighting vector w ∈ R d that maximizes the following Fisher separability criterion:

w T (µ c -µ s )(µ c -µ s ) T w w T (Σ c + Σ s )w . ( 2
)
Few calculations show that maximizing the Fisher criterion on the training data, C trn and S trn , gives the following weighting vector w:

w = Σ c + Σ s -1 ( µ c -µ s ) (3)
with

µ c = 1 N trn N trn n=1 c trn n , µ s = 1 N trn N trn n=1 s trn n Σ c = 1 N trn -1 (C trn -µ c )(C trn -µ c ) T ,
and

Σ s = 1 N trn -1 (S trn -µ s )(S trn -µ s ) T .
The power of the ensemble classifier comes from using L different FLD classifiers all built on randomly selected subsets of d sub features denoted F 1 , . . . , F L . It is worth noting that the vector v ∈ R L of all L projections from (1) can be written:

v = Pf , (4)
where, again, f ∈ R d is a feature vector to be classified and P is a "sparse" matrix of size L × d whose l-th row contains zeros for all features not included in F l while it contains the weighting vector from the corresponding l-th base learner in all remaining elements. In the ensemble classifier as developed for steganalysis [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF], the projection v is then thresholded, to obtain a vote from each base learner, as follows:

sign (v -b) ,
where b ∈ R L represents the vector of thresholds of all L base learners (1) and the sign(x) function, applied element-wise, is 1 if x is positive, -1 is x is negative and 0 in the (unlikely) case of x = 0.

In this paper, we model the distribution of v using the multivariate Gaussian distribution and cast the process of reaching the ensemble decision within the framework of hypothesis testing. This will allow us to design optimal detectors and establish their performance. In Section IV, we extend this approach to the case of multi-class steganalysis (multiple hypothesis testing). Because we also changed the implementation of the ensemble classifier for steganalysis, we provide a detailed comparison with the original ensemble in Section III-C.

III. OPTIMAL BINARY DETECTOR USING ENSEMBLE CLASSIFIER

Let us assume that the vector of base learners' projections v, see Equation (4), follows the distribution P θ0 under the null hypothesis H 0 (features are extracted from cover images) and P θ1 under the alternative hypothesis H 1 (features extracted from stego-images with data hidden with a known payload R and a known embedding method). This constitutes the ideal scenario for a steganalyst as he/she knows the probability distribution under both hypotheses, the embedding method, and the payload R. Accepting for a moment this ideal setting, steganalysis amounts to choosing between the two following simple hypotheses:

H 0 : {v ∼ P θ0 } , H 1 : {v ∼ P θ1 } . (5)
A statistical test is a mapping δ : [START_REF] Ferguson | Mathematical Statistics: A Decision Theoretic Approach[END_REF], [START_REF] Lehmann | Testing Statistical Hypotheses[END_REF] for details). The present paper focuses on the Neyman-Pearson bicriteria approach that aims at minimizing the missed-detection probability for a guaranteed false-alarm probability. Hence, let:

R L → {H 0 , H 1 }, such that hypothesis H i is accepted if δ(v) = H i (see
K α0 = {δ : P H0 (δ(v) = H 1 ) ≤ α 0 } , (6)
be the class of tests with a false-alarm probability upperbounded by α 0 . Here, P Hi (A) stands for the probability of event A under hypothesis H i , i = {0, 1}. Among all tests in K α0 , it is necessary to find a test δ that maximizes the power function defined by the correct detection probability:

β δ = P H1 (δ(v) = H 1 ) , (7)
which is equivalent to minimizing the missed-detection probability α 1 (δ) = 1 -β δ .

When the hypotheses are simple, it follows from the Neyman-Pearson Lemma [26, Theorem 3.2.1] that the Most Powerful (MP) test in the class K α0 (6) is the Likelihood Ratio test (LRT) defined as:

δ lr (v) =        H 0 if Λ lr (v) = p θ1 (v) p θ0 (v) ≤ τ lr , H 1 if Λ lr (v) = p θ1 (v) p θ0 (v) > τ lr , (8)
where p θ0 and p θ1 denote the joint probability density function (pdf) associated with the distributions P θ0 and P θ1 , respectively, and τ lr is the solution of the equation P H0 Λ lr (v) > τ lr = α 0 to ensure that the LRT is in the class K α0 , see Equation [START_REF] Cogranne | A local adaptive model of natural images for almost optimal detection of hidden data[END_REF].

The choice of the Neyman-Pearson criterion of optimality is justified by practical consideration; in fact, when analyzing a large number of digital images the most difficult challenge is to guarantee a (very) low false-alarm probability, which is exactly the goal of the Neyman-Pearson approach that maximizes the detection accuracy under the constraint of a prescribed falsealarm probability. Other criteria, such as Fishers' separability criterion (2), Bayesian criterion, see Corollary 1, do not indicate how to set the threshold in order to guarantee a false-alarm probability and, consequently, do not provide an expression for the achievable detection accuracy for a given prescribed false-alarm rate.

The ideal scenario of testing simple hypotheses is addressed in the remainder of this section. Then, two extensions are presented within the same framework that aim at designing a detector with established statistical properties to guarantee a prescribed false-alarm probability. First, in Section III-E a practical application of the proposed test when the payload is unknown is addressed. Second, the problem of extending the proposed approach of optimal detection to multi-class classification is addressed in Section IV. This corresponds to classifying images with data hidden with different embedding methods.

A. Statistical Model of Ensemble Classifier

In the present paper, it is proposed to model the vector v of base learners' projections by a multivariate normal distribution. Fundamentally, it is hardly possible to formally prove that this model holds true whatever the features might be. However, since the number of features used by each base learner is usually quite large the use of the multivariate normal distribution is supported by invoking Lindeberg's central limit theorem (CLT) [START_REF] Lehmann | Testing Statistical Hypotheses[END_REF]Theorem 11.2.5]. This is later confirmed experimentally. Using this statistical model, v ∼ N (µ 0 , Σ 0 ), under the null hypothesis H 0 , and v ∼ N (µ 1 , Σ 1 ) under the alternative hypothesis H 1 . Here µ i and Σ i represent the expectations and covariances of base learners' projections under hypotheses H i , i = {0, 1}.

In order to simplify the presentation of the proposed test, we transform the base learners' projections as follows:

v = Σ -1/2 0 (v -µ 0 ) , (9)
where the symbol Σ -1/2 0 denotes the symmetric matrix satisfying

Σ -1/2 0 Σ -1/2 0 = Σ -1 0 .
It is straightforward to note that it is equivalent to computing the normalized projections v from base learners projections v or from vb, used to compute the votes, as the mean µ 0 is also modified in the same way. This is why, within the proposed methodology, there is no need to use any thresholds of the individual base learners. The affine transformation [START_REF] Zitzmann | Hidden information detection based on quantized Laplacian distribution[END_REF] guarantees that, under the hypothesis H 0 , the "normalized" base learners' projections v follow a multivariate normal distribution with zero mean and identity covariance matrix: v ∼ N (0, I L ) with I L the identity matrix of size L. It is important to note that the family of multivariate normal distributions N (µ 0 , Σ 0 ) remains invariant under such a transformation, see [START_REF] Ferguson | Mathematical Statistics: A Decision Theoretic Approach[END_REF]Chap. 4] and [START_REF] Lehmann | Testing Statistical Hypotheses[END_REF]Chap. 6] for details about the invariance principle in statistical decision theory.

In this paper it is further assumed that the covariance matrices Σ 0 and Σ 1 are equal; this assumption is approximately true for small payloads R, which are the focus of the present paper because it is the most difficult case for detection.

Let us denote

θ 1 = Σ -1/2 0 (µ 1 -µ 0 ) . (10)
The steganalysis detection problem can be rewritten as a choice between the two following simple hypotheses:

H 0 : { v ∼ N (0, I L )} , H 1 : { v ∼ N (θ 1 , I L )} . (11)
We have conducted a wide range of experiments to confirm the validity of this model. Some are presented among the numerical results in Section V-B.

B. Optimal LRT and Study of its Statistical Performance

As discussed in the introduction of Section III, for simple hypotheses [START_REF] Farid | Detecting hidden messages using higher-order statistical models[END_REF] the optimal statistical test that guarantees a false-alarm probability and maximal power function is the LRT [START_REF] Cogranne | An asymptotically uniformly most powerful test for LSB matching detection[END_REF]. In our case, the Likelihood Ratio (LR) between the tested hypotheses can be simplified as (see Appendix A):

Λ lr ( v) = θ T 1 v θ 1 , (12)
where,

θ 1 2 = θ T 1 θ 1 .
Note that the proposed LR, Λ lr ( v), is related to the Linear Discriminant Analysis (LDA) because both share the underlying model, the homoscedastic multivariate Gaussian distribution of data, and reach the decision in a linear fashion based on the projection vector θ 1 , see Eq. [START_REF] Thai | Statistical model of quantized DCT coefficients : Application in the steganalysis of jsteg algorithm[END_REF]. From the properties of the multivariate normal distribution, it immediately follows from the distribution of v under hypotheses H 0 and H 1 , see Equation [START_REF] Farid | Detecting hidden messages using higher-order statistical models[END_REF], that the LR Λ lr ( v), Equation [START_REF] Pevný | Steganalysis by subtractive pixel adjacency matrix[END_REF], follows the following distribution:

Λ lr ( v) = θ T 1 v θ 1 ∼ N (0, 1) under H 0 N ( θ 1 , 1) under H 1 . (13)
From here, it is straightforward to establish the statistical properties of the proposed LRT (8): Proposition 1. For any false alarm probability α 0 ∈ (0, 1), it follows from (13) that the following decision threshold:

τ lr = Φ -1 (1 -α 0 ), (14)
guarantees that P H0 Λ lr ( v) > τ lr = α 0 . Here Φ and Φ -1 denote the normal cumulative distribution function (cdf) and its inverse.

From the expression for the threshold τ lr (14), and the statistical distribution of the LR Λ lr ( v), Equation (13), the power function of the most powerful LRT δ lr is given by:

β δ lr = P H1 Λ lr ( v) > τ lr = 1 -Φ τ lr -θ 1 (15) = 1 -Φ Φ -1 (1-α 0 ) -θ 1 .
Proof. Proposition 1 is proved in Appendix A.

Two essential elements can be deduced from Proposition 1. First, thanks to the normalization of the base learners' projections, see [START_REF] Zitzmann | Hidden information detection based on quantized Laplacian distribution[END_REF], and of the LR Λ lr ( v) through the multiplication by θ 1 -1 (12), the decision threshold only depends on the prescribed false-alarm probability. Second, the performance of the proposed optimal LRT is entirely given by θ 1 , the norm of the expectation under H 1 .

Remark 1. Note that the proposed LRT is in fact a linear classifier. It is indeed straightforward from Equation ( 4) and ( 9) that for a feature vector f the proposed LRT ( 12) accepts the alternative hypothesis H 1 if, with the previous notation:

θ T 1 Σ -1/2 0 (Pf -µ 0 ) θ 1 > τ lr .
In fact, the only non-linearity in the original ensemble classifier for steganalysis comes from the majority voting, which is not used in the proposed methodology. The fact that the proposed linear decision rule performed virtually identically within our experiments (see Section V-D) makes the usefulness of the non-linearity due to majority voting in the original ensemble questionable. This claim is also supported in [START_REF] Cogranne | Is Ensemble Classifier Needed for Steganalysis in High-Dimensional Feature Spaces?[END_REF] which shows that, provided the regularization parameter is correctly set, a linear classifier can be constructed with the same performance as that of the original ensemble. This indicates that the fusion of base learners in the ensemble acts as a regularization, which seems to help the detection more that the non-linear majority vote rule.

Finally, from Proposition 1, one can also compute the decision threshold τ PE that minimizes the total probability of error under equal Bayesian priors, P E = 1 /2 (α 0 + 1 -β(α 0 )) as follows, see details in Appendix 1:

Corollary 1. The threshold given by:

τ PE = θ 1 2 , ( 16
)
minimizes the total probability of error P E . Using the threshold given in Equation ( 16), the power and the false-alarm probability of the optimal LRT, at the minimal P E , are given by:

α 0 = β δ lr = 1 -Φ θ 1 2 . (17)

C. Comparison with Original Ensemble Classifier for Steganalysis

First of all, it worth noting that the proposed methodology fundamentally differs from the majority voting rule originally proposed for the FLD ensemble for the following three reasons. First, the covariance between the base learners is taken into account. Second, while the majority voting gives the same weight to all base learners, the proposed framework, as well as some other prior works [START_REF] Chaumont | Steganalysis by ensemble classifiers with boosting by regression, and post-selection of features[END_REF], allows giving more importance to base learners that better distinguish the two classes. Last, as stated above, the proposed optimal LRT is a linear classifier. The main advantage of the proposed LRT is that, thanks to the statistical model detailed in Section III-A, the performance of our test is established in Proposition 1. Hence, one can not only design a test that guarantees a given false-alarm probability but also compute the associated threshold as well as the highest detection performance that can be expected. Moreover, the theoretically expected performance of the test can be compared with the empirically obtained one (see Section V-D).

Additionally, we also adjusted the search for the optimal parameters, the number of features used by each base learner d sub as well as the number of base learners L, see also discussion in Section III-D. The original FLD ensemble minimizes the total probability of error under equal Bayesian priors, P E = 1 /2 (P M D + P F A ), with P M D and P F A the probability of missed detection and false alarm, respectively (see the formal definitions in Section III-A). Since we proposed a statistical model for the ensemble classifier, we also use it to determine the optimal values of the parameters d sub and L in order to maximize the theoretically established detection performance, see Proposition 1. In fact the theoretical performance only depends on the expected norm of the normalized projection vector θ 1 , see Equation [START_REF] Thai | Statistical model of quantized DCT coefficients : Application in the steganalysis of jsteg algorithm[END_REF]. Hence, we proposed to maximize directly θ 1 when searching for optimal values of d sub and L, instead of determining these parameters by maximizing the empirical total probability P E as proposed in the original version of ensemble classifier.

Finally, we wish to emphasize that the computational cost of the proposed LRT is very similar to the original ensemble. We note that the training of a single FLD base learner requires to evaluate the scatter matrix denoted Σ c + Σ s in Equation 3and its inversion. These two operations are respectively of complexity O(N trn d 2 sub ) and O(d 3 sub ). Hence, for a fixed value of the parameters d sub and L the training complexity is O(N trn Ld 2 sub + Ld 3 sub ), where N trn is the size of the training set. To evaluate the performance through the mean projection θ 1 the proposed methodology requires to estimate the covariance matrix Σ 0 between base learners' projections and then inverting this matrix. These operations are, respectively, of complexity O(N cv L 2 ) and O(L 3 ), with N cv the size of the cross-validation set (in our implementation, the training set is, by default, split into subsets of equal size for training and cross validation). Hence, the proposed methodology is slightly more computationally expensive. Note, however, that the difference is of order O(N cv L 2 ) + O(L 3 ), which is rather negligible compared the to overall complexity of the ensemble classifier, which is O(N trn Ld 2 sub + Ld 3 sub ) since usually d sub >> L 2 . As for the original ensemble, the overall complexity remains much smaller than others popular machine learning algorithms (e.g. SVM).

D. Implementation of the Proposed Methodology

The implementation of the proposed LRT is similar to the original ensemble classifier as detailed in [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF]. Here, we emphasize the main implementation differences. The flowchart in Figure 1 represents the proposed implementation for the training (and testing) with the proposed methodology. First, as in [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF], the features are divided into two subsets for training and testing, both equal in size by default. The testing set is used to compute the numerical results in this paper, see Figure 1, but of course, in "real life" all the samples will be used for training, the testing set will be provided by data to classify for which the "label" (either cover or steganographic images) remains unknown. Then, the training of each base learner is performed only on a subset of the training set, while the remaining subset of the training set is used for crossvalidation to measure the performance of the base learners. In contrast to the original ensemble, we did not use bootstrapping but simply split the training set into two subsets of equal size (hence each containing one quarter of the entire feature set). All these subsets are kept disjoint in both the original ensemble as well as the proposed method.

As briefly discussed in Section III-C, we use the proposed statistical model for the ensemble classifier in order to search for the optimal values of the parameters d sub and L. To this end, one only needs to measure the base learners' projection covariance Σ 0 and the expectations µ 0 and µ 1 under each hypothesis H 0 and H 1 , respectively. Hence, for a fixed value of d sub , each base learner is added to the ensemble as follows (see the bottom row of Figure 1): (1) a randomly selected subset of features of size d sub is selected (not shown in Figure 1) (2) the training set is used to obtain the projection vector w, see Equation (3), from an FLD base learner (each base learner is trained as a usual FLD except that we do not need the threshold value b); (3) the cross-validation set is used, with the projection vector w, to update the base learners' projection covariance Σ 0 and expectations µ 0 and µ 1 . Once these values have been updated, it is straightforward to recompute θ 1 using Equation [START_REF] Thai | Statistical model of quantized DCT coefficients : Application in the steganalysis of jsteg algorithm[END_REF]. Because we reused the search method used to find the parameters dsub and L from the original ensemble classifier [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF] (shown by rhombuses in Figure 1), we refer the reader to this paper for further details.

It also worth noting that in the original ensemble a bootstraping is done so that, for each added base learner, the crossvalidation is performed on a different subset of the training set and all the training samples are considered to estimate the performance using the out-of-bag (OOB) estimate of the testing error. Because this can hardly be done with the base learners' projections, we change the subset used for the crossvalidation only when the number of features d sub is changed and keep the split of the training set unchanged to find the number of base learners L.

Finally, note that the estimation of the parameters of the multivariate normal distribution is also difficult in this context. While the estimation of expectations is rather simple, the estimation of the covariance matrices becomes difficult when the number of base learners increases: for L base learners, 1 /2L(L + 1) covariances have to be estimated. Since the estimation of the covariance matrix of projections Σ 0 plays a crucial role, once the optimal values of d sub and L are found, we estimate the mean µ 0 and the covariance matrix Σ 0 on ten random cross-validation splits. This is especially important for the covariance matrix estimation: an accurate estimation ensures the validity of the theoretical results for low false-alarm probability α 0 around 10 -3 , see Section V-D.

E. Extension to Steganalysis of Unknown Payload

Until now, the most powerful LRT was designed based on the assumption that the payload R was known. In practice, however, the steganalyst usually has little or no information about the payload size. In this case, the statistical problem of detecting the presence of hidden data becomes a test between the following composite hypotheses, reusing the notation from Section III-A:

H 0 : {v ∼ P θ0 } , H 1 : {v ∼ P θ R , R ∈ (0, 1]} , (18)
with R the payload expressed in bits per pixel (or per non-zero AC DCT coefficient for JPEG images). The ultimate goal of steganalysis is to design a statistical test that is Uniformly Most Powerful (UMP): a test which coincides with the Most Powerful LRT whatever the embedded payload might be. Generally speaking, such a test rarely exists except when the hypotheses have a monotone Likelihood Ratio of a scalar parameter, see [START_REF] Lehmann | Testing Statistical Hypotheses[END_REF]Theorem 3.4.1]. We adopt here the assumption that, whatever the payload used for training, when testing an image, the expectations of base learners projections are increasing with the payload. This assumption is sometimes referred to as the "shift hypothesis" and was recognized for the first time in [START_REF] Ker | Batch steganography and pooled steganalysis[END_REF]. We further assumed that the covariance matrix remains the same for any (small) payload; this is reasonable for small payloads, which is the focus of this paper. With the proposed statistical model, the problem of detecting messages of unknown length can be formally written as a choice between the following composite hypotheses:

H 0 : { v ∼ N (0, I L )} , H R : { v ∼ N (f (R)θ 1 , I L )} , (19)
with f : [0, 1] → R + a monotone increasing function. Roughly speaking f (R) represents the impact of the payload R, how much it "pushes" all the expectations of all base learners' projections v, see Equation ( 9).

Proposition 2.

(1) Assuming that the model ( 19) holds true, then for any α 0 ∈ (0, 1) the LRT [START_REF] Cogranne | An asymptotically uniformly most powerful test for LSB matching detection[END_REF] with the LR defined as in [START_REF] Pevný | Steganalysis by subtractive pixel adjacency matrix[END_REF] is UMP in the class K α0 for testing hypotheses [START_REF] Jiang | Similarity-based online feature selection in content-based image retrieval[END_REF] provided that the decision threshold is chosen as in Equation ( 14).

(2) For any payload R ∈ (0, 1] and for any α 0 ∈ (0, 1), the power function of the UMP LRT ( 8) is given by:

β δ lr = P H1 Λ lr ( v) > τ lr = 1 -Φ τ lr -f (R) θ 1 (20) = 1-Φ Φ -1 (1-α 0 ) -f (R) θ 1 Proof. Proposition 2 is proved in Appendix A.
It is important here to stress that, in practice, the function f (R) is not used in the proposed UMP LRT. The proposed approach requires only the knowledge of θ 1 and the covariance matrix Σ 0 , which can both be estimated for any fixed payload as in the case of simple hypotheses. The function f (R) is primarily introduced to formalize the condition under which the optimality of the proposed approach holds, which is when a given embedding scheme pushes the projections of all base learners' projections along the same direction θ 1 regardless of the payload. In practice, f (0) = 0 and is non-linearly increasing with the payload; it behavior depends on the steganographic embedding algorithm, its content adaptivity, the embedding domain (spatial or JPEG), and many other attributes. Besides, since the power of the proposed UMP LRT [START_REF] Su | Hierarchical ensemble of global and local classifiers for face recognition[END_REF] depends on f (R), in the case of an unknown payload, R, the power is also unknown. This expression is provided in order to (a) contrast it with the case of simple hypothesis [START_REF] Pevný | Multiclass detector of current steganographic methods for jpeg format[END_REF] and (b) fully characterize the statistical properties of the UMP LRT.

Note that there exist other possibilities to construct a test that is optimal regardless of the payload. It has been proposed in [START_REF] Zitzmann | Statistical decision methods in hidden information detection[END_REF] to use the Local Asymptotic Normality [START_REF] Cam | Asymptotic Methods in Statistical Decision Theory[END_REF] in order to design an asymptotically UMP test around a chosen payload R . This approach can however hardly be used for the problem addressed in this paper because 1) we explicitly wish to address the case in which the payload is unknown and no prior information on this is available and, 2) it is hardly possible to indefinitely increase the number of base learners. A more practical approach has been proposed in [START_REF] Pevný | Detecting messages of unknown length[END_REF] by training the classifier with a mixture of payloads. The author has shown that the best results are obtained using a uniform distribution of payload. This approach is contrasted with the proposed UMP test in Section V-E for two different embedding algorithms. Note that we tested many other embedding algorithms but, for brevity, only two are presented. The results obtained show the relevance of the proposed test. While the proposed UMP test performs very well, either when trained on a mixed payload or with a wrong payload, the original ensemble with majority voting performs significantly worse, see Section V-E.

IV. OPTIMAL CRITERIA AND DESIGN OF STATISTICAL TESTS FOR MULTI-CLASS STEGANALYSIS

The previous Section III focused on binary hypothesis testing, that is when the steganalysts' goal is to detect a specific embedding scheme. The goal of this section is to extend the proposed methodology to classification of multiple steganographic algorithms. Before presenting the proposed optimal statistical test, let us first formally state the problem of multi-class classification (or multiple hypotheses testing) and present the criterion of optimality that will be used, and that has never been studied for steganalysis.

A. Multi-class Classification: Problem Statement

In the context of multi-class classification a set of M different steganographic algorithms is suspected by the steganalyst. As in Sections III-A -III-B, let us assume that the payload R is known and that the base learners have been trained and their projection vectors normalized, see Equation [START_REF] Zitzmann | Hidden information detection based on quantized Laplacian distribution[END_REF]. Hence, we have a set of M + 1 possible hypotheses:

           H 0 : { v ∼ N (0 , I L )} H 1 : { v ∼ N (θ 1 , I L )} . . . . . . H M : { v ∼ N (θ M , I L )} , (21)
where, again, v are the normalized base learners' projections defined in [START_REF] Zitzmann | Hidden information detection based on quantized Laplacian distribution[END_REF] and θ m , m = {1, . . . , M }, represents the expected value of the normalized base learners' projections under hypothesis H m .

Remark 2. It is worth noting that the hypotheses testing problem ( 21) is similar to the detection of signal in Gaussian noise, which has been extensively studied but seldom within the framework of hypothesis testing, such as in [START_REF] Middleton | An introduction to statistical communication theory[END_REF]- [START_REF] Van Trees | Detection, Estimation, and Modulation Theory[END_REF]. The problem of multi-class classification is usually addressed using a vector decision function δ mc : R L → {0, 1} M +1 defined by M + 1 "sub-decision functions" δ mc = (δ mc 0 , . . . , δ mc M )

T such that hypothesis H m , m = {0, . . . , M },

is accepted if δ mc m ( v) = 1 and ∀ v ∈ R L , M m=0 δ mc m ( v) = 1.
With this notation, let us formally define the following probabilities, useful for evaluating the average performance of a statistical multi-class (multiple hypotheses) test:

     α 0 = P (F A) = E H0 (1 -δ mc 0 ( v)) P Hm (M D) = E Hm (δ mc 0 ( v)) β m = P Hm (CC) = E Hm (δ mc m ( v)) . (22)
Here, P (F A) corresponds to the probability of false-alarm, P Hm (M D) denotes the probability of missed detection under hypothesis H m , and P Hm (CC) represents the probability of correct classification under hypothesis H m with the probability of erroneous classification defined by

α m = 1 -β m = P Hm (EC) = E Hm (1 -δ mc m ( v)).

B. Minimax criterion and Optimal Minimax Test

The Neyman-Pearson criterion of optimality is no longer usable for multi-class classification. A natural extension of the Neyman-Pearson approach for multi-class classification is to seek a test that, first, guarantees a prescribed false alarm probability α 0 and, second, maximizes the worst correctclassification probability β m with respect to all possible alternative hypotheses H 1 , . . . , H M . The idea behind focusing on the worst case is similar to the example given in [START_REF] Ker | A mishmash of methods for mitigating the model mismatch mess[END_REF]. Consider two tests, one that provides a 60% correct classification probability for all alternative hypotheses and one that has a correct classification probability of 80% for the first half of the alternatives hypotheses and a correct classification probability of 45% for the second half of alternative hypotheses. Which test is more desirable to use in practice? It is argued in [START_REF] Ker | A mishmash of methods for mitigating the model mismatch mess[END_REF] that the first test is more stable. Another crucial argument is that the steganographer may choose the embedding method that the steganalyst detects the worst: one for which the correct classification probability is 45%.

In hypothesis testing theory, a test that achieves a given false alarm probability α 0 while maximizing the worst case correct-classification probability β m , is referred to as the "constrained minimax test" [START_REF] Baygün | Optimal simultaneous detection and estimation under a false alarm constraint[END_REF]. Formally, for solving the detection problem defined in [START_REF] Fan | Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation[END_REF], let us define the class of all tests with a false-alarm probability bounded by α as:

D α0 = δ : R L → {0, 1} M +1 , M m=0 δ mc m ( v) = 1 , (23)
E H0 (1 -δ mc 0 ( v)) ≤ α . (24)
Among all the tests in ( 23), we try to find a test that maximizes the smallest correct-classification probability β m , or, equivalently, minimizes the maximal α m . Hence, the test δ mc = (δ mc 0 , . . . , δ mc M )

T is formally said to be an optimal constrained minimax test if, for any other decision function δ mc = δ mc 0 , . . . , δ mc M T , δ mc satisfies:

max m>0 E Hm 1 -δ mc m ( v) ≤ max m>0 E Hm 1 -δ mc m ( v) . ( 25
)
Note that while the usual minimax criterion aims at finding a test that minimizes the maximal α m [START_REF] Bas | Break our steganographic system -the ins and outs of organizing boss[END_REF] (over all M + 1 hypotheses), the constrained minimax test introduces a constraint on the false-alarm probability α 0 , which is similar to the Neyman-Pearson criterion of optimality for the binary case.

A practical way to find the constrained minimax test follows from the following theorem [START_REF] Baygün | Optimal simultaneous detection and estimation under a false alarm constraint[END_REF].

Theorem 1. The test δ mc = (δ mc 0 , . . . , δ mc M )

T with the assignments

   δ mc 0 ( v) = 1 if max m∈{1,...,M } w m +θ T m v ≤ τ mc δ mc k ( v) = 1 if max m∈{1,...,M } w m +θ T m v = w k +θ T k v > τ mc (26
) is an optimal constrained minimax test provided the weights w m are such that the following necessary and sufficient conditions hold: 1) δ mc ∈ D α0 ; 2) all the erroneous classification probabilities are equal:

E H1 1 -δ mc 1 ( v) = . . . = E H M 1 -δ mc M ( v) . ( 27
)
Such a test is referred to as an "equalizer test" in the sense that ∀m ∈ {1, . . . , M } , P Hm

(EC) = E Hm (1 -δ mc m ( v)) = α mc .
We note that the log-likelihood ratio between the hypotheses H m and H 0 can be written as θ T m v, see [START_REF] Pevný | Steganalysis by subtractive pixel adjacency matrix[END_REF] and Appendix A. Hence, the term w m +θ T m v corresponds to the likelihood ratio between hypotheses H m and H 0 scaled by e wm . Remark 4. A formal proof of Theorem 1 can be found in [START_REF] Baygün | Optimal simultaneous detection and estimation under a false alarm constraint[END_REF]. One way way to understand this theorem is that if all probabilities α m were not equal, then one could randomly accept the k-th hypothesis, with k = arg max m∈{1...M } α m (the one with the highest erroneous classification rate), instead of the j-th hypothesis, with j = arg min m∈{1...M } α m (which has the smallest erroneous classification rate) to design a test with a smaller maximal erroneous classification probability.

Remark 5. It also worth noting that the proposed minimax test coincides with the Bayesian test for the testing problem [START_REF] Fan | Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation[END_REF], see [START_REF] Ferguson | Mathematical Statistics: A Decision Theoretic Approach[END_REF]Chap. 6] when the prior probabilities p 0 , . . . , p M satisfy w m = log(p m /p 0 ). The prior distribution, which corresponds to the minimax test, is sometimes referred to as the "least favorable distribution" since it is the one for which the correct classification probability is the smallest, see [START_REF] Lehmann | Testing Statistical Hypotheses[END_REF]Chapter 3.8].

C. Discussion and Implementation of Proposed Minimax Test

Before discussing the results of the proposed minimax test, in Section V-F, let us first recall that such a criterion of optimality, that guarantees a false-alarm rate for multi-class case, has never been studied in steganalysis. Let us also briefly describe the implementation of the proposed minimax test which is depicted in Figure 2. Again, we have chosen to train an ensemble for each pair H k against H m , with k = m, see Remark 3 and the training step in Figure 2. Then all the base learners are aggregated: that is the features to be classified are projected onto projection vectors of all base learners from all classifiers; this in fact does not cause computation issue even though the ensembles may have different base learners which may be build on subset of features of different dimension. Again, as in the binary testing case discussed in Section III-D, we note that in fact, the proposed methodology only needs to estimate the base learners' projection expectations and covariances. Hence, from the aggregated projection vectors, it is straightforward to estimate the covariance matrix Σ 0 and the expectation µ 0 on the cross-validation subset. This ensures that all projections are normalized and that the proposed model [START_REF] Fan | Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation[END_REF] holds true. This is the normalization step in Figure 2. Finally, the means of the normalized projections are computed from the cross-validation subset for each alternative hypothesis. From this last step, the application of the proposed minimax test is straightforward. Note that because of the higher number of projections involved in this case, the accuracy of the covariance matrix estimate is extremely important, hence the necessity of estimating it with many different cross-validation splits described in Section III-C. Note that the classifiers trained without H 0 are used in the same way, and, from experiments, we noted that their use substantially helps to decrease the misclassification of embedding schemes (or hypotheses).

It should be noted that the weights w m , which are required to "equalize the erroneous classification probability" [START_REF] Cogranne | Is Ensemble Classifier Needed for Steganalysis in High-Dimensional Feature Spaces?[END_REF], are computed numerically because it is hardly possible to find an analytic formula for the weights w m that "equalize" erroneous classification probabilities. Indeed, let us define:

Λ mc = θ T 1 v, . . . , θ T M v T ,
which correspond to the M values of the log-likelihood ratios between hypotheses H m , m ∈ {1, . . . , M }, and H 0 .

It is straightforward to establish from the statistical model of normalized base learners' projections v that Λ mc follows, under hypothesis H m , a multivariate normal distribution with expectation

E Hm θ T n v = θ T n θ m and covariance Cov θ T m v, θ T n v = θ T m θ n .
Even though some accurate approximations of multivariate normal probabilities may be found [START_REF] Gassmann | Computing multivariate normal probabilities: A new look[END_REF], [START_REF] Genz | Computation of multivariate normal and t probabilities[END_REF], analytically computing the threshold that guarantees that

P H0 θ T 1 v ≤ τ mc ∩ . . . ∩ θ T M v ≤ τ mc = 1- α 0 is hardly possible. Similarly, the correct classification prob- ability P Hm n =m θ T n v < θ T m v ∩ θ T m v > τ mc
requires integrating a multivariate normal pdf over a complicated domain, which is quite difficult to achieve even numerically. We thus opted to compute the weights numerically using a gradient-free Nelder-Mead (downhill simplex) method [START_REF] Nelder | A simplex method for function minimization[END_REF] on the training samples. The initial weights are set to zero while the threshold is fixed at τ mc = Φ -1 (1 -α 0 /M ), which is easily shown to guarantee the prescribed false-alarm probability α 0 in the case of orthogonal vectors θ m . Then, on the cross-validation subset, we numerically compute the "weights" w m that (1) guarantee the prescribed false-alarm rate α 0 and (2) equalize the erroneous-classification probability α m , ∀m ∈ {1, . . . , M }. This was done using Matlab built-in optimization functions by minimizing a weighted sum of the two objective functions.

V. NUMERICAL SIMULATIONS AND RESULTS

A. Common core of all experiments

All results presented in this paper are obtained on BOSSbase 1.01 [START_REF] Bas | Break our steganographic system -the ins and outs of organizing boss[END_REF] containing 10, 000 512×512 gray-scale images. The detection errors are always computed by averaging over 10 different random database splits. Four spatial domain embedding schemes were used: HUGO [START_REF] Pevný | Using high-dimensional image models to perform highly undetectable steganography[END_REF] with bounding distortion (HUGO-BD) implemented using the Gibbs construction [START_REF] Filler | Gibbs construction in steganography[END_REF], the Wavelet Obtained Weights (WOW) [START_REF] Holub | Designing steganographic distortion using directional filters[END_REF] algorithm, the spatial version of UNIversal WAvelet Relative Distortion (S-UNIWARD) [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF], and the recent scheme based on statistical detectability [START_REF] Sedighi | Content-Adaptive Steganography by Minimizing Statistical Detectability[END_REF], [START_REF] Sedighi | Content-adaptive pentary steganography using the multivariate generalized gaussian cover model[END_REF]. For spatial domain steganalysis, we used the second-order Subtractive Pixel Adjacency Matrix (SPAM) [START_REF] Pevný | Steganalysis by subtractive pixel adjacency matrix[END_REF] feature set of dimensionality 686, the Spatial Rich Model (SRM) [START_REF] Fridrich | Rich models for steganalysis of digital images[END_REF] with dimensionality 34, 671 and its selection-channel-aware version (maxSRMd2) [START_REF] Denemark | Selection-channel-aware rich model for Steganalysis of digital images[END_REF]. Several embedding methods for JPEG domain have also been included in our tests, namely, nsF5 [START_REF] Fridrich | Statistically undetectable jpeg steganography: Dead ends challenges, and opportunities[END_REF], the Entropy-Based Steganography (EBS) [START_REF] Wang | An efficient jpeg steganographic scheme based on the block entropy of dct coefficients[END_REF], the Uniform Embedding Distortion (UED) [START_REF] Guo | An efficient jpeg steganographic scheme using uniform embedding[END_REF], and the JPEG domain version of UNIWARD referred to as J-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF]; the two last being the state-ofthe-art in JPEG domain steganography that does not use side information. For JPEG domain embedding scheme with side information we used the Perturbed Quantization (PQ) [START_REF] Fridrich | Statistically undetectable jpeg steganography: Dead ends challenges, and opportunities[END_REF], the side-informed version of EBS (SI-EBS) [START_REF] Wang | An efficient jpeg steganographic scheme based on the block entropy of dct coefficients[END_REF], and the sideinformed version of UNIWARD, SI-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF]. The four different feature sets that have been used for JPEG image steganalysis are the Cartesian-calibrated JPEG Rich Model (CC-JRM) [START_REF] Kodovskỳ | Steganalysis of JPEG images using rich models[END_REF] with 22, 510 features, the CF [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF] feature set with 7, 850 features, the spatial rich model with fixed quantization (SRMQ1) [START_REF] Fridrich | Rich models for steganalysis of digital images[END_REF] of dimensionality 12, 753, and the union of SRMQ1 and CC-JRM, referred to as JSRM [START_REF] Kodovskỳ | Steganalysis of JPEG images using rich models[END_REF], whose dimensionality is 35, 263. All feature extractors used in this paper and most embedding algorithms can be downloaded from the DDE website at http://dde.binghamton.edu/download.

Note that the payload is measured in bits per pixel (bpp) for spatial domain steganography and in bits per non-zero AC coefficients (bpnzAC) for JPEG domain steganography. The JPEG images were created using the imwrite function from Matlab with quality factors 75 and 90 for both side-informed and non-side informed schemes. The precover was either the uncompressed image (for SI-UNIWARD and SI-EBS) or a JPEG image compressed with quality factor 90 and 100 (for the PQ algorithm). Finally, note that for HUGO-BD, the switch T was set to 255 to remove the weakness identified during the BOSS contest. The stabilizing constant σ for UNIWARD was set to 1 for the spatial version and to 10 -6 for JPEG versions to prevent the attack proposed in [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF].

Finally, note that when the detection accuracy is measured as the total probability of error under equal Bayesian priors, P E = 1 /2 (P M D + P F A ), we used the threshold τ PE as given in Corollary 1, Equation ( 16).

B. Experiment and Comparison on Simulated Data

To verify the sharpness of the theoretical results, we selected the covariance matrix corresponding to five randomly selected base learners for the experiment on multiple hypothesis testing reported in Table X. It used the SRM features and three different embedding schemes, WOW, S-UNIWARD, and HUGO-BD, all with payload R = 0.4. The estimated covariance under hypothesis H 0 is given by:

Σ 0 =       0.
     
, which clearly shows a very high correlation between the base learners' projections. Then, a total of 10 7 samples following a multivariate normal distribution with this covariance have been randomly generated for each hypothesis and the proposed optimal binary test has been applied and compared with the theoretically established power function β δ lr , see [START_REF] Pevný | Multiclass detector of current steganographic methods for jpeg format[END_REF]. Figure 3 shows the ROC curves for the binary test of H 0 (covers) versus hypotheses H 1 and H 2 (corresponding to WOW and S-UNIWARD at R = 0.4). These results indicate that the proposed optimal LRT performs almost exactly as theoretically established. For comparison, we also include the results obtained with the original ensemble classifier with the majority voting (denoted EC-MV). Note that for a very low L, the optimal LRT performs much better, see also Figure 6. Most importantly, however, establishing the statistical properties of the EC-MV detector is quite expensive. Even in this toy example, drawing the ROCs required 32 evaluations of the multivariate normal cumulative distribution function (cdf) to establish the probability of having at least N c ∈ 0, . . . , 5 base learner votes. More precisely, in this toy example, one needs to compute the probability that exactly N c base learners among 5 classify an image as stego. Since there exists 5 Nc possibilities that exactly N c base learners classify an image as stego, this requires evaluating the same number of time a normal mutivariate cdfs. For establishing the performance of the original ensemble classifier with a variable threshold on the number of votes, this should be done for N c ∈ 0, . . . , 5. This example can be extended easily to L base learners when one wants to establish the probability of false alarm and the Fig. 4: Comparison between the theoretical normal distribution and the empirical distribution of the proposed LR under H 0 on one half of BOSSbase [START_REF] Bas | Break our steganographic system -the ins and outs of organizing boss[END_REF] used for testing.

power as a function of the number of base learner votes using a multivariate Gaussian model or, in fact, any statistical model.

For each threshold N c on the number of base learners that classify an image as stego, there is L Nc possibilities. So if one wants to establish the power and the false-alarm probability for all possible values of N c , so that it can select the one that suits it needs in terms of the false-alarm probability for instance, this requires a total of 2 L possibilities. A similar complication occurs when optimizing the threshold of each base learner to maximize the detection accuracy. This will also be extremely time consuming to compute using a statistical model of base learners for the same reason. A numerical joint optimization over L thresholds of all base learners will certainly be computation expensive as well.

C. Relevance of the Proposed Model

A wide range of numerical experiments have been performed to confirm the assumption of mutivariate normality of base learners' projections. For brevity and clarity, in this section we include numerical results that support the claim that the proposed LR Λ lr ( v) follows a normal distribution with 0 mean and unit variance under H 0 , which is crucial to guarantee a prescribed false-alarm probability, see Proposition 1.

First, Figure 4 shows a comparison between the theoretical Gaussian distribution of the LR, Λ lr ( v), and the empirical distribution obtained with optimal d sub and L for three different algorithms (WOW [START_REF] Holub | Designing steganographic distortion using directional filters[END_REF], S-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF], and HUGO-BD [START_REF] Pevný | Using high-dimensional image models to perform highly undetectable steganography[END_REF]). This provides visually interpretable results that show the match between the empirical distribution and the theoretical model.

Next, Tables I ans II show the χ 2 goodness-of-fit (GOF) score obtained when comparing the empirical LR Λ lr ( v) distribution and one with the assumed pdf N (0, 1) under hypothesis H 0 . Because of the rather high number of testing data, we have used 100 bins to apply the χ 2 GOF test. Hence, this score should follow a χ 2 distribution with 100 degree of freedom, that has an expectation of 100 with variance 200, with higher scores indicating a larger deviation from the assumed model. [START_REF] Sedighi | Content-Adaptive Steganography by Minimizing Statistical Detectability[END_REF] and gathering SRM and maxSRMd2 [START_REF] Denemark | Selection-channel-aware rich model for Steganalysis of digital images[END_REF] features.

We hypothesize that the normal distribution of base learners' projection is a consequence of CLT. Hence, the model should be more accurate when increasing the number of features D, the number of features used by each base learner d sub , and the number of base learners L. Table I shows the χ 2 GOF score for the content-adaptive scheme [START_REF] Sedighi | Content-Adaptive Steganography by Minimizing Statistical Detectability[END_REF] when using both SRM and maxSRMd2 features [START_REF] Denemark | Selection-channel-aware rich model for Steganalysis of digital images[END_REF], to be able to increase the number of features. As shown in Table I, when the number of base learners L increases, the assumed model becomes more accurate. Similarly, when d sub increases the goodness also generally increases, however, one can note that when it becomes too large the empirical distribution starts diverging significantly from the expected model. This can be explained by the fact that in this case the base learners become strongly correlated, which destabilizes the inversion of the covariance matrix Σ 0 .

Next, we wanted to simulate the use of a larger database, so that we would be able to learn more accurately the diversity of [START_REF] Guo | An efficient jpeg steganographic scheme using uniform embedding[END_REF] and JSRM [START_REF] Kodovskỳ | Steganalysis of JPEG images using rich models[END_REF] features. images in BOSSbase. To this end, Table II shows the χ 2 GOF score when the training size is increased. The results presented in Table II have been obtained using UED [START_REF] Guo | An efficient jpeg steganographic scheme using uniform embedding[END_REF] and JSRM [START_REF] Kodovskỳ | Steganalysis of JPEG images using rich models[END_REF] features with the ratio of data used for training ranging from 0.3 to 0.9. These results confirm that when the diversity of images can be learned more accurately, the proposed model also becomes more accurate.

Tables I and II also show the limit of the proposed model because a χ 2 GOF score of about 200 shows a non-negligible deviation from the assumed model. The proposed model is in fact not perfect and especially we note that, because the learning is carried out over a range of images, some images act as outliers because they have singular properties (very smooth content, underexposed scenes, etc. . . . ). We also note, see Table II that the accuracy of the model is estimated on the testing set, while parameters are estimated using the training set and that those may be slightly different.

D. Numerical Results for the Binary Case

While the main goal of the present paper is to analytically establish the statistical properties of the ensemble classifier within the proposed framework of hypothesis testing and to extend its scope, it is also very important to ensure that the proposed optimal test does not perform significantly worse than the original EC-MV detector [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF]. To this end, Figure 5 shows a comparison between the EC-MV and the proposed optimal LRT. These results were obtained by searching for the optimal values of the parameters d sub and L for each detector separately, that is by minimizing P E for the EC-MV and maximizing the theoretical power function β δ lr for the proposed LRT, see Section III-C. For diversity, the top row of Figure 5 shows a comparison between the EC-MV and the proposed optimal LRT for spatial domain steganography: the same feature set (SRM) is used with three different embedding schemes: WOW in Figure 5a, S-UNIWARD in Figure 5b, and HUGO-BD in Figure 5c. The bottom row of Figure 5 shows a comparison between the proposed optimal LRT and the EC-MV for J-UNIWARD using three different feature sets: the CC-JRM in Figure 5d, the CF in Figure 5e, and the JSRM in Figure 5f. Note that the detection performance is measured in both figures using the usual mean probability of error P E even though the proposed LR is designed to maximize the detection power under a false-alarm probability constraint. Figure 5 shows that the proposed optimal LRT achieves basically the same performance as the EC-MV detector. However, both detectors behave differently with respect to the parameters d sub and L. This is demonstrated in Figure 6, which shows the total probability of error P E as a function of L for a few fixed values of d sub . As discussed at the beginning of Section V, the proposed optimal LRT performs much better for small values of L or for small values of d sub . For large values of L and d sub the performance of both detectors becomes very similar. The results presented in Figure 6 were obtained with the CC-JRM feature set and J-UNIWARD at payload R = 0.4 bpnzAC. Similar trends have been observed for other feature sets and embedding methods.

Finally, we wanted to verify the accuracy of theoretically established results, false alarm probability and power function, see Proposition 1. To this end, Figure 7 presents a comparison between the theoretically established optimal LRT power β δ lr , see [START_REF] Pevný | Multiclass detector of current steganographic methods for jpeg format[END_REF], as a function of the false-alarm (ROC curve), and the empirical power function obtained on the testing set. The results were obtained using the JSRM feature set for J-UNIWARD with payload R = 0.4 bpnzAC. Similar results have been obtained with other feature sets and embedding methods in both the spatial and JPEG domain. Figure 7 shows that the theoretically established power function almost perfectly matches the empirical power function. However, as discussed above, using a very high number of features for each base leaner d sub , typically beyond the optimally found parameter, would certainly decrease the accuracy of the theoretical results.

Finally, we would like to emphasize the possibility of the proposed approach to guarantee a prescribed false-alarm probability. To this end, Table III compares the empirical and the theoretical false-alarm probabilities for three different threshold values that correspond respectively to α 0 = {2, 1, 0.5} • 10 -3 . The results presented in Table III have been obtained with the parameters L and d sub always set to their optimal values. We note that, generally, the empirical false-alarm probabilities are close the theoretical ones. However, these results have been obtained with 10 equally sized random splits of BOSSbase for training and testing, giving us a total of 50, 000 LR values. Thus, it is important to note that the very low prescribed false-alarm probability corresponds to the limit of what can be obtained within this setup as measuring a false-alarm probability of 10 -3 corresponds to 50 images. In practice, it would be interesting to study even much lower falsealarm probabilities, but in such a case the use of the CLT is disputable as it is not relevant to model the tails of distribution.

Similarly to Table III, Figure 8 shows a comparison between the theoretically established false-alarm probability as a function of the decision threshold, 1 -Φ(τ lr ) = α 0 , see [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF], and the empirically measured false-alarm probability obtained on the testing set. Again, for brevity, only the results obtained from J-UNIWARD with payload R = 0.4 bpnzAC and steganalysis using the JSRM feature set are shown. From this figure, the empirical and theoretical false-alarm probabilities are very close and similar trends can be found for other embedding methods and feature sets, see Table III.

The results presented in Figure 8 and in Table III clearly demonstrate that it is feasible in practice to accurately guarantee even a low false-alarm rate (typically around α 0 = 10 -3 ).

Feature

Embedding

α 0 = α 0 = α 0 = set algorithm 0.2% 0.1% 0.05% SRMQ1 [43]
WOW [START_REF] Holub | Designing steganographic distortion using directional filters[END_REF] 0.204% 0.134% 0.094% S-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF] 0.206% 0.120% 0.062% SRM [START_REF] Fridrich | Rich models for steganalysis of digital images[END_REF] WOW [START_REF] Holub | Designing steganographic distortion using directional filters[END_REF] 0.224% 0.146% 0.100% S-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF] 0.234% 0.150% 0.086% maxSRMd2 [START_REF] Denemark | Selection-channel-aware rich model for Steganalysis of digital images[END_REF] MiPOD [START_REF] Sedighi | Content-Adaptive Steganography by Minimizing Statistical Detectability[END_REF] 0.156% 0.120% 0.094% CF [START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF] EBS [START_REF] Wang | An efficient jpeg steganographic scheme based on the block entropy of dct coefficients[END_REF] 0.194% 0.112% 0.076% SI-EBS [START_REF] Wang | An efficient jpeg steganographic scheme based on the block entropy of dct coefficients[END_REF] 0.238% 0.138% 0.086% J-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF] 0.244% 0.142% 0.102% SI-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF] 0.200% 0.118% 0.078% nsF5 [START_REF] Fridrich | Statistically undetectable jpeg steganography: Dead ends challenges, and opportunities[END_REF] 0.140% 0.100% 0.074% UED [START_REF] Guo | An efficient jpeg steganographic scheme using uniform embedding[END_REF] 0.210% 0.120% 0.076% JSRM [START_REF] Kodovskỳ | Steganalysis of JPEG images using rich models[END_REF] EBS [START_REF] Wang | An efficient jpeg steganographic scheme based on the block entropy of dct coefficients[END_REF] 0.252% 0.172% 0.120% SI-EBS [START_REF] Wang | An efficient jpeg steganographic scheme based on the block entropy of dct coefficients[END_REF] 0.216% 0.134% 0.092% J-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF] 0.244% 0.136% 0.076% SI-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF] 0.180% 0.122% 0.084% nsF5 [START_REF] Fridrich | Statistically undetectable jpeg steganography: Dead ends challenges, and opportunities[END_REF] 0.222% 0.142% 0.094% UED [START_REF] Guo | An efficient jpeg steganographic scheme using uniform embedding[END_REF] 0.234% 0.130% 0.062%

E. Results for Composite Hypotheses: Unknown Payload

Tables IV and VI show the detection power β, or correct detection probability, Equation [START_REF] Thai | Statistical detection of data hidden in least significant bits of clipped images[END_REF], obtained with the optimal LRT that knows the embedding rate (the clairvoyant test), the LRT when the ensemble classifiers were trained on a uniform mixture of payloads, and the proposed UMP test when the ensemble classifiers were trained with a fixed payload R = 0.4. This should be contrasted with Tables V and VII also showing the detection power under the same setting with the original ensemble classifier trained to minimize P E . While the detection power for clairvoyant detectors produces almost the same results, when either trained on payload mixture or with a fixed payload R = 0.4, the original ensemble performs significantly worse. This can be explained in part by the non-linearity of the majority vote, which tends to classify steganographic images with a smaller payload than the one used for training as covers. Note that here, the false-alarm probability α 0 is again set at 0.075 i.e. 7.5%. Note also that, for the original ensemble classifier, we replaced the majority voting rule so that the hypothesis H 1 is accepted when reaching a given "number of votes" N c determined to satisfy the false-alarm constraint on the cross-validation subset. Similar results have been obtained with the original majority voting rule and the usual minimal P E measure of detection accuracy with other embedding schemes and feature sets. The results given in Tables VI and VII are also interesting because the steganalysis features utilize the knowledge of the selection channel, the probabilities of changing each pixel during embedding [START_REF] Denemark | Selection-channel-aware rich model for Steganalysis of digital images[END_REF]. For such features, assuming two different payloads for training and testing may thus create an additional mismatch for the detector because the change probabilities used for feature extraction will also be different. We note that in this case the proposed LRT trained for R = 0.4 still achieves good performance as compared to the clairvoyant LRT, though the loss of detection accuracy is larger.

F. Extension to Multi-Class Steganalysis

The performance of the proposed constrained minimax test is shown on selected cases in Tables VIII-XI for a prescribed false-alarm rate of α 0 = 0.075. The training was carried out as described in Section IV-C. The parameters d sub and L were set to their optimal values as described in Section III-C. Tables VIII and IX present the detection performance for JPEG domain steganographic schemes using JSRM [START_REF] Kodovskỳ | Steganalysis of JPEG images using rich models[END_REF] features. The results presented in Table VIII show the detection accuracy for distinguishing J-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF], UED [START_REF] Guo | An efficient jpeg steganographic scheme using uniform embedding[END_REF], and EBS [START_REF] Wang | An efficient jpeg steganographic scheme based on the block entropy of dct coefficients[END_REF], all with payload R = 0.4 bpnzAC.

The results presented in Table IX are the same except for J-UNIWARD replaced by nsF5 [START_REF] Fridrich | Statistically undetectable jpeg steganography: Dead ends challenges, and opportunities[END_REF] with payload R = 0.2 bpnzAC. Interestingly, the correct classification probability for UED and EBS increased in this case. In fact, since the detection of J-UNIWARD is much harder than the detection of nsF5, the proposed equalizer test focuses mainly (in the first set of results) on maximizing the correct classification of J-UNIWARD to the detriment of UED and EBS. The probability of mis-classifying UED and EBS as J-UNIWARD is much larger in Table VIII than the probability of mis-classifying EBS and nsF5 as UED in Table IX. The results presented in Tables X-XI show the detection accuracy for spatial domain steganography: classification of WOW [START_REF] Holub | Designing steganographic distortion using directional filters[END_REF], S-UNIWARD [START_REF] Holub | Universal distortion function for steganography in an arbitrary domain[END_REF], and HUGO with bounding distortion [START_REF] Pevný | Using high-dimensional image models to perform highly undetectable steganography[END_REF], [START_REF] Filler | Gibbs construction in steganography[END_REF] using the SRM [START_REF] Fridrich | Rich models for steganalysis of digital images[END_REF]. The results shown in Table X indicate that at payload R = 0.4 bpp the above state-of-the-art algorithms can be correctly classified with a probability as high as 35% for a false-alarm probability as low as 0.075 despite the fact that all these algorithms are very similar, in the sense that they all place the embedding changes adaptively based on content complexity, and hence are hard to discern from each other.

We note that, however, the proposed minimax test implies a higher missed-detection rate for each individual embedding scheme compared with the individual binary tests with known payload. This is natural because the test must guarantee a prescribed false-alarm rate with respect to several alternative hypotheses. For the same reason, the correct classification rate of the proposed minimax test decreased for each alternative hypothesis. This is because in the multi-class case one can mis-classify a steganographic algorithm as another one, which cannot happen in the binary case.

Finally and most importantly, Tables VIII-XI testify that the prescribed false-alarm probability of 0.075 can indeed be achieved on the testing set, which was one of the main goals of the proposed optimal minimax test.

VI. CONCLUSION

This paper proposes a statistical model of base learners' projections in an ensemble of linear classifiers for steganalysis of digital images. The main assumptions adopted here are that the base learners' projections follow a multivariate normal distribution and that the covariance matrix remains constant under information hiding, at least for small payloads. This statistical model is used within the framework of hypothesis testing theory to achieve the following three main goals. First, the statistical properties of the optimal LRT designed for binary case, i.e., targeted steganalysis, are analytically established. Second, the LRT is extended to an optimal detector of steganography when the payload is unknown. Last but not least, the proposed framework permits extending the ensemble to multi-class steganalysis. The validity of the proposed statistical model based on which the sharpness of the theoretically established results holds has been confirmed by extensive and diverse experiments. Finally, because the proposed LRT is linear, this work also questions the usefulness of the non-linear majority vote by showing that a linear classifier can achieve roughly same performance for steganalysis using current stateof-the-art high dimensional rich feature spaces.

Fig. 1 :

 1 Fig. 1: Schematic representation describing the implementation of the proposed methodology. Note that rectangles represent data, circles are associated with operations, and rhombuses represent loops and associated tests.

Remark 3 .

 3 Two main settings have been used for training the ensemble: 1) gather FLD base learners when training H 0 against H m , giving us M sets of base learners, 2) train an ensemble for each pair H k against H m , with k = m, giving us M ×(M +1)/2 sets of FLD base learners. Since the second approach performs better, it is the only one used in this paper.

Fig. 2 :

 2 Fig. 2: Schematic representation describing the application of proposed multi-hypotheses minimax test.

Fig. 3 :

 3 Fig.3: ROC curves for the ensemble classifier implemented with majority voting and the proposed optimal LRT comparing empirical and theoretical results: toy example with five randomly chosen base learners (see the text for more details).

  Distribution of Λ lr ( v) under H0 Distribution of Λ lr ( v) under H0 Distribution of Λ lr ( v) under H0 Theoretical distribution N (0, 1)

5 P 5 P

 55 E of proposed LRT P E for majority voting (a) Comparison between EC-MV and the proposed LRT in terms of P E for WOW, SRM feature set. Comparison between EC-MV and the proposed LRT in terms of P E for S-UNIWARD, SRM feature set. Comparison between EC-MV and the proposed LRT in terms of P E for HUGO-BD, SRM feature set. E of proposed LRT P E for majority voting (d) Comparison between the EC-MV and the proposed LRT in terms of P E for J-UNIWARD, CC-JRM feature set. Comparison between the EC-MV and the proposed LRT in terms of P E for J-UNIWARD, SRMQ1 feature set. Comparison between the EC-MV and th proposed LRT in terms of P E for J-UNIWARD, JSRM features set.

Fig. 5 :Feature

 5 Fig. 5: Comparison between the proposed LRT and the majority vote decision rule for different spatial domain embedding schemes, top row, and for J-UNIWARD with different feature sets, bottom row.

36 PFig. 6 : 1 β 40 d sub = 120 L = 20 Fig. 7 :

 366140207 Fig. 6: Comparison between the performance of the proposed optimal LRT and the EC-MV detector as a function of L for selected values of d sub .

1 Theoretical

 1 P F A : α0 = 1 -Φ(τ ) Emp. P F A for d sub = 1800, L = 160 Emp. P F A for d sub = 450, L = 40 Emp. P F A for d sub = 120, L = 20

Fig. 8 :

 8 Fig. 8: Comparison between the theoretically established and the empirical probability of false-alarm as a function of the decision threshold τ .

TABLE I :

 I Values of χ 2 GOF for several values of d sub and L. Results obtained with MiPOD

TABLE II :

 II Values of χ 2 GOF as a function of the training ratio for UED

TABLE III :

 III Comparison between the empirical and the theoretical false-alarm probabilities for a wide-range of steganographic algorithms and feature sets.

TABLE IV :

 IV Power β, see Equation7, of the proposed optimal UMP test for false-alarm probability α 0 = 0.075 and UED[START_REF] Guo | An efficient jpeg steganographic scheme using uniform embedding[END_REF] embedding scheme with CF[START_REF] Kodovsky | Ensemble classifiers for steganalysis of digital media[END_REF] features; d sub and L set at the optimal values.

	Payload Clairvoyant test Payload mixture Trained for R = 0.4
	0.4	0.9009	0.8521	0.9008
	0.3	0.6403	0.5932	0.6211
	0.2	0.3355	0.2495	0.2449
	0.1	0.1514	0.1427	0.1113
	0.05	0.1029	0.0990	0.0867

TABLE V :

 V Power β of the original ensemble classifier with the same settings as in TableIV.

	Payload Clairvoyant test Payload mixture Trained for R = 0.4
	0.4	0.5493	0.5663	0.5484
	0.3	0.4538	0.4555	0.4472
	0.2	0.3415	0.3340	0.3169
	0.1	0.2268	0.2045	0.1866
	0.05	0.1603	0.1489	0.1383

TABLE VI :

 VI Power β, see Equation7, of the proposed optimal UMP test for false-alarm probability α 0 = 0.075 and the content-adaptive scheme[START_REF] Sedighi | Content-Adaptive Steganography by Minimizing Statistical Detectability[END_REF] with maxSRMd2 features[START_REF] Denemark | Selection-channel-aware rich model for Steganalysis of digital images[END_REF]; d sub and L set at the optimal values.

	Payload Clairvoyant test Payload mixture Trained for R = 0.4
	0.4	0.5382	0.5641	0.5438
	0.3	0.4356	0.4416	0.3902
	0.2	0.3187	0.2934	0.1843
	0.1	0.2064	0.1319	0.1164
	0.05	0.1418	0.0896	0.0891

TABLE VII :

 VII Power β of the original ensemble classifier with same settings as in Table VI.

  True/accept H 0 (Cover) H 1 (J-UNIW) H 2 (UED)

					H 3 (EBS)
	H 0	0.9203	0.0665	0.0067	0.0065
	H 1	0.5154	0.4509	0.0170	0.0167
	H 2	0.1161	0.3835	0.4679	0.0325
	H 3	0.0465	0.4600	0.0201	0.4734

TABLE VIII :

 VIII Classification accuracy for the proposed multiclass minimax test, JPEG embedding schemes, payload R = 0.4, JSRM features.

	True/accept H 0 (Cover) H 1 (nsF5) H 2 (UED)	H 3 (EBS)
	H 0	0.9202	0.0198	0.0409	0.0190
	H 1	0.0661	0.8259	0.0987	0.0093
	H 2	0.0969	0.0127	0.8228	0.0676
	H 3	0.0379	0.0063	0.1317	0.8241

TABLE IX :

 IX Classification accuracy for the proposed multiclass minimax test, JPEG embedding schemes, payload R = 0.2 for nsF5 and R = 0.4 for UED and EBS, JSRM features.True/accept H 0 (Cover) H 1 (WOW) H 2 (S-UNIW) H 3 (HUGO-BD)

	H 0	0.9228	0.0363	0.0277	0.0131
	H 1	0.5347	0.3601	0.0879	0.0174
	H 2	0.4653	0.1524	0.3531	0.0292
	H 3	0.5022	0.0613	0.0566	0.3800

TABLE X :

 X Classification accuracy for the proposed multi-class minimax test, spatial domain embedding schemes, payload R = 0.4, SRM features. True/accept H 0 (Cover) H 1 (WOW) H 2 (S-UNIW) H 3 (HUGO-BD)

	H 0	0.9239	0.0313	0.0249	0.0199
	H 1	0.7164	0.1763	0.0828	0.0245
	H 2	0.6857	0.0841	0.2003	0.0300
	H 3	0.6964	0.0445	0.0496	0.2096

TABLE XI :

 XI Classification accuracy for the proposed multiclass minimax test, spatial domain embedding schemes, payload R = 0.2, SRM features.

R.COGRANNE, AND J.FRIDRICH, "MODELING AND EXTENDING THE ENSEMBLE CLASSIFIER FOR STEGANALYSIS OF DIGITAL IMAGES ...", c IEEE, 2013.

We also point out that in steganalysis the training set must always consist of pairs of cover-stego images as it has been shown that preserving those pairs can significantly increase the detection performance[START_REF] Schwamberger | Simple algorithmic modifications for improving blind steganalysis performance[END_REF].

Since the FLD is a well-known tool, it is only briefly described in this paper; the reader is referred to[START_REF] Duda | Pattern classification[END_REF] for a more detailed exposition.

R.COGRANNE, AND J.FRIDRICH, "MODELING AND EXTENDING THE ENSEMBLE CLASSIFIER FOR STEGANALYSIS OF DIGITAL IMAGES ...", c IEEE, 2013.

ACKNOWLEDGEMENTS

The authors would like to thank Vojtěch Holub for providing the code for steganographic algorithms as well as Tomáš Denemark and Vahid Sedighi for fruitful discussions. An open-source Matlab demo code, used for this paper, is available online within download section of DDE website at dde.binghamton.edu/download/ and on the UTT-LM2S website.

This work was supported by Air Force Office of Scientific Research under the research grant number FA9950-12-1-0124. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation there on. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied of AFOSR or the U.S. Government.

APPENDIX A EXPRESSION FOR THE LIKELIHOOD RATIO AND PROOF OF OPTIMAL PROPERTIES

The goal of this appendix is threefold. First, elements of calculus are provided to establish the form of the LR [START_REF] Pevný | Steganalysis by subtractive pixel adjacency matrix[END_REF]. Then, proofs of Corollary 1, and Proposition 2 are given.

A. Expression of the Likelihood Ratio for Simple Hypothesis and Proof of Proposition 1

Let us recall that after the transformation [START_REF] Zitzmann | Hidden information detection based on quantized Laplacian distribution[END_REF], the problem of detecting hidden data may be described as a choice between the following hypotheses (11)

Here, the pdf of the multivariate normal distribution of L jointly distributed random variables is denoted

) with µ and Σ the expectation and the covariance matrix respectively and |Σ| denotes the determinant of matrix Σ.

From the above definition of tested hypotheses, one can get the following expression for the LR between the hypotheses:

Here, taking any strictly increasing function of the LR p(x;0,I L ) p(x;θ1,I L ) does not change the properties of the test, up to the decision threshold; the logarithm of the LR, permits us to write

Similarly, since 1 2 θ T 1 θ 1 and θ 1 are constants whatever the true hypothesis may be, removing the first term and scaling by the second does not change the optimality of the test and immediately yields [START_REF] Pevný | Steganalysis by subtractive pixel adjacency matrix[END_REF].

From the properties of the multivariate normal distribution, it is immediate to obtain the distribution of the LR

as given in ( 13), as it is essentially a weighted sum of uncorrelated normally distributed random variables. The proof of Proposition 1 thus immediately follows from the distribution (13) as it straightforward that:

with the last equality following from the fact that the standard normal cdf Φ is strictly increasing, while the equality in the inequality ensures the maximization of the power function.

Similarly, the power function of the LRT δ lr immediately follows from distribution [START_REF] Pevný | From blind to quantitative steganalysis[END_REF] as

B. Computing the Threshold that Minimizes P E , Corollary 1

Using the definition of the proposed LRT, the probability of false alarm ( 14) and the power function [START_REF] Pevný | Multiclass detector of current steganographic methods for jpeg format[END_REF], also provided in [START_REF] Cam | Asymptotic Methods in Statistical Decision Theory[END_REF] and ( 34) respectively, as a function of the decision threshold τ lr can be written:

The threshold τ lr that minimizes the total probability of error under equal Bayesian priors, P E (τ lr ) = 1 /2 α 0 (τ lr ) + 1 -β δ lr (τ lr ) , can be obtained by using Equation [START_REF] Middleton | An introduction to statistical communication theory[END_REF] and differentiating with respect to τ lr . From the definition of Φ(x) is it immediate that

from which it follows that

(37) Setting the derivative of P E (37) to zero to find the threshold value τ lr that minimizes P E leads to

which proves the Corollary 1.

C. Proof of Uniformly Most Powerful Property, Proposition 2

Next, to prove that the LRT δ lr is also Uniformly Most Powerful for the case in which the payload R is unknown, it is worth noting that the pdf p(x; f (R)θ 1 , I L ), see [START_REF] Ker | Batch steganography and pooled steganalysis[END_REF], of the normalized base learners' projections v can be written:

which, from the factorization (Fisher-Neyman) Theorem [26, Chapt. 1.9] immediately proves that the proposed LR Λ lr ( v) is a sufficient statistic for R.

Then, recalling that f : [0, 1] → R + is an increasing function, it is immediate to note that Λ lr (x) is also an increasing function of R and hence, it follows from [26, Theorem 3.4.1] that the proposed LRT ( 8) is a Uniformly Powerful Test (UMP) for solving the hypothesis testing problem [START_REF] Cogranne | Theoretical model of the FLD ensemble classifier based on hypothesis testing theory[END_REF] with a composite alternative hypothesis H R .