Member, IEEE Velotiaray Toto-Zarasoa

Member, IEEE Aline Roumy

Senior Member, IEEE Christine Guillemot

Source modeling for Distributed Video Coding

Keywords: Distributed Source Coding, Hidden Markov Process, Parameter Estimation, Distributed Video Coding

This paper studies source and correlation models for Distributed Video Coding (DVC). It first considers a twostates HMM, i.e. a Gilbert-Elliott process, to model the bit-planes produced by DVC schemes. A statistical analysis shows that this model allows us to accurately capture the memory present in the video bit-planes. The achievable rate bounds are derived for these ergodic sources, first assuming an additive binary symmetric correlation channel between the two sources. These bounds show that a rate gain can be achieved by exploiting the sources memory with the additive BSC model. A Slepian-Wolf (SW) decoding algorithm which jointly estimates the sources and the source model parameters is then described. Simulation results show that the additive correlation model does not always fit well with the correlation between the actual video bit-planes. This has led us to consider a second correlation model (the predictive model). The rate bounds are then derived for the predictive correlation model in the case of memory sources, showing that exploiting the source memory does not bring any rate gain and that the noise statistic is a sufficient statistic for the MAP decoder. We also evaluate the rate loss when the correlation model assumed by the decoder is not matched to the true one. An a posteriori estimation of the correlation channel has hence been added to the decoder in order to use the most appropriate correlation model for each bit-plane. The new decoding algorithm has been integrated in a DVC decoder, leading to a rate saving of up to 10.14% for the same PSNR, with respect to the case where the bit-planes are assumed to be memoryless uniform sources correlated with the SI via an additive channel model.

I. INTRODUCTION

Distributed Source Coding (DSC) refers to the problem of separate encoding and joint decoding of correlated sources. From the Shannon's theorem [START_REF] Shannon | Coding theorems for a discrete source with a fidelity criterion[END_REF], the minimum achievable rate for the lossless compression of two statistically dependent memoryless sources X and Y is given by their joint entropy H(X, Y ). To approach this rate bound, traditional systems encode and decode the two sources jointly. In 1973, Slepian and Wolf (SW) have established [START_REF] Slepian | Noiseless coding of correlated information sources[END_REF] that, for two dependent binary sources X and Y , this lossless compression rate can be achieved by a separate encoding of the two sources, provided that the respective rates of X and Y are greater than their conditional entropies, H(X|Y ) and H(Y |X), and joint decoding is performed. The lossy equivalent of the SW theorem for two correlated continuous-valued sources has been formulated by Wyner and Ziv (WZ) [START_REF] Wyner | The rate-distortion function for source coding with side information at the decoder[END_REF]. First practical DSC solutions were based on channel codes, e.g., convolutional codes, turbo codes [START_REF] Garcia-Frias | Compression of correlated binary sources using Turbo codes[END_REF], [START_REF] Aaron | Compression with side information using Turbo codes[END_REF] or Low-Density-Parity-Check (LDPC) codes [START_REF] Liveris | Compression of binary sources with side information at the decoder using LDPC codes[END_REF]. In the sequel, we consider a SW encoder based on LDPC codes, whose performance is very close to the SW bound [START_REF] Liveris | Compression of binary sources with side information at the decoder using LDPC codes[END_REF], and whose decoding complexity is linear with the code length.

This research was partially supported by the French European Commission in the framework of the FP7 Network of Excellence in Wireless COMmunications NEWCOM++ (contract n.216715).

Video compression has been recast into a DSC framework, leading to the emergence of Distributed Video Coding (DVC) systems. The video sequence is structured into Groups Of Pictures (GOP) in which key frames are intra-coded and intermediate frames (also called "WZ frames") are WZ-coded. Each WZ frame is encoded independently of the other frames. The WZ data is transformed and quantized, then the quantized coefficients are binarized. Each resulting bit-plane is encoded by a channel encoder to yield syndrome bits. The DVC decoder constructs the side information (SI) via a motioncompensated interpolation of the previously decoded key frames. The encoder first sends a subset of syndrome bits for each bit-plane. If the decoder cannot properly decode the current bit-plane, more syndrome bits are requested from the encoder.

Most DVC practical systems assume the SW encoded bitplanes to be memoryless sources. Here, we instead model the video bit-planes as two-state Hidden Markov sources, with the help of the Gilbert-Elliott (GE) process. The probability of a given symbol is hence dependent only on the current state. This two-state model has already been considered to model a communication channel in [START_REF] Garcia-Frias | Decoding of Low-Density Parity-Check codes over finite-state binary Markov channels[END_REF], [START_REF] Eckford | Designing good LDPC codes for Markov-modulated channels[END_REF], [START_REF]Analysis of Low-Density Parity-Check codes for the Gilbert-Elliott channel[END_REF] as well as the correlation channel in DSC [START_REF] Garcia-Frias | Turbo decoding of Gilbert-Elliot channel[END_REF], [START_REF] Garcia-Frias | LDPC codes for compression of multiterminal sources with Hidden Markov correlation[END_REF], [START_REF] Ser | Asymmetric joint sourcechannel coding for correlated sources with blind HMM estimation at the receiver[END_REF], [START_REF] Bhattad | A decision feedback based scheme for Slepian-Wolf coding of sources with Hidden Markov correlation[END_REF], [START_REF]Distributed joint source-channel coding for arbitrary memoryless correlated sources and source coding for Markov correlated sources using LDPC codes[END_REF], assuming the input sources to be uniform. However, this GE process is used here to model the sources and not the correlation channel. One can thus model a source with infinite memory with few parameters. Raptor codes [START_REF] Shokrollahi | Raptor codes[END_REF] have been used in [START_REF] Fresia | Distributed source coding using raptor codes for hidden Markov sources[END_REF] for distributed coding of hidden Markov sources. However, the approach is validated with synthetic sources and the parameters of the source are assumed to be known to the decoder, which is not the case in practical DVC setup where the source parameters differ from bit-plane to bit-plane and must be estimated on-line. A statistical analysis of the burst lengths in the bit-planes has shown in [START_REF] Toto-Zarasoa | Hidden markov model for distributed video coding[END_REF] that the twostate GE model was reliably capturing the source memory; the correlation between the two sources was assumed to follow an additive model. This paper first derives the achievable entropy-rate for ergodic correlated sources, when the correlation channel is additive, showing a theoretic rate gain with respect to memoryless sources. The paper then describes a joint estimation-decoding algorithm based on the Expectation Maximization (EM) algorithm [START_REF] Rabiner | A tutorial on hidden Markov models and selected applications in speech recognition[END_REF], which jointly estimates the source parameters and the source symbols. When the model parameters need to be estimated by the decoder as in actual DVC systems, the initialization of the estimator of the source model parameters can make use of the knowledge of the SI, which is not the case when using this model for the correlation channel. We derive properties of the source model (Lemma 1) which allows us to efficiently initialize the EM algorithm. More precisely, a Baum-Welch algorithm is performed in order to estimate the parameters and the state realizations of the SI bit planes, which are then used to initialize the parameters and the state realizations of the SW-encoded source.

The first simulation results have shown that, although the GE model was well capturing the bit-planes memory, exploiting the source memory while decoding the bit-planes in the DVC decoder was not always improving the rate-distortion (RD) performances of the system. This is explained by the fact that the additive correlation model does not always fit well with the correlation between the actual video bit-planes. This has led us to then consider another correlation model, the predictive correlation model together with HMM memory sources. Achievable entropy-rates have thus been also derived for ergodic correlated sources, when the correlation channel is predictive. It is shown, similarly to the case of non-uniform Bernoulli sources, in [START_REF] Toto-Zarasoa | Non-uniform source modeling for distributed video coding[END_REF], that, if the correlation channel is additive, then the compression rate can be reduced by exploiting the source memory. However, for the predictive correlation model, exploiting the source memory does not reduce the compression rate. Actually, we show that the noise statistics are sufficient for the MAP estimation in the predictive case and that the source statistics do not play any role in the performance bounds. We further study the impact on the DVC decoding performance of a possible mismatch between the assumed correlation model and the actual one. One extra step, i.e., an a posteriori estimation of the type of correlation channel, has thus been added to the SW decoder. These results extend those presented in [START_REF] Toto-Zarasoa | Non-uniform source modeling for distributed video coding[END_REF] obtained for an i.i.d binary source model. We show that the joint source estimationdecoding algorithm together with the estimation of the channel correlation type (predictive or additive) per bit-plane leads to significant rate saving compared with classical DVC schemes.

This paper is structured as follows. Section II, after reviewing the theory behind DSC, presents the GE source model, and defines its parameters. Section III describes the LDPCbased estimation-decoding EM algorithm which takes into account the statistics of the source. Its performance is first assessed with synthetic sources. Section IV demonstrates the accuracy of the GE source model for DVC bit-planes, and presents how the EM algorithm can be used in a DVC codec. Finally, Section V introduces the predictive correlation model, the corresponding rate bounds with ergodic sources as well as the way the decoder operates to estimate the correlation model to be used in the decoding process. The RD performances obtained with the modified DVC codec are then presented by exploiting the complete HMM model as well as a simplified version which amounts to exploiting only the non-uniformity of the sources.

II. DISTRIBUTED SOURCE CODING OF CORRELATED

BINARY HIDDEN MARKOV SOURCES This Section first revisits the principle of asymmetric DSC with the classical binary symmetric correlation model, referred to as additive BSC. The rate bounds are then derived in the case where the correlated sources have infinite memory.

In the sequel, a source is modeled as a random process. Uppercase sequences {X n } n≥1 , in short ({X n }, {Y n }, {Z n }, {Σ n }), refer to random stochastic processes, uppercase variables (X n , Y n , Z n , Σ n ) refer to random variables at instant n, and lowercase variables (x n , y n , z n , σ n ) refer to their realizations; bold uppercase variables

(X = X N 1 , Y = Y N 1 , Z = Z N 1 , Σ = Σ N 1 )
refer to vectors of random variables, and bold lowercase variables

(x = x N 1 , y = y N 1 , z = z N 1 , σ = σ N 1 )
refer to vectors of their realizations. By abuse of notation, uppercase variables (X, Y, Z, Σ) refers to i.i.d. processes. The symbol "⊕" stands for the modulo-two addition of binary symbols. The bold H stands for the parity-check matrix of channel code. Finally, H(X) stands for the entropy of the i.i.d. source, and H(X) [START_REF] Cover | Elements of Information Theory[END_REF] stands for the entropy-rate of the ergodic source.

A. Asymmetric DSC: source coding with side information at the decoder only DSC refers to the separate compression of two correlated sources assuming the decoding is performed jointly. In the asymmetric DSC setup, one of the sources (say {X n }) has to be compressed, while the other (say {Y n }) is available at the decoder. {Y n } therefore serves as a side-information (SI) for the decoding of {X n }. When the sources {X n } and {Y n } are i.i.d. with finite alphabet, [START_REF] Slepian | Noiseless coding of correlated information sources[END_REF] shows that the source {X n } can be compressed at the conditional entropy: H(X|Y ), such that knowing the SI at the encoder does not help reducing the compression rate. In the case of ergodic random processes {X n } and {Y n } with discrete (in the sense of infinite countably) alphabets, [START_REF] Cover | A proof of the data compression theorem of Slepian-Wolf for ergodic sources[END_REF] shows that the minimum compression rate for the source {X n } is the conditional entropy-rate H(X|Y).

Let us now consider two binary correlated sources: {X n }, {Y n }. The correlation can be modeled by a virtual channel as defined below. Definition 1. An (X, Y, p) additive BSC is a channel with binary ergodic input {X n }, binary ergodic output {Y n }. The noise is an i.i.d. binary process Z ∼ B(p). Z is independent of the channel input, and the channel output is obtained by

Y n = X n ⊕ Z n , ∀n.
If the correlated binary sources {X n }, {Y n } are i.i.d., [START_REF] Wyner | Recent results in the Shannon theory[END_REF] shows that the compression rate for {X n }, H(X|Y ) [START_REF] Slepian | Noiseless coding of correlated information sources[END_REF], can be achieved by use of channel codes. This scheme is called the syndrome approach. More precisely, the (N, K) linear code C, defined by its (N -K) × N parity check matrix H, defines a partition of the N -long source sequences into cosets, where all the sequences in the same coset C s share the same syndrome s, i.e. C s = {x : Hx = s}. To encode a particular vector x, the encoder transmits its syndrome s x = Hx, achieving a compression ratio of N : (N -K); the side-information y is available at the decoder. The decoder's estimation x consists in finding the closest sequence to y having syndrome s x . [START_REF] Wyner | Recent results in the Shannon theory[END_REF] shows that if the code defined by the parity check matrix H achieves the capacity of the (X, Y, p) additive BSC, then the syndrome approach with the same code achieves the compression rate for {X n } i.e. lim N →+∞ (N -K)/N = H(X|Y ).

B. Binary memory sources: Gilbert-Elliott modeling

We now consider binary sources with memory, modeled as a two-state Hidden Markov Model (HMM) (the Gilbert Elliott (GE) process). We will see in the sequel that the GE process accurately models the video bit-planes which are SW-encoded in a practical DVC system. This model is well suited to our problem since it builds infinite memory sources with very few parameters. This section states a property which will be useful in the model parameters estimation to be performed by the decoder.

Let {Σ n } be a finite Markovian process with memory of order one, having two realizations s and d, where s stands for "sparse source", and d stands for "dense source". We define the transition probabilities t ss , t sd , t ds and t dd , between the states, as shown in Fig. 1. Since t ss = (1 -t sd ) and t dd = (1 -t ds ), the set of parameters of the model is θ X = (p s , p d , t ds , t sd ). Those parameters are defined by:

p s = P θ X (X n = 1|Σ n = s) p d = P θ X (X n = 1|Σ n = d) t ds = P θ X (Σ n = s|Σ n-1 = d) t sd = P θ X (Σ n = d|Σ n-1 = s) (1)
where Σ = Σ N 1 is the underlying N -long state sequence, and σ = σ N 1 ∈ {s, d} N is its realization. These equations lead to the following property of the source:

Property 1. ∀n ∈ [1, N ], given the state Σ n , {X n } is a memoryless Bernoulli process of parameter p Xn = P θ X (X n = 1|σ n ), where p Xn = p s if σ n = s, and p Xn = p d if σ n = d.

C. Gilbert-Elliott modeling in DSC: properties and rate bounds

We now turn back to the DSC problem. Let {X n } be a GE source and let a second source {Y n } be correlated to {X n }, where the correlation is modeled as a virtual additive BSC (X, Y, p) (see Def. 1). We assume that ∀n, Y n = X n ⊕Z n , with P(Y n = X n ) = P(Z n = 1) = p. Given the characteristics of the correlation model, we state the following Lemma 1 which characterizes the GE nature of the side-information {Y n }.

Lemma 1. Let {X n } be a GE process of parameter θ X = (p s , p d , t ds , t sd ). {Σ n } denotes the underlying hidden state process. Let {Y n } be a source correlated to {X n } according to the additive channel model (X, Y, p). Therefore, there exists an i.i.d. Bernoulli process Z independent of {X n } s.t. ∀n, Y n = X n ⊕ Z n , and {Y n } is a GE source with the same underlying state process {Σ n } as {X n }.

Proof: When conditioned on the state process {Σ n }, the source {X n } is memoryless according to Property 1. Since Z is memoryless too, the source {Y n } is memoryless when conditioned on the same state process {Σ n }. Moreover, ∀n, given the state Σ n , {Y n } is a (memoryless) Bernoulli process of parameter p Yn = p Xn (1 -p) + (1 -p Xn )p, where p Xn is described in Property 1. Therefore {Y n } is a Gilbert Elliott source with the same state process as for {X n }.

We now turn back to the asymmetric DSC problem, where {Y n } is available at the decoder only, and {X n } is transmitted at a rate greater than its conditional entropy-rate H(X|Y). It has been shown in [START_REF] Toto-Zarasoa | Non-uniform source modeling for distributed video coding[END_REF] that, if the source {X n } is memoryless but not uniform, the minimum coding rate for {X n } can be reduced by an amount of H(Y ) -H(X) ≥ 0 with respect to the uniform memoryless case. In the following Lemma 2, we extend this result to the case of memory sources.

Lemma 2. Let {X n } and {Y n } be two correlated binary ergodic sources, where the correlation is modeled as a virtual (X, Y, p) additive BSC. We consider the asymmetric DSC problem, where {Y n } is available at the decoder and {X n } is transmitted at a rate greater than its conditional entropy-rate H(X|Y).

The minimum coding rate for

{X n } is H(X|Y) = H(Z) - [H(Y) -H(X)]. If the source {X n } is uniform (and therefore memoryless), this rate H(X|Y) = H(Z). Thus, since H(Y) - H(X) ≥ 0, H(X|Y) is reduced by H(Y) -H(X) compared to the minimum coding rate H(X|Y) = H(Z) for a uniform source. Proof: Since the BSC is additive, ∃ Z an i.i.d. binary process (of parameter p), independent of {X n } s.t. ∀n, Y n = X n ⊕ Z n .
The conditional entropy-rate of the source {X n } is computed as:

H(X|Y) = H(X, Y) -H(Y) = H(X) + H(Y|X) -H(Y) = H(Z) -[H(Y) -H(X)] (2)
where the last equality follows from the independence between {X n } and Z.

If {X n } is uniform, so is {Y n }, and H(Y) = H(X) = 0.5 i.e. H(X|Y) = H(Z). Now, consider the case of an ergodic process {X n }. On the one hand we have H(X) = H(X ⊕ Z|Z), and on the other hand we have H(Y) = H(X ⊕ Z). Since conditioning reduces entropy [20, Theorem 2.6.5], it implies H(X) ≤ H(Y) with equality if, and only if, {Y n } and Z are independent i.e. if p = 0 or 1, or if the source {X n } is uniform.

As H(Y) -H(X) ≥ 0 (with equality if, and only if, the source {X n } is uniform or in the degenerate case p = 0 or 1), the lower transmission rate bound of a GE source is lower than that of a uniform source, and the rate gain is given by H(Y) -H(X).

The dotted curve in Fig. 2 shows the theoretical rates H(X|Y) that can be achieved for a GE source with parameter θ X = (t ds = 0.03, t sd = 0.01, p s = 0.07, p d = 0.7), when p varies in [0, 1], and the correlation is modeled as a virtual additive BSC. If instead the same source is modeled as a memoryless source (with parameter pst ds +p d t sd t ds +t sd ), the minimum achievable rate (solid line) is greater than the one with the GE modeling (dotted curve). Therefore, knowing the GE source distribution, one can achieve smaller compression rates.

III. JOINT ESTIMATION-DECODING FOR DISTRIBUTED CODING OF GE SOURCES: THE EM ALGORITHM

This section describes the algorithm used to jointly estimate the source {X n } and its model parameters. The approach is based on the Expectation-Maximization (EM) algorithm. The EM algorithm has already been used in [START_REF] Garcia-Frias | Turbo decoding of Gilbert-Elliot channel[END_REF], [START_REF] Garcia-Frias | Decoding of Low-Density Parity-Check codes over finite-state binary Markov channels[END_REF], [START_REF] Ser | Asymmetric joint sourcechannel coding for correlated sources with blind HMM estimation at the receiver[END_REF] [9], and [START_REF] Eckford | The factor graph EM algorithm: applications for LDPC codes[END_REF] to estimate the HMM parameters of a communication channel or of the correlation channel in DSC problems. Instead, here the EM algorithm is used to estimate the HMM parameters of the source; the initialization of the estimator relies on the source properties stated in Lemma 1. A Baum-Welch algorithm is performed in order to estimate the parameters and the state realizations of the SI, which are then used to initialize the parameters and the state realizations of the source {X n }.

A. Formalization of the maximization problem

The SW decoder estimates the source realization x from its syndrome s x and the side-information y, assuming that {X n } is a GE source. However, the decoder is not aware neither of the parameter θ X nor of the hidden state sequence {Σ n }. This estimation is carried out by an Expectation-Maximization (EM) algorithm, which is an optimization procedure that learns new parameters from the observed variables (y and s x ), and a previously estimated set of hidden variables (x, σx and θX ).

Let l be the current decoding iteration, and θ l X the current estimate of the GE source parameters. Then, the updated value θ (l+1) X for the next iteration is computed so as to maximize the mean log-likelihood function:

θ (l+1) X = arg max θ X E X,Σx|Y,s X ,θ l X log P θ X (y, x, σx, sx) (3)
To simplify the notation, in the sequel we denote "P(X = x|θ X )" by "P θ X (x)".

Since the logarithm is a strictly increasing function, the value θ (l+1) X that maximizes P θ l X (y, x, σ x , s x ) also maximizes log P θ l X (y, x, σ x , s x ) . The algorithm converges since it increases the likelihood at each iteration [START_REF] Wu | On the convergence properties of the EM algorithm[END_REF].

We now consider that the code used to compress the source X is a syndrome-based LDPC code. LDPC codes can be equivalently represented by their sparse parity-check matrix or by their factor graph. For an LDPC code yielding a compression rate N : (N -K), let H = (h mn ) m∈[1,(N -K)],n∈ [1,N ] be the sparse matrix of size (N -K) × N . Fig. 3 presents the factor graph [START_REF] Kschischang | Factor graphs and the sum-product algorithm[END_REF] that describes the dependencies between the observed and the hidden variables of the problem. We introduce the following notation for the variables and the messages that are passed on the factor graph during the estimation-decoding process: • V n are the messages passed from the variable nodes to the state nodes.

All the messages are Log-Likelihood Ratio (LLR), they are labeled (in) or (out) if they come to or from the considered node. In the following, we give the update rules for the messages that are passed on the graph.

B. Expectation step: computation of the mean log-likelihood function

First, we expand the likelihood function using the Bayes rule:

P θ l X (y, x, σ x , s x ) = P(y, s x |x, σ x )P θ l X (x, σ x ) (4)
where P(y, s x |x, σ x ) is independent of θ l X . Then, the log-likelihood function reduces to:

log P θ l X (x, σ x ) = log P θ l X (σ 1 ) + N n=2 d i=s d j=s δ σn-1=i,σn=j log(t l ij ) + N n=1 d i=s δ σn=i,xn=1 log(p l i ) + δ σn=i,xn=0 log(1 -p l i ) (5)
where δ bool = 1, if bool = true 0, otherwise Finally, the mean log-likelihood function is obtained by taking the expectation of the log-likelihood [START_REF] Aaron | Compression with side information using Turbo codes[END_REF], where

E X,Σx|Y,s X ,θ l X δ σn=i,xn=k = P θ l X (Σ n = i, X n = k|y, s x ) = P θ l X (Σ n = i|y, s x )P θ l X (X n = k|Σ n = i, y, s x ) E X,Σx|Y,s X ,θ l X δ σn-1=i,σn=j = P θ l X (Σ n-1 = i, Σ n = j|y, s x ) (6)
See Equation 26in Appendix A for the expanded expression of the mean log-likelihood function.

C. Maximization step

Here, the mean log-likelihood function is maximized versus θ X , given the estimate θ l X [START_REF] Toto-Zarasoa | Non-uniform source modeling for distributed video coding[END_REF], and under the constraints, ∀i, j ∈ {s, d}: p i ∈ [0, 1], and t ij ∈ ] 0, 1 [ , and j∈{s,d}

t ij = 1 (7)
The new parameters, solutions to the maximization problem (3), are:

p (l+1) i = N n=1 P θ l X (Σn = i|y, sx)P θ l X (Xn = 1|Σn = i, y, sx) N n=1 P θ l X (Σn = i|y, sx) t (l+1) ij = N n=2 P θ l X (Σn-1 = i, Σn = j|y, sx) N -1 n=1 P θ l X (Σn = i|y, sx) (8)
See Appendix A for a detailed demonstration of the results in Equation [START_REF] Eckford | Designing good LDPC codes for Markov-modulated channels[END_REF].

Therefore, the maximization step needs the current a posteriori probabilities (APP) of the states Σ and the source X. Due to the cycles in the graph, these quantities are too complex to compute, but they can efficiently be approximated by a belief propagation algorithm run on the graph shown in Fig. 3. The algorithm proceeds in two steps explained below: the soft estimates of the states result from a forward-backward-like algorithm (see Appendix B for detailed presentation of the algorithm), and the soft estimates of the source symbols result from an LDPC-decoding-like algorithm.

D. LDPC belief propagation for the soft estimate of X

Given the current estimate of the parameters θ l X and the soft estimate of the states σ l

x , we find the best approximation of the a posteriori probabilities (APP) P θ l X (X n = k|σ n , y, s x ), n ∈ [1, N ], k ∈ {0, 1} needed for the parameters updates in Equation ( 8) of the maximization step. As side products, we also obtain the estimate x l . Here, we describe the update rules for the belief-propagation run on the graph in Fig. 3.

1) Messages from the state nodes to the variable nodes:

S n = log P θ l X (X n = 0) P θ l X (X n = 1) = log 1 -p l Xn p l Xn ( 9
)
where p l Xn is obtained from the states probabilities P θ l X (σ n |y, s x ) (from the forward-backward algorithm), and the current estimate θ l X . More precisely, ∀n ∈ [1, N ]:

p l Xn = i∈[s,d] p l i • P θ l X (Σ n = i|y, s x ) (10)
2) Intrinsic information of the BSC nodes:

I n = log P(Z n = 0) P(Z n = 1) = log 1 -p p (11)
Here, the value of p is assumed to be known prior to decoding.

3) Messages from the BSC nodes to the variable nodes:

B n = (1 -2y n )I n (12)
4) Messages from the variable nodes to the check nodes:

E (out)
n,e

= B n + dxn k=1,k =e E (in) n,k + S n , (13)
Each

E (out) n,e
is mapped to the corresponding Q

(in)
m,e according to the connections in the factor graph.

5) Messages from the check nodes to the variable nodes:

Q (out) m,e = 2 tanh -1   (1 -2s n ) dsm k=1,k =e tanh Q (in) m,e 2   (14)
Each

Q (out)
m,e is mapped to the corresponding E (in) n,k . 6) Messages from the variable nodes to the state nodes:

We compute the extrinsic LLR E n for each x n , as:

E n = B n + dxn k=1 E (in) n,k (15)
Then, the variable-to-state messages are given by:

   V n (0) = e En 1 + e En V n (1) = 1 -V n (0) (16)
For this LDPC decoding, we have decided to propagate LLR, which implies their conversion to probabilities, in [START_REF] Fresia | Distributed source coding using raptor codes for hidden Markov sources[END_REF], for use in the maximization step. So far, the values of V n (0) and V n (1) are the best guess on the a posteriori probabilities P θ l X (X n = 0|σ n , y, s x ) and P θ l X (X n = 1|σ n , y, s x ).

7) Decision:

After each iteration of the EM algorithm, a hard decision is made on V n (1) to get the estimated symbols of x (l+1) .

∀n ∈ [1, N ], x (l+1) n = 1, if V n (1) ≥ 0.5 0, otherwise (17)
E. Stopping criteria: syndrome check, convergence test, and maximum number of iterations

The estimation-decoding EM algorithm stops either if the estimated x (l+1) satisfies the parity check equations (defined by Hx (l+1) = s x ), or if the syndrome test has failed while no symbol of x has been updated during the current iteration (we decide that the decoder has converged to a wrong word), or if the maximum number of iterations has been reached (100 iterations is a good compromise between performance and complexity).

F. Initialization of the EM algorithm

As exhibited in Lemma 1, the side-information {Y n } is a GE source that has the same states as the source {X n }. Therefore, to initialize the EM algorithm, we use the estimated GE parameters associated to y, { θY , σy }; that is the best guess on {θ 0 X , σ 0 x } so far. To estimate these parameters, we use a simplified version of the EM algorithm: the Baum-Welch algorithm [START_REF] Rabiner | A tutorial on hidden Markov models and selected applications in speech recognition[END_REF]. In the Baum-Welch algorithm, the hard values of the bits are directly used, instead of their probabilities, and no LDPC decoding is performed. To that end, we take P(Y n = 1) = y n , P(Y n = 0) = (1 -y n ) in Equation ( 8) of Section III. This Baum-Welch algorithm is initialized with the arbitrary values:

θ 0 X = (p 0 s = 0.49, p 0 d = 0.51, t 0 ds = 0.1, t 0 sd = 0.1) (18)
Since there is no a priori on the Bernoulli parameters p s and p d , they have to be initialized by a value close to 0.5. However they cannot be initialized with the same value p 0 = 0.5, since the parameters would not be updated from an iteration to the next (this is a consequence of Equation ( 8)). Moreover, since by definition of the GE process p s ≤ p d , this order must be kept for the initialization. As for the initialization of the transition probabilities, we observed that the value 0.1 allows to speed the convergence rate of the EM algorithm.

G. Simulation results: Distributed coding of GE sources

We consider a GE source {X n } with parameters θ X = (p s = 0.07, p d = 0.7, t ds = 0.03, t sd = 0.01), having realization x of length N = 1584 (same length as the video bit-planes, in the DVC experiments with QCIF sequences reported in Section IV). We also consider a memoryless binary noise Z ∼ B(p) with realization z, and a second source {Y n }, with realization y, correlated to {X n } s.t. ∀n, Y n = X n ⊕Z n (additive BSC). Therefore, the entropy of the noise is

H(Z) = H(p) = -p log(p) -(1 -p) log(1 -p)
. The entropyrates H(X) and H(Y) are calculated using the statistical approach in [START_REF] Rezaeian | Computation of capacity for Gilbert-Elliott channels, using a statistical method[END_REF]. The Slepian-Wolf bound for the coding of {X n }, i.e. its conditional entropy-rate, is thus obtained by

H(X|Y) = H(Z) -[H(Y) -H(X)].
To prove the enhanced performance of the proposed DSC decoder, the syndrome of x, as well as the side-information y, are transmitted to three different decoders:

• a the standard decoder, which views {X n } as a uniform source; • b the proposed decoder, which assumes that {X n } is a GE source and uses the EM algorithm to iteratively estimate its parameter θ X ; • c a genie-aided decoder, which assumes that {X n } is a GE source and knows the parameter θ X . All three decoders have the same variable degree distribution Λ(x) = 0.457827x + 0.323775x 2 + 0.0214226x 3 + 0.0592851x 5 +0.0389015x 6 +0.0248109x 7 +0.00884569x 8 + 0.0176697x 18 +0.04746251x 19 , and check degree distribution Φ(x) = x 8 . The BER of {X n } corresponding to the three decoders are plotted in Fig 4, the estimated parameters from the decoder b are plotted in Fig. 5.

First, we see in Fig. 4 that the proposed decoder b performs considerably better than the original decoder a . In particular, the source is retrieved error-free for H(p) = 0.5, which corresponds to the SW bound for uniform sources. Moreover, the plots b and c in Fig. 4 show that knowing the true θ X is not essential to the decoder, since it does not improve the performance of the decoder in a significant amount: the rate improvement from b to c is less than 0.02 bit for any value taken by p.

It is shown in Fig. 5 that the EM algorithm manages to closely retrieve the parameters of {X n } in the range where the BER is small (i.e. (H(p) < 0.65), see Fig. 4).

IV. DISTRIBUTED VIDEO CODING EXPLOITING THE

MEMORY OF THE WZ BIT-PLANES We consider the transform-domain DVC codec described in [START_REF] Artigas | The DISCOVER codec: Architecture, techniques and evaluation[END_REF], developed by the European IST-DISCOVER project [START_REF]DISCOVER project web page[END_REF], and which will be referred to as the DISCOVER codec in the sequel. A. Review of the DISCOVER codec Fig. 6 shows the DISCOVER codec block diagram. The encoder first splits the video frames into key frames and WZ frames. The key frames are conventionally encoded using an H264/AVC encoder and transmitted to the decoder. The WZ frames are first transformed with a Discrete Cosine Transform (DCT), and the obtained transform coefficients are quantized. The quantized coefficients are organized into bands, where every band contains the coefficients associated to the same frequency in different blocks. Then, the quantized coefficients bands are fed bit-plane by bit-plane to a SW encoder, which computes their syndromes.

At the decoder, the key frames are first decoded using a conventional video decoder. Then a motion compensated interpolation between every two closest key frames is performed, in order to produce the SI for a given WZ frame. The WZ frame is also split into blocks which are then DCT transformed. The correlation channel between the WZ and SI DCT coefficients is approximated by a Laplacian model [START_REF] Brites | Correlation noise modeling for efficient pixel and transform domain WynerZiv video coding[END_REF]. Knowing the sideinformation and the syndrome bits, the decoder estimates the transmitted bit-planes. If the number of syndrome bits is not sufficient to have a BER lower than 10 -4 at the output of the decoder, more syndrome bits are requested from the encoder.

B. Accuracy of the GE model

The question we deal with here is whether the bit-planes are better approximated by the proposed GE source model (Section II-B) or by a non-uniform source model. To that end, we investigate the distribution of the bursts of 1's, i.e. the number of consecutive 1's, in the bit-planes.

First, consider a binary source X without memory. It can be modeled as a Bernoulli process with parameter p X = P(X = 1). In this case, the burst length distribution, (P k ) k≥1 , is defined as the probability of having a sequence with k consecutive 1's, given that the sequence starts with a 0 (i.e. 0 1 . . . 1 k 0). For the Bernoulli process,

P k = (1 -p X )p k-1 X (19)
Therefore, for Bernoulli sequences, the log-scale burst distribution log(P k ) is linear in the burst length. Now, consider a GE source. Its steady-state distribution is Π = (1-t sd ) t sd t ds (1-t ds ) be the state transition probability matrix. Let I = 1 0 0 1 be the identity matrix. We also introduce the following notation C = 1-ps 1-p d and I = 1 1 . Then, the burst length distribution is:

P k = Π (I -B) (P B) k C Π (I -B) (P B) 2 I ( 20
)
Let us consider the bit-plane sequences obtained within the DISCOVER codec. For those sequences, we plot the empirical burst length distributions and the theoretical ones. More precisely, the empirical distribution is obtained by counting the occurrence of each burst length directly from the bit-plane. Given the GE parameters estimated from a given bit-plane (that are estimated thanks to the Baum-Welch algorithm), the theoretical burst length distribution is computed from Equation [START_REF] Cover | Elements of Information Theory[END_REF]. Fig. 7 shows the comparison of the empirical distribution of a given bit-plane (bold line) with two theoretical distributions (dashed lines): one obtained by assuming that the bit-plane has no memory, from Equation [START_REF] Toto-Zarasoa | Non-uniform source modeling for distributed video coding[END_REF], and the other obtained by taking into account the memory, from Equation [START_REF] Cover | Elements of Information Theory[END_REF]. Interestingly, the empirical distribution of the bit-plane matches well with the memory-aware theoretical distribution.

The plots in Fig. 7 only prove that the GE model is accurate for the particular bit-plane that is analyzed. To quantify the memory in a larger number of bit-planes, we look at the behavior of the parameter θ X in the 100 first bit-planes of some video sequences. The bit-planes are taken one after another, following the decoding order in the DISCOVER decoder. More precisely, for each bit-plane, we observe t ds , t sd , and the ratio p d ps . Memory is present in the bit-planes if the three following criteria are satisfied: Fig. 6. Block diagram of the IST-DISCOVER DVC codec. The approach is rate-adaptive, meaning that the syndrome is transmitted incrementally to avoid re-encoding the data. (a) t ds 0.5, (b) t sd 0.5, (c)

Virtual Channel Model T T -1 T Q -1 and
p d p s 1 (21)
The criteria (a) and (b) imposing low transition probabilities mean that the memory is persistent; the criterion (c) imposing a high ratio of the two Bernoulli parameters means that the two states are clearly distinguishable. Otherwise, the two states are similar and the EM algorithm is not able to differentiate them.

The estimated transition parameters are shown in Fig. 8 for the three video sequences Hall Monitor, Foreman and Soccer. Note that the transition probabilities can be very low, mainly for the sequence Soccer; this does account for a huge persistence of the states, hence of the memory in the bit-planes.

The ratio of the estimated Bernoulli parameters are shown in Fig. 9 for the same video sequences, and the same 100 first bit-planes. Here, we note that the ratio can be greater than 5 in some of the bit-planes.

The combination of the low transition probabilities (Fig. 8) and the huge Bernoulli parameters ratio (Fig. 9) accounts for the presence and for the persistence of memory in the WZ bitplanes. The amount of WZ bit-planes that fulfill the criteria stated in ( 21) is large enough to justify the modeling.

C. Exploiting the bit-planes memory together with the Laplacian correlation channel

In the DISCOVER codec, the WZ bit-planes, modeled as the realizations of GE sources, are obtained from the binarization of the quantized coefficients resulting from the DCT transform applied on the WZ frames. First, we obtain the SI bit-planes from the binarization of the quantized SI DCT coefficients. Then, the WZ syndromes and the SI bit-planes are input to the EM algorithm (see Section III), which estimates the WZ bit-planes of the current frame. The EM algorithm estimates the statistics of the source while the statistics of the (BSC) correlation noise is estimated with [START_REF] Toto-Zarasoa | Maximum likelihood BSC parameter estimation for the Slepian-Wolf problem[END_REF].

However, the binarization of the SI is suboptimal since only partial information of the SI is used. Therefore, we now consider the Laplacian model between the obtained WZ DCT coefficients X k and the SI DCT coefficients Y k introduced in [START_REF] Brites | Correlation noise modeling for efficient pixel and transform domain WynerZiv video coding[END_REF], i.e. Y k = X k + Z k , where Z k stands for the Laplacian noise, and k for the frequency band. The density function of the noise is given by:

p Z k (z) = αk 2 e -αk |z| (22)
where αk is the Laplacian parameter estimated from each frequency band k of the SI. We further assume that the noise Z k is independent of the WZ coefficients X k , as in the definition of the additive channel. Note that this definition differs from the assumption in [31, Proposition 1], where the noise is assumed to be independent of the SI. It follows that the a priori probabilities have to be recomputed with respect to previous DVC decoders taking into account the GE model and the independence between the WZ coefficients and the noise. More precisely, ∀n ∈ [1, N ] and for each DCT frequency band k, the WZ coefficient x k,n is quantized into B bits. The b-th bit denoted x b k,n has an associated a priori probability that takes into account the previously decoded bit-planes. The corresponding LLR is given by: log

  P X b k,n = 1|y k,n , x 1 k,n , . . . , x b-1 k,n P X b k,n = 0|y k,n , x 1 k,n , . . . , x b-1 k,n   = log    x∈Q(1) p Z k (y k,n -x)p(x|x 1 k,n , . . . , x b-1 k,n )dx x∈Q(0) p Z k (y k,n -x)p(x|x 1 k,n , . . . , x b-1 k,n )dx    (23)
where

• y k,n is the n-th DCT coefficient of the frequency band k; • Q(m) = Q(m, x 1 k,n , . . . , x b-1 k,n
) is the set of all the quantization intervals corresponding to the quantized symbols, which b most significant bits are (x 1 k,n , . . . , x b-1 k,n , m). At iteration l of the EM algorithm, p(x|x 1 k,n , . . . , x b-1 k,n ) can be computed from the current probabilities of the GE source and its parameters. More precisely, the density of the WZ coefficient is assumed to be constant on each quantization interval. It follows that [START_REF] Eckford | The factor graph EM algorithm: applications for LDPC codes[END_REF] Xn is given in [START_REF] Garcia-Frias | Turbo decoding of Gilbert-Elliot channel[END_REF]. This equation [START_REF] Wu | On the convergence properties of the EM algorithm[END_REF] replaces "B n + S n " in Equation [START_REF] Bhattad | A decision feedback based scheme for Slepian-Wolf coding of sources with Hidden Markov correlation[END_REF].

D. First Performance Analysis

In order to assess the performance achieved by the proposed Slepian-Wolf decoder exploiting the memory in the bit-planes, we implement the LDPC-based estimation-decoding EM algorithm for the DISCOVER codec. All the bit-planes do not have memory: some can be i.i.d. binary sources. But this model is a particular case of the GE model, so the decoder adapts by itself to any of the models.

We show in Fig. 10 the performance of the modified decoder if the noise between the WZ coefficients and the SI is assumed to be additive i.e. independent of the WZ coefficients. For the sequence Soccer, the rate gain is almost 10% at the highest PSNR, with respect to the standard SW decoder of the DISCOVER codec. However, we see that there is not always a rate gain when exploiting the bit-planes memory; E.g., for the sequence Foreman, the standard and the proposed decoder have almost the same performance; for the sequence Hall Monitor, the performance of the proposed decoder is degraded compared to that of the standard decoder. the performance are given by the curve b in Fig. 4. From Lemma 4, the decoder that assumes that the correlation type is predictive is the standard decoder a (in Fig. 4), that does not take into account the source statistics. Therefore, the rate loss due to a mismatch between a true additive and assumed predictive correlation channel is the distance between the curves a and b in Fig. 4. Now we consider the case where the true correlation channel is predictive between the two GE sources {X n } and {Y n }, i.e. ∀n, X n = Y n ⊕ Z n . If the decoder is matched to the correlation type, the BER of X is given by the curve a in Fig. 11. If the decoder is mismatched and assumes that the correlation type is additive, the BER is greater (see curve b in Fig. 11). Therefore, we conclude that a mismatch between the true predictive and the assumed additive correlation channel degrades the performance of the decoder. It is therefore important to estimate the correlation type. In the following, we propose an estimator for the correlation type to be used in a DVC codec.

C. DVC RD performance when estimating the correlation model

From the results above, it is important to use in the LDPC decoding algorithm the most appropriate correlation model. Therefore, we add an extra step to the DVC decoder to estimate on-the-fly the correlation model. To that end, for each syndrome bits request made by the decoder, the decoding is first performed with the additive channel assumption; if this decoding fails, (see the condition in Section III-E), another decoding is carried out, assuming that the channel is predictive.

Fig. 12 shows the results obtained for all the WZ frames of the video sequences Hall Monitor, Foreman and Soccer. Interestingly, we now observe that for all video sequences, the proposed algorithm decreases the rates, which was not the case when the correlation model was always assumed to be additive (see Fig. 10). The decrease of rate for the sequence Hall Monitor is 2.54kbps (-2.72%) at the highest PSNR, while it is 8.76kbps (-4%) for Foreman, and 29.55kbps (-10.14%) for the sequence Soccer. This proves that it is worth taking into account the memory estimated from the bit-planes, and that the additive channel is not sufficient to model the correlation in the bit-planes generated by the DVC codec. In Table I we show the percentage of use of the additive channel model during the decoding of the three sequences. These figures have been obtained a posteriori, after the decoding has ended. The combination of the results presented in Fig. 12 and in Table I shows that the more the additive channel model is used, the more gain is obtained for the decoding of the corresponding test sequence.

D. Non-uniform source modeling of the bit-planes

The GE process is a simple yet accurate model for the video bit-planes. However the estimation of the states of the HMM increases significantly the decoder complexity. Therefore, we propose here a simplified model, where only the non-uniformity of the source is exploited. Note that the nonuniform source model is a particular case of the hidden Markov model, with the particular values p s = p d = p W Z .

We first investigate the statistics of the WZ bit-planes by assessing off line their Bernoulli distributions. To that end, we measure the Hamming weight of each WZ bit-plane divided by its length. We show in Fig. 13 the empirical probabilities of 1's in the 200 first WZ bit-planes from the three video sequences Hall monitor, Foreman, and Soccer when the PSNR is the highest. These Bernoulli distributions confirm that the bit-planes are non-uniformly drawn. Their distribution varies from bit-plane to bit-plane. Moreover, the parameters have a periodic structure, of period 64 bit-planes; this corresponds to the 64 bit-planes that represent each frame of the video sequence.

The results presented here are obtained for all the WZ frames of the video sequences Hall monitor, Foreman, and Soccer, with a GOP length of 2. The RD performance of the proposed SW decoder that uses the non-uniformity is compared to the standard DISCOVER's SW decoder in Fig. 14. The correlation channel model (additive or predictive) is unknown by the decoder, and has to be assessed as explained above for infinite memory sources (Fig. 12 in section V-C). The proposed decoder that exploits the non-uniformity, while assessing the type of the channel, needs less rate than the standard one to the videos at the same PSNR.

The rate decrease is 0.94kbps (-1%) for the sequence Hall monitor at the highest PSNR; it is 5.88kbps (-2.68%) for Foreman; and 16.57kbps (-5.7%) for Soccer. That decrease proves that it is worth taking into account the non-uniformity of the bit-planes although the gain is less than by exploiting their memory. The advantage is that only one parameter has to be estimated by the decoder (instead of four parameters for the GE modeling).

E. Markov source modeling of the bit-planes

The Markov source model is the simplest representation of the memory that lies in the binary bit-planes. The Markov source is a particular instance of the two-state hidden Markov source, since it corresponds to the case where p s = 0 and p d = 1. This implies that the states are directly observed from the source realization itself, and the transition parameters can also be directly computed from the observed source realization.

We modify the estimation-decoding EM algorithm to estimate the state parameters and the transition probabilities of this Markov model. Then, we place the modified SW decoder in the DISCOVER codec. The results that we obtained show that there is practically no rate gain for the coding of the WZ bit-planes, with respect to the standard DISCOVER codec. More precisely the rate improvement is only 0.27kbps for the sequence Hall Monitor at the highest PSNR; it is 0.18kbps for Foreman and 1.15kbps for Soccer. It is worth noting that the non-uniform source model (Section V-D) brings more improvement than the Markov one, in terms of the PSNR versus rate performance of the DISCOVER codec. This might be counter-intuitive, but for two correlated Markov sources X and Y , the states of X and Y do not correspond (Lemma 1 do not stand for Markov sources); we can say that the knowledge of the states of Y is misleading for the decoding of X.

Therefore, the Markov source model is not suited to the memory that lies in the WZ bit-planes generated by the DVC codec.

VI. CONCLUSION We have proposed a new source model for DVC bit-planes: a Gilbert-Elliott model that takes into account the memory in the original video bit-planes, complemented with a correlation model that takes into account the nature of the noise. More precisely, the correlation channel is modeled as a predictive or an additive BSC (or Laplacian channel). We have also shown that a mismatch between the true and the assumed correlation model degrades the compression rate. Therefore, we have proposed a test to make a decision between the two models. A joint estimation-decoding EM algorithm is performed for the parameter estimation and the decoding. For the DVC setup, we demonstrated the accuracy of the source model for the WZ bitplanes; the rate gain is up to 10.14% when the bit-planes are considered as GE sources, with respect to the case where they are considered as uniform sources. Finally, a simplified model with non-uniform sources has been proposed that achieves an improvement by up to 5.7%. APPENDIX A. Details for the maximization step of the EM algorithm Here, we derive the parameters update rules in Equation ( 8) for the maximization problem (3). From Equations ( 5) and ( 6), the mean log-likelihood function is expressed as:

Fig. 1 .

 1 Fig. 1. The GE model: a two-state HMM.

Fig. 2 .

 2 Fig.2. Comparison of the source conditional entropies, when X is uniform, and when X is a GE process.

Fig. 3 .

 3 Fig. 3. Factor graph describing the joint estimation-decoding EM.

Fig. 4 .Fig. 5 .

 45 Fig. 4.Performances of the three decoders, for a GE source {Xn} of parameter θ X = (ps = 0.07, p d = 0.7, t ds = 0.03, t sd = 0.01). When exploiting the memory, H(X|Y) = 0.5 occurs for H(p) = 0.88 (p = 0.299), instead of H(p) = 0.5 (p = 0.11) when considering the source as uniform.

1 t

 1 ds +t sd [p s t ds p d t sd ]. Let B = ps 0 0 p d be the matrix which diagonal values are the Bernoulli parameters. Let P =

Fig. 7 .

 7 Fig. 7. Distribution of the bursts of 1's for a selected bit-plane of the video sequence Soccer.

Fig. 8 .

 8 Fig.8. The estimated transition parameters (t sd and t ds ) from the 100 first bit-planes from the video sequences Hall Monitor, Foreman and Soccer.

Fig. 9 .

 9 Fig. 9. The ratio of the estimated Bernoulli parameters ( p d ps ) from the video sequences Hall Monitor, Foreman and Soccer.

2 e

 2 = 1|y k,n , x 1 k,n , . . . , x b-1 k,n P X b k,n = 0|y k,n , x 1 k,n , . . . , x b-αk |y k,n -x| dx x∈Q(0)αk 2 e -αk |y k,n -x| dx

Fig. 10 .

 10 Fig.10. Behavior of the proposed DVC decoder when the Laplacian channel is assumed to be always additive. Exploiting the memory that is present in the bit-planes does not always improve the performance of the codec.

Fig. 11 .

 11 Fig. 11. Performances of the standard decoder a and the proposed decoder b exploiting the source memory. The mismatch degrades the performance when the memory is exploited.

Fig. 12 .

 12 Fig.[START_REF] Ser | Asymmetric joint sourcechannel coding for correlated sources with blind HMM estimation at the receiver[END_REF]. The decoder that exploits the bit-planes memory needs less rate to render the videos at the same PSNR.

Fig. 13 .

 13 Fig.[START_REF] Bhattad | A decision feedback based scheme for Slepian-Wolf coding of sources with Hidden Markov correlation[END_REF]. Probability of 1's in the 200 first WZ bit-planes of each video sequence taken individually. This corresponds to a little more than the 3 first images. The WZ bit-planes of the three video sequences are mostly nonuniformly distributed, which justifies the model adopted here.

  Fig. 14.The proposed decoder that exploits the non-uniformity, while assessing the type of the channel, needs less rate than the standard one to the videos at the same PSNR.

  n-1 = i, Σ n = j|y, s x ) log(t l ij ) n = i|y, s x )P θ l X (X n = 1|y, s x ) log(p l i ) + P θ l X (Σ n = i|y, s x )P θ l X (X n = 0|y, s x ) log(1 -p l i )

  The GE process {X n } is a binary source which depends on the Markov process {Σ n }. More precisely, X n only depends on Σ n . Therefore, Σ n represents the hidden state of the random process {X n }. In each state s and d, the source is drawn according to a Bernoulli law of parameter p s and p d respectively (Fig1), with p s ≤ p d by definition. p s and p d respectively stand for the probability of having X n = 1 in states s and d.

	t ss	t sd
	s		d	t dd
	Bernoulli(p s )	t ds	Bernoulli(p d )

  z n are the noise symbols, represented by the BSC nodes; • s m are the syndrome symbols, represented by the check nodes. x n is connected to s m in the bipartite graph if h mn = 1 in the parity-check matrix; • d xn is the degree of x n , i.e. the number of check nodes connected to it; • d sm is the degree of s m , i.e. the number of variable nodes connected to it; • I n is the intrinsic information for the node z n ; • E n,e , e ∈ [1, d xn ] are the messages passed from the variable nodes, on their e-th edge, to the check nodes;

• x n are the source symbols, represented by the variable nodes; their estimates are denoted xn ; • y n are the side-information symbols, represented by the side-information nodes; • • E n is the a posteriori Log-Likelihood Ratio of xn ; • Q m,e , e ∈ [1, d sm ] are the messages passed from the check nodes, on their e-th edge, to the variable nodes; • B n are the messages passed from the BSC node z n to the variable node x n ; • S n are the messages passed from the state nodes to the variable nodes;

  Reconst.

		3		Wyner-Ziv Encoder		8		Wyner-Ziv Decoder
		3a	3b	3c	3d	3f	8a	8b	8c	8d
			Q	Bit Ordering	Channel Encoder	Buffer	Channel Decoder	Decoder Success / Failure
					3e Minimum Rate		7	7b	7a	6
					Estimation			Soft input
								Computation
	1	WZ and			2		4	5	Side-
		Conventional			Conventional	Conventional	Information
		Video Splitting			Video Encoder	Video Decoder	Extraction

TABLE I PERCENTAGE

 I OF USE OF THE ADDITIVE CHANNEL MODEL.

V. PREDICTIVE CORRELATION CHANNEL: RATE BOUNDS AND PRACTICAL DVC RD PERFORMANCE We have seen above that, although the GE model fits well with the bit-planes distributions, exploiting the memory in the video bit-planes does not always bring a rate gain, when assuming the correlation model to be additive. We thus consider here a second correlation model, called the predictive model. Definition 2. An (X, Y, p) predictive BSC is a channel with binary ergodic input {X n }, binary ergodic output {Y n }. The noise is an i.i.d. binary process Z ∼ B(p). Z is independent of the channel output s.t. ∀n, X n = Y n ⊕ Z n .

This model has been introduced in [START_REF] Zamir | Nested linear/lattice codes for structured multiterminal binning[END_REF], where it is called "the reverse channel", and in [START_REF] Cheng | Successive refinement for the Wyner-Ziv problem and layered code design[END_REF]. This model assumes that the correlation noise Z is independent of the SI, instead of being independent of the source {X n } as in the additive model. Note that the additive model is the model considered for the correlation noise in most work so far in DSC [START_REF] Garcia-Frias | Compression of correlated binary sources using Turbo codes[END_REF], [START_REF] Aaron | Compression with side information using Turbo codes[END_REF], [START_REF] Liveris | Compression of binary sources with side information at the decoder using LDPC codes[END_REF], [START_REF] Garcia-Frias | LDPC codes for compression of multiterminal sources with Hidden Markov correlation[END_REF], [START_REF] Ser | Asymmetric joint sourcechannel coding for correlated sources with blind HMM estimation at the receiver[END_REF], [START_REF] Bhattad | A decision feedback based scheme for Slepian-Wolf coding of sources with Hidden Markov correlation[END_REF], [START_REF]Distributed joint source-channel coding for arbitrary memoryless correlated sources and source coding for Markov correlated sources using LDPC codes[END_REF]. Interestingly, most DVC codecs assume that the bitplanes to be compressed are i.i.d. uniform processes. In this case, as shown in the next section (see Lemma 5), the correlation type does not influence neither the decoder nor the achievable compression bound. Therefore, the authors do not explicitly state what the assumed correlation model is. However, [START_REF] Puri | A new robust video coding architecture based on distributed compression principles[END_REF], [START_REF] Girod | Distributed video coding[END_REF], [START_REF] Nayak | Spatial prediction in distributed video coding using Wyner-Ziv DPCM[END_REF] use the duality between channel coding and asymmetric DSC, which only holds for additive BSC. So it can be deduced that an additive correlation channel is assumed. However, in [START_REF] Brites | Encoder rate control for transform domain Wyner-Ziv video coding[END_REF] a reference to [31, Proposition 1] is made, where the channel is predictive.

A. Rate bounds for the predictive correlation channel

In this section, we study the case when the correlation between the binary sources {X n } and {Y n } is a predictive channel.

Lemma 3. Let {X n } and {Y n } be two correlated binary ergodic sources, where the correlation is modeled as a virtual (X, Y, p) predictive BSC i.e. there exists an i.i.d. binary process (noise) Z ∼ B(p) s.t. ∀n, X n = Y n ⊕Z n . We consider the asymmetric DSC problem, where {Y n } is available at the decoder and {X n } is transmitted at a rate greater than its conditional entropy-rate H(X|Y).

The minimum transmission rate for {X n } only depends on the noise statistics. More precisely H(X|Y) = H(Z). Therefore the minimum transmission rate for {X n } is not reduced, with respect to that of a uniform source.

Proof: Here Z is independent of {Y n }. Therefore, H(X|Y) = H(Z) and the minimum coding rate for {X n } only depends on the noise statistics. So, the minimum transmission rate for {X n } is not reduced, with respect to that of a uniform source.

Lemma 4. Let {X n } and {Y n } be two correlated binary ergodic sources, where the correlation is modeled as a virtual predictive channel i.e. there exists an i.i.d. binary process

The MAP estimate for the source {X n }, when {Y n } is available at the decoder, only depends on the noise statistics. Therefore, for a given SI realization y, whatever the binary source model, the performances of the MAP decoders are the same.

Proof: The MAP detection problem

needs the a priori probabilities P (x n |y n ), which only depend on p Z (y n -x n ) and not on the source statistics. The noise statistics is therefore a sufficient statistic for the MAP detection problem [START_REF] Kschischang | Factor graphs and the sum-product algorithm[END_REF]. Therefore any MAP decoder with a predictive correlation model, whatever the source statistics, give the same performance. This result is coherent with Lemma 3: since the decoding algorithm is the same whatever the binary source model, the minimum transmission rate is also the same.

Lemma 5. Let X and Y be two correlated binary, uniform i.i.d. sources. We consider the asymmetric DSC problem, where Y is available at the decoder and X is transmitted at a rate greater than its conditional entropy H(X|Y ).

The compression rate for X and the MAP decoder only depends on the noise statistics and is independent of the correlation channel type (predictive or additive BSC).

Proof: The MAP detection problem (25) needs the a priori probabilities P (x n |y n ), which is p Z (y n -x n ) if the correlation model is predictive and 0.5 * p Z (y n -x n ) if additive. Thus, the MAP estimate only depends on the noise statistics and is therefore independent of the correlation type.

Moreover, from Lemma 2 (additive BSC) and Lemma 3 (predictive BSC), the compression rate for X is H(Z), whatever the correlation type (additive or predictive BSC).

From Lemma 4 and Lemma 5, we conclude that the three decoders with the assumptions detailed below give the same performances: -either the sources are memoryless with an additive correlation model, -or the sources are memoryless with a predictive correlation model, -or the sources have memory with a predictive correlation.

B. Rate loss in case of mismatch between the additive and the predictive correlation models

In the previous Section, we have shown the minimum transmission rate when the correlation channel is predictive (Lemma 3). These bounds are derived under the hypothesis that the decoder is matched to the source distribution and to the correlation type. When the decoder is mismatched, there is a rate loss, which is evaluated below.

Let us first assume that {X n } and {Y n } are two GE sources and that the correlation model is additive. We consider the same parameter setup as in Section III-G and would like to estimate the rate loss when the decoder assumes that the correlation type is predictive, whereas the true one is additive. If the decoder assumes that the correlation type is additive, Then the partial derivatives with respect to the Bernoulli parameters are, ∀i ∈ {s, d}:

Then, the derivative ( 27) is zero when, ∀i ∈ {s, d}:

For the transition parameters, given that t ss = (1 -t sd ) and t dd = (1 -t ds ) (1), the partial derivatives are given by, ∀i, j ∈ {s, d}, i = j:

Then, the derivative (29) is zero when, ∀i, j ∈ {s, d}, i = j:

Note that the solutions ( 28) and (30) satisfy the constraints [START_REF] Garcia-Frias | Decoding of Low-Density Parity-Check codes over finite-state binary Markov channels[END_REF] of our optimization problem (3).

B. Forward-backward algorithm for the soft estimate of Σ

Here, the probabilities of Σ are updated according to the values of the estimate θ l X . The aim is to compute, ∀n ∈ [1, N ],

To that end, we decompose the following expression, to retrieve the equations corresponding to the forward-backward recursions:

The forward-backward algorithm is run on the trellis in Fig. 15 defined by the states s and d generating the source symbols, with two branches between the states, labeled by the two possible values x n = 0 and x n = 1. We define

i∈{s,d} α (n-1) i

• γ

(n-1),n i,j

• γ n,(n+1) i,j is the transition probability between the states i at position n and j at position (n + 1).

• α n j is the forward probability for the source to be in state j at position n; • β n i is the backward probability for the source to be in state i at position n. Now we define the states APP:

• γ

(n-1),n i,j

• β n j (34) Normalizing P θ l X (σ n , y|s x ) and P θ l X (σ n-1 , σ n , y|s x ), we get P θ l X (σ n |y, s x ) and P θ l X (σ n-1 , σ n |y, s x ).


Mental health problems in youths committed to juvenile institutions: prevalences and treatment needs

Ola Ståhlberg, Henrik Anckarsäter,

Introduction

Many international studies report high prevalences of mental disorders among adolescents in coercive care. Teplin and coworkers [START_REF] Teplin | Psychiatric disorders in youth in juvenile detention[END_REF] showed that almost two-thirds of male and three-quarters of female juvenile detainees fulfilled criteria for one or more mental disorders. Excluding conduct disorder (CD) as a tautological definition in research on the causes of institutional care since it is a direct description of the kind of behaviors that tend to warrant institutionalization, 60 % of boys and 66 % of girls still met the diagnostic criteria for at least one major psychiatric disorder. Studies from the United Kingdom have compared the prevalence of mental disorders among young people in care to adolescents in the general population and found a three-to five-fold increase among the former [10,[START_REF] Meltzer | The mental health of young people looked after by local authorities in England. The Stationery Office[END_REF], with mental disorders significantly affecting at least one in two, and even higher rates among the subjects in residential care compared to those in more family-like forms of intervention [START_REF] Meltzer | The mental health of young people looked after by local authorities in England. The Stationery Office[END_REF].

Teplin and coworkers [START_REF] Teplin | Psychiatric disorders in youth in juvenile detention[END_REF] found the most common disorders among both boys and girls to be substance use disorders, disruptive behavior disorders (oppositional defiant disorder (ODD), and conduct disorder (CD)), followed by anxiety disorders, mood disorders, and attention-deficit/hyperactivity disorder (AD/HD). A recent meta-analysis based on 25 studies mainly confirmed these findings and also found a ten-fold increase in psychotic disorders among institutionalized adolescents compared to the overall population [START_REF] Fazel | Mental disorders among adolescents in juvenile detention and correctional facilities: a systematic review and metaregression analysis of 25 surveys[END_REF].

High degrees of overlap across mental disorders have also been reported among juvenile detainees. Abram and colleagues showed that more than half of girls and almost half of boys fulfilled the criteria for two or more disorders (often including severe mental disorders and substance abuse in combination, sometimes referred to as "double diagnoses") [START_REF] Abram | Comorbid psychiatric disorders in youth in juvenile detention[END_REF].

In a clinically based Swedish study, Anckarsäter and coworkers [START_REF] Anckarsater | Prevalences and configurations of mental disorders among institutionalized adolescents[END_REF] showed that 53 % of institutionalized adolescents fulfilled criteria for at least one neuropsychiatric disorder (defined as autism spectrum disorders (ASDs), AD/HD, tic disorder, mental retardation (MR), and other learning disabilities). AD/HD, affecting 39 % of the adolescents, was the most common disorder. Moreover, 66 % of the adolescents had a psychiatric disorder requiring specialist treatment (AD/HD, ASD, MR, complicated depression and/or psychoses).

Disruptive behavior disorders are conditions that not only severely affect children's general health and education [START_REF] Foster | The high costs of aggression: public expenditures resulting from conduct disorder[END_REF] but also carry a dramatically increased risk for mental health problems, substance abuse, and criminality in adulthood [START_REF] Hofvander | Continuity of aggressive antisocial behavior from childhood to adulthood: The question of phenotype definition[END_REF][START_REF] Kim-Cohen | Prior juvenile diagnoses in adults with mental disorder: developmental followback of a prospective-longitudinal cohort[END_REF]. It is therefore of vital importance, both for those afflicted and for society at large, that the period when a troubled child is in institutional care (if not before) is used to identify vulnerable individuals and implement effective preventive and treatment programs to counteract the negative prognosis. One proposed risk indicator is the age at onset of CD and substance abuse [START_REF] Gustavson | Age at onset of substance abuse: a crucial covariate of psychopathic traits and aggression in adult offenders[END_REF][START_REF] Moffitt | Childhood-onset versus adolescent-onset antisocial conduct problems in males[END_REF], with the most severe mental health problems and poorest prognosis in those with an early onset of CD (defined as pre-pubertal or before 10 years of age).

These circumstances illustrate the need for further studies with a broad diagnostic approach, and the present study was designed to explore findings from comprehensive neuropsychiatric and psychiatric assessments of subjects in Swedish juvenile institutions organized under the authority of the National Board of Institutional Care (SiS). The specific aims were: a) to provide detailed data on the prevalence and constellation of mental health problems, operationally defined according to the DSM-IV [START_REF]Diagnostic and statistical manual of mental disorders : DSM-IV[END_REF] with special focus on AD/HD, ASDs, substance abuse, and CD (with early or late onset), b) to describe the results of intelligence tests in different diagnostic subgroups, and c) to quantify previous contacts with the child and adolescent psychiatric services in this group of institutionalized adolescents. A previously published study on similar groups [START_REF] Anckarsater | Prevalences and configurations of mental disorders among institutionalized adolescents[END_REF] will be used for comparisons of prevalences and configurations.

Subjects and methods

Subjects

In Sweden, three different laws regulate the placement of adolescents in specialized SiS institutions:

1) Care of Young Persons (special provisions) Act (SFS 1990:52) [27] ("young persons act", referred to as "YPA"), a law applied if the adolescent, due to his/her behavior or environment, is at risk of coming to harm.

2) Care of Young Offenders Act (SFS 1998:60) [28] ("young offenders act", referred to as "YOA"), a law that allows courts of law to sentence offenders between the ages of criminal responsibility (15 years) and maturity (18 years) to incarceration in special youth institutions.

3) The Social Services Act (SFS 2001:453) [29], a law applied if the adolescent, due to his/her behavior or environment, is at risk of coming to harm and there is consensus between the authorities and the subject/parents that treatment is needed.

During 2007, a total of 1 396 individuals (916 boys, 480 girls) were placed in specialized state-run institutions in Sweden: 1 242 (787 boys, 455 girls) of these according to the YPA, 81 (76 boys, 5 girls) according to the YOA, and 73 (53 boys, 20 girls) according to the Social Service Act). The total population in Sweden included 504 544 males and 477 990 females between the ages of 12 and 19 years, and about 1.25 per thousand adolescents and year were thus placed in institutions.

The present study group consisted of adolescents committed to specialized youth institutions in the southwest of Sweden between September 2004 and February 2007.

Inclusion in the study required that the referring authorities, the court or the Social Services, had requested psychosocial and psychiatric assessments. Adolescents scheduled for no more than a short stay at the institutions due to emergency placements were excluded. Each subject gave his or her written consent to participate in the study. All nine juvenile SiS institutions in the Swedish region of Västra Götaland were invited to participate in the study. Two of the four institutions that accepted participation contributed 95 of a total of 103 consecutively committed adolescents meeting the inclusion criteria (rate of consent 92 %). The other two institutions contributed sporadic cases (n=15), giving a total study group of 110 adolescents.

As only three of the 110 initially included subjects were committed according to the Social Services Act, these subjects were excluded from further analyses. Seven other subjects were excluded because of missing data.

Methods

Based on the investigational proceedings normally followed at the studied institutions, clinical information was collected by teams consisting of psychologists, psychiatric social workers, and ward personnel on the regular staff and psychiatric specialists called in as consultants to conduct the medical investigations. The ratings were made by four specialists in psychiatry, eight psychologists, and three social workers. In addition, one of the authors (HA) was consulted in all complicated cases. He also scrutinized all medical files and made final decisions on the diagnostic work-up, when needed in consensus with the investigating teams. Three different study protocols were used: Protocols A, B, and C. Protocol A, containing demographic and other background information, included detailed data on school achievement, criminal history, substance abuse, family situation, ethnical origin, and was filled out by the social workers or, in a few cases, by the team psychologist. Protocol B, which contained data on neurocognitive function (i.e. the Wechsler Adult Intelligence Scale (WAIS) [34] or the Wechsler Intelligence Scale for Children (WISC) [START_REF] Wechsler | Manual for the Wechsler Intelligence Scale for Children-Third Edition[END_REF] test results) as well as information on personality factors, such as results on the Temperament and Character Inventory (TCI) [START_REF] Cloninger | A psychobiological model of temperament and character[END_REF], Junior Temperament and Character Inventory (J-TCI) [START_REF] Luby | The Junior Temperament and Character Inventory: preliminary validation of a child self-report measure[END_REF], Beck Youth Inventories [START_REF] Beck | Beck Youth Inventories of Emotional & Social Impairment manual[END_REF], The Autism -Tics, AD/HD and other Comorbidities inventory (A-TAC) [5], and Youth Self-Report (YSR) [START_REF] Achenbach | Manual for the ASEBA School-Age Forms & Profiles[END_REF], was completed by the psychologist assigned to the case. The variation in instrumentation was due to the large age span among the subjects and to the fact that some of the instruments target only subpopulations in the study. Psychologists and/or social investigators rated 34 of the subjects with the Hare Psychopathy Checklist-Revised (PCL-R) [START_REF] Hare | The Hare Psychopathy Checklist-Revised[END_REF], and 73 completed a self-rating questionnaire (Youth Psychopathic Inventory (YPI) [START_REF] Andershed | Psychopathic traits in nonreferred youths: Initial test of a new assessment tool[END_REF]) aimed at capturing psychopathic characteristics. Protocol C, which included structured checklists with DSM-IV criteria from the specialized psychiatric investigations was filled out by the consultant psychiatrist after discussions (and in consensus) with the investigation team. Based on the medical evaluation files, protocol C thus provided DSM-IV-based diagnostics for Axis I, including data on child neuropsychiatric conditions (AD/HD, ASD, tics, MR etc). Protocols A and C were completed for all subjects, while protocol B was incomplete in four cases.

Diagnostic algorithms defined for the pervasive developmental disorders (PDDs) in the DSM-IV were followed. In accordance with the most widely used clinical terminology, these disorders are referred to as autism spectrum disorders (ASDs), including autistic disorder (defined as fulfilling a total of at least six DSM-IV criteria distributed as at least two criteria under "impairment in social interaction" and at least one criterion under "impairment in communication" and at least one criterion under "restricted repetitive and stereotyped patterns of behavior, interests and activities"), Asperger's disorder (defined as fulfilling at least two DSM-IV criteria under "impairment in social interaction" and at least one criterion under "restricted repetitive and stereotyped patterns of behavior, interests and activities"), and PDD not otherwise specified (PDD NOS, defined as fulfilling at least three criteria distributed among any of the three DSM-IV autism areas).

The ASDs were mutually exclusive and arranged in a hierarchical order where Asperger's disorder and PDD NOS were subordinated to autistic disorder and PDD NOS was subordinated to Asperger's disorder. AD/HD was classified into three subgroups according to the specifications in DSM-IV: 1) AD/HD predominantly inattentive type, ADD (defined as fulfilling at least six inattention DSM-IV criteria), 2) AD/HD predominantly hyperactive-impulsive type (hyperkinetic disorder, HD) (defined as fulfilling at least six hyperactivity-impulsivity DSM-IV criteria), and AD/HD combined type (defined as fulfilling at least six hyperactivity-impulsivity criteria and at least six inattention criteria). The AD/HD subgroups were mutually exclusive. For all other disorders, DSM-IV criteria limiting the possibility of assigning other comorbid psychiatric diagnoses were disregarded to allow comprehensive recording of the pattern of comorbidity. Systematic assessments of possible concomitant medical disorders, such as brain MRIs or chromosomal analyses were not performed unless indicated by clinical findings.

The Wechsler Adult Intelligence Test III (WAIS-III) or the Wechsler Intelligence Scale

for Children IV (WISC-IV) were used to assess the IQ and factor scores. WAIS-III was used in 58 and WISC-IV in 38 cases, 16 years or older for WAIS-III vs. younger for WISC-IV. This means that 96 % of all subjects had a test-based assessment of cognitive profiles. The full-scale IQ (FSIQ) could not be calculated in four subjects in whom all sub-tests were not administered, but 92 % of all cases had a FSIQ determined by the Wechsler tests based on Swedish normative data provided by the publisher. In the statistical analyses, data from WAIS-III and WISC-IV were pooled to yield collapsed Wechsler scores. MR was defined as having a FSIQ equal to or below 70.

Statistics

To explore the differences between groups, non-parametric statistics were used (Chisquare test for independence, Mann-Whitney U-test). Ninety-five percent confidence intervals and two tailed p-values with a significance level set at the 5 % level (p ≤ 0.05) were used throughout the study. All calculations were made with the Statistical Package for the Social Sciences (SPSS) version 16.0.

Results

Seventy-three (73 %) subjects fulfilled the criteria for at least one major DSM-IV disorder, including ASD and AD/HD, but not counting CD and substance abuse. Tables 1 and2 present comprehensive figures of prevalences and patterns of overlap between disorders.

Almost half of the subjects had some form of AD/HD, with the combined type being the most frequent, followed by the inattentive and the hyperactive-impulsive types, respectively (see Table 1 for a detailed presentation). Overall, one subject in six met the criteria for an ASD, most often in the form of PDD NOS (primarily by fulfilling criteria within the "impairment in social interaction" area, a domain where 100 % of all subjects with an ASD met at least one criterion) (see Table 1 for a detailed presentation).

There were significantly more AD/HD-cases in the YPA than in the YOA group, 58 % vs 9 % (χ 2 (df = 1, n = 100) = 14.38, p < .001, phi = .403). Prevalences for ASDs and AD/HD were similar among boys and girls.

Eleven subjects (11 %) had both ASD and AD/HD, which means that 65 % of the subjects with ASD had comorbid AD/HD, and that 23 % of those with AD/HD had comorbid ASD. Overlaps between diagnoses are shown in Table 2.

Seventy-seven subjects (77 %) fulfilled the criteria for CD. The onset was reported as late (after 10 years of age) in 66 cases (86 %) and as early (before 10 years of age) in 11 ( 14%). Patterns of psychiatric disorders in these groups are described in Table 2. There were no significant associations (by Chi-square test for independency, with Yates Continuity Correction -data provided by the author upon request) between CD and AD/HD, ASD, sex, or legal category (YPA or YOA). Likewise, there was no significant association between early onset of CD and ASD, AD/HD, sex, or legal category.

Fifty-five (55 %) of all subjects had some kind of substance abuse (including alcohol).

There was no significant association between CD and substance abuse (chi-square test as previously).

The mean full-scale IQ (FSIQ) in the study group was 85.3 (SD ±14.3, range 45-121), i.e. almost exactly one standard deviation below the population mean. The corresponding figures for boys (n=92) and girls (n=8) were 85.7 (SD ±14.2, range 45-121) and 80.9 (SD ±16.1, range 54-106), respectively, showing no significant sex difference in FSIQ (U = 270.0, z = -.915, p = .360). In the YPA group (n=78) and the YOA group (n=22), the mean FSIQ figures were 83.7 (SD ±14.3, range 45-121) and 92.1 (SD ±12.7, range 71-114), respectively. This difference was statistically significant (U = 421.0, z = -2.180, p = .029). Eleven subjects (11 %) had a FSIQ equal to or below 70, meeting the criteria for MR, and 30 subjects (30 %) fulfilled the criteria for DSM-IV borderline intellectual functioning (FSIQ=71-84). The AD/HD group (n=46) had a mean FSIQ of 81.8 (SD ±13.6, range 45-121), which was significantly lower than the mean FSIQ of 88.7 (SD ±14.4, range 53-114) in the group without AD/HD (n= 46) (U = 711.0, z = -2.712, p = .007). In the group with ASD (n=17), the mean FSIQ was 84.4 (SD ±15.6, range 45-113), and the subjects with ASD did not differ from those without ASD (n= 75), whose mean FSIQ was 85.5 (SD ±14.1, range 47-121).

Besides the differences seen in FSIQ, only a few differences between diagnostic groups were noted in the factors (Verbal IQ (VIQ) and Performance IQ (PIQ)) and the four secondary indices (Verbal Comprehension Index (VCI), Perceptual Reasoning Index (PRI), Working Memory Index (WMI), and Processing Speed Index (PSI)) of the Wechsler scales. The groups with ADD, AD/HD combined form, and any form of AD/HD had significantly lower WMI than the group without AD/HD (U = 121.5, z = -2.13, p = .033; U = 339.5, z = -3.883, p = <.001 and U = 535.5, z = -3.729, p = <.001 respectively). The groups with AD/HD combined form or any form of AD/HD also had significantly lower FSIQ (U = 493.5, z = -2.642, p = .008 and U = 711.0, z = -2.712, p = .007 respectively) and VIQ (U = 495.0, z = -2.743, p = .006 and U = 764.5, z = -2.433, p = .015 respectively). AD/HD, regardless of subtype, was thus associated with significantly lower FSIQ, VIQ, and WMI. Among the groups with different forms of ASD no significant differences were found between the factors or indices. The YOA group scored significant higher than the YPA group in FSIQ, PIQ, PRI, and WMI (U = 421.0, z = -2.180, p = .029; U = 499.5, z = -2.007, p = .045; U = 398.0, z = -2.413, p = .016 and U = 330.0, z = -2.201, p = .028 respectively). The same pattern was found between FSIQ, VIQ, VCI, WMI, and PSI in the group with and the group without substance abuse (U = 557.5, z = -2.342, p = .019; U = 580.5, z = -2.318, p = .020; U = 601.0, z = -2.014, p = .044; U = 521.0, z = -2.279, p = .023 and U = 555.0, z = -2.558, p = .011 respectively). There were no significant differences in any factor or index between subjects with late-onset CD and early-onset CD (Table 3).

Overall, more than one in four (27 %) of the subjects had a severe mental disorder (ASD, MR, and/or psychotic disorder), which according to the Swedish legislation (the Special Support and Service Act SFS 1993:387) would make them entitled to special assistance.

The overall proportion of individuals in need of psychiatric specialist treatment (AD/HD, ASD, MR, psychotic disorder, and/or complicated depression) was 63 %. In addition to these problems, a few subjects fulfilled criteria for sleep disorder, eating disorder, and/or tics (Table 1).

Regardless of diagnosis, substance use, and neurocognitive function, almost all subjects (97 %) reported major difficulties in school, most often in the form of truancy, bullying, learning problems, and special tuition and supervision (Table 4).

More than half of the subjects had been in contact with child and adolescent psychiatric services (CAP) at least once before admittance to the institution, and more than one-third of these more than once. One in five had been in contact with CAP within a period of one year prior to the index referral to the youth institution. Eighty-eight percent of the subjects with ASD had been in contact with CAP at least once, and the corresponding figures for subjects with AD/HD, MR, and CD was 70 %, 54 %, and 52 %, respectively.

The important message conveyed by these figures is that considerable numbers of children with severe mental problems, including as many as half of those with MR, had not come to the attention of the CAP services before being committed to specialized institutions (Table 4 gives a thorough presentation of CAP contacts in the various diagnostic groups).

Discussion

Overall, 63 % of the adolescents in the studied youth institutions had a psychiatric disorder generally considered to require specialist attention. More than one in four, 27 %, had a psychotic disorder or a neurocognitive disability (ASD and/or MR) severe enough to entitle to special rights to assistance according to the Swedish legislation. These figures are generally consistent with the results of other surveys [START_REF] Meltzer | The mental health of young people looked after by local authorities in England. The Stationery Office[END_REF][START_REF] Nicol | Mental health needs and services for severely troubled and troubling young people including young offenders in an N.H.S. region[END_REF][START_REF] Teplin | Psychiatric disorders in youth in juvenile detention[END_REF][START_REF] Timmons-Mitchell | Comparing the mental health needs of female and male incarcerated juvenile delinquents[END_REF] and exceed by far the figures found for comparable normal populations [10,[START_REF] Kadesjo | Neuropsychiatric and neurodevelopmental disorders in a young school-age population. Epidemiology and comorbidity in a school health perspective[END_REF][START_REF] Meltzer | The mental health of young people looked after by local authorities in England. The Stationery Office[END_REF]. In a total population study of young schoolchildren from a middle-sized Swedish town, clinically severely impairing AD/HD was found in 3.7 % and autism and Asperger's disorder in another 1.1 % [START_REF] Kadesjo | Neuropsychiatric and neurodevelopmental disorders in a young school-age population. Epidemiology and comorbidity in a school health perspective[END_REF]. The rate of ASD in the general population was found to be 1.2 % [START_REF] Kadesjo | Brief report: autism and Asperger syndrome in seven-year-old children: a total population study[END_REF], which was in line with the range reported in other countries. That the general population figures are about ten times lower than the prevalences in our sample signals a considerable over-representation of severe mental health problems among institutionalized adolescents in Sweden. The diagnostic panorama seems to vary considerably across studies, however, depending on the criteria or terminology used.

Special assessments for ASDs seem to be rare in surveys outside Scandinavia and the UK, where, in contrast, considerable prevalences of especially PDD NOS/atypical autism are consistently reported. Assessments have to target not only narrow syndromes among the ASDs but also the atypical forms or even broader phenotypes in future studies of mental health problems among adolescents referred to specialized institutions in order to capture adolescents with severe social dysfunctions on the autism spectrum [START_REF] Anckarsater | Autism spectrum disorders in institutionalized subjects[END_REF].

The majority of adolescents with diagnoses of mental disorder fulfilled criteria for more than one such disorder. This was especially the case when the primary disorder was in the category describing neurocognitive problems, such as ASDs, MR, and/or AD/HD. The high degree of comorbidity corresponds to findings from other studies [START_REF] Kadesjo | The comorbidity of ADHD in the general population of Swedish school-age children[END_REF]30], and may imply a broad range of still unresolved treatment problems [START_REF] Gillberg | Co-existing disorders in ADHD --implications for diagnosis and intervention[END_REF]. Institutions for adolescents like those we investigated clearly need access not only to medical support but also to professionals especially trained in child and adolescent psychiatry and neuropsychiatry to provide diagnostic evaluations, neurological examinations, and pharmacological treatment both of common disorders, such as AD/HD, and rare conditions, such as autism, tic disorders, and various subtypes of MR.

Criteria for ASDs have to be adapted to adolescents in institutions, where social interaction problems and non-verbal communication abnormalities stand out as the hallmarks of the disorder, whereas classical Asperger-associated, fact-based special interests or elaborate repetitive routines are rare. Stereotyped behaviors may instead be found in repetitive crime patterns, collections of knives or other criminal paraphernalia, or a strong interest in drugs without indications of substance use disorders. A pervasive inability to take other people's perspectives and social "oddity", even when compared to other adolescents with similar lifestyles, are also recurrent features in ASD.

We found no support for the hypothesis that age of onset of CD would distinguish between clinical subgroups (with more severe problems corresponding to early-onset CD). One possible reason for this disagreement with previous findings [START_REF] Moffitt | Childhood-onset versus adolescent-onset antisocial conduct problems in males[END_REF] may be lack of reliable information regarding time of onset of conduct problems in our subjects.

Another explanation may be that the relevance of time of onset is obscured by the complex pattern of associated problems in this heavily affected group. If so, the age at onset of CD may be a marker of especially problematic cases in the general population, which loses its predictive power in groups enriched by severe problems from onset, such as ours.

In line with studies of adolescents mental health service use [START_REF] Abram | Perceived barriers to mental health services among youths in detention[END_REF][START_REF] Merikangas | Prevalence and treatment of mental disorders among US children in the 2001-2004 NHANES[END_REF] the subjects in this study showed the same pattern in regards to contact with child and adolescent psychiatric health service characterized by an unbalance between need and use. Not surprisingly it was the adolescents with substance abuse problems, conduct disorder and mental retardation who had the least extensive mental health service use. Almost all of the subjects had showed, and in most cases still show, severe school related problems. In the light of the high prevalence of conduct disorder and AD/HD, this is of course not surprisingly, but also shows the urgent need for developing special educational programs for adolescents with these kinds of problems.

The current literature and our own studies demonstrate that there is a need to include care and treatment programs based on neuropsychiatric knowledge along with the traditional psychosocial support programs provided in special youth institutions. The low level of use of mental health services despite the extensive needs in this group clearly call for the establishment of a closer collaboration with the child and adolescent mental health services in order to ascertain that adolescents committed to institutions receive appropriate treatment.

F. AD only 4

1

  Care of Young Offenders Act SFS 1998:603,2 Care of Young Persons (special provisions) Act SFS 1990:52,3 Autism spectrum disorder (autistic disorder, Asperger's disorder, Pervasive developmental disorder not otherwise specified),4 Attention-deficit/hyperactivity disorder (AD/HD) (AD/HD Predominantly Inattentive Type, AD/HD Predominantly Hyperactive-Impulsive Type or Attention-deficit/hyperactivity disorder, combined type), 5 according to the Special Support and Service Act SFS 1993:387

Table 1 .

 1 Prevalences and constellations of diagnoses. Figures collected in an earlier study on similar groups in 2002 are included within square brackets for comparison[START_REF] Anckarsater | Autism spectrum disorders in institutionalized subjects[END_REF].

	All subjects	Boys	Girls	YPA 8 group	YOA 9 group	CD	CD late onset	CD early
	(n=100)	(n=92)	(n=8)	(n=78)	(n=22)	(n=77)	(n=66)	onset
		[81 %]	[19 %]					(n=11)

Table 2 .

 2 Crosstabulation of comorbidity between diagnoses. Each condition is represented by the same bold letter in the rows and columns, and the figures give number of subjects with overlapping conditions (percentage of overlap within brackets)

Table 3 .

 3 Distribution of full-scale IQ (FSIQ), Verbal IQ (VIQ), Performance IQ (PIQ), Verbal Comprehension Index (VCI), Perceptual Reasoning Index (PRI), Working Memory Index (WMI), and Processing Speed Index (PSI) among different diagnoses.

		Mean FSIQ	Mean VIQ	Mean PIQ	Mean VCI	Mean PRI	Mean WMI	Mean PSI
		(±SD)	(±SD)	(±SD)	(±SD)	(±SD)	(±SD)	(±SD)
	All subjects (n=100)	85.3 (14.3)	86.8 (13.8)	86.6 (15.4)	87.0 (14.0)	91.2 (16.3)	87.0 (12.6)	85.0 (12.5)
		(n=92)	(n=93)	(n=94)	(n=92)	(n=92)	(n=89)	(n=93)
	Boys (n=92)	85.7 (14.2)	87.6 (13.6)	86.5 (15.4)	87.7 (13.8)	91.3 (16.4)	87.6 (12.8)	85.1 (12.5)
		(n=84)	(n=85)	(n=86)	(n=84)	(n=84)	(n=81)	(n=85)
	Girls (n=8)	80.9 (16.1)	78.8 (14.5)	88.0 (15.7)	80.0 (14.7)	90.8 (15.6)	83.1 (9.9)	83.1 (13.3)
		(n=8)	(n=8)	(n=8)	(n=8)	(n=8)	(n=8)	(n=8)
	YOA 1 group (n=22)	92.1 (12.7)	92.0 (11.2)	93.6 (13.3)	90.4 (10.3)	100.0 (12.4)	95.9 (15.3)	85.6 (10.1)
		(n=17)	(n=18)	(n=19)	(n=17)	(n=17)	(n=14)	(n=19)
	YPA 2 group (n=78)	83.7 (14.3)	85.6 (14.1)	84.9 (15.4)	86.3 (14.7)	89.3 (16.5)	85.6 (11.4)	84.8 (13.1)
		(n=75)	(n=75)	(n=75)	(n=75)	(n=75)	(n=75)	(n=74)
	ASD any form 3 (n=17)	84.4 (15.6)	87.9 (15.9)	83.4 (14.5)	89.4 (16.1)	87.3 (17.2)	85.8 (9.7)	87.2 (8.8)
		(n=17)	(n=17)	(n=17)	(n=17)	(n=17)	(n=17)	(n=16)
	AD/HD any type 4 (n=47)	81.8 (13.6)	83.5 (13.8)	83.5 (14.8)	84.8 (14.0)	88.0 (16.2)	82.5 (10.3)	84.8 (13.0)
		(n=46)	(n=46)	(n=46)	(n=46)	(n=46)	(n=46)	(n=45)
	Drug abuse (n=55)	88.7 (12.0)	90.0 (12.8)	89.3 (12.3)	90.4 (13.0)	94.1 (12.5)	88.9 (12.2)	87.8 (10.9)

Autism spectrum disorder (autistic disorder, Asperger's disorder,

Pervasive developmental disorder not otherwise specified, PDD NOS),

Attentiondeficit/hyperactivity disorder (AD/HD

),[START_REF]Diagnostic and statistical manual of mental disorders : DSM-IV[END_REF] AD/HD, predominantly inattentive

type, 5 AD/HD, predominantly hyperactive-

impulsive type,[START_REF] Anckarsater | Autism spectrum disorders in institutionalized subjects[END_REF] AD/HD,

combined

type,[START_REF] Anckarsater | Prevalences and configurations of mental disorders among institutionalized adolescents[END_REF] Full-scale IQ,[START_REF] Andershed | Psychopathic traits in nonreferred youths: Initial test of a new assessment tool[END_REF] Care of Young Persons (special

provisions) Act SFS 1990:52,[START_REF] Beck | Beck Youth Inventories of Emotional & Social Impairment manual[END_REF] Care of Young

Offenders Act SFS 1998:603, 10 according to the Special Support and Service Act SFS 1993:387.

Acknowledgements

This work was supported by grants to HA from The National Board of Institutional Care (SiS), Sweden, to TN from the National Board of Forensic Medicine, and by support to OS from the Region of Västra Götaland. The following clinical psychologists, psychiatrists, and social workers are highly appreciated for their contributions to the collection of clinical data: Iraj Abedini, Agneta Andersson, Susanne Durehed, Anders Elverfors, Anders W Eriksson, Mats Gustafsson, Eva Hallberg, Sabina Johansson, Mats Johnson, Peter Sand, Dan Sjöholm, and Elisabet Åkerlund. Monika Montell and Agneta Brimse were excellent research secretaries.

Declaration of interest

None

AD/HD combined form 6 33 30 (

I. Substance abuse

L. Conduct

M. CD early onset

O. Borderline intellectual functioning

1 Autism spectrum disorders, 2 Pervasive developmental disorder not otherwise specified, 3 Attention-deficit/hyperactivity disorder, 4 Attentiondeficit/hyperactivity disorder, predominantly hyperactive-impulsive type, 5 Attention-deficit/hyperactivity disorder, predominantly inattentive type, 6 Attention-deficit/hyperactivity disorder, combined type