David Brie

Rémi Klotz

Sebastian Miron

Saïd Moussaoui
email: saïd.moussaoui@irccyn.ec-nantes.fr

Christian Mustin

Philippe Bécuwe

Stéphanie Grandemange

Rémy Klotz

Philippe Becuwe

Joint Analysis of Flow Cytometry Data and Fluorescence Spectra as a Non-Negative Array Factorization Problem

Keywords: Flow cytometry, fluorescence spectroscopy, mixture of multivariate probability density functions, non-negative block Candecomp/Parafac decomposition, non-negative matrix factorization, mitochondrial membrane potential, JC-1 probe

. Also a formal link between flow cytometry data and fluorescence spectra is provided allowing the joint processing of both data. To illustrate the effectiveness of the approach, a study of the T47D cell line mitochondrial membrane potential as a function of the CCCP decoupling agent concentration is performed. The main advantages of the method are : (i) the flow cytometry data compensation is no longer necessary, (ii) the cell sorting capacity of the method is significantly improved as compared to classical clustering methods. As a byproduct, it was possible to observe directly on the result of the processing, the dependence of the cell mitochondrial membrane potential with respect to the cell cycle phase. The proposed method is quite general provided that it is possible to design an experiment allowing to observe the response of cell populations to an environmental/chemical/biological parameter.

I. INTRODUCTION

Flow cytometry is an investigation technique widely used in biology and medicine for the characterization and quantification of the morphological, density and fluorescence properties of cells. A highly insightful approach, often used in biology and biomedical studies, consists in studying the evolution (response) of a cell population with respect to environmental/chemical/biological parameters (e.g. temperature, chemical drugs, gene expression). The recent technological progress of cytometers allow the design of complex multiparameter experiments yielding a large amount of data. The classical flow cytometry data analysis methods are no longer adapted to these data and hence there is a need for new algorithms that can efficiently retrieve the relevant information from this large amount of data [START_REF] Bashashati | A survey of flow cytometry data analysis methods[END_REF]. Classical flow cytometry data processing consists in a sequence of procedures mainly relying on the user expertise, and is, therefore, somewhat subjective. In general, the different operations are as follows:

1) Gating: consists in manually selecting a cell population of interest within the dot plot.

2) Compensation: aims at minimizing the influence of the spectral overlapping for different cell subpopulations. It consists in applying a linear transformation to the data, whose parameters are user-defined.

3) Clustering: is a cell sorting operation based on cell characteristics mentioned above. In most manufacturer provided softwares, this operation consists in manually defining quadrants on the dot plots.

Most of these methods require a user decision step which may strongly affect the relevance of the results. This is especially the case when the sub-population distributions strongly overlap, as often in practical applications. In the last years, more effective clustering algorithms were proposed mainly based on k-means related methods (e.g. [START_REF] Wilkins | Comparaison of five clustering algorithms to classify phytoplankton from flow cytometry data[END_REF]- [START_REF] Lo | Automated gating of flow cytometry data via robust model-based clustering[END_REF]) or Gaussian mixture models (e.g. [START_REF] Frelinger | Flow: Statistics, visualization and informatics for flow cytometry[END_REF], [START_REF] Pedreira | A multidimensional classification approach for the automated analysis of flow cytometry data[END_REF]- [START_REF] Boedigheimer | Mixture modeling approach to flow cytometry data[END_REF]). In [START_REF] Lo | Automated gating of flow cytometry data via robust model-based clustering[END_REF], Lo et al. propose the use of t-distributions mixtures instead of Gaussian mixtures, as they allow to better handle outliers, due to their heavier tail.

In [START_REF] Carter | Information preserving component analysis: Data projections for flow cytometry analysis[END_REF], [START_REF] Carter | Information-geometric dimensionality reduction[END_REF], an information geometry approach allows to define similarity measures between sets of cytometry data facilitating data interpretation by clinicians and resulting in a low dimension representation of the data. This approach is somewhat related to the one proposed here since it jointly considers multiple cytometry data sets.

The contributions of this work are twofold: i) we introduce a cell population sorting method based on a non-negative block tensor decomposition of the data histograms. The key point is that it is a fully multidimensional approach in which cell sorting is done according to the response of the different sub-populations to a parameter (the CCCP concentration in this paper). The model identifiability is also studied. The main advantages of our methods are: it is almost fully unsupervised (the only input parameter is the number of sub-populations sought in the data), it is non-parametric (there is no underlying parametric March 27, 2014 DRAFT probability density function) and it is not much affected by the overlapping of the sub-population distributions. ii) we prove that there is a relationship between flow cytometry and bulk spectroscopy data, and propose a joint processing approach of the two data modalities. This improves the separation accuracy and provides a more complete description of the analyzed cell populations. This joint modality data analysis can be regarded as a data fusion approach, as proposed in [START_REF] Miron | Joint processing of the parallel and crossed polarized Raman spectra and uniqueness in blind nonnegative source separation[END_REF] in the context of polarized Raman spectroscopy. This type of data fusion approach has gained a lot of interest recently in various domains of application (see e.g. [START_REF] Acar | Understanding data fusion within the framework of coupled matrix and tensor factorizations[END_REF], [START_REF] Sorensen | Coupled tensor decompositions for applications in array signal processing[END_REF]).

The remainder of this paper is organized as follows: in the next section, we introduce the notations and some general assumptions that are used throughout this paper; in section III, the data model for the proposed approach is derived and the link between spectroscopy and cytometry data is highlighted; in section IV we analyze the identifiability of the proposed model and propose a three-step algorithm for sorting the cell sub-populations; section V gives an illustration of applicability of the proposed approach to the study of the T47D cell line mitochondrial membrane potential as a function of the CCCP decoupling agent concentration. It includes the experiment description and the results of the proposed approach applied to different datasets; some conclusions are given in section VI.

II. PRELIMINARIES

Lowercase letters (x, y, . . . ) denote scalars, boldface lowercase (x, y, . . . ) are used for vectors, boldface capitals (X, Y, . . . ) symbolize matrices and tensors are written in boldface calligraphic capital letters (X , Y, . . . ). A tensor or N -way array (N ≥ 3) can be seen as the generalization of matrices to the multidimensional case. The number of dimensions N is called the order of the tensor. Thus, a vector is a first order tensor, a matrix is a second order tensor, etc. Consider a 3-way data array (third order tensor) X (I × J × K) admitting the following decomposition in a sum of K terms:

X = K k=1 a k • b k • c k (1)
where a k (I × 1), b k (J × 1) and c k (L × 1) are vectors and "•" denotes the outer product. The three dimensions of X are referred to as modes. The quantity a k • b k • c k represents a rank-1 tensor and the decomposition in (1) is commonly known as CANDECOMP/PARAFAC (CP) [START_REF] Carroll | Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart-Young" decomposition[END_REF], [START_REF] Harshman | Foundations of the PARAFAC procedure: Models and conditions for an 'explanatory' multimodal factor analysis[END_REF]. If K is the minimum number of rank-1 tensors that yield exactly X , than K is called the rank of the CP decomposition. An alternative notation for (1) is For simplicity, throughout this paper, the noise/error term in data model expressions will be ignored, which nothing detracts from the generality of the presented method. To introduce the theoretical data model in section III, continuous probability density functions (pdfs) should be employed. However, in practice, the recorded data are represented by histograms, implying discretized versions of these pdfs. For the clarity of the presentation, we will use lowercase letters to denote the continuous pdfs and boldface lowercase for their discretized versions. The length of the vectors representing the discretized pdfs (i.e.

X = [[A, B, C]], (2)
the number of bins) will not be explicitly mentioned, unless it is crucial for the comprehension of the presentation. Also, for simplification, the distinction continuous/discretized pdf will not always be made in the text, but can be easily deduced from the context. having, for each dimension, a number of points equal to the number of bins on which the histogram was calculated. The number of bins should be large enough to have a sufficient resolution (i.e the ability to distinguish different pdfs) but not too large to have a reasonable noise (error). For all experiments, the number of bins was fixed to 50 along each dimension.

III. DATA MODEL

A. The probability density function of N-dimensional flow cytometry data

B. The bilinear model of sequence of N-D pdfs

Let us consider a cell population composed of K different sub-populations. A sub-population is defined as a set of cells exhibiting identical/similar behaviors with respect to the variation of a physical parameter.

We represent the N -D data points in the analyzed sample by the pdf p(x) of the measurement vector x.

This pdf is expressed as a mixture of K density functions f k , corresponding to the K sub-populations: with K k=1 α k = 1. Assume that we study the response of this cell population to the variation of some physical parameter, denoted s hereafter. For each value of s, a flow cytometry dataset can be recorded, resulting in a sequence of flow cytometry data matrices. Hence, each data matrix obtained for a given physical condition yields a pdf denoted by p(x, s), that can be modeled as:

P (x m = x) = p(x) = K k=1 α k f k (x) (3)
p(x, s) = K k=1 α k (s)f k (x). (4)
The sequence of N -D histograms, obtained for different values of s, can thus be gathered into a (N + 1)-D array (tensor) denoted by P. By unfolding this tensor along the dimensions corresponding to the N different wavelengths, we obtain a matrix P that admits the following bilinear factorization:

P = AF T . (5)
In [START_REF] Zeng | Feature-guided clustering of multi-dimensional flow cytometry datasets[END_REF], P has a number of rows equal to the number of values of the parameter s; the columns of A, symbolized by a k (k = 1, . . . , K), contain the mixing coefficients of the K sub-populations for the different values of s; the columns of F are the "unfolded" N -D density functions f k of the K sub-populations.

C. The Block-CANDECOMP/PARAFAC model of sequence of N -D pdfs

Assuming the independence of each coordinate of

x = [x 1 , • • • , x N ] T , the multivariate pdf f k (x) can
be factorized as a product of N univariate pdfs:

f k (x) = f 1 k (x 1 ) • f 2 k (x 2 ) • • • f N k (x N ).
Thus the data array can be written as a CP model of order N + 1:

P = K k=1 a k • f k,1 • • • • • f k,N . (6)
March 27, 2014 DRAFT Equation ( 6) clearly expresses a N + 1 CP model of rank K which can be alternatively written as:

P = [[A, F 1 , • • • , F N ]], (7)
where A = [a 1 , . . . , a K ] and F n = [f 1,n , . . . , f K,n ], with n = 1, . . . , N . The link between models (7) and

(5) is obtained by unfolding P into a matrix, according to:

P = A(F 1 ⊙ • • • ⊙ F N ) T
, where "⊙" stands for the Khatri-Rao product. Thus, we have [START_REF]Decompositions of a higher-order tensor in block terms -Part II: Definitions and uniqueness[END_REF]. By doing so, it is possible to consider the more general case of non-separable pdfs. In fact, this is nothing but performing a low rank approximation of the (discretized) pdfs. Hence, the (N + 1)-D data array can be written as:

F = F 1 ⊙ • • • ⊙ F N .
P = K k=1 a k • E k , (8)
where the rank L k matrices E k can be decomposed as:

E k = Lk l=1 f l k,1 • f l k,2 = Lk l=1 f l k,1 f l k,2 T . (9)
A graphical illustration of this model is given on figure 2.

P E 1 E K a 1 a K f LK K,1 f LK K,2 T f 1 K,1 f 1 K,2 T f 1 1,1 f 1 1,2 T f L1 1,1 f L1 1,2 T Fig. 2: Graphical illustration of the rank-(L k , L k , 1) BCP model of P.
The BCP model can be seen as a CP model in which some of the loading vectors (columns) of the matrix A are collinear. The model in (8) can be easily generalized to higher dimensional (N > 2) data.

The higher-dimensional case does not yield more complicated data processing situations, since it is known that higher-order CP models require less restrictive identifiability conditions. For example, some 4-order CP models, with collinear loading in at most three modes, are provably identifiable [START_REF] Brie | An uniqueness condition for the 4-way CANDECOMP/PARAFAC model with collinear loadings in three modes[END_REF].

March 27, 2014 DRAFT

D. Joint analysis of fluorescence cytometry and bulk spectroscopy data

The question investigated in this subsection is the following: is there a formal link between the fluorescence spectra and the cytometry data for a given cell population? We are going to show that the answer is actually yes, allowing us to propose a joint analysis of the two types (modalities) of data.

A bulk fluorescence spectrum corresponds to the fluorescence measured on a large number of cells. In that respect, it can be seen as an average of all the individual cell fluorescence spectra. Thus, considering the variation of the same physical parameter s, the measured spectra m(s) can be written as:

m(s) = x p(x, s)dx. (10)
The vector x contains x as a "sub-vector", and therefore we can write 1 :

p(x, s) = K k=1 α k (s)f k (x), (11)
where the mixing coefficients α k (s) are the same as in eq. [START_REF] Boedigheimer | Mixture modeling approach to flow cytometry data[END_REF]. By replacing [START_REF] Carter | Information preserving component analysis: Data projections for flow cytometry analysis[END_REF] in [START_REF] Boedigheimer | Mixture modeling approach to flow cytometry data[END_REF], we obtain:

m(s) = K k=1 α k (s) xf k (x)dx fk = K k=1 α k (s) fk . (12)
If we regroup the set of spectra (after normalization to unit energy) recorded for the different values of s on the rows of a matrix M, the following mixture model can be written:

M = A FT . (13)
In [START_REF] Miron | Joint processing of the parallel and crossed polarized Raman spectra and uniqueness in blind nonnegative source separation[END_REF], the columns of F are the vectors fk and correspond to spectra characterizing the (averaged) spectral response of the K cell sub-populations and the mixing matrix A is exactly the same as in the cytometry data model [START_REF] Zeng | Feature-guided clustering of multi-dimensional flow cytometry datasets[END_REF]. We are now able to propose the joint model by gathering the two models [START_REF] Zeng | Feature-guided clustering of multi-dimensional flow cytometry datasets[END_REF] and ( 13) into a single one. This is possible because of the common mixing matrix A. We follow an approach quite similar to the one proposed in [START_REF] Miron | Joint processing of the parallel and crossed polarized Raman spectra and uniqueness in blind nonnegative source separation[END_REF] which consists in concatenating the data matrices P and M according to:

[P M] = A   F F   T . ( 14
)
The joint use of a sequence of fluorescence flow cytometry and spectroscopy data provides a very complete description of cell population. It yields a decomposition of the whole population into homogeneous subpopulations characterized by their common:

1 In reality, the data vector x is obtained by integrating the emitted light over a wavelength interval ∆λ around the different "colors" used by the cytometer. However, for small values of ∆λ, eq. ( 11) is a good approximation of p(x, s).

IV. MODEL IDENTIFIABILITY AND DATA PROCESSING

The problem at hand can be embedded in the general framework of non-negative approximation of non-negative tensors using multilinear decompositions. This is still an open problem but a milestone was reached with the work of Lim and Comon [START_REF] Lim | Nonnegative approximations of nonnegative tensors[END_REF] where it is proved that non-negativity ensures the well posedness of the non-negative tensor approximation. As mentioned in [START_REF] Lim | Nonnegative approximations of nonnegative tensors[END_REF], this can actually be associated to the sparse naive Bayes probabilistic model for pdf [START_REF] Lowd | Naive bayes models for probability estimation[END_REF], since the underlying probabilistic model is a mixture of densities having independent variables. Here we go one step further since, by using the BCP, we can relax the independence assumption. However, the question of the validity of the non-negative BCP as an approximation tool is an open problem which would deserve to be studied. We did not pay further attention to this point, but as a first attempt to illustrate the regularization property of the nonnegativity, we show that rank (L k , L k , 1) exact non-negative BCP decomposition can be unique without any additional assumption. This is not the case for general BCP unless some other constraints (such as orthogonality) are enforced. In section III, it was shown that NMF was involved in all the different models considered. In that respect, uniqueness of the NMF plays a central role and in the sequel we recall some results on the NMF uniqueness and we give a sufficient condition which allows to check directly on the data if a NMF is likely to be unique. We then use these results to study the uniqueness of the non-negative BCP and we give some practical consequences of theses results. To conclude this section, we present the three different steps of the data processing algorithm.

A. Identifiability of the non-negative bilinear model

In this section, we address the identifiability of the bilinear model which arises in different contexts in this work, in particular for models ( 5) and ( 13). Such non-negative bilinear models are also involved in BCP decomposition, as we will show in section IV-B. Assume that a non-negative matrix W admits an exact bilinear model representation:

W = HG T . ( 15
)
Depending on the considered case, the matrix W may represent different quantities: matrix P for the model ( 5), matrix M for (5), matrix [P M] for model [START_REF] Acar | Understanding data fusion within the framework of coupled matrix and tensor factorizations[END_REF], and matrix E k for the BCP model [START_REF] Pedreira | A multidimensional classification approach for the automated analysis of flow cytometry data[END_REF].

March 27, 2014 DRAFT It is well known, that in general the bilinear decomposition [START_REF] Sorensen | Coupled tensor decompositions for applications in array signal processing[END_REF] does not admit a unique solution since for any non-singular matrix T it is possible to write:

W = HT T -1 G T = H GT (16)
which is another admissible solution. Regularization is thus needed to obtain an unique decomposition.

Among the possible additional constraints which can be considered, we focus on the non-negativity assumption, leading to the Non-negative Matrix Factorization (NMF) problem [START_REF] Lee | Learning the parts of objects by non-negative matrix factorization[END_REF]. In order to discuss the identifiability of the NMF model ( 15), the notion of simplicial cone needs to be introduced.

Definition 1 (Simplicial cone):

The simplicial cone generated by a family of vectors {g n } N n=1 is

C({g n }) = w : w = n α n g n , α n > 0 .
The order of a simplicial cone is the dimension of the subspace span({g n } N n=1 ). Based on the definition above, a necessary and sufficient condition for NMF identifiability has been provided by Chen in [START_REF] Chen | Nonnegative rank factorisation of nonnegative matrices[END_REF]: Theorem 1 (Necessary and sufficient uniqueness condition): Denoting K the convex hull of the data matrix W, the decomposition of W according to

W = HG T , H ≥ 0, G ≥ 0 is unique if and only if the simplicial cone C(G), such as K ⊂ C(G), is unique 2 .
Clearly, Theorem 1 does not provide any numerical conditions to check if a NMF is unique or not. This motivated the work of [START_REF] Donoho | When does non-negative matrix factorization give a correct decomposition into parts?[END_REF] from which it appears that uniqueness relies on the number of zero entries in both matrices H and G. New results can also be found in the very recent paper [START_REF] Huang | Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition[END_REF]. Unfortunately, even if these approaches do give numerical rules to check if a NMF model is identifiable, they do not provide any effective mean to check directly from the data if the NMF is unique. This is the goal of the following proposition and corollary. However, before going any further, it is necessary to introduce the notion of monomial matrix and and some related properties.

Definition 2 (Monomial matrix):

A positive matrix T of dimension (p, p) is called a monomial matrix if every row and every column of this matrix contains exactly one non-null element [START_REF] Van Den Hof | Realization of positive linear systems[END_REF], that is ∀i = 1, . . . , p, ∃k; t ik > 0 and t jk = 0 ∀j = i.

(

) 17
Property 1 (see [START_REF] Berman | Nonnegative matrices in the Mathematical Sciences[END_REF]): An arbitrary positive square matrix T has a positive inverse matrix if and only if T is a monomial matrix. Then T -1 is also monomial.

Property 2 (see [START_REF] Berman | Nonnegative matrices in the Mathematical Sciences[END_REF]): Each monomial matrix T may be decomposed as T = ∆U, where ∆ is a positive diagonal matrix and U is a permutation matrix, that is, a monomial matrix whose non-zero elements are equal to 1. Such a transformation, when applied to G yields the scaling and ordering indeterminacies.

We are now ready to formulate a sufficient condition for the uniqueness of NMF, from which we derive another sufficient condition, which can be applied directly to the data.

Proposition 1 (Sufficient Uniqueness Condition):

The decomposition of W into H and G according to

W = H G T , with G 0, H 0, (18)
is unique if the following conditions are satisfied:

(B1) There exists a submatrix of H of dimension (K, K) which is monomial.

(B2) There exists a submatrix of G of dimension (K, K) which is monomial.

Proof : See the appendix.

From this result, we may immediately deduce the following corollary which gives a sufficient condition on W to admit a unique non-negative factorization.

Corollary 1: The decomposition of W into H and G according to

W = H G T with H 0, G 0, (19)
is unique if the following condition is satisfied:

(C1) After line and column permutations, the matrix of W can be written as:

W =      W 11 . . . W 12 . . . . . . . . . W 21 . . . W 22     
, where W 11 is a non-singular diagonal matrix of dimension (K, K).

The proof of this corollary is trivial using the fact that a non-negative monomial matrix can be factorized only as a product of two other non-negative monomial matrices of the same size.

B. Identifiability of the non-negative BCP model

Before addressing the BCP model identifiability, some uniqueness results of the CP decomposition must be presented. A key notion to the uniqueness of the CP decomposition is due to Kruskal [START_REF]Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics[END_REF], and relies on the concept of "Kruskal-rank" or simply k-rank. The k-rank of an I × K matrix A, denoted by March 27, 2014 DRAFT k A , is the maximum value l ∈ N such that every l columns of A are linearly independent. By definition, the k-rank of a matrix is less than or equal to its classical rank. Kruskal proved that [START_REF]Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics[END_REF]

k A + k B + k C ≥ 2K + 2 (20)
is a sufficient condition for ensuring the uniqueness of the CP decomposition in [START_REF] Lathauwer | Decompositions of a higher-order tensor in block terms -Part I: Lemmas for partitioned matrices[END_REF]. Furthermore, it becomes a necessary and sufficient condition in the case K = 2 or 3.

This condition no longer holds when one matrix (say A) has a k-rank equal to 1, that is, when the matrix A has collinear columns. Unfortunately, this is what happens for the BCP model. In this case, we have to resort to the notion of partial uniqueness which means that only "part of the model" can be unique (see [START_REF] Berge | Partial uniqueness in CANDECOMP/PARAFAC[END_REF], [START_REF] Guo | Uni-mode and partial uniqueness conditions for CANDECOMP/PARAFAC of three-way arrays with linearly dependent loadings[END_REF] for details). Restricting our attention to the BCP decomposition at hand, and based on the results of [START_REF] Guo | Uni-mode and partial uniqueness conditions for CANDECOMP/PARAFAC of three-way arrays with linearly dependent loadings[END_REF], the identifiability of A is ensured if:

r A + k B + k C ≥ 2K + 2, (21)
where r A is the classical rank of matrix A. In particular, in the case considered, if B and C are fullcolumn rank matrices, the identifiability of A requires only r A ≥ 2. Some other identifiability results

for A can be found in [START_REF] Guo | Uni-mode and partial uniqueness conditions for CANDECOMP/PARAFAC of three-way arrays with linearly dependent loadings[END_REF].

The key point is that, provided A can be uniquely estimated from the data, theorem 3.1 of [START_REF] Guo | Uni-mode and partial uniqueness conditions for CANDECOMP/PARAFAC of three-way arrays with linearly dependent loadings[END_REF] ensures that the identifiability of the entire CP model can be assessed by checking the identifiability of several independent lower-rank CP models. Coming back to the BCP problem (8) at hand, this means that the uniqueness of the decomposition can be assessed by investigating the uniqueness of each bilinear sub-problem [START_REF] Chan | Statistical mixture modeling for cell subtype identification in flow cytometry[END_REF]. In general, uniqueness cannot be guaranteed and the bilinear problem is unique up to rotational ambiguities. This is the essential uniqueness of BCP, as introduced by De Lathauwer in [START_REF]Decompositions of a higher-order tensor in block terms -Part II: Definitions and uniqueness[END_REF].

However, in the particular case of non-negative pdfs, the results of section 3 can be used. This means that the BCP decomposition ( 8) is unique if the uniqueness of A is ensured and the non-negative bilinear factorizations [START_REF] Chan | Statistical mixture modeling for cell subtype identification in flow cytometry[END_REF] are unique. This result clearly shows the interest of non-negativity for the uniqueness of CP-like decompositions.

C. From theory to practice

The theoretical identifiability results presented in the previous subsections, all involve the identifiability of the NMF model. However, they may be a bit difficult to interpret for users who are not familiar with matrix factorization. In this subsection, we first give some graphical illustrations corresponding to practical situations for which the NMF identifiability is ensured or not. The figure 3 gives 4 examples of data March 27, 2014 DRAFT matrices consisting in the superposition of 3 rank-1 non-negative matrices. The left data matrix of the top row satisfies the condition of corollary 1, meaning that its non-negative rank-3 decomposition is unique.

That on the right of the top row gives an example where Donoho's condition [START_REF] Donoho | When does non-negative matrix factorization give a correct decomposition into parts?[END_REF] is fulfilled; thus, its non-negative rank-3 decomposition is also unique. On the contrary, the two bottom row matrices do not admit a unique solution since the necessary condition of [START_REF] Huang | Non-negative matrix factorization revisited: Uniqueness and algorithm for symmetric decomposition[END_REF] is not fulfilled (see also [START_REF] Moussaoui | Non-negative source separation: Range of admissible solutions and conditions for the uniqueness of the solution[END_REF]). This discussion on the identifiability of the BCP model naturally brings up the following question: is the uniqueness of the BCP decomposition really important in practical applications? This is actually a question which deserves to be discussed in detail since, for our problem of approximating the 2-D pdfs represented by the matrices E k , the rotational ambiguities do not matter. Indeed, regardless of the rotational ambiguities, the reconstructed E k remains the same. Nevertheless, if the objective is the biological interpretation of the results, then uniqueness does matter because it turns out that pdfs having complex shapes result from mixtures of sub-populations behaving in a similar way. To illustrate the idea, let us consider a simple analogy: consider a population composed of children and adults having to run away from danger and observed with a device measuring the height and the weight of each individual.

W = W = W = W =
Roughly, 3 types of behavior are expected to be observed: a fast running sub-population (the adults), a slow running sub-population (children) and a last sub-population composed of adults holding children's hands and therefore running faster than children but slower than adults. From a decomposition point of view this is a single sub-population since they are all moving at the same speed, but if the goal is to interpret the experiment, the uniqueness of the BCP decomposition would allow the population of adults and children running together to be decomposed into adults only and children only, without any ambiguity. March 27, 2014 DRAFT

D. Algorithms for data processing

The data processing method proposed in this paper consists of three steps. The first two steps deal with the processing of the flow cytometry data while the third addresses the coupling (fusion) of the flow cytometry data with the cell fluorescence spectra.

1) Estimation of the probability density functions: An estimation of the flow cytometry data pdf is obtained by computing the N -D histograms of the data. The developed matlab function requires to define the number of bins (one for each dimension) on which the histograms are calculated.

2) Non-negative BCP decomposition of the data: For the needs of this step, the non-negative BCP algorithm developed in the tensorlab toolbox [START_REF] Sorber | Tensorlab v1.0[END_REF] can be used. We also developed a procedure to estimate the ranks L k and K of the decomposition. It first consists in performing a non-negative high-rank CP decomposition of the data. Thanks to the partial uniqueness properties of the CP model [START_REF] Guo | Uni-mode and partial uniqueness conditions for CANDECOMP/PARAFAC of three-way arrays with linearly dependent loadings[END_REF], the matrix containing the collinear loadings is ensured to be unique. In practice, because of the noise/error terms, the estimated loading may not be strictly collinear. The "most" collinear loadings, i.e. those presenting a correlation greater than a specified threshold (typically 0.9), are collapsed into a single loading by an averaging procedure. The K resulting averaged loadings are gathered into a full column rank matrix A.

The corresponding loadings of the other two modes are gathered to form matrices E k , k = 1, . . . , K. This results in a non-negative BCP decomposition which is used as initial solution for the non-negative BCP algorithm of [START_REF] Sorber | Tensorlab v1.0[END_REF]. Once the non-negative BCP decomposition is achieved, the corresponding N -D pdfs are normalized to have unit sum and the normalization factor is then transferred on the corresponding loadings, representing the responses of the K sub-populations to the physical parameter.

3) Coupling of the flow cytometry data with fluorescence spectra:

There are at least two ways to couple the two datasets. A first approach is to estimate first the mixing matrix A using only the cytometry fluorescence data. Once matrix A is estimated, the source spectra are obtained from the bulk spectroscopy data by a least-squares procedure under non-negativity constraints. A second approach is to actually combine the data into a single data matrix following model ( 14) and then to decompose the large data matrix using a non-negative factorization algorithm (e.g. the Bayesian Positive Source Separation algorithm developed in [START_REF] Moussaoui | Separation of non-negative mixture of non-negative sources using a Bayesian approach and MCMC sampling[END_REF]). In the next section, only the results corresponding to the first approach of step 3 are presented. In fact, no significant difference was observed between the two different approaches.

The reason is that, in the considered example, the uniqueness of the corresponding rank-2 non-negative matrix factorization is provably unique. The interested reader is referred to [START_REF] Moussaoui | Non-negative source separation: Range of admissible solutions and conditions for the uniqueness of the solution[END_REF], where a necessary and sufficient condition for having the uniqueness of the rank-2 NMF is provided.

March 27, 2014 DRAFT

The next section illustrates the effectiveness of the proposed approach on real flow cytometry and spectroscopy data. These data result from an experiment aiming at studying the response of the mitochondrial membrane potential of a particular cell line to a decoupling agent.

V. ANALYZING MITOCHONDRIAL MEMBRANE POTENTIAL WITH JC-1

A. Mitochondrial membrane potential

Mitochondrial membrane potential (∆Ψm) is an important indicator of the mitochondrial membrane integrity and mitochondrial efficacy through the coupling between oxidative phosphorylation and ATP synthesis. Indeed, ∆Ψm is an indicator of cell viability since a drastic decrease of this potential is associated with cytochrome c release during apoptosis [START_REF] Zamzami | Mitochondrial implication in accidental and programmed cell death: apoptosis and necrosis[END_REF], [START_REF] Cossarizza | Analysis of mitochondria during cell death[END_REF]. The membrane permeant dye JC-

1 (5 ′ , 6, 6 ′ -tetrachloro-1, 1 ′ , 3, 3 ′ -tetraethylbenzimidazolylcarbocyanine iodide
) is largely used to monitor this mitochondrial parameter [START_REF] Reers | J-aggregate formation of a carbocyanine as a quantitative fluorescent indicator of membrane potential[END_REF]- [START_REF] Mathur | Evaluation of fluorescent dyes for the detection of mitochondrial membrane potential changes in cultured cardiomyocytes[END_REF]. This lipophilic cationic dye enters cells and accumulates in mitochondria as monomers or oligomers (J-aggregates) that exhibit two different emission spectra often referred to as green and red respectively 3 . JC-1 monomers are associated with depolarized mitochondria whereas J-aggregates are formed when ∆Ψm is high. Thus the mitochondrial membrane potential can be estimated by following the red/green ratio of the JC-1 dye to distinguish between mitochondria with high or low ∆Ψm. Qualitative and quantitative analysis of ∆Ψm is usually performed by flow cytometry after an excitation of the probe at 488nm with an argon laser. After excitation, JC-1 monomers fluorescence is measured in the FL1 channel 515 -545nm and the JC-1 aggregates in the FL2 channel 564 -606nm.

Because of the overlap of the two emission spectra, compensation is needed, around 20-30% of the green signal (FL1) has to be subtracted from the red signal (FL2). This compensation value needs to be well calibrated in each experiment. Recently, Perelman et al. [START_REF] Perelman | Jc-1: alternative excitation wavelengths facilitate mitochondrial membrane potential cytometry[END_REF] have demonstrated that a new generation cytometer, equipped with another excitation laser, in particular at 405nm, can be used for JC-1 measurements. These new lasers considerably reduce the overlap of the monomers fluorescence (green) with the J-aggregates fluorescence (red). These findings simplify the procedure since the fluorescence compensation can be avoided. However, it is worth mentioning that, regardless of the laser excitation, there will be situations where the peak overlapping cannot be completely avoided. It is also clear that changing the laser excitation is not always possible.

The present experimental study aims at validating the proposed data processing approach on real multicolor flow cytometry data corresponding to the response of a cell line to a widely used and well 3 We will see, in the sequel, that this has to be understood as "mostly green" or "mostly red" since the emission spectra are not purely green or red.

March 27, 2014 DRAFT understood uncoupling agent (carbonyl cyanide p-chlorophenylhydrazone -CCCP, Sigma-Aldrich). In particular, we address the following questions:

1) What is the gain of coupling the flow cytometry together with the fluorescence spectroscopy?: This is an original point since, so far, no previous work proposed to couple the two techniques.

2) Is it really necessary to perform the data compensation?: We believe that this is a very important practical issue since, from our own experience, depending on the way the compensation is performed, the analysis results may be strongly affected. Avoiding this pre-processing step may certainly represent a major step in the development of quantitative analysis in flow cytometry.

3) Does the proposed approach bring new insights into the analysis and the understanding of cytometry data?: Here, the stake is to evaluate, from a biological perspective the benefit of an accurate separation of the contributions of the different cell sub-populations.

B. Cell culture and data acquisition

Human ductal breast epithelial tumor cell line, T47D (from ATCC) was grown in RPMI 1640 medium supplemented with 10% fetal calf serum, 2mM L-glutamine and 5 µg/ml Gentamicin at 37 • C in a humidified atmosphere containing 5% CO 2 . The mitochondrial membrane potential-sensitive dye JC-1 was prepared as a stock solution in dimethyl sulfoxide (DMSO, Sigma-Aldrich) and stored at -20 • C . Before use, JC-1 stock solution was diluted 100× in assay buffer (delivered by manufacturer). Cells were stained following the manufacturer specifications. Briefly, 1 ml of each cell suspension was centrifuged at 400g for 5 minutes at RT. The pellets were resuspended in 0.5ml of JC-1 freshly diluted and containing various concentrations of carbonyl cyanide p-chlorophenylhydrazone (CCCP, Sigma-Aldrich). CCCP is an ionophore used to uncouple oxidative phosphorylation in mitochondria. It causes a mitochondrial proton leak, leading to a depolarization of the mitochondrial membrane. Thus, it is frequently used as a negative control in mitochondrial membrane potential measurements by flow cytometry. The concentration of CCCP for which it is well accepted that the cells are fully depolarized, ranges between 50 and 100 µM . Thus, we chose 6 CCCP concentrations varying between 0 and 100 µM ([CCCP ] = 0, 5, 10, 25, 50, 100) to ensure that the whole CCCP response range is observed.

The samples were incubated for 15 minutes at 37 • C in CO 2 incubator. At the end of the incubation period, each tube was washed twice with assay buffer and cells were resuspended in 0.3 ml of culture medium. Half of the cells were analyzed by flow cytometry (BD FACSCalibur) and the rest by fluorescence plate reader (Safas). Figure 4 gives an example of experimental data. This is a sequence of six cytometry data sets, each one corresponding to a given CCCP concentration.

March 27, 2014 DRAFT

The first step of the processing consists in estimating the 2-D histograms. For each dimension, the number of bins is fixed to 50 resulting in a 50 × 50 data matrix. Then the six matrices are gathered into a 3-way array of dimension (50×50×6). For this data set, the compensation was fixed to have a maximum separation along the horizontal axis. Figure 5 shows the corresponding sequence of fluorescence spectra.

All the spectra are normalized to have a unit energy.

C. Results and discussion

Figure 6 shows the results of the non-negative BCP decomposition corresponding to the dataset of figure 4 (see section III-C) and figure 7 shows the corresponding spectral source estimated by the joint March 27, 2014 DRAFT analysis (see section III-D).

The number of block-component is expected to be equal to 2: a highly polarized cell sub-population whose response to CCCP is expected to decrease and a depolarized cell sub-population whose response is expected to increase with the CCCP concentration. The experimental parameters L k of BCP were determined after successive trials. The first block rank is (3, 3, 1) while the second block rank is (2, 2, 1).

The responses of the two sub-populations are in very good agreement with what was expected.

From a biological point of view, the top left-hand side plot on figure 6 represents the distribution of cells with a low mitochondrial membrane potential (referred to as "green" fluorescence) while the top right-hand side figure corresponds to the cells with high mitochondrial membrane potential (referred to as "red" fluorescence). The associated responses show that the low mitochondrial membrane potential subpopulation increases with the concentration of CCCP while the high mitochondrial membrane potential sub-population decreases. In particular, it can be observed that the full cell population depolarization is reached after a concentration of 50 µM which is corresponding to the value generally accepted by practitioners.

As one can see on figure 7 (a), the estimated spectra for the two sub-populations are highly correlated.

From a signal processing point of view, separating these two spectral signatures using only the bulk spectroscopy data on figure 5, is a very difficult and challenging problem. The coupling of the two data modalities makes this separation possible without imposing additional constraints on the source parameters. From the biological point of view, the estimated spectra provide interesting insights into the understanding of the average behavior of the two sub-populations. Each sub-population, treated or not with CCCP, exhibits green and red fluorescence corresponding to JC-1 monomers and JC-1 aggregate, respectively. Cell sub-population with high (respectively low) mitochondrial membrane potential is more red than green (respectively more green than red). The estimated spectra also show that, for the two sub-populations, there are no other discriminant wavelengths in their emitted fluorescence light. This could represent an interesting tool for the efficient choice of the adequate wavelengths in flow cytometry.

The joint analysis of cytometry and spectroscopy data yield a complete characterization of the two types When using standard cytometry data analysis tools (as provided by the manufacturer), the red and green fluorescence analysis of JC-1 always require a user defined compensation procedure which may strongly affect the quantitative interpretation of the data. As mentioned in [START_REF] Rajwa | Just compensation?[END_REF], compensation is in fact a linear transformation of the data and therefore, in our experiments, it is not supposed to affect the response to CCCP. In the following experiments, the data have been acquired without and with compensation.

Having a closer look at the pdfs of the cell sub-populations revealed that they were not unimodal, which was rather surprising from a biological point of view, since a single cell line is considered. Our conjecture was that the cell asynchronicity was responsible of this multimodal distribution, each mode corresponding to a particular mitochondrial membrane potential associated with a specific cell cycle phase. Indeed, the dependance of the mitochondrial membrane potential of the cell cycle phase was already mentioned in To synchronize cells at the G1 phase of the cell cycle, cells were exposed to 2 mM thymidine (Sigma-Aldrich) for 48h. Then, synchronized cells were collected, counted (TC10 Automated Cell Counter, Bio-Rad) and adjusted to the density of 300 000 cells/ml for analysis of mitochondrial membrane potential. The results of the data processing are shown in figure 10 (a,b,c,d). On the one hand, comparing figure 10 (a) and 10(c) as well as 10 (b) and 10 (d), it can be observed that the compensation does not affect much the shape of the response to CCCP. In other words, by using the proposed approach, compensation is no longer necessary. On the other hand, it appears that the synchronization modifies the shape of the cell population distributions. Indeed, the not-synchronized low mitochondrial membrane potential subpopulation includes two main modes, one centered on (350, 300) and a second centered on (450, 400).

This second mode significantly decreases after synchronization in G1 phase, resulting in a shift toward the low value of the red and thus increasing the relative importance of the green. This is much more visible on the distribution of the high mitochondrial membrane potential sub-population. This can be attributed to the fact that cells in G1 phase have a lower mitochondrial membrane potential that those in S and G2 phases which is in accordance with the literature. Also, looking at the response to CCCP, it seems to indicate that dynamic of the responses of the low and high mitochondrial membrane potential sub-population is stronger after synchronization.

VI. CONCLUSIONS

In this paper, we proposed a novel flow cytometry data analysis methodology, based on a non-negative block-Candecomp/Parafac model of the data, and highlighted the link between bulk spectroscopy and flow cytometry data. A sufficient condition allowing to guarantee the uniqueness of data decomposition was also derived. The joint processing of the two data modalities results in an effective tool that reveals March 27, 2014 DRAFT The main underlying idea is to exploit the different behavior of cell sub-populations with respect to a physical parameter (the CCCP concentration in this paper). Unlike the classically employed clustering methods, strongly relying on the user expertise, our method requires only the knowledge of the number of sub-populations to be extracted from the data, and is very little sensitive to the overlapping of cell sub-population distributions. Thus, similarly to Perelman et al. [START_REF] Perelman | Jc-1: alternative excitation wavelengths facilitate mitochondrial membrane potential cytometry[END_REF], our method also yields an effective way to avoid compensation, but without requiring the use of a different laser excitation adapted to the March 27, 2014 DRAFT mitochondrial membrane potential measurement with JC-1 probe. The ability of the method to separate overlapping densities has revealed two sub-populations within a single cell line in both high and low mitochondrial membrane potential cell populations. This unexpected but insightful side result has been attributed to cells being in different cell cycle phases, having slightly different mitochondrial membrane potentials. The price to pay for these interesting features is an increased complexity of the experiments generating the data. However, the joint data processing and experiment design is a promising research direction in which biologists and data analysts may develop very fruitful collaborations.

APPENDIX PROOF OF PROPOSITION1

Suppose that conditions (B1) and (B2) are satisfied. After a possible permutation of its columns, the matrix G can be rewritten as: Similarly, after a possible permutation of its rows, the matrix H can be re-written as: from which it turns out that TG T 0 and HT -1 0 if and only if T > 0 and T -1 > 0. From property 1, this is equivalent to say that T is a monomial matrix and according to property 2, the solution is unique up to scaling and ordering indeterminacies.

G =     
H =           

March 27 ,

 27 2014 DRAFT where A = [a 1 . . . a K ], B = [b 1 . . . b K ] and C = [c 1 . . . c K ] denote the component/loading matrices. All these notions, defined here for the 3-way case, generalize straightforwardly to N -way arrays (N > 3).

  Consider N-dimensional flow cytometry data. Each of the analyzed cells yields a length N vector measuring the amplitudes at N different wavelength values of the emitted fluorescence light. The set of measurements collected on a population of M different cells can be gathered in a N × M matrix X = [x 1 • • • x M ], where x m = [x m (1), • • • , x m (N )] T and m = 1, . . . , M . As illustrated in figure 1, this data matrix X can be characterized by its N -variate pdf denoted by p(x) = P (x m = x). An estimation of this pdf can be obtained by calculating the histogram of data which is a N-dimensional (N-D) array

X matrix 2 Fig. 1 :

 21 Fig. 1: Transforming the (N × M ) matrix X into N-D histograms; illustrations for N = 2 and 3.

Fig. 3 :

 3 Fig. 3: Four data matrices admitting a NMF decomposition. The two matrices at the top of the figure admit a unique decomposition. Nothing can be said by examining the bottom line matrices.

Fig. 4 :Fig. 5 :

 45 Fig. 4: A sequence of flow cytometry data showing the response of T47D cells to CCCP. The upper line figures correspond to the data and the lower line figures are the corresponding 2-D histograms.

Fig. 6 :

 6 Fig. 6: Non-negative BCP decomposition results of flow cytometry dataset corresponding to the first experiment

Fig. 7 :

 7 Fig. 7: Estimation of the pure fluorescence spectra (a) corresponding to the first experiment. The mixing coefficient (b) are those obtained from the non-negative BCP decomposition.

Fig. 8 : 6 MarchFig. 9 :

 869 Fig. 8: Non-negative BCP decomposition of the two datasets obtained by repeating the same experiment as for the dataset on figure 6

Fig. 10 :

 10 Fig. 10: Non-negative block decomposition results of datasets after cell synchronization in G1 phase or not and compensation or not

  Let us consider the regular(K × K) matrix T = [t ij ] whose inverse is noted T -1 = t ♯ ij .We have t 11 g 11 . . . t 1K g KK . . . . . . . . . t K1 g 11 . . . t KK g KK . . .

						
				h 11	0
				. . .		  
						
				0	h KK	  	.
				h (K+1)1 . . . h (K+1)K . . . . . .	  
						
	TG T =	  				  	.
						
	HT -1 =	       	h 11 t ♯ 11 . . . h KK t ♯ . . . 1K . . . h KK t ♯ . . . h 11 t ♯ 1K . . . KK . . .	       	.
	March 27, 2014					DRAFT

March 27, 2014 DRAFT

The notation H ≥ 0 means that each entry of the matrix H is non-negativeMarch 27, 2014 DRAFT


Antoine Lavignotte

Christophe Gravier

Julien Subercaze

Jacques Fayolle

Quality of Experience in 3D Multimedia

Keywords: Quality of Experience, QoE, 3D Video, subjective tests, quality metrics

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

A lot of technologies and services which contain HCI (Human Computer Interface) can be deployed due to their ready-to-use technology. Most of them struggle to emerge because they fall short of users' expectations.

The users' perception (QoE) is a key element to take into account during the development of a new technology or service, especially in Multimedia [START_REF] Jain | Quality of experience[END_REF]. ITU-T (International Telecommunication Union) defines QoE by: -"The overall acceptability of an application or service, as perceived subjectively by the end-user" (12,2009). That short definition contains two additional notes: -Quality of Experience includes the complete end-toend system effects (client, terminal, network, services infrastructure, etc).

-Overall acceptability may be influenced by users' expectations and context.

A more general notion of QoE is proposed by Alben to define all the different interaction aspects between a product and a person [START_REF] Alben | Defining the criteria for effective interaction design[END_REF]. Unlike Quality of services (QoS), QoE has subjectivity, expressed or bound to a given context.

A technology is currently trying to be adopted by users: the third dimension (3D). More and more TV manufacturer offers this feature in their product range. A rapid adoption will strongly depend on its impact on the user. 3D will have to therefore significantly enhance the user experience in a multimedia broadcast. Taking into account QoE notion during 3D service development will be a crucial step to its durability.

The rest of this paper is organized as follows. Section 2 describes the different approaches to improve the QoE measured by a user during a movie diffusion. In section 3, a review of the existing works on QoE associated with 3D is provided. Section 4 presents the various national or international projects whose work concerns the study of the QoE associated with 3D. Finally, the paper is summarized in section 5.

QOE APPROACHES

Several studies have been carried out on QoE. We can actually identify two principal approaches to evaluate QoE: The objective methods and the subjective methods.

The objective evaluation

The objective evaluation is based on objective metrics associated with the final service. There are two main families of studies: Model-based or Feature-based evaluations. The model-based family recreates the human perception and study the impact of artifacts and degradations on a subject. The Feature-based method consist of studying the impact of video's degradation by signal processing methods. In this case, the most often used metrics are MSE (Mean Squared Error) and PSNR (Peak Signal to Noise Ratio) [START_REF] Zepernick | Perceptual-based quality metrics for image and video services: A survey[END_REF]?;?). These metrics are widely used due to their ease of computation. However, [START_REF] Sheikh | Image information and visual quality[END_REF]?) demonstrate a moderate correlation between PSNR and human perception. Another type of model-based method consists in focusing on altering effects produced during the content transport. This approach traditionally focuses on QoS parameters [START_REF] Varga | Correlating user perception and measurable network properties: Experimenting with qoe[END_REF]?;?). This view ignores the most important part of QoE: the end user.

Subjective evaluation

Subjective evaluation is usually based on tests with a group of users who need to grade different configurations to which they are exposed [START_REF] Chen | A crowdsourceable qoe evaluation framework for multimedia content[END_REF]?;?). In scientific and industrial fields, subjective evaluation is the most direct way to evaluate human perception of a service and allows one to optimize 3DTV systems. Several tests using this method have been established during the last few decades. This kind of evaluation can be divided into two groups: psycho-perceptual and user-centered approaches.

The psycho-perceptual approach needs a controlled laboratory environment and study the relation between physical stimuli and sensorial experience. Concerning 2D subjective video quality assessment methodologies, the ITU-R BT.500 (ITU- R, 2002) recommendation is widely used for assessing television pictures quality. Another very popular method in 2D is subjective assessment methodology for video quality (SAMVIQ) [START_REF] Brotherton | Subjective Multimedia Quality Assessment[END_REF].

The user-centered approach is quite different due to the perspective chosen to conduct this kind of studies. We need to put ourselves in the user's place rather than that of the system. This is achieved by taking into account typical users, required system characteristics, context of potential usage scenario and goal-related evaluation tasks.

The figure 1

QoE methods

A more complete description of the different tests can be found in (Goldmann and Ebrahimi, 2010) In the following part, we will explain the difference between 2D and 3D insofar as QoE is concerned.

FINDINGS IN 3D QOE

One may assume that 3D is just a sum of a 2D technology and the depth added to the pictures. If this were the case, one could simply extends existing 2D QoE assessment methods to take depth into account. Unfortunately, As Goldman states in (Goldmann and Ebrahimi, 2010), it is not so simple.

The 3D content visualization criteria

The third dimension tries to imitate the human visual system. Unfortunately, each person's 3D perception system is unique. Unlike other technological leaps, such as the transition from black and white to color where everybody was in agreement on the improvement provided by this new technology, for 3D, it is completely different. Some viewers complain about visual discomfort, fatigue, eye strain, problems of visual quality and distortions in 3DTV broadcasting, . . . In order to understand the cause of these problems, we must define the different criteria that play a role in the perceived quality by a user.

As stated in [START_REF] Bulat | Emerging Research Directions on 3D Video Quality Assessment[END_REF], we can find four main criteria in the literature: image quality, naturalness, viewing experience, and depth perception. The first three criteria are traditional: the same than in 2D QoE. However, Depth perception is the new criteria introduced by 3D. It includes several important characteristics that are described in the following paragraph.

3.1.1 Binocular disparity and stereopsis [START_REF] Woo | Absolute stereoscopic thresholds as measured by crossed and uncrossed disparities[END_REF]?):

This refers to the difference seen by the left and the right eye for the same object. The brain uses the binocular disparity to extract the depth information from images perceived by the two eyes.

3.1.2 Accomodation and vergeance [START_REF] Ukai | Visual fatigue caused by viewing stereoscopic motion images: Background, theories, and observations[END_REF]; ?):

The accommodation is the adaptive optical modification that permits a clear image for different vision distances. The vergence allows for the measurement of the focusing properties of the users.

Asymmetrical binocular combination and individual differences:

One must take into account the specificities of 3D perception of each user. The last aspect has a large research filing which contain:

Stereoblindness: Richards [START_REF] Richards | Stereopsis and stereoblindness[END_REF] shows that a part of the population was unable to see in 3D using stereo vision due to the inability to perceive depth information by combining and comparing images from their two eyes. A study by Eyecaretrust, an English health organism, showed that 12% of the english population has a visual impairment and can not enjoy 3D films in all their glory 1 .

Strabismus: A part of the population is inflicted with a non alignment of the optical axes of the two eyes. The gaze of each eye is not perfectly aligned on the point in the space and affects the depth perception.

Interpupillary distance: Dodgson pointed out in [START_REF] Dodgson | Variation and extrema of human interpupillary distance[END_REF] and [START_REF] Bahn | Effects of Interpupillary Distance and AC/A Ratio on Binocular Fusion and Depth Perception[END_REF], the mean 1 http://www.eyecaretrust.org.uk/view.php? item_id=566 interpupillary distance (IPD) is an important and oft-quoted measure in stereoscopic works. He proved that the mean IPD is around 63mm, but a range 40-80mm has to be taken into account to cover the whole population (adults and children).

Age: Age is also an important variable. Norman [START_REF] Norman | The effects of age upon the perception of depth and 3-D shape from differential motion and binocular disparity[END_REF] & Hayslip [START_REF] Hayslip | Adult Development and Aging[END_REF]) demonstrated that older adults can extract depth and shape from optical patterns containing differential motion or binocular disparities but these abilities are often manifested with lower levels of performance.

Display duration:

The test display duration has to be defined. [START_REF] Tam | Display duration and stereoscopic depth discrimination[END_REF] and [START_REF] Patterson | The effect of testing method on stereoanomaly[END_REF] investigated the role of display duration in stereoscopic depth perception.

Results confirmed large individual differences in the display duration required for a stereoscopic depth perception.

Specificity of 3D assessments

As explained in 2.0.2, ITU-R BT.500 (ITU-R, 2002) is a recommendation widely used to assess the quality of 2D videos. The previously exposed singularities of 3D over 2D make that recommendation inappropriate for 3D assessments. Chen et al propose an extend of the ITU recommendation greater completeness and to take into account the specificity of 3D which is next presented. In this part, we will present the particularities of 3D assessment exposed by [START_REF] Chen | New requirements of subjective video quality assessment methodologies for 3DTV[END_REF].

General viewing conditions

We can list several new important conditions for a good experience in 3D:

Problems of luminance and contrast ratio: They appears du to the use of additional optical instruments for 3D viewing (glasses and filters).

Greater importance of the display position: Has to be taken into account compared to 2D tests. The perception of real background and the perceived display depth depend on it.

Room illumination: It has to be specified precisely to avoid conflicts and perturbations with the media broadcasted to the user.

Monitor resolution:

The monitor resolution is also an important parameter.

Recommendations: Minimum values for spatial and temporal view resolution and stereoscopic resolution must be addressed [START_REF] Holliman | 3D display systems[END_REF].

Preferred Viewed Distance: The PVD recommended in BT-500 does not take into account the depth perception. It should certainly be added in the recommendation [START_REF] Patterson | Human factors of 3-D displays[END_REF].

Viewing position: is an important parameter which can cause luminance reduction depending of the viewing angle or motion parallax on multiview autostereoscopic displays.

Depth rendering: Finally, the depth rendering has a significant influence on the QoE for autostereoscopic displays [START_REF] Barkowsky | Influence of depth rendering on the quality of experience for an autostereoscopic display[END_REF]

Source signals

Actually, several video formats are available for 3D videos as "video plus depth" [START_REF] Fehn | A 3D-TV system based on video plus depth information[END_REF], "Multi-View video plus depth representation and Coding(MVC)" [START_REF] Merkle | Efficient Prediction Structures for Multiview Video Coding[END_REF] and "Layer Depth Video (LDV)" (S. [START_REF] Jolly | 3D Content Requirements and Initial Acquisition Work[END_REF], etc... In 2011, the MPEG 3D committee issued a Call for Proposal (CfP) on 3D video coding technology with the objective to "define a data format and associated compression technology to enable the high-quality reconstruction of synthesized views for 3D displays". Unlike many studies that have focused on the stereoscopic technology, this CfP also contained the auto-stereocopic multi-view display technology. The results are not actually known but a decision will be taken during the year 2012.

Selection of test materials

Main elements of 2D video complexity defined by ITU-T P910 (ITU, ) are the Spatial perceptual Information (SI) and the Temporal perceptual Information (TI). In 3D, new elements as Depth perceptual information (DI) should also be added [START_REF] Chen | New requirements of subjective video quality assessment methodologies for 3DTV[END_REF].

Test methods

A new element is revealed during 3D assessments: the visual discomfort. It is an important point. It can be measured by optometric tests, ERP (event-related potential) [START_REF] Li | Measurement of 3D visual fatigue using event-related potential (ERP): 3D oddball paradigm[END_REF], eye tracking or questionnaires.

Observers

The number of 15 participants defined in ITU-BT.500 can be discussed du to the inter-individual differences [START_REF] Ukai | Visual fatigue caused by viewing stereoscopic motion images: Background, theories, and observations[END_REF]. Moreover, about 10-15% of the population cannot perceive the depth in 3D due to eyes malfunctions. We are therefore not capable of addressing all the population with only one test.

Test duration

During a 3D diffusion, an accommodation time is needed to the viewer. The 10 seconds proposed by the recommendation BT.500 seems too short for good results.

Collaboration for future research

The domain is currently structured. The different actors work more and more in tandem to advance their research. In this part, we will present important work to propose a base for future research.

A standard for evaluation tests

An European COST Action named QUALINET (European Network on Quality of Experience in Multimedia Systems and Services) has been invited by MPEG for the CfP to participate during the evaluation part of the different 3D video coding algorithms.

The main scientific objective of QUALINET is the development of methodologies for subjective and objective quality metrics by taking into account current and new trends in multimedia communication systems as witnessed by the appearance of new types of contents and interactions.

The ability to compare the different video quality assessment methods is crucial, therefore the development of a standard certification mechanism is critical. To demonstrate the importance of a certification procedure, a group of four institutions (e.g. NTNU, EPFL, UBC and Acreo) conducted a cross-laboratory analysis to estimate the correlation of quality scores obtained by each laboratory for an identical test [START_REF] Perkis | Towards certification of 3D video quality assessment[END_REF]. Results demonstrated that different laboratories employing different subjects can still produce highly correlated results, as they follow similar guidelines to carry out assessments.

That is an interesting basis for future QoE tests that will be able to be carried out by many laboratories in the same conditions for greater impartiality.

3D Databases

To unify the future research in the domain, EPFL (Goldmann et al., 2010) proposed a comprehensive stereoscopic video database that contains a large variety of scenes and different captured parameters. The database also provides subjective quality scores. It can be used to evaluate the performance of visual quality metrics as well as for the comparison and design of new metrics. More information can be found at http://mmspg.epfl.ch/3dvqa.

Current projects & reflexions for QoE 3D

Since 2010, national and international projects on 3D have been launched around the world. The QoE is a key element in the production chain of a 3D movie. So, it is often an important part of these projects. In this chapter, we will scan all the past and actual projects over the world that address the QoE notion in 3D.

3D Live

This french project was composed of 9 academic and industrial partners namely Orange, Institut Telecom, Technicolor, INRIA, AMP, Binocle 3D, Thales Angenieux, Thomson and Grass Valley. The project covered all the 3D diffusion chain from coding to the quality perceived by the user. Their main objective was to improve the user experience by working on all the 3D creation chain. Their research covered: -specific equipments as cameras or optics -software development for disparity correction, content adaptation (upscaling, downscaling) and data protection -metadata management (depth, camera type) -text and graphical insertions without conflicts in the scene depth (subtitles, logos) The program finished on march 2012.

Informations and publications can be found at www. 3dlive-project.com.

Jedi 3D

The project Jedi 3D is a consortium of 21 partners in 3 countries (Belgium, France and Spain). It aims to study and develop a high quality end to end 3DTV for consumer, with a user centric approach, with stereoscopic as basis, toward multi-view. Two work-packages are interesting for the QoE community: WP3 "3D User experience" and WP4: "Demonstrations & User assessment" which are led by the societies NXP and Barco. More information can be found at http://www.jedi-itea2.org/

Future Internet Engineering

The "Future internet Engineering" is a Polish national project financed by the European Union. The project covers the development and testing of infrastructure and services for future Internet generation. One part of the project consist of developing a measurement methodology for 3D video and services. Several interesting publications has been produced on QoE [START_REF] Stankiewicz | A survey of QoE assurance in converged networks[END_REF]?). The project just ended on december 2012.

3D4YOU

3D4YOU is an European project funded by ICT (Information and Communication Technologies), a thematic priority for research and development under the specific program "Cooperation" of the Seventh Framework Program 2007-2013. Industrial and academic partners that work on multimedia joined together on this project (Philips NL, BBC UK, Fraunhofer Heinrich Hertz Institut DE, Thomson R&D FR, Orange France Telecom FR, Christian Albrechts Universitat DE and KUK Filmproduktion DE). Their objective is to develop the key elements of a practical 3D television system. A first approach of the content requirements for subjective testing is presented in (3D4YOU, 2008) A second document (3D4YOU, 2011), much more complete, has been produced in march 2011 to list the consortium proposals to achieve subjectives quality tests for 3D multimedia.

VQEG & 3DTV

VQEG (Video Quality Experts Group) is an expert group born in 1997 to deal with the field of video quality assessment by investigating new and advanced subjective and objective techniques (subjective assessment methods and objective quality metrics). Several projects are actually in progress. One of them is called 3DTV. It's goal is to investigate how to assess 3DTV subjective video quality. This activity is linked to the ITU-R Question 128/6. A first draft is actually available1 . The ITU's schedule provides a project completion in 2015.

QUALINET

The European COST named Qualinet is an expert group which want to create a network for multidisciplinary QoE research in Europe: Their objectives are: -the development of methodologies to assess the perceived QoE for multimedia presentations, -the identification of measurable features which are relevant for the subjectively perceived QoE, -the development of metrics to measure the QoE perceived by a human observer, -the development of tools to optimize existing systems, -and the design of new interaction models between users and content multimedia. It is actually composed of 24 COST countries and 6 non-european. Every year, they organize workshops on QoE as QoMEX (Quality of Multimedia Experience). A white paper written on QoE's definition is available at http://www.qualinet.eu/images/ stories/whitepaper_v1.1_dagstuhl_output_ corrected.pdf. More details can be found at http://www.qualinet.eu/

Conclusion

The presented permits to discover the 3D QoE basics. It explains the new characteristics of 3D assessments compared to 2D. We also introduce national, european and international projects. We hope these elements will enable each researcher to better understand the concept and the importance of QoE in 3D multimedia.

  depicts these QoE assessment methods.

		QoE
		Methods
	Subjective	Objective
	tests	tests
	Psycho	User	Model	Feature
	Perceptual	Centered	Based	Based
	Methods:			Methods:
	ITU-R.500 SAMVIQ			PSNR MSE QoS parameters

http://www.its.bldrdoc.gov/vqeg/ project-pages/3dtv/3dtv.aspx


Antoine Lavignotte

Christophe Gravier

Julien Subercaze

Jacques Fayolle

Quality of Experience, a very personal experience !

Keywords: HDTV, TV Broadcasting, Human Factors, Human Behavior, Measuring, Monitoring

At the Pervasive Computing area, end users expect to receive a multimedia service with an acceptable quality, anytime, and anywhere. Measuring this acceptability is usually referred to Quality of Experience (QoE). Unlike Quality of Service (QoS) which focuses on allocating expected systems and network resources, QoE is concerned with optimizing the perceived quality of a service by end-users. The appraisal of the user's acceptability thresholds is a key factor for service providers to perform adaptation decisions on their products (VoD, IPTV, online games, etc). While QoE is individualized, no study has yet examined to what extent. In this paper, we report an empirical study to understand if QoE should be managed globally, per cluster of users, or personally. We prove that every user has his very own vision of a same service, therefore that future QoE-based adaptive systems should take into account this property.

I. INTRODUCTION

A lot of technologies and services which contain HCI (Human Computer Interface) can be deployed due to their ready-to-use technology. Most of them struggle to emerge because they fall short of users' expectations although they do satisfy some QoS (Quality of Service) thresholds. The reason is that QoS focuses on technical features and does not capture the end-user's expectations. The users' perception (QoE) is a key element to take into account during a new technology or service development, especially in Multimedia [START_REF] Jain | Quality of experience[END_REF]. ITU-T defines QoE by: "The overall acceptability of an application or service, as perceived subjectively by the end-user" [START_REF]Itu-t rec[END_REF]. A more general notion of QoE is proposed by Alben to define all the different interaction aspects between a product and a person [START_REF] Alben | Defining the criteria for effective interaction design[END_REF]. The most important fact to understand is that QoE contains an important degree of subjectivity, in a given context.

The paper is organized as follows. Section 2 defines the objectives and the experimental setup of this study. Results are reported and analyzed in section 3. Finally, section 4 concludes.

II. EXPERIMENTS

A. Objectives

The main objective of the study is to define how to take into account the QoE in an adaptive system. To answer this question, we conducted a test on a users group. In a first approach, we tried to understand the perception difference between users by comparing grades associated on a video set by each subject. The next step, important to secure the results' consistency, was to ensure that user's perception did not change during the test. Finally, thanks to the acceptability threshold asked to each subject, we were able to compare the video quantity which met each user's threshold for a traditional multimedia session.

B. Experimental recommendations

We recreated a personal living room environment in order to carry out tests in a context close to real world situations. To match a great test environment, we decided to follow most of the ITU-R recommendations regarding subjective evaluation methods. All specifications can be found in [4].

This environment was composed of: 1) An HD TV 40" as the terminal, 2) A sofa 2.97 meters away in order to satisfy the PVD (Preferred Viewing Distance [4]), 3) And a room plunged into the dim light to cancel other room illumination. This experiment is independent of a given optimal context. However, the context of the experience for each subject was strictly identical. That means: No light in the room during the test, the same TV configuration for each user (brightness, contrast, ratio, luminance,...) and no noise during the test.

C. Materials and methods

1) Selection of test materials:

To emphasize QoE perception of the users in a certain context, we have chosen to base our test on a single movie encoded in different ways. The first parameter is the image size. Since each terminal has its own power of resolution, so the same service should be distributed with various parameters. The most common resolutions have been chosen: 480p, 720p, 1080p. Keeping the same ratio size was important to respect the image proportions during the diffusion. Each video was broadcast in a full HD TV, that is to say 1080p. The second parameter is the video encoding bitrate. To determine the interesting encoding levels, we conducted preliminary tests to highlight the most significant bit rate levels. In a 3G connection (UMTS, HSDPA), the bandwidth can easily vary from 10Mbps to few kbps during hard handover or telco roaming. We chose a range between 10Mpbs and 250Kbps which fits with a real bandwidth variation on a cellular network. Different steps were also defined to propose multiple video modalities (in Kbps): {250, 500, 800, 1000, 1500, 2000, 9653}, where 9653 Kbps was the source encoding bitrate. These steps were chosen to be closed to different adaptive live streaming recommendations [START_REF] Apple | Technical note tn2224[END_REF].

2) Methodology used: This test is based on a derivative of the Single Stimulus Methodology (SS method). It is the most widely used among visual quality researchers evaluating Image Quality Assessments (IQA) algorithms [START_REF] Redi | Comparing subjective image quality measurement methods for the creation of public databases[END_REF]. This method has been formalized in the ITU-R BT500-11 and ITU-T P.910 recommendations. To better understand the difference of perception and to discover the quality threshold for each user, some changes has been made on the traditional methodology. Unlike many studies where video clips duration varied from 8 to 30 seconds [START_REF] Winkler | Maximizing audiovisual quality at low bitrates[END_REF], we decided to use a full video trailer (Thor movie) lasting 2 minutes and 26 seconds. This choice allowed us to provide a movie with various types of scene (slow parts for discussions, fast parts for action,...). Although the time of each video is important, the user can switch from one video to another at any time. This is more closed of the real life where the user can stop the session if the quality is not good enough. He is therefore not forced to watch the entire video every time. Moreover, to respect the ITU-R BT.500-11 recommendations, a time limitation of 30 minutes has been done. To respect this limitation, the use of only one video was necessary. We chose focus on quality (number of modalities) compared to the quantity (number of videos). To test the influence of previously seen videos and the objectivity of each grade per video, we included two video repetitions into the playlist. A presentation of the playlist is presented in Table I. The videos from the playlist are all downloadable 1 . Repeated videos are respectively marked by an * and **. 2) Scoring Method: Subjects were asked to use the MOS (Mean of Opinion Score) scale to grade each video. This is a way to quantify numerically the outcomes of a subjective experiment. It has been defined by ITU [4]. The MOS scale contains 5 grades. Each grade is designed to reflect a judgement of users concerning video quality. No half grades were available during the test. Table II reproduces the MOS scale for the reader, as it was presented to the subjects. The MOS scale was chosen because it is already widely used in the scientific literature, therefore amenable to comparison with previously obtained results. This method yields a distribution of judgements across a scale of categories for the different videos proposed.

3) Test Environment: To ensure that users cannot be influenced by others, all tests were taken in isolation. The tests subjects were seated on a couch exactly in front of the 40" HD TV. Each user had first to give his/her name, age and gender. Then, the first video started to show. At any moment, as soon as the user had chosen a grade, he/she was able to select it by pressing the appropriate key on the keyboard. Automatically, the next video was started and the previous one stopped. A security of 10 seconds was added to prevent any keyboard manipulation error. This continued until the end of the playlist. Before stopping the tests, one important thing to do was to ascertain the users' threshold of acceptability and more specifically, at what quality he/she was ready to watch the movie for a normal showing lasting an hour and a half. It was really important, in terms of analysis, to know how many videos had an acceptable level of quality for the user within this context.

III. RESULTS & ANALYSIS

A. Mark distribution per video examples 1) Exemple for a 480p video:

To begin with, we propose to study the users' perception of the fifteenth video of the playlist. As a reminder, this video is encoded with a 1500 Kbps bitrate for a size of 480p. The Figure 1 presents a graph detailing the results for the same fifty-two views. It integrates the MOS value assigned by each user on this video. An initial quick scan of this graph shows that the grade distribution is important. Approximately 23% of the tested people chose grade 3. 48% 4 and 27% 5. Only one of them chose grade 2 and nobody grade 1. These results show that clients do not really have the same perception of a service. A quarter of the people think they are watching a perfect image movie with no distortion. Another quarter are watching a movie with only a fair level of quality! If we were in a context where an operator provides a video on demand to his client, a general adaptation decision based on a global users acceptability threshold could push 25% of the users aside because the quality is not good enough.

2) Example for a 720p video: For illustration, we have chosen the third video which is encoded with a 1000Kbps bitrate for a size of 720p. Results can be found in Figure 2. The video still differentiates the users. 10% of the people have chosen grade 5, 23% 4,48% preferred 3 and 17% the 2. Only one person has chosen the mark 1. Compared to the previous video, we noticed that almost half of the users 3) Example for a 1080p video: Finally, results for a 1080p video are represented in Figure 3. We have chosen the eighteenth video which is encoded with a 2000Kbps bit rate for a size of 1080p. 6% of the users have chosen the grade 2, 33% grade 3, 44% grade 4 and 15% grade 5.

4) Results comment: Figure 1, 2 & 3 show that MOS is distributed along the quality axis, with a rather big standard deviation. We observe here Thurstones's law on comparative judgements [START_REF] Thurstone | A law of comparative judgment[END_REF], which is based on the assumption that subjective scores are distributed around the true value. But on the use case of video on demand (VoD) or video streaming, this results are very important. A bad quality reception could lead to stop in the use of the video service. That also implies that everybody does not have a same perception of a service. But we do not actually know if the service has an enough good quality to be seen by each user.

B. Mark distribution analysis

To differentiate user expectations and understand the above results, further analysis is needed than just a superficial reading.

1) Global Mark Evaluation:

To have a better view of the distribution mark for each video, we have represented the results in a box plot graph which is displayed in Figure 4. We recall that the reading mode for column graphs is: On each box, the central point is the median. The edges of the box are the 25th and 75th percentiles. The whiskers extend to the most extreme data points. Outliers are not considered by the whiskers, but they are plotted individually. What we saw before in the example presentation of results (480p, 720p and 1080p) is repeated. Almost all the results are the same. The disparity of acceptability is very important in each video. 50% of videos have a distribution over 4 2) Repeated videos, gap rating: To know if the users' opinion can change over the time for the same video, we repeated two trailers during the test. As a reminder, the first repeated video is 720p size encoded with a 800Kbps bitrate. They were shown 13th and 16th in the sequence. Their broadcast is just separated by a trailer with a low quality (1080p & 500Kbps bitrate). Results are shown in Figure 5. The graph represents the difference between the two grades awarded by the users. It is ordered by users. 60% have exactly the same opinion of a given video. If we work in absolute values, 96% find a difference less to 1 between the two videos. These results mean that the personal scale is quite stable. But we could say that the difference is small due to the proximity of the two broadcasted videos. To be sure of the first results, a second video was repeated. It was a 1080p size encoded with a 2000Kbps bitrate. The first display had been broadcast in the first quarter of videos (video 4). Then the replay was played in the fourth quarter (video 18). The results are shown in Figure 6. 46% have exactly the same opinion of the video. Working in absolute values, 85% discern a difference less to 1between the two videos.

Comparing with the first pair of videos, results are in good agreement. Only 10% of people are scattered in relation to the previous test. This tends to illustrate that users are not very affected by previously streamed views when they are introduced to a new multimedia stream. This is an interesting result when studying QoE during zapping sessions, although further tests may have to be conducted to draw stronger conclusions.

3) 3D Representation: The results can be represented using 3D modeling. This representation allows a quick overview of user's expectations in terms of QoE during this lab tests. Due to the space limitations, only three graphs are depicted in the paper but all the results can be found at http://datasets-satin.telecom-st-etienne.fr/alavignotte/ QoeExperience/. Figure 7, shows the QoE modeling of User 31. Two thresholds appear: 800 & 1200Kbps. It can be observed that the user perception is not very sensitive with the resolution but much more with the bitrate. On Figure 8, results are drastically different. Just one threshold appears in high resolution (1.5Mbps). Compared to the previous user, the resolution is more important. For example, we have a 2 MOS gap with a same bitrate but with different resolution. In this context, the adaptation choice could be decisive on the user decision to continue the viewing or not. Another user shows another result (Figure 9). Only one threshold appear at 800Kbps and later, the grade changes proportionally to the bitrate growth. Sometimes, playing with the resolution can up the MOS score and could better satisfy the user (800Kbps / 480p & 1500Kbps / 1080p). Finally, Figure 10 shows the user's average representation. With a such result's disparity, we cannot base our adaptation's choices on an average grade which is not representative of all users. 4) Accepted videos: After all these graphs, we have yet to answer the main question: Does the QoE need to be addressed personally ? A good way of answering this question would be to differentiate the number of accepted videos for each user2 and to compare them. These results are shown in Figure 11. Never more than 13% people agree on the same number of accepted videos. There is a range from 5 to 19 accepted videos for the same context. How is it possible, therefore, to base the provision of multimedia on the results from a group of users with only two or three quality levels without actually knowing the individual user's expectations ? At least it illustrates the fact that treating QoE globally (for instance with a MOS threshold shared by all users) may not be the best way to maximize the personal multimedia experience. We can say that QoE based adaptation would be better performed in a personal manner and a clustering method can not be a solution to maximize the number of satisfied users.

IV. CONCLUSION

In this article, we have provided a new point of view on the uses of QoE. To conduct this study, we designed an experiment to address the video broadcast problem. The results lead to different findings. The first one is that each subject has its own vision of a particular service, except for poor QoE multimedia, which are globally perceived as poor. If a service operator needs to adapt his video due to network congestion, he needs to consider that his clients will not have the same perception of the delivery content. This could result in the broadcast cancellation for a certain proportion of his clients. The second one shows that people's perception of a service is quite stable and will not significantly change during the test. This means that if you are aware of the users' acceptability levels, you would have all the necessary information for continuous adaptation to this user for a long time. You may not have to recover the personal QoE scale too regularly. The third and final one supports the results of the first one. It shows that users do not have the same scale of expectations. We can just prepare a few different quality levels for a service and make decisions according to the network vagaries. But we have to take in consideration the real personal scale of the users to make the right decision to maximize QoE.

D. Subjective assessment 1 )

 1 User Panel: Fifty-two subjects took part in this study. They ranged between 18 and 58 years old. The average age is 33 and the median is 29. Concerning the gender distribution, 1 http://datasets-satin.telecom-st-etienne.fr/alavignotte/QoeExperience/

Figure 1 .

 1 Figure 1. Video 15: Users distribution per MOS Value

Figure 2 .Figure 3 .

 23 Figure 2. Video 3. Users distribution per MOS Value

Figure 5 .Figure 6 .

 56 Figure 5. Difference between video 13 & 16 (ordered per user)

  Figure 7. User 31-3D Representation

  It tends to show that poor QoE is globally perceived, while enhanced QoE is perceived differently by users, and the difference is significant. These results suggest that everybody has their own personal scale. But is this personal scale, in the same context, fixed or does it change over time ?

		5
		4
	MOS Value	3
		2
		1
		1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23
																							Video
		Figure 4. Box plots summary for MOS values obtained per video
	grades, 30% over 3 grades. All the others have a distribution
	over 2 grades at least. All users agree with mark 1 for
	videos with very low quality. (videos 17, 21 and 23).

Based on the MOS threshold for an acceptable video streaming as declared at the end of the session by each subject.