ASSESSING MARINE RESOURCE DAMAGE AND THE CLEAN-UP COST OF OIL SPILLS

This paper presents a discussion of some of the social costs which resulted from the Amoco Cadiz's oil spill which damaged 350 kilometers of French coastline in 1978. This case is particularly interesting for several reasons.

(i) The Amoco Cadiz is the largest vesse! spill in history, over five times the amount spilled by the Exxon Valdez in 1989, and this is a reference point.

(ii) It reveals the difficulties for economists in evaluating natural damage costs since these costs are highly dependent on the assumptions made in order to calculate them, the models used and the scarcity of data available.

(iii) As the lawsuits were settled only very recently (1992), this case allows for the first longitudinal study of the issues involved in the economic valuation of a massive oil spill. This paper is divided into two sections. The first one is devoted to the non-market components of marine resource damage. Recreation and amenity losses are considered first.

The various estimates which were produced are discussed . Then, ecological damage due to a perturbation of an ecosystem is defined. A monetarization of biomass Joss based on a trophic chain mode! is given. The second section gives an overview of the main problems which were posed during the clean-up operations. The limitations of a static analysis of such clean-up are illustrated and a dynamic approach is proposed.

The wreck of the Braer in the Shetland Islands in January 1993, coming so soon after December 1992 oil spill off Northwest Spain has focused attention on oil tanker traffic risk and natural resource damage. Four years ago, the Exxon Valdez oil spill in Alaska's Prince William Sound was a high-profile case that resulted in one of the largest environmental settlements in the US history (about$ 1 billion). This wreck led to the passage of the US Oil Pollution Act of 1990 which extends the scope of recoverable damages for oil spills. In fact the worst oil spill to reach land occurred in Brittany (France) in March 1978 when the Amoco Cadiz ran aground along the coast of Brittany. The amount of oill spilled into the ocean from the Amoco Cadiz was over five times the amount spilled from the Exxon Valdez (220 000 metric tons of crude oil).

After every oil spill the same questions arise : what can be done to limit the risk of oil spills ? How can we contain and limit the extent of disasters when they happen ? How can we evaluate the damage ? Each oil spill has its specific features but there are always lessons to be leamt in various fields. Marine ressource damage valuation and the management of clean-up operations pose very difficult problems [START_REF] Bonnieux | Impact socio-économique de la marée noire provenant de l'Amoco-Cadiz[END_REF][START_REF] Bonnieux | catastrophe écologique et dommages économiques : problèmes d'évaluation à partir de l'Amoco-Cadiz[END_REF] which are reviewed in this paper. Comments are based upon our own experience with the Amoco Cadiz case. As the lawsuits were settled only very recently (1992), this case allows for the first longitudical study of the issues involved in the economic valuation of a massive oil spi Il.

MARINE RESOURCE DAMAGE

The damage to marine resource resulting from a discharge of oil on the shoreline stems from the reduction in the service flow from the environment as a consequence of contamination by hydrocarbons. Services are defined in reference to the functions that marine resources support. Iwo main categories of services are provided by marine resources. The first category is related to use values and non-use values. It refers to the economic concept of damage : how much money it would take to make everyone as well off as they were before the oil spill occurred [START_REF] Freeman | Assessing damages from the Valdez oil spill[END_REF]. The second category relates to services which are provided by the normal functioning of the marine ecosystem. The stress suffered by the lower trophic level organisms creates an ecological imbalance. This imbalance induces biological and economical consequences. There are obvious links between these two categories e.g. commercial and sport fishing are supported by the biological productivity of the ecosystem. This section is restricted to two non-market components of the total damage : (i) recreation and amenity losses (ii) assessment of the perturbation of the ecosystem.

Recreation and amenity losses

There are important links between outdoor recreation and the environment. While ail outdoor recreation does not depend on the natural environment, much of it does. In Brittany, a good deal of outdoor recreation relies upon marine resources and marine assets that cannot be easily reproduced. The attractiveness of resorts and the quality of the shoreline constitute important determinants of any demand for recreation.

Pollution from the wreck of the Amoco Cadiz adversely affected the physical characteristics of a lot of recreation sites in Brittany, it therefore induced a decline in benefits for both users and non users. Welfare losses from any environmental deterioration for an individual can be measured in monetary tenns. The conceptual basis for determining this monetary value rests on the plausible assumption that when confronted with two alternative situations an individual can indicate which s/he prefers, or state s/he has no preference.

In order to show the way in which losses from an environmental disaster can be measured let us consider a single recreation site such as a beach. There is a demand function which relates the quantity of beach services demanded, measured in beach-days per season, to the price of these services and other variables. This demand function can also be interpreted as a marginal willingness to pay function, relating the marginal value of a beach-day to the quantity ofbeach-days and other variables.

In figure 1, D 1 is the demand curve in beach-days before pollution. Suppose the price of admission to this beach is OA per day (OA can be zero), then the recreational use will be ON l • The value of this beach to users, given its initial quality, is the consumers surplus as measured by area ABC. Assume that beach quality has deteriorated, users would be willing to pay less at the margin to use this polluted beach. In economic terms the effect is to shift the demand curve to the left. The new demand curve is shown by D2 and the consumers surplus is now measured by the area ADE. Therefore the decrease in consumers surplus gives the net loss due to beach pollution; it is measured by the area BCED.

The net loss can be divided into two components. The first is the decrease in value to those users who came the beach even at the low levels of quality resulting from the pollution. This is the area BDEF. This area represents the decreased willingness to pay to visit this beach rather than do without, eg tourists who came to Brittany in 1978. The second results from the lesser attractiveness of this beach relative to alternative beaches and alternative expenditures, eg tourists who did not corne to Brittany in 1978. It cornes from the decrease in the number of beach-days which declines to ON2_ The net loss associated with this decline is the area CEF.

A significant amount of work dealing with non market valuation is now available. It is usual to consider benefit transfers (i .e. the application of monetary values obtained from a particular non market goods analysis to an alternative or secondary policy decision setting) as a relevant approach in damage assessment [START_REF] Brookshire | Benefit transfers: conceptual and empirical issues[END_REF]Ward and Duffield, 1992). But at the time of the Amoco Cadiz oil spill, the valuation of non market damages received comparatively little economic attention, especially in France. At that time, the investigators involved in this case had to develop original studies. Although economists would always prefer to have more resources and more time, the reality of the litigation setting is that in this case this was not possible. So we had to do our best in a limited period of time and with a limited amount of money. A variety of estimates were produced. They can be classified according to three categories : (i) estimates based upon observed behaviour (implementation of the travel cost approach) ; (ii) estimates based upon preference elicitation mechanisms (use of contingent behavior, insurance, compensation and willingness to pay) ; (iii) estimates based upon expert opinion (use of unit value per beach day lost). Table 1 summarizes the main results of these studies which dealt with recreation and amenity lasses. We must emphasize that the estimates were quite consistent, except for the output of the travel cost model for tourists who did corne in 1978. These results, despite some restrictive assumptions and some weakness of data, show that non market damage is very high and is comparable with market damage [START_REF] Bonnieux | catastrophe écologique et dommages économiques : problèmes d'évaluation à partir de l'Amoco-Cadiz[END_REF].

Assessment of the perturbation of the ecosystem

The central issue which arises when assessing ecological losses following a perturbation of an ecosystem is : what should be counted as damage, and how should this damage be measured ? In order to understand what ecological losses mean from an economic point of view, the ecological impact has to be determined and quantified. Only then, can we attempt to monetarize the ecosystem perturbation.

An important point in any quantification of damage to the ecosystem is the resource recoverability as measured in reference to baseline level services, i.e. the conditions existing before the oil discharge. It corresponds to the amount of time needed for an injured resource to recover and reach its initial level. The ecosystem recovers at natural recovery rate which depends on the magnitude and the toxicity of the oil discharge. This rate can be accelerated by human operations. (Ward and Duffield, 1992, p. 153). [START_REF] Glemarec | Ecological impact of an oil spill : utilization of biological indicators[END_REF]. Among the sites which were monitored, the most interesting ones are downstream sands in a small estuary located Jess than 10 kilometers from the place where the supertanker ran aground (Aber Wrac'h). In this area both macrofauna and oil contamination of the sediments were studied. Figure 3 shows that one year after the wreck the oil contamination was still more than 1000 ppm (1 g of oil per kg of sediment). Even though the decontamination process was very rapid in the beginning, four years were needed to meet an acceptable level of oil in the sediments. The ecological impact is characterized by two elements : (i) mortality, (ii) recovery in population species. Mortality is a direct result of oil toxicity. This toxicity was increased by the severe stonn which caused the grounding of the vesse!. The oil was dissolved in the entire column of water, so the oil content in deep water was almost as high as in water nearer the surface. Substitute population species appeared in the gap left. Three distinct phases can be distinguished in a post spill marine ecosystem (Le [START_REF] Moal | Perturbation and recovery after two major oil spills (Amoco Cadiz and Tanio) in Terrestrial and aquatic ecosystems[END_REF].

-In the initial phase, the degradation phase, the light fractions of crude oil and the aromatic hydrocarbons cause the destruction of the ecosystem. Opportunistic species develop after six months.

-The recolonization phase occurs in two successive stages. The first one is a primary recolonization which is partial and incomplete because only a few spec1es of the original species are present. Four years after the accident, the second recolonization stage begins.

-Thirdly the restructuration phase, which is observed six years after the accident, is a regulation process. The community shows readjustments, the competition between species becomes controlled by predators, and a new equilibrium is achieved.

The biological process which explains the recovery can be described usmg three synthetic parameters : the number of species S characterizing how rich the ecosystem is; A, the abundance ( or density) and the biomass B. These three parameters drastically decrease reaching a minimum 8 months after the accident as shown by figure 4. From 8 to 28 months these three parameters and especially the abundance increase. At 36 months there is a drop corresponding to the ecotonal stage. The stabilization is achieved after a period of six years [START_REF] Glemarec | Ecological impact of an oil spill utilization of biological indicator[END_REF]. br2.pre Three conclusions can be drawn from the above. Firstly, a comparison between figure 3 and figure 4 shows that the decontamination required about 70 months while the ecosystem took 84 months to reach its new equilibrium. Secondly, if the decontamination follows a linear or quasi-linear time function, the evolution of the synthetic parameters is not a simple function of time. Thirdly, biomass is the simplest parameter to use to quantify the ecological impact because its evolution is the most simple over time. It is a proxy variable which can be used to estima te the perturbation of the ecosystem in a reasonable delay ( about one year after the oil spill).

Biomass can be considered as the main component of the marine food chain, the process whereby living organisms obtain food energy by consuming other organisms. The food energy is transferred in the form of living matter at each step up to commercial marine products. Any loss of biomass represents a potential decrease of commercial species which would have been produced. If we know the rate at which food energy flows through the ecosystem and the path it follows, it is possible to value the impact of a perturbation to the ecosystem, (Isard, 1972, p.55).

A simple four-staged mode! quantifying the biological relationships of the marine food system is considered. The zero trophic level relates to the phytoplankton and sea weeds which are the main producer plants in coastal waters. Phytoplankton production depends mainly on the rate of photosynthesis and the rate of respiration. This is the level of the producers. The first trophic level includes the primary consumers, the herbivores, mainly zooplankton and also fish such as the grey mullet. The second trophic level corresponds to the primary carnivores.

The third trophic level corresponds to the secondary carnivores (species eating camivorous invertebrates or fish) .

Let Y; the potential production at level i (i = 0, 1,2,3). The amount of phytoplankton required to produce one unit of herbivore is a 01 . In the same way the functional relationship from a trophic level to the upper one is a 11 , a 1 J . However for the first and second trophic levels there are commercial and non commercial species, so other transformation coefficients /J, and /3 1 are introduced. Then non commercial output at the first level is /J, Yi and the commercial output is ( 1-PiJYi . The non commercial output at the second level is /3 2 Yi and the commercial output is (1 -/3 2 ) Yi-It is supposed that total phytoplankton production is non commercial, while total secondary carnivore production is commercial (Chart 1 ). The mode! can be written with five equations :

Y, = f31Y, + (1-/J)Y, ½ = /32½ +(J -f32JI'i Y, = ao1Yo ½ = a12/J1Y, ½ =a23/J2I'i
and potential commercial output equals : ( 1-/3 1 ) Y, + ( 1-/3 2 ) +Y;, this amount is used to value food chain flows.

Concerning the transformation coefficients a, we admit the ratio of 10 : 1 as suggested by Petersen's studies, (Isard 1972 p. 246). In order to estimate /3 1 and /3 2 it was supposed that the structure of landings in the polluted area before the oil spill, reflected the structure of potential commercial output. Let r = /3 2 I'i I ( 1-f3 2 )I'i the ratio between fish belonging to the third trophic level and fish belonging to the second level, and <5 =()-/3 1 )Y, / (J-/3 2 )Y 2 the ratio between herbivorous fish and fish belonging to the second trophic level. y and <5 being given by published data then, /3 1 = 0.994 and /3 2 = 0.534. Chart 2 summarizes the results on the basis of a potential production at Y 0 of 1000 metric tons.

Chart 2. An application of the marine food chain to the polluted area (potential output in metric tons) According to marine biologists, [START_REF] Glemarec | Ecological impact of an oil spill utilization of biological indicator[END_REF], the Joss of non commercial marine biomass reached 260 000 metric tons of wet dead weight during the first six months. Over the six years necessary to obtain a new equilibrium the total biomass Joss was 600 000 tons corresponding to 31 500 tons of commercial species. Using the catch average price the potential damage was 195 millions 1978 Francs.

CLEAN-UP AND EMERGENCY RESPONSE : MARGINAL AND COST-EFFECTIVENESS

ANALYSES

When the supertanker Amoco-Cadiz grounded on March 16th 1978 she was loaded (220000 metric tons of Arabian and Iranian crude oil). As the result of winds blowing in the onshore direction and the proximity of the wrecking to the shore about 35 percent of the oil was driven onto the coast. The oil and water formed an emulsion, termed "mousse", containing 20 to 30 percent oil. About 250 thousand tons of mousse were deposited along 350 km of coastline injuring na tu rai resources and affecting the economic activity of the area.

Because clean-up activities represent the largest component of social costs of any large oil spill, these activities raise a major question for decision-makers : how many goods and services have to be used to restore the shoreline ? Basically there are three different positions to operate the cleanup :

-the laisser-faire. Wave and tidal action is supposed to allow a good self-cleaning along rocky and high energy beaches. Man's intervention using dispersants is considered to be harmful to marine life. This is the position of the tanker owners, who prefer the natural recovery without any clean-up effort [START_REF] White | Considérations pratiques concernant le coût des marées noires in OCDE[END_REF].

-the full restoration. Conservationists, fishermen, oystermen, hotel and restaurant owners and residents who direct losses, are in faveur of using maximum effort, against the spill in order to minimize the reduction in losses which result from the oil discharge. These people are in faveur of a rapid and total cleanness of the coast.

-the rational clean-up effort. laisser faire and complete cleanness are corner solutions.

Since there is a trade-off between increasing clean-up costs and reducing losses in use and in non-use of the shoreline and coastal resources, the problem is to determine the efficient level of the clean-up effort.

A static analysis of cleanup

The value of the shoreline and of the marine resources for producers (fishermen, oyster men, cafe-hotel-restaurant owners) and consumers (residents and tourists) can be captured with a simple social welfare function. Total welfare depends on the quality of the environment :

cleanness can be used as a proxy variable for this complex parameter. Total welfare W is supposed to be a decreasing function of the amount of oil spilled (Figure 2). This relationship can be viewed another way ; welfare increases with the quantity of oil removed and the marginal cost of damage increases with the amount of oil spilled (up to a level above which non-convexity can occur). [START_REF] Bellier | Lutte contre les pollutions marines accidentelles par les hydrocarbures: l'expérience de l[END_REF]. The most severely affected and most accessible parts of the coast were cleaned first, followed by regions of lower priority where hydrocarbon deposition was either not so great or where access for heavy equipment was more difficult.

During the first ten days of pumping, liquid mousse pumped per man fell very quickly from 1.2 cubic meters on the first day to between 0.07 and O. 16 cubic meters on the fifth day [START_REF] Meade | La marée noire de l'Amoco-Cadiz : analyse des coûts des opérations d'intervention d'urgence, de nettoiement et de remise en état de l'environnement[END_REF].

Figure 6. Optimal level of cleanup A static analysis is of little use to the decision maker since specific information at a given moment is needed to allow decisions to be made. Every decision involves forecasting how the benefits from the measures taken will evolve over time. For example it was decided to place booms directly across estuaries to protect them from the oil. But weather conditions greatly reduced their effectiveness because of the magnitude of the waves. In the same way, the scraping of sand from a lot of beaches can be criticized ex post since these beaches were not endangered of being polluted. Basically, in a major emergency it is not possible to state

0 A pollution -•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
with certainty what consequences a particular measure of protection will have. Will a given set of measures actually achieve its goal ? And on the demand side, is it certain that the services from an injured resource will actually be desired in the future ? The main implication of uncertainly on both sides, supply side and demand side, is the existence of what is called an "option price".

Decision-makers cannot know in advance wether the decision taken will be a success or a failure. However the risk is that the action taken may be too much rather than too little since officiais have an important risk aversion and fear public discontent. For the Amoco-Cadiz this problem was increased by the existence of sensitive areas such as bird reserves with auks, puffins, murres and razorbills for which the injury could have irreversible effects.

Three issues must be taken into account : (i) decisions are taken under uncertainly, (ii) the decision process is sequential : it is possible to obtain further information about the consequences of a choice, (iii) the possibility that a choice will have irreversible consequences (no choice is a choice).

In these circumstances when the decision-maker can make the decision after any uncertainty is resolved, quasi-option value has to be considered [START_REF] Henry | Option values in the economics of irreplaceable assets[END_REF]. The concept of quasi-option value refers to the increase in expected net benefits of not undertaking a project which could have irreversible consequences. In emergency response to an oil spill it is the reverse, since the laisser-faire or the minimum clean-up costs would lead to irreversible effects.

Decisions involving irreversible consequences are therefore more conservationist once we open up the possibility of learning more by choosing protective solutions. In ttùs case the emergency response is always more costly than it would have been with perfect information,. but the conservationist choice induces a better saving of natural resources.

As shown in figure 7

Figure 1 .

 1 Figure 1. Beach demand and losses due to water quality deterioration

Figure 2

 2 Figure 2 illustrates a simple time profile which describes the Joss of natural resource services after the oil spill. The baseline service level is reached after a timespan which depends on the natural recovery rate or human action (restoration recovery).

Figure 2 .

 2 Figure 2. Loss of natural resource services after an oil spill and their recovery rate Natural Resource Services

Fig. 3 .

 3 Fig. 3. Level of oil in sediments over the time in the dowstream sands of the Aber Wrac'h (in ppm)

Figure 4 .

 4 Figure 4. The marine ecosystem after the oil spill : evolution of syntheticparameters (in the Aber Wrac'h)

Chart 1 .

 1 Simplified mode! of marine food chain Trophic level 0 _________ . . . . __. . . . . _ ______________ -=pr=o=du=c=u=• o=n=Y= 3 ================

Fig. 5 .

 5 Fig. 5. Social welfare as a fonction of cleanness w

restoration

  This decline in productivity per man-day is illustrated in Figure6by the CmN curve which shows that the cleanup effort per metric ton of oil spilled is Iow for a large amount of hydrocarbon (right side of the horizontal axis). Conversely the cost of removing one more metric ton of oil 2.2. A dynamic analysis of clean-up becomes very high when the shoreline is nearly clean ( close to zero on the horizontal axis). The CmD curve shows the marginal benefits from the clean-up activity in terms of services provided by marine natural resources. The optimal level of cleaning will be reached when the marginal clean-up cost is exactly equal to the marginal social benefit, point B. At this point the quantity AC of oil has been removed. The total clean-up cost associated is equal to the triangle ABC. At this point the amount AO of oil remams. This amount AO corresponds to residual damage which equals the compensatable triangle ABO. In theory compensatable damage includes the clean-up costs (triangle ABC) plus the residual damage (triangle ABO).

Figure

  Figure 7. The shift of the optimum level of effort in a dynarnic analysis

Table 1 .

 1 Recreation and amenity lasses

	Valuation in nùllion


est destinée au dépôt

  The time-course evolution of viability and competence for proliferation of woody plant protoplasts following cold-storage Sergio Ochatt, E.M. Patat-Ochatt To cite this version: Sergio Ochatt, E.M. Patat-Ochatt. The time-course evolution of viability and competence for proliferation of woody plant protoplasts following cold-storage. 8. International Protoplasts Symposium, 1991, Uppsala, Sweden. 1 p., 1991. hal-02849603


Keywords: Classification non supervisée, Mélanges gaussiens, Sélection de variables, Critère pénalisé, Entropie à crochet

Un critère pénalisé non asymptotique pour la sélection de modèle de mélanges gaussiens Résumé : Des mélanges gaussiens de formes spécifiques sont considérés pour résoudre un problème de sélection de variables en classification non supervisée. Un critère pénalisé non asymptotique est proposé pour sélectionner le nombre de composantes du mélange et l'ensemble des variables pertinentes pour cette classification. Le contraste Kullback-Leibler ayant un comportement non linéaire sur les mélanges gaussiens, un théorème général de sélection de modèles pour l'estimation de densités par maximum de vraisemblance du à Massart (2007) est utilisé pour déterminer la forme de la pénalité. Ce théorème nécessite le contrôle de l'entropie à crochet des familles de mélanges gaussiens étudiées. Le cas des variables ordonnées et celui des variables non ordonnées sont tous deux considérés dans cet article.

Introduction

Model-based clustering methods consist of modelling clusters with parametric distributions and considering the mixture of these distributions to describe the whole dataset. They provide a rigorous framework to assess the number of mixture components and to take into account the variable roles.

Currently, cluster analysis is more and more concerned with large datasets where observations are described by many variables. This large number of predictor variables could be beneficial to data clustering. Nevertheless, the useful information for clustering can be contained into only a variable subset and some of the variables can be useless or even harmful to choose a reasonable clustering structure. Several authors have suggested variable selection methods for Gaussian mixture clustering which is the most widely used mixture model for clustering multivariate continuous datasets. These methods are called "wrapper" since they are included into the clustering process. [START_REF] Law | Simultaneous Feature Selection and Clustering Using Mixture Models[END_REF] have introduced the feature saliency concept. Regardless of cluster membership, relevant variables are assumed to be independent of the irrelevant variables which are supposed to have the same distribution. [START_REF] Raftery | Variable selection for model-based clustering[END_REF] recast variable selection for clustering into a global model selection problem. Irrelevant variables are explained by all the relevant clustering variables according to a linear regression. The comparison between two nested variable subsets is performed using Bayes factor. A variation of this method is proposed in [START_REF] Maugis | Variable selection for clustering with Gaussian mixture models[END_REF] where irrelevant variables can only depend on a relevant clustering variable subset and variables can have different sizes (block variables). Since all these methods are based on a variable selection procedure included into the clustering process, they do not impose specific constraints on Gaussian mixture forms. On the contrary, [START_REF] Bouveyron | High-dimensional data clustering[END_REF] consider a suitable Gaussian mixture family to take into account that data live in low-dimensional subspaces hidden in the original space. However, since this dimension reduction is based on principal components, it is difficult to deduce from this approach an interpretation of the variable roles.

In this paper, a new variable method for clustering is proposed. It recasts variable selection and clustering problems into a model selection problem in a density estimation framework. Suppose that we observe a sample from an unknown probability distribution with density s. A specific collection of models is defined: a model S (K,v) corresponds to a particular clustering situation with K clusters and a clustering "relevant" variable subset v. A density t in S (K,v) has the following form: its projection on the relevant variable space is a Gaussian mixture density with K components and its projection on the space of the other variables is a multidimensional Gaussian density. Definitions of models S (K,v) are precised in Section 2.1. The problem can be formulated as a choice of a model among the collection since this choice automatically leads to a data clustering and a variable selection. Thus, a data-driven criterion is needed to select the "best" model among the model collection. With a non asymptotic point of view, the "best" model is the one whose the associated maximum likelihood estimator of s gives the lowest estimation error. In this sense, the variable selection is not only beneficial to the clustering problem but also to the estimation problem.

In the density estimation framework, the principle of selecting a model by penalizing a loglikelihood type criterion has emerged during the seventies. [START_REF] Akaike | Second International Symposium on Information Theory[END_REF] proposed the AIC criterion (Akaike's information criterion) and [START_REF] Schwarz | Estimating the dimension of a model[END_REF] suggested the BIC (Bayesian Information Criterion). These two classical criteria assume implicitly that the true distribution belongs to the model collection (see for instance [START_REF] Burnham | Model selection and multimodel inference[END_REF]. With a different point of view, the criterion ICL (Integrated Completed Likelihood) proposed by [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] takes into account the clustering aim. Although the behaviours of these asymptotic criteria were tested in practice, their theoretical properties are few or not proved. For instance, the BIC consistency is only stated for cluster number under restrictive regularity assumptions and assuming that the true density belongs to the considered Gaussian mixture family [START_REF] Keribin | Consistent estimation of the order of mixture models[END_REF].

A non asymptotic approach for model selection via penalization has emerged during the last ten years, mainly with works of [START_REF] Birgé | From model selection to adaptive estimation[END_REF] and [START_REF] Barron | Risk bounds for model selection via penalization[END_REF]. An overview is available in [START_REF] Massart | Concentration inequalities and model selection[END_REF]. The aim of this approach is to define penalized data-driven criteria which lead to oracle inequalities. The belonging of the true density to the model collection is not required. The penalty function depends on the parameter number of each model and also on the complexity of the whole model collection. This approach has been carried out in several frameworks where penalty functions are explicitly assessed. In our context, a general model selection theorem for maximum likelihood estimation (MLE) is used to obtain a penalized criterion and an associated oracle inequality. This theorem proposed by [START_REF] Massart | Concentration inequalities and model selection[END_REF] is a version of Theorem 2 in [START_REF] Barron | Risk bounds for model selection via penalization[END_REF]. Its application requires to control the bracketing entropy of the considered Gaussian mixture models.

The paper is organized as follows: Section 2 gives the model selection principles. The Gaussian mixture models considered in this paper are described in Section 2.1 and principles of non asymptotic theory for density estimation based on Kullback-Leibler contrast are reviewed in Section 2.2. This section is completed by the statement of the general model selection theorem. The main results are stated in Section 3 and a discussion is given in Section 4. Results of Section A.1 recast the control of the bracketing entropy of mixture families into a control of the bracketing entropy of Gaussian density families. This bracketing entropy is upper bounded for Gaussian mixtures with diagonal variance matrices in Appendix A.2 and with general variance matrices in Appendix A.3. The proof of the main results is given in Appendix B.

Model selection principles

Framework

Centered observations y = (y 1 , . . . , y n ), with y i ∈ R Q are assumed to be a sample from a probability distribution with unknown density s. This target s is proposed to be estimated by a finite mixture model in a clustering purpose. Note that s itself is not assumed to be a Gaussian mixture density. Model-based clustering consists of assuming that the data come from a source with several subpopulations, which are modelled separately and the overall INRIA population is a mixture of them. The resulting model is a finite mixture model. When the data are multivariate continuous observations, the parameterized component density is usually a multidimensional Gaussian density. Thus, a Gaussian mixture density with K components is written

K k=1 p k Φ(.|η k , Λ k )
where the p k 's are the mixing proportions (∀k = 1, . . . , K, 0 < p k < 1 and

K k=1 p k = 1) and Φ(.|η k , Λ k ) denotes the Q-dimensional Gaussian density with mean η k and variance matrix Λ k . The parameter vector is (p 1 , . . . , p K , η 1 , . . . , η K , Λ 1 , . . . , Λ K ).
The mixture model is an incomplete data structure model: the complete data are ((y 1 , z 1 ), . . . , (y n , z n )) where the missing data are z = (z 1 , . . . , z n ) with z i = (z i1 , . . . , z iK ) such that z ik = 1 iff y i arises from the component k. The vector z defines an ideal clustering of the data y associated to the mixture model. After an estimation of the parameter vector thanks to the EM algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF], a data clustering is deduced from the maximum a posteriori principle:

ẑik = 1 if pk Φ(y i |η k , Λk ) > pl Φ(y i |η l , Λl ), ∀l = k 0 otherwise.
Currently, statistics deals with problems where data are explained by many variables. In principle, the more information we have about each individual, the better a clustering method is expected to perform. Nevertheless, some variables can be useless or even harmful to obtain a good data clustering. Thus, it is important to take into account the variable role in the clustering process. To this aim, Gaussian mixtures with a specific form are considered. On irrelevant variables, data are assumed to have an homogeneous behavior around the null mean (centered data) allowing not to distinguish a possible clustering. Hence the data density is modelled by a spherical Gaussian joint law with null mean vector on these variables. On the contrary, the different component mean vectors are free on relevant variables. Moreover, the variance matrices restricted on relevant variables are either taken completely free or are chosen in a specified set of definite positive matrices. This idea is now formalized.

Let V be the collection of the nonempty subsets of {1, . . . , Q}. A Gaussian mixture family is characterized by its number of mixture components K ∈ N and its relevant variable index subset v ∈ V whose cardinal is denoted α. In the sequel, the set of index couples (K, v) is M = N × V. Consider the decomposition of a vector x ∈ R Q into its restriction on relevant variables x [v] = (x j1 , . . . , x jα ) and its restriction on irrelevant variables x [v c ] = (x l1 , . . . , x l Q-α ) where v = {j 1 , . . . , j α } and v c = {l 1 , . . . , l Q-α } = {1, . . . , Q}\v. On relevant variables, a Gaussian mixture f is chosen among the following mixture family

L (K,α) = K k=1 p k Φ(.|µ k , Σ k ); ∀k, µ k ∈ [-a, a] α , (Σ 1 , . . . , Σ K ) ∈ ∆ + (K,α) (λ m , λ M ) 0 < p k < 1, K k=1 p k = 1 RR n°6549
where 0 < a, 0 < λ m < λ M and ∆ + (K,α) (λ m , λ M ) denotes a family of K-uples of α × α symmetric definite positive matrices whose eigenvalues belong to the interval [λ m , λ M ]. The family ∆ + (K,α) (λ m , λ M ) is related to the Gaussian mixture shape specified hereafter. The associated set of Gaussian densities composing mixtures of L (K,α) is denoted F (α) . On irrelevant variables spherical Gaussian density g is considered, belonging to the following family

G (α) = Φ(.|0, ω 2 I Q-α ); ω 2 ∈ [λ m , λ M ] . (1)
Finally, the family of Gaussian mixtures associated to (K, v) ∈ M is defined by

S (K,v) = x ∈ R Q → f (x [v] ) g(x [v c ] ); f ∈ L (K,α) , g ∈ G (α) . (2)
The dimension of the model S (K,v) is denoted D(K, v) and corresponds to the free parameter number common to all Gaussian mixtures in this model. It only depends on the number of components K and the number of relevant variables α.

In this paper, two collections of Gaussian mixtures are considered but the same notation S (K,v) is used for the two model collections to make easier the reading of the article.

• For the diagonal collection: the variance matrices Σ k on relevant variables are assumed to be diagonal matrices. Thus the relevant variables are independent conditionally to mixture component belonging. In this context, the set of Gaussian densities composing mixtures of L (K,α) is defined by

F (α) = Φ(.|µ, Σ); µ ∈ [-a, a] α , Σ = diag(σ 2 1 , . . . , σ 2 α ), σ 2 1 , . . . , σ 2 α ∈ [λ m , λ M ] (3)
and the dimension D(K, v) of model S (K,v) is equal to K(2α + 1).

• For the general collection: the variance matrices are assumed to be totally free. The variance matrices belong to the set D + (α) (λ m , λ M ) of α × α positive definite matrices whose eigenvalues are to the interval [λ m , λ M ]. The relevant variables are thus admitted to be correlated conditionally to mixture component belonging. The Gaussian density family composing mixtures is

F (α) = w ∈ R α → Φ(w|µ, Σ), µ ∈ [-a, a] α , Σ ∈ D + (α) (λ m , λ M ) (4)
and the dimension of the family

S (K,v) is equal to D(K, v) = K 1 + α + α(1+α) 2 .
Whereas a mixture with diagonal variance matrices can be seen as an element of the general collection, its number of free parameters is lower than the dimension of the corresponding general model for a couple (K, v). It is interesting to consider the two different collections since the results obtained further are stated in function of the model dimension. Furthermore this distinction allows to have several mixture collections to cluster datasets in practice. Moreover in order to extend the application field, the cases of ordered and nonordered variables are both addressed in this paper. If variables are assumed to be ordered,

INRIA

the relevant variable subset is v = {1, . . . , α} and can be assimilated to its cardinal α. Thus, in order to distinguish between the two cases, Gaussian mixture families are written S (K,α) and their dimensions are denoted D(K, α) when variables are ordered.

These Gaussian mixture families allow to recast clustering and variable selection problems in a global model selection problem. A criterion is now required to select the best model according to the dataset. We propose a penalized criterion using a non asymptotic approach whose principles are reminded in the following section.

Non asymptotic model selection

Density estimation deals with the problem of estimating an unknown distribution corresponding to the observation of a sample y. In many cases, it is not obvious to choose a model of adequate dimension. For instance, a model with few parameters tends to be efficiently estimated whereas it could be far from the true distribution. In the opposite situation, a more complex model easily fits data but estimates have larger variances. The aim of model selection is to construct data-driven criterion to select a model of proper dimension among a given list. A general theory on this topic, with a non asymptotic approach is proposed in the works of Birgé and Massart (see for instance Birgé and Massart, 2001a,b). This model selection principle is now described in our density estimation framework.

Let S be the set of all densities with respect to the Lebesgue measure on R Q . The contrast γ(t, .) = -ln{t(.)} is considered, leading to the maximum likelihood criterion. The corresponding loss function is the Kullback-Leibler information. It is defined for two densities s and t in S by

KL(s, t) = ln s(x) t(x) s(x) dx
if sdx is absolutely continuous with respect to tdx and +∞ otherwise. Noticing that being the unique minimizer of the Kullback-Leibler function on S, s satisfies

s = argmin t∈S -ln{t(x)}s(x) dx.
Consequently, s is also a minimizer over S of the expectation of the empirical contrast defined by

γ n (t) = - 1 n n i=1 ln {t(y i )} .
A minimizer of the empirical contrast γ n over a model S, a subspace of S, is denoted ŝ. Substituting the empirical criterion γ n to its expectation and minimizing γ n on S, it is expected to obtain a sensible estimator of s, at least if s belongs (or is close enough) to model S.

A countable collection of models (S m ) m∈M with a corresponding collection (ŝ m ) m∈M of estimators is now considered. The best model is the one presenting the smallest risk

m(s) = argmin m∈M E[KL(s, ŝm )].
However, the function ŝm(s) , called oracle, is unknown since it depends on the true density s. Nevertheless, this oracle is a benchmark: a data-driven criterion is found to select an estimator such that its risk is close to the oracle risk. The model selection via penalization procedure consists of considering some proper penalty function pen : m ∈ M → pen(m) ∈ R + and of selecting m minimizing the associated penalized criterion

crit(m) = γ n (ŝ m ) + pen(m).
The resulting selected estimator is ŝ m. The final purpose of this non asymptotic approach is to obtain a penalty function and an associated oracle inequality, allowing to compare the risk of the penalized MLE ŝ m with the benchmark inf

m∈M E[KL(s, ŝm )].
Commonly, in order to find a suitable penalty function, one begins by writing the following inequality (see Massart, 2007, p.9): For all m ∈ M and

s m ∈ S m , KL(s, ŝ m) ≤ KL(s, s m ) + pen(m) -pen( m) + γn (s m ) -γn (ŝ m)
where γn is the centered empirical process defined by γn

(t) = γ n (t) -E[γ n (t)].
The penalty function has to be chosen to annihilate the fluctuation of γn (s m ) -γn (ŝ m). The aim is to obtain an uniform control of γn (s m ) -γn (ŝ m ) with respect to m in M. This quantity is controlled by its expectation using a Talagrand's inequality [START_REF] Talagrand | Concentration of measure and isoperimetric inequalities in product spaces[END_REF][START_REF] Talagrand | New concentration inequalities in product spaces[END_REF][START_REF] Massart | Concentration inequalities and model selection[END_REF], for an overview). Next, two different situations occur. In some situations, the expectation in the Talagrand's inequality can be efficiently connected to the model dimension, and an oracle inequality with explicit constants is deduced. This is the case in the context of histogram density estimation [START_REF] Castellan | Modified Akaike's criterion for histogram density estimation[END_REF] and of density estimation via exponential model [START_REF] Castellan | Density estimation via exponential model selection[END_REF]. For situations when these sharp calculations are impossible to obtain, Massart (2007, Section 7.4) proposes a general theorem which gives the form of penalties and associated oracle inequalities in terms of the Kullback-Leibler and Hellinger losses. This theorem is based on the centered process control with the bracketing entropy, allowing to evaluate the "size" of models. For Gaussian mixture models, we can only follow the second alternative because of the non linear behavior of the logarithm function on Gaussian mixture densities. Another requirement to apply concentration inequalities is to fulfill an hypothesis of boundness as γn (s m ) -γn (t) ∞ is bounded by a constant for all t ∈ S m . But in our context, it is impossible to bound uniformly all the ratios of two Gaussian mixtures. After remembering the definition of the Hellinger distance and specifying some notation, this general MLE selection model theorem (Massart, 2007, Theorem 7.11) is stated in a restricted form, which is sufficient for our study.

The norm √ f -√ g 2 between two nonnegative functions f and g of L 1 is denoted

d H (f, g).
We note that if f and g be two densities with respect to the Lebesgue measure on R Q , d H (f, g) is the Hellinger distance between f and g. In the following, d H (f, g) is improperly called Hellinger distance even if f and g are not density functions. An εbracketing for a subset S of S with respect to d H is a set of integrable function pairs (l 1 , u 1 ), . . . , (l N , u N ) such that for each f ∈ S, there exists j ∈ {1, . . . , N } such that l j ≤ f ≤ u j and d H (l j , u j ) ≤ ε. The bracketing number N [.] (ε, S, d H ) is the smallest number

INRIA

of ε-brackets necessary to cover S and the bracketing entropy is defined by

H [.] (ε, S, d H ) = ln N [.] (ε, S, d H ) .
Since S is the density set, the bracket extremities can be chosen as nonnegative functions in L 1 . Let (S m ) m∈M be some at most countable collection of models, where for each m ∈ M, the elements of S m are assumed to be probability densities with respect to Lebesgue measure. Firstly, the following separability assumption allows to avoid measurability problems. For each model S m , assume that there exists some countable subset S m of S m such that for all t ∈ S m , there exists a sequence (t k ) k≥1 of elements of S m such that for x ∈ R Q , ln{t k (x)} tends to ln{t(x)} when k tends to infinity. Secondly H [.] (ε, S m , d H ) is assumed to be integrable at 0 for each m and we also assume that there exists a function Ψ m on R + fulfilling the following properties

[I]: Ψ m is nondecreasing, x → Ψ m (x)/x is nonincreasing on ]0, +∞[ and for ξ ∈ R + and all u ∈ S m , denoting S m (u, ξ) = {t ∈ S m ; d H (t, u) ≤ ξ}, ξ 0 H [.] (x, S m (u, ξ), d H ) dx ≤ Ψ m (ξ).
Theorem 1 [START_REF] Massart | Concentration inequalities and model selection[END_REF]). Let y 1 , . . . , y n be i.i.d. random variables with unknown density s with respect to Lebesgue measure on R Q . Let (S m ) m∈M be some at most countable collection of models fulfilling the previous properties and let (ŝ m ) m∈M be the corresponding collection of MLEs. Let (ρ m ) m∈M be some family of nonnegative numbers such that m∈M e -ρm = Υ < ∞.

(5)

For every m ∈ M, considering Ψ m with properties [I], ξ m denotes the unique positive solution of the equation

Ψ m (ξ) = √ n ξ 2 .
Let pen : M → R + and consider the penalized loglikelihood criterion

crit(m) = γ n (ŝ m ) + pen(m).
Then, there exists some absolute constants κ and C such that whenever for all m ∈ M,

pen(m) ≥ κ ξ 2 m + ρ m n
some random variable m minimizing crit over M does exist and moreover, whatever the density s,

E d 2 H (s, ŝ m) ≤ C inf m∈M {KL(s, S m ) + pen(m)} + Υ n , (6)
where KL(s, S m ) = inf t∈Sm KL(s, t) for every m ∈ M.

RR n°6549

Inequality ( 6) is not exactly an oracle inequality since the Hellinger risk is upper bounded by the Kullback bias KL(s, S m ). Nevertheless, this last term is of the order of d 2 H (s, S m ) if ln( s/t ∞ ) is uniformly bounded on ∪ m∈M S m according to Lemma 7.23 in [START_REF] Massart | Concentration inequalities and model selection[END_REF]. In our context, this condition can be achieved even if it means assuming that all densities are defined on compact support.

Main results

As announced previously, Theorem 1 is applied to our specific framework described in Section 2.1. The ensuing theoretical results are now addressed, for the ordered and non-ordered variable cases separately. These provide a non asymptotic penalized criterion to select the number of clusters K and the variable subset v used in the Gaussian mixtures. Moreover, these results give an oracle inequality which is fulfilled by the associated penalized estimator.

Ordered variable case

In this section, variables are assumed to be ordered and we recall that the model collection is denoted (S (K,α) ) (K,α)∈M in this case. In the two types of Gaussian mixtures, the following theorem gives the form of penalty functions and the associated oracle inequalities for this model collection.

Theorem 2. For the diagonal and general mixture collections, there exists two absolute constants κ and C such that, if

pen(K, α) ≥ κ D(K, α) n 1 + 2 A + ln 1 1 ∧ D(K,α) n A where A is a function of Q, λ m , λ M and a such that A = O( √ ln Q) as Q goes to infinity , then the model ( K, α) minimizing crit(K, α) = γ n (ŝ (K,α) ) + pen(K, α) over M exists and E d 2 H (s, ŝ( K, α) ) ≤ C inf (K,α)∈M {KL(s, S (K,α) ) + pen(K, α)} + 1 n .
This theorem is proved in Section B.1. It requires to control the bracketing entropy of Gaussian mixture families. In Section A.1, this problem is recast into the control for Gaussian density families. Section A.2 and Section A.3 are then devoted to the bracketing entropy control of Gaussian density families in the diagonal and the general cases respectively. Note that in order to apply Theorem 1, the local bracketing entropy H [.] (x, S (K,α) (u, ξ), d H ) has to be controlled. Nevertheless, it is difficult to characterize the subset S (K,α) (u, ξ) in INRIA function of the parameters of its mixtures. Therefore a global study of the entropy bracketing is proposed in the theorem proof since H

[.] (x, S (K,α) (u, ξ), d H ) ≤ H [.] (x, S (K,α) , d H ).
Several remarks can be given about this result. First, the deduced penalty function has an expected form since it is proportional to the model dimension D(K, α). This shows the interest of considering separately the two collections since the model dimensions are different. For the diagonal mixture family, the risk bound is more accurate when this family is the diagonal collection than if it is considered as a subset of the general collection. Second, the constant A is made explicit in the theorem proof (see Section B.1) and its expression is different between the diagonal case and the general case (see Equations ( 22) and ( 23)). In the both cases, it depends on parameters λ m , λ M , a and Q with A = O( √ ln Q) as Q goes to infinity. This number of variables Q has to have a reasonable order in the constant A so that the upper bound in the oracle inequality remains meaningful. Contrary to classical criteria for which Q is fixed and n tends to infinity, our result allows to study cases for which Q increases with n. For specific clustering problems where the number of variables Q is of the order of n or even larger than n, the oracle inequality is still significant. Thirds, since the multiplicative constants are not explicit, a practical method is necessary. This will be addressed in the discussion section.

Non-ordered variable case

Theorem 2 can be generalized to the non-ordered variable case. In this context, a model S (K,v) is characterized by its number of mixture components K and its subset v ∈ V of relevant variable indexes. This model is related to the model S (K,α) of the ordered case by

S (K,v) = {x ∈ R Q → f • τ (x), f ∈ S (K,α) }
where τ is a permutation such that (τ (x) 1 , . . . , τ (x) α ) = x [v] . Moreover, the dimension D(K, v) of S (K,v) is equal to D(K, α). Consequently, the model S (K,v) has the same complexity as S (K,α) and thus has the same bracketing entropy. However, the model set {S (K,v) } (K,v)∈M contains more models per dimension than in the ordered case. This richness of the model family involves to define following new weights in penalty function:

ρ (K,v) = D(K, v) 2 ln 8eQ {D(K, v) -1} ∧ (2Q -1)
.

Consequently, in the following theorem which is the analog of Theorem 2 for the non-ordered case, the associated penalty functions have an additional logarithm term depending on the dimension.

Theorem 3. For both diagonal and general mixture collections, there exists two absolute constants κ and C such that, if

pen(K, v) ≥ κ D(K, v) n 2 A + ln 1 1 ∧ D(K,v) n A + 1 2 ln 8eQ {D(K, v) -1} ∧ (2Q -1) RR n°6549
where A is the same constant as ordered case then the model ( K, v) minimizing crit(K, v) = γ n (ŝ (K,v) ) + pen(K, v) on M exists and

E d 2 H (s, ŝ( K,v) ) ≤ C inf (K,v)∈M {KL(s, S (K,v) ) + pen(K, v)} + 2 n .
The theorem proof given in Section B.2 only consists of justifying the form of new weights and finding an upper bound of the weight sum since S (K,v) has the same bracketing entropy as S (K,α) . This non-ordered case is more attractive for practical use but this result is difficult to apply when the number of variables becomes too large since an exhaustive research of the best model is then untractable.

Discussion

In this paper, specific Gaussian mixtures are considered to take into account the role of variables in the clustering process. Main results are stated for diagonal and general Gaussian mixture forms for ordered and non-ordered variables. A data-driven penalized criterion is proposed to select the number of clusters and the clustering relevant variable subset. Oracle inequalities satisfied by the associated estimator ŝ( K,v) are also obtained. The main interest of these results is to give the shape of an adequate penalty in this particular framework in which loglikelihoods are difficult to control. Proofs of these results require to control the bracketing entropy of multidimensional Gaussian density families and to determinate weights taking into account the richness of the model collection. Similar results for non-Gaussian mixtures can be obtained as soon as the bracketing entropy of the new component density family can be controlled.

Usually, the Gaussian mixture clustering problem is recast as a selection problem of the number of mixture components and besides, of the mixture shape among a mixture shape collection. A complete collection of twenty eight parsimonious models is available, used for instance in Mixmod software [START_REF] Biernacki | Model-based cluster and discriminant analysis with the mixmod software[END_REF]. These models are obtained by imposing conditions on the proportions and the elements of variance matrix eigenvalue decomposition (see [START_REF] Banfield | Model-based Gaussian and non-Gaussian clustering[END_REF][START_REF] Celeux | Gaussian parsimonious clustering models[END_REF]. Commonly, an asymptotic criterion as BIC [START_REF] Schwarz | Estimating the dimension of a model[END_REF] or ICL [START_REF] Biernacki | Assessing a mixture model for clustering with the integrated completed likelihood[END_REF] is used to solve this model selection problem. In this paper, our main results allows to propose a non asymptotic criterion to select the number of clusters (the subset v is fixed to the complete variable set). Moreover, we focus on two mixture forms but similar results can be stated for several of the mixture shapes. It is thus possible to obtain a non asymptotic criterion which besides allows to select the mixture form. A comparison of our criterion with BIC, ICL and AIC is proposed in our framework in [START_REF] Maugis | Slope heuristics for variable selection and clustering via Gaussian mixtures[END_REF] and also for the selection of the mixture component number in [START_REF] Baudry | Clustering through model selection criteria[END_REF].

For practical purposes, theoretical results stated in this paper are not immediately usable since they depend on unknown constants and mixture parameters are not bounded. Nevertheless, they are required to justify the shape of penalties and ensure that the number of variables Q can be large. [START_REF] Birgé | Minimal penalties for Gaussian model selection[END_REF] propose their so-called "slope INRIA heuristics" (see also Massart, 2007, Section 8.5) to calibrate these constants. This heuristics consists of assuming that twice the minimal penalty is almost the optimal penalty. Theoretically, this rule of thumb is proved in [START_REF] Birgé | Minimal penalties for Gaussian model selection[END_REF] in the framework of Gaussian regression in a homoscedastic fixed design and generalized by [START_REF] Arlot | Slope heuristics for heteroscedastic regression on a random design[END_REF] in the heteroscedastic random design case for histograms. The slope heuristics is also the subject of several practical studies. For example, it has been successfully applied for multiple change point detection by [START_REF] Lebarbier | Detecting multiple change-points in the mean of gaussian process by model selection[END_REF], for clustering [START_REF] Baudry | Clustering through model selection criteria[END_REF], for Gaussian Markov random fields [START_REF] Verzelen | Adaptive estimation of regular Gaussian Markov random fields[END_REF], for estimation of oil reserves [START_REF] Lepez | Potentiel de réserves d'un bassin pétrolier : modélisation et estimation[END_REF] and genomics [START_REF] Villers | Tests et sélection de modèles pour l'analyse de données protéomiques et transcriptomiques[END_REF]. In the context of Gaussian mixtures, this heuristics is carried out on simulated and real datasets in [START_REF] Maugis | Slope heuristics for variable selection and clustering via Gaussian mixtures[END_REF]. It is shown that the developed procedure allows to obtain efficient clusterings and variable selection, for instance in the curve clustering context.

Appendices

A Tools: bound on bracketing entropies

A.1 The bracketing entropy of mixture density family

In order to use Theorem 1, it is necessary to control the bracketing entropy of Gaussian mixture families to define a suitable function ψ (K,α) fulfilling properties [I]. [START_REF] Ghosal | Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities[END_REF] and [START_REF] Genovese | Rates of convergence for the Gaussian mixture sieve[END_REF] have proposed an upper bound of the bracketing entropy of unidimensional Gaussian mixtures in order to obtain convergence rates in Hellinger distance for density estimation using the Gaussian mixtures. We first tried to follow the strategy proposed in [START_REF] Ghosal | Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities[END_REF] to control the bracketing entropy of our multidimensional Gaussian mixture models. But the control obtained this way has a too large dependency in Q. We propose instead a method inspired by the work of [START_REF] Genovese | Rates of convergence for the Gaussian mixture sieve[END_REF]. They state the next theorem allowing to bound the bracketing number of a mixture set according to the product of the bracketing numbers of the associated mixture component families. For all k in {1, . . . , K}, let C k = {f θ k , θ k ∈ Θ k } be a family of densities with respect to Lebesgue measure on R Q . The following family of mixture distributions based on C k is considered

W K := K k=1 p k f θ k , θ k ∈ Θ k ∀k = 1, . . . , K, p = (p 1 , . . . , p K ) ∈ T K-1
where T K-1 is the K -1 dimensional simplex defined by T K-1 := p = (p 1 , . . . , p K ), ∀k = 1, . . . , K, p k ≥ 0,

K k=1 p k = 1 .
Theorem 4. With the previous notation, for all K and all ε ∈ (0, 1],

N [.] (ε, W K , d H ) ≤ K(2π e) K 2 3 ε K-1 K k=1 N [.] ε 3 , C k , d H .
Here we want to take into account the specific form of the multidimensional mixtures studied in this paper. A new result is deduced from Theorem 4 for the general and diagonal cases. For all f ∈ S (K,v) and x ∈ R Q , Equation (2) gives that

f (x) = Φ(x [v c ] |0, ω 2 I Q-α ) K k=1 p k Φ(x [v] |µ k , Σ k ) INRIA where Φ(.|0, ω 2 I Q-α ) belongs to G (α)
and where Gaussian densities Φ(.|µ k , Σ k ) belong to F (α) (see Section 2.1). In order to bound the bracketing entropy of the mixture family S (K,v) by a sum with the bracketing entropies of the simplex, of G (α) and of F (α) , the following proposition is stated.

Proposition 1. For the diagonal and general cases, for all ε ∈ (0, 1], the bracketing number of the density family S (K,v) is bounded by

N [.] (ε, S (K,v) , d H ) ≤ K(2π e) K 2 9 ε K-1 N [.] ε 3 , G (α) , d H N [.] ε 9 , F (α) , d H K .
Proposition 1 allows to deduce that

H [.] (ε, S (K, v) , d H ) ≤ C(K)+(K -1) ln 1 ε +H [.] ε 3 , G (α) , d H +K H [.] ε 9 , F (α) , d H (7)
with C(K) = ln(K) + K 2 ln(2πe) + (K -1) ln( 9). Therefore, Inequality (7) recasts the bracketing entropy control problem of S (K,v) in the bracketing entropy control of the two families F (α) and G (α) . Bracketing entropy upper bounds of these two sets are assessed for two mixture forms in Section A.2 and Section A.3.

Proof. According to Theorem 4, for all δ ≤ 1,

N [.] (δ, L (K,α) , d H ) ≤ K(2πe) K 2 3 δ K-1 K k=1 N [.] δ 3 , F (α) , d H .
If we prove that for all ε ≤ 1,

N [.] (ε, S (K,v) , d H ) ≤ N [.] ε 3 , G (α) , d H N [.] ε 3 , L (K,α) , d H (8)
then we obtain the result

N [.] (ε, S (K,v) , d H ) ≤ K(2π e) K 2 9 ε K-1 N [.] ε 3 , G (α) , d H N [.] ε 9 , F (α) , d H K .
Thus, it remains to check Inequality (8). It is done by the following adaptation of a result proof given by [START_REF] Genovese | Rates of convergence for the Gaussian mixture sieve[END_REF].

Let δ ∈ [0, 1] and h ∈ S (K,v) , decomposed into h(x) = f (x [v] )g(x [v c ]
) where f ∈ L (K,α) and g ∈ G (α) . Let [l, u] and [ l, ũ] be two δ-brackets of L (K,α) and G (α) containing f and g respectively. Then, the two functions defined by

L(x) = l(x [v] ) l(x [v c ] ) and U (x) = u(x [v] )ũ(x [v c ] ) (9)
constitute a bracket of S (K,v) containing h. The size of this bracket is now calculated. First of all, Lemma 3 from [START_REF] Genovese | Rates of convergence for the Gaussian mixture sieve[END_REF] gives that

u(x [v] )dx [v] ≤ 1 + 3δ ũ(x [v c ] )dx [v c ] ≤ 1 + 3δ. ( 10
)
Then the squared Hellinger distance between L and U is equal to

d 2 H (L, U ) = u(x [v] )ũ(x [v c ] ) -l(x [v] ) l(x [v c ] ) 2 dx = ũ(x [v c ] ) u(x [v] ) -l(x [v] ) + ũ(x [v c ] ) -l(x [v c ] ) l(x [v] ) 2 dx = ũ(x [v c ] ) dx [v c ] u(x [v] ) -l(x [v] ) 2 dx [v] + ũ(x [v c ] ) -l(x [v c ] ) 2 dx [v c ] l(x [v] ) dx [v] + 2 ũ(x [v c ] ) ũ(x [v c ] ) -l(x [v c ] ) dx [v c ] u(x [v] ) -l(x [v] ) l(x [v] ) dx [v] .
According Cauchy-Schwarz inequality and (10),

u(x [v] ) -l(x [v] ) l(x [v] ) dx [v] ≤ 1 × d H (l, u) ≤ δ and ũ(x [v c ] ) ũ(x [v c ] ) -l(x [v c ] ) dx [v c ] ≤ √ 1 + 3δ × d H ( l, ũ) ≤ 2 δ .
Thus,

d 2 H (L, U ) ≤ d 2 H (l, u) ũ(x [v c ] ) dx [v c ] + d 2 H ( l, ũ) + 4δ 2 ≤ (1 + 3δ) δ 2 + δ 2 + 4δ 2 ≤ 9δ 2 .
Finally, with δ = ε/3 and according to the bracket definition ( 9), the number of brackets for

S (K,v) is upper bounded by N [.] (ε, S (K,v) , d H ) ≤ N [.] ε 3 , G (α) , d H × N [.] ε 3 , L (K,α) , d H .

A.2 The bracketing entropy of Gaussian mixture family with diagonal variance matrices

In this section, we consider the case where variance matrices of the Gaussian densities in the mixtures are diagonal. The following proposition gives an upper bound of the bracketing entropy of the two families F (α) and G (α) defined by ( 3) and ( 1) respectively. It allows us to deduce an upper bound of the bracketing entropy of S (K,v) according to Inequality (7).

INRIA Proposition 2. Set c 1 = 5 1 -2 -1 4 /8. For all ε ∈ (0, 1], H [.] (ε, F (α) , d H ) ≤ α ln 2 a 2 c 1 λ m + α ln 8 λ M λ m + 2α ln( √ 2 Q) + 2α ln 1 ε (11)
and

H [.] (ε, G (α) , d H ) ≤ ln 8 λ M λ m + ln √ 2 Q + ln 1 ε . ( 12
)
Thus,

H [.] (ε, S (K,α) , d H ) ≤ C(K) + α(2Kα + 1) ln(9 √ 2 Q) + (Kα + 1) ln 8 λ M λ m +Kα ln a 8 c 1 λ m + D(K, α) ln 1 ε ( 13
)
where C(K) = ln(K) + K 2 ln(2πe) + (K -1) ln( 9). Proof. According to Assertion ( 7), the upper bound on the bracketing entropy of S (K,α) , given by Inequality ( 13), is deduced from upper bounds on the bracketing entropy of F (α) and G (α) , respectively expressed in Inequalities ( 11) and ( 12). These two inequalities are now proved successively.

The proof of Inequality ( 11) is adapted from [START_REF] Genovese | Rates of convergence for the Gaussian mixture sieve[END_REF] who prove similar results for unidimensional Gaussian mixture families. The main idea is to define a lattice over the parameter space

B = {(µ, σ 2 1 , . . . , σ 2 α ) ∈ [-a, a] α × [λ m , λ M ] α
} and next to deduce a bracket covering of F (α) according to the Hellinger distance.

First, consider ε ∈ (0, 1] and δ = ε/( √ 2Q). For all j ∈ {2, . . . , r}, set

b 2 j = (1 + δ) 1-j 2 λ M with r = 2 ln n λ M (1+δ) λm o ln(1+δ) in order to have b 2 r ≤ λ m < λ M = b 2 2 .
h denotes the smallest integer greater than or equal to h. Then, for all J = (j(1), . . . , j(α)) ∈ {2, . . . , r} α , a diagonal matrix B J is defined by

B J = diag(b 2 j(1) , . . . , b 2 j(α) ) .
We also consider vectors ν J = (ν

(J) 1 , . . . , ν (J) α ) ∈ [-a, a] α such that ∀q ∈ {1, . . . , α}, ν (J) q = c 1 λ M δ (1 + δ) 1-j(q) 4 s q , where s q ∈ Z ∩ [-A, A] with A = a δ -1 (1+δ) -1-j(q) 4 √ c1 λM
. Thus, the set R(ε, α) of all such couples (ν J , B J ) forms a lattice on B.

This set R(ε, α) allows to construct brackets that cover F (α) . For a function f (.) = Φ(.|µ, Σ) of F (α) , the two following functions are considered:

l(x) = (1 + δ) -α Φ(x|ν J , (1 + δ) -1 4 B J+1 ) u(x) = (1 + δ) α Φ(x|ν J , (1 + δ) B J ).
The index set J = (j(1), . . . , j(α)) is taken to satisfy b 2 j(q)+1 ≤ σ 2 q ≤ b 2 j(q) for all q in {1, . . . , α} and ν J can be chosen such that

(µ -ν J ) B -1 J+1 (µ -ν J ) ≤ c 1 α δ 2 (14)
where J + 1 := (j(1) + 1, . . . , j(α) + 1). Then we check that the bracket [l, u] contains f . Inequality ( 14) implies that

(µ -ν J ) B -1 J (µ -ν J ) ≤ α 4 δ 2 . ( 15
)
The use of Corollary 2, which allows to bound the ratio of two Gaussian densities with diagonal variance matrices, together with (15) leads to

f (x) u(x) = Φ(x|µ, B) (1 + δ) α Φ(x|ν J , (1 + δ) B J ) ≤ (1 + δ) -α 4 exp 1 2 δ (µ -ν J ) B -1 J (µ -ν J ) ≤ 1. The function h : δ → 1 -(1 + δ) -1 4 being concave, it yields 1 -(1 + δ) -1 4 ≥ δ(1 -2 -1 4
). With Corollary 2 and ( 14), this shows that l ≤ f since

l(x) f (x) = (1 + δ) -α Φ(x|ν J , (1 + δ) -1 4 B J+1 ) Φ(x|µ, B) ≤ (1 + δ) -5α 8 exp (µ -ν J ) B -1 J+1 (µ -ν J ) 2[1 -(1 + δ) -1 4 ] ≤ 1.
Therefore, [l, u] contains the function f . To prove that [l, u] is an ε-bracket, it remains to check that d H (l, u) ≤ ε. According to Corollary 3,

d 2 H (l, u) = d 2 H (1 + δ) -α Φ(.|ν J , (1 + δ) -1 4 B J+1 ), (1 + δ) α Φ(x|ν J , (1 + δ) B J ) = (1 + δ) -α + (1 + δ) α -2 2 (1 + δ) -7 8 + (1 + δ) 7 8 α 2 = 2 cosh(α ln[1 + δ]) -2 (i) + 2 -2 cosh 7 8 ln(1 + δ) -α 2 (ii)
.

INRIA

The upper bounds of terms (i) and (ii) separately lead to

d 2 H (l, u) ≤ sinh(1) + 49 128 α 2 δ 2 ≤ 2 α 2 δ 2 ≤ ε 2 .
Consequently, the parameter family R(ε, α) induces an ε-bracketing family over F (α) .

An upper bound on the bracketing number of F (α) is then deduced from an upper bound of the cardinal of R(ε, α)

N [.] (ε, F (α) , d H ) ≤ Card(R(ε, α)) ≤ J∈{2,...,r} α α q=1 2 a √ c 1 λ M δ (1 + δ) 1-j(q) 4 ≤ 2 a(1 + δ) r-1 4 √ c 1 λ M δ α (r -1) α .
According to the definition of r, (1 + δ)

r-1 4 ≤ λ M (1 + δ)/λ m . Hence, N [.] (ε, F (α) , d H ) ≤ 2 a δ 1 + δ c 1 λ m α   2 ln λM λm (1 + δ) ln(1 + δ)   α ≤ 2 √ 2 a √ c 1 λ m α 8 λ M λ m α δ -(2α) ≤ 2 √ 2 a √ c 1 λ m α 8 λ M λ m α √ 2Q ε 2α
that implies Inequality (11). Using a similar proof, the upper bound of the bracketing entropy of G (α) given by Inequality ( 12) is obtained. To check this result, the variance family

{b 2 j = (1 + δ) 1-j 2 λ M , ∀2 ≤ j ≤ r} and brackets [ l, ũ] defined on R Q-α by l(x) = (1 + δ) -(Q-α) Φ(x|0, (1 + δ) -1 4 b 2 j+1 I Q-α ) ũ(x) = (1 + δ) Q-α Φ(x|0, (1 + δ) b 2 j I Q-α ) are considered.
A.3 The bracketing entropy of Gaussian mixture family with general variance matrices

We now consider the case of general Gaussian mixture collection. The following proposition gives an upper bound of the bracketing entropy of the Gaussian density family F (α) defined by (4).

Proposition 3. For all ε ∈ (0, 1],

H [.] ε, F (α) , d H ≤ α(α + 1) 2 ln 6 √ 3 λ M λ m + α ln 6 a √ λ m + α(α + 1) 2 + α ln(Q 2 ) + α(α + 1) 2 + α ln 1 ε .
This result together with Inequality (7) and the upper bound of G (α) (see Inequality ( 12)) gives the following corollary.

Corollary 1. For all ε ∈ (0, 1],

H [.] (ε, S (K,α) , d H ) ≤ C(K) + ln 8 √ 2 λ M λ m + K α(α + 1) 2 ln 54 √ 3 λ M λ m + Kα ln 54 a √ λ m + V (K, α) ln(Q 2 ) + D(K, α) ln 1 ε
where C(K) = ln(K) + K 2 ln(2πe) + (K -1) ln(9) and V (K, α) = K α(α+1)

2

+ Kα + 1.

To prove Proposition 3, the method used in the diagonal case cannot be extended to this general situation. Considering the eigenvalue decomposition of the variance matrices, a countable covering on the spectrum could be build as in the diagonal case. An explicit countable covering over the orthogonal matrix set is also necessary to obtain an upper bound of the bracketing entropy of F (α) . Nevertheless, this last point is tricky thus an alternative method is proposed. It consists of defining an adequate covering over the space D + (α) (λ m , λ M ) with respect to the uniform norm, and then of using it to construct a bracket covering of F (α) . The following notation is used for matrix norms:

B ∞ = max 1≤i,j≤α |B ij | and |||B||| = sup x 2=1 |x Bx| = sup λ∈vp(B)
|λ| where vp(B) denotes the spectrum of B.

The variance matrix lattice

Let β > 0 and let R(β) be a β-covering on D + (α) (λ m , λ M ) for the uniform norm . ∞ , composed of symmetric matrices and defined by

R(β) = A = (A ij ) 1≤i,j≤α ; A ij = a ij β; a ij = a ji ∈ Z ∩ - λ M β , λ M β .

INRIA

Thus, for all Σ in D + (α) (λ m , λ M ), there exists A in R(β) such that

A -Σ ∞ ≤ β. (16)
The following lemma allows to compare the eigenvalues of Σ with respect to those of its associated matrix A.

Lemma 1. Let Σ ∈ D + (α) (λ m , λ M ) and A ∈ R(β) such that Σ -A ∞ ≤ β.
Let λ 1 , . . . , λ α and τ 1 , . . . , τ α be respectively the eigenvalues of Σ and A, ranked in increasing order and counted with their multiplicity.Then, for all q ∈ {1, . . . , α}, τ q -βα ≤ λ q ≤ τ q + βα.

Proof. Since Σ -A ∞ ≤ β, we have |||Σ -A||| ≤ βα. Moreover, according to Theorem of Rayleigh, given for instance in Serre (2002, Theorem 3.3.2 p49),

λ q = min dim(F )=q max x∈F \{0} x Σx x 2 2 and τ q = min dim(F )=q max x∈F \{0} x Ax x 2 2
where F is a linear subspace of R α . Then, for all q ∈ {1, . . . , α}, τ q -βα ≤ λ q ≤ τ q + βα.

Covering F (α) with a family of ε-brackets Based on the set R(β), ε-brackets for the Gaussian density family F (α) are now constructed. Consider f = Φ(.| µ, Σ) be a function of F (α) with µ ∈ [-a, a] α and Σ ∈ D + (α) (λ m , λ M ). For β > 0, there exists a matrix A ∈ R(β) such that A -Σ ∞ ≤ β according to (16). Then the two following functions are considered

u(x) = (1 + 2δ) α Φ (x| ν, (1 + δ)A) (17) and l(x) = (1 + 2δ) -α Φ x| ν, (1 + δ) -1 A (18)
where the vector ν and the positive number δ are adjusted later in order that [l, u] is an ε-bracket of F (α) containing the function f . Next lemma allows to fulfill hypothesis necessary to use Proposition 6. The resulting bounds on Gaussian density ratios are given in Lemma 3.

Lemma 2. Assume that 0 < β < λ m /(3α) and set δ = 3βα/λ m . Then, (1 + δ)A -Σ and Σ -(1 + δ) -1 A are both positive definite matrices. Moreover, for all x in R α ,

x {(1 + δ)A -Σ}x ≥ βα x 2 2 (19) and x {Σ -(1 + δ) -1 A}x ≥ βα x 2 2 . ( 20
)
Proof. For all x = 0, since |||A -Σ||| ≤ αβ,

x {(1 + δ)A -Σ}x = (1 + δ)x (A -Σ)x + δx Σx ≥ -(1 + δ) |||A -Σ||| x 2 2 + δ λ m x 2 2 ≥ {δ λ m -(1 + δ)αβ} x 2 2 ≥ 2 3 δ λ m -αβ x 2 2 because αβ ≤ λ m /3. Then x {(1 + δ)A -Σ}x ≥ αβ x 2 2 > 0 according to the definition of δ. Similarly, x {Σ -(1 + δ) -1 A}x = (1 + δ) -1 x (Σ -A)x + {1 -(1 + δ) -1 }x Σx ≥ δ λ m -αβ 1 + δ x 2 2 ≥ 2αβ 1 + δ x 2 2 ≥ αβ x 2 2 > 0.
Lemma 3. Assume that β < λ m /(3α) and set δ = 3βα/λ m . Then,

f (x) u(x) ≤ (1 + 2δ) -α 2 exp µ -ν 2 2 2βα and l(x) f (x) ≤ (1 + 2δ) -α 2 exp µ -ν 2 2 2βα .
Proof. According to Proposition 6, since (1 + δ)A -Σ is a positive definite matrix from Lemma 2,

f (x) u(x) ≤ (1 + 2δ) -1 |(1 + δ)A| |Σ| exp 1 2 (µ -ν) {(1 + δ)A -Σ} -1 (µ -ν) . Inequality (19) implies that |||{(1 + δ)A -Σ} -1 ||| = {inf λ} -1 ≤ (βα) -1
where the infimum is taken over all eigenvalues of (1 + δ)A -Σ. Then, since

(µ -ν) {(1 + δ)A -Σ} -1 (µ -ν) ≤ |||{(1 + δ)A -Σ} -1 ||| µ -ν 2 2 ,
this leads to

(µ -ν) {(1 + δ)A -Σ} -1 (µ -ν) ≤ µ -ν 2 2 αβ .

INRIA

Moreover, according to Lemma 1,

|(1 + δ)A| |Σ| = (1 + δ) α α q=1 τ q λ q ≤ (1 + δ) α α q=1 1 + βα λ q ≤ (1 + δ) α 1 + βα λ m α ≤ (1 + 2δ) α . Then f (x) u(x) ≤ (1 + 2δ) -α 2 exp µ -ν 2 2 2βα .
Similarly, using Proposition 6, (20) and Lemma 1, we obtain

l(x) f (x) ≤ (1 + 2δ) -α 2 exp µ -ν 2 2 2βα .
Next proposition finishes the construction of an ε-bracket covering of F (α) .

Proposition 4. For all ε ∈ (0, 1], we define δ = ε/( √ 3 α) and

β = λ m ε/(3 √ 3α 2 ). The following set [l, u]; u(x) = (1 + 2δ) α Φ (x| ν, (1 + δ)A) l(x) = (1 + 2δ) -α Φ x| ν, (1 + δ) -1 A ; A ∈ R(β), ν ∈ X (ε, a, λ m , α)
where

X (ε, a, λ m , α) = ν = (ν 1 , . . . , ν α ); ν q = √ λ m ε 3α s q ; s q ∈ Z ∩ - 3 a α √ λ m ε , 3 a α √ λ m ε , is an ε-bracket set over F (α) . Proof. Let f (x) = Φ(x|µ, Σ) be a function of F (α) where µ ∈ [-a, a] α and Σ ∈ D + (α) (λ m , λ M ). There exists A in R(β) such that Σ -A ∞ ≤ β and ν in X (ε, a, λ m , α) satisfying, for all q in {1, . . . , α}, |µ q -ν q | ≤ √ λ m ε/(3α).
Consider the two associated functions l and u defined in ( 17) and ( 18) respectively. Since µ -ν 2 2 ≤ λ m ε 2 /(9α), using Lemma 3,

f (x) u(x) ≤ (1 + 2δ) -α 2 exp √ 3 ε 6 .
Thus, noting that for all x in [0, 2], ln(1 + x) ≥ x/2, it leads to

ln f (x) u(x) ≤ - α 2 ln 1 + 2 ε √ 3α + √ 3 ε 6 ≤ - α 2 ε √ 3α + ε 2 √ 3 ≤ 0.
Similarly, ln {l(x)/f (x)} ≤ 0 and thus for all x ∈ R α , l(x) ≤ f (x) ≤ u(x). It remains to bound the size of bracket [l, u] with respect to Hellinger distance. According to Proposition.7,

d 2 H (l, u) = (1 + 2δ) α + (1 + 2δ) -α -2 -d 2 H (Φ(.|ν, (1 + δ)A), Φ(.|ν, (1 + δ) -1 A)) = 2 cosh{α ln(1 + 2δ)} -1 + 1 -[cosh{ln(1 + δ)}] -α 2 ≤ 2 sinh(1)α 2 δ 2 + 1 4 α 2 δ 2 ≤ 3α 2 δ 2 = ε 2 .

Proof of Proposition 3

Proof. Since the set of ε-brackets over F (α) , described in the Proposition 4 is totally defined by the parameter spaces R(β) and X (ε, a, λ m , α), an upper bound of the bracketing number of F (α) is deduced from an upper bound of the two set cardinals.

N [.] ε, F (α) , d H ≤ card{R(β)} × card{X (ε, a, λ m , α)} ≤ 2 λ M β α(α+1) 2 6 aα √ λ m ε α ≤ 6 √ 3 λ M α 2 ε λ m α(α+1) 2 6 aα √ λ m ε α .
Thus, since ln(α) and ln(α 2 ) are bounded by ln(Q 2 ),

H [.] ε, F (α) , d H ≤ α(α + 1) 2 ln 6 √ 3 λ M λ m + α ln 6 a √ λ m + α(α + 1) 2 + α ln(Q 2 ) + α(α + 1) 2 + α ln 1 ε .
INRIA B Proofs of the main results

B.1 Proof of Theorem 2

Theorem 1 is applied in order to prove Theorem 2. It requires to find a convenient function ψ (K,α) which is deduced from the bracketing entropy upper bounds established in Section A. We state the following technical result which is used hereafter: For all ε ∈ (0, 1],

ε 0 ln 1 x dx ≤ ε ln 1 ε + √ π . ( 21
)
This inequality is deduced from an integration by part and the following concentration inequality (Massart, 2007, p.19): If Z is a centered standard Gaussian variable then P (Z ≥ c) ≤ e -c 2 2 for all c > 0. First, we consider the diagonal mixture form with ordered variables. For all positive real number ξ, using ( 21

), ξ 0 H [.] (x, S (K,α) , d H ) dx ≤ ξ    C(K) + Kα ln a 8 c 1 λ m + (Kα + 1) ln 8 λ M λ m    + ξ (2Kα + 1) ln(9 √ 2 Q) + ξ∧1 0 D(K, α) ln 1 x dx ≤ ξ    C(K) + Kα ln a 8 c 1 λ m + (Kα + 1) ln 8 λ M λ m    + ξ (2Kα + 1) ln(9 √ 2 Q) + ξ D(K, α) ln 1 1 ∧ ξ + √ π .
According to Inequality (21) and Proposition 2, we get

ξ 0 H [.] (x, S (K,α) , d H ) dx ≤ ξ D(K, α) (2) + ln 1 1 ∧ ξ with (2) = C(K) D(K, α) + Kα D(K, α) ln a 8 c 1 λ m + (Kα + 1) D(K, α) ln 8 λ M λ m + (2Kα + 1) D(K, α) ln(9 √ 2 Q) + √ π.
Moreover, since C(K) D(K,α) ≤ ln(18πe 2 ) and Kα D(K,α) , Kα+1 D(K,α) and 2Kα+1 D(K,α) are all smaller than 1, (2) is bounded by a constant A(λ m , λ M , a, Q) denoted only A hereafter and defined by

A(λ m , λ M , a, Q) := √ π + ln (18πe 2 )+ ln a 8 c 1 λ m + ln 8 λ M λ m + ln(9 √ 2 Q). (22)
In the same way, we obtain in the general case that for all ξ > 0 ξ 0

H [.] (x, S (K,α) , d H ) dx ≤ ξ D(K, α) A + ln 1 1 ∧ ξ
where the constant is equal to

A = ln(18πe 2 ) + ln 8 √ 2 λ M λ m + ln 54 √ 3 λ M λ m + ln 54 a √ λ m + ln(Q 2 ) + √ π.
(23) Consequently in the two cases the following function

Ψ (K,α) : ξ ∈ R + → ξ D(K, α) A + ln 1 1 ∧ ξ which satisfies condition [I]
of Theorem 1 can be considered. Next we need to find ξ such that Ψ (K,α) (ξ ) = √ n ξ 2 to deduce the penalty function. This is equivalent to solve

D(K, α) n A + ln 1 1 ∧ ξ = ξ .
Noticing that the quantity ξ =

D(K,α) n A satisfies ξ ≤ ξ , we get ξ ≤ D(K, α) n A + ln 1 1 ∧ ξ and so ξ 2 ≤ D(K, α) n 2 A 2 + ln 1 1 ∧ D(K,α) n A 2 .
Finally, according to the low bound of penalty functions in Theorem 1, it remains to define the weights ρ (K,α) . The considered weights ρ (K,α) = D(K, α) depend on the model dimension and their sum Υ is equal to 1 since

B.2 Proof of Theorem 3

Apart from the weight definition step, the proof of Theorem 3 is the same as in the ordered case. The following Lemma is used to define weights for this family which has a greater richness. Recall that D(K, v) denotes the dimension of S (S,v) which is equal to K{2 card(v) + 1}

and K 1 + card(v) + card(v)(card(v)+1) 2 in diagonal and general cases respectively.

Lemma 4. The quantity card {(K, v) ∈ N × V; D(K, v) = D} is upper bounded by

   2 Q if Q ≤ D-1 2 2eQ D-1 D-1 2 otherwise .
Proof.

1. In the diagonal case,

= ∞ K=1 Q α=1 Q α 1I K(2α+1)=D ≤ ∞ α=1 Q α 1I α≤Q∧ D-1 2 . If Q ≤ D-1 2 , ∞ α=1 Q α 1I α≤Q∧ D-1 2 = 2 Q .
Otherwise, according to Proposition 2.5 in [START_REF] Massart | Concentration inequalities and model selection[END_REF],

∞ α=1 Q α 1I α≤Q∧ D-1 2 ≤ f D -1 2
where f (x) = eQ x x is an increasing function on [1, Q]. Noticing that Q is an integer, it leads that

Q∧ D-1 2 α=1 Q α ≤    2 Q if Q ≤ D-1 2 2eQ D-1 D-1 2 otherwise .
2. In the general case, card (K, v) ∈ N × V; K 1 + card(v) + card(v){card(v)+1} 2 = D is upper bounded by

Q α=1 Q α 1I 1+ 3 2 α+ α 2 2 ≤D ≤ Q α=1 Q α 1I α≤ D-1
2 hence the result is the same as the diagonal case.

Proposition 5. Consider the following weight family ρ (K,v) (K,v)∈N ×V defined by

ρ (K,v) = D(K, v) 2 ln 8eQ {D(K, v) -1} ∧ (2Q -1)
.

Then we have C Results for multivariate Gaussian densities C.1 Ratio of two Gaussian densities Proposition 6. Let Φ(.|µ 1 , Σ 1 ) and Φ(.|µ 2 , Σ 2 ) be two Gaussian densities . If Σ 2 -Σ 1 is a positive definite matrix then for all x ∈ R Q ,

Φ(x|µ 1 , Σ 1 ) Φ(x|µ 2 , Σ 2 ) ≤ |Σ 2 | |Σ 1 | exp 1 2 (µ 1 -µ 2 ) (Σ 2 -Σ 1 ) -1 (µ 1 -µ 2 ) .
Proof. The ratio between the two Gaussian densities is equal to

Φ(x|µ 1 , Σ 1 ) Φ(x|µ 2 , Σ 2 ) = |2πΣ 1 | -1 2 |2πΣ 2 | -1 2 exp - 1 2 (x -µ 1 ) Σ -1 1 (x -µ 1 ) -(x -µ 2 ) Σ -1 2 (x -µ 2 ) .
The matrix Σ -1 1 -Σ -1 2 = Σ -1 1 (Σ 2 -Σ 1 )Σ -1 2 is nonsingular since Σ 1 , Σ 2 and Σ 2 -Σ 1 are positive definite matrices. Defining µ = (Σ -1 1 -Σ -1 2 ) -1 (Σ -1 1 µ 1 -Σ -1 2 µ 2 ), (x-µ 1 ) Σ -1 1 (x-µ 1 )-(x-µ 2 ) Σ -1 2 (x-µ 2 ) = (x-µ ) (Σ -1 1 -Σ -1 2 )(x-µ )+(µ 1 -µ 2 ) (Σ 2 -Σ 1 ) -1 (µ 1 -µ 2 ) . Finally,

Φ(x|µ 1 , Σ 1 ) Φ(x|µ 2 , Σ 2 ) = |Σ 2 | |Σ 1 | exp - 1 2 (x -µ ) (Σ -1 1 -Σ -1 2 )(x -µ ) + (µ 1 -µ 2 ) (Σ 2 -Σ 1 ) -1 (µ 1 -µ 2 ) ≤ |Σ 2 | |Σ 1 | exp 1 2 (µ 1 -µ 2 ) (Σ 2 -Σ 1 ) -1 (µ 1 -µ 2 ) .
Corollary 2. Let Φ(.|µ 1 , B 1 ) and Φ(.|µ 2 , B 2 ) be two Gaussian densities. Their variance matrice are assumed to have the following diagonal form B i = diag (b 2 i1 , . . . , b 2 iQ ) for all i = 1, 2 such that b 2 2q > b 2 1q > 0 for all q ∈ {1 . . . , Q}. Then, for all x ∈ R Q , the ratio of the two densities is bounded by

Φ(x | µ 1 , B 1 ) Φ(x | µ 2 , B 2 ) ≤ Q q=1 b 2q b 1q exp 1 2 (µ 1 -µ 2 ) diag 1 b 2 21 -b 2 11 , . . . , 1 b 2 2Q -b 2 1Q (µ 1 -µ 2 ) . RR n°6549

  card {(K, α) ∈ N × {1, . . . , Q}; D(K, α) = D} ≤ D INRIA and (K,α) e -ρ (K,α) ≤ D≥1 D e -D ≤ 1. Therefore according to Theorem 1, if the penalty function satisfies the inequality pen(K, α) α) of crit(K, α) = γ n (ŝ (K,α) ) + pen(K, α) on M exists and E d 2 H (s, ŝ( K, α) ) ≤ C inf (K,α)∈M{KL(s, S (K,α) ) + pen(K, α)} + 1 n .

  card {(K, v) ∈ N × V; D(K, v) = D} = card [(K, v) ∈ N × V; K{2 card(v) + 1} = D]

  (K,v)∈N ×V e -ρ (K,v) ≤ 2.Proof. According to Lemma 4, ≤ exp {-(D -1) ln(2)} since D ≤ 2Q. In the second sum, since D ≥ 2Q +

RR n°6549

Acknowledgements

The authors are grateful to Sylvain Arlot (Université Paris-Sud 11) and Pascal Massart (Université Paris-Sud 11) for helpful discussions and valuable comments.

C.2 Hellinger distance between two Gaussian densities

The following proposition gives the expression of the Hellinger distance between two Gaussian densities.

Proposition 7. Let Φ(.|µ 1 , Σ 1 ) and Φ(.|µ 2 , Σ 2 ) be two Gaussian densities. The Hellinger distance between these two densities d 2 H (Φ(.|µ 1 , Σ 1 ), Φ(.|µ 2 , Σ 2 )) has the following expression:

Proof. According to the definition of the Hellinger distance,

Finally, the distance is equal to

that entails the concluding result.

Corollary 3. Using the notation of the Corollary 2, the Hellinger distance of two Gaussian densities with diagonal variance matrices is given by the following expression

INRIA