Sokol Koço
email: sokol.koco@lif.univ-mrs.fr

Cécile Capponi
email: cecile.capponi@lif.univ-mrs.fr

A boosting approach to multiview classification with cooperation

Keywords: Boosting, Multiview Learning, Supervised Learning, Classification

In many fields, such as bioinformatics or multimedia, data may be described using different sets of features (or views) which carry either global or local information. Some learning tasks make use of these several views in order to improve overall predictive power of classifiers through fusion-based methods. Usually, these approaches rely on a weighted combination of classifiers (or selected descriptions), where classifiers are learned independently. One drawback of these methods is that the classifier learned on one view does not communicate its failures within the other views. This paper deals with a novel approach to integrate multiview information. The proposed algorithm, named Mumbo, is based on boosting. Within the boosting scheme, Mumbo maintains one distribution of examples on each view, and at each round, it learns one weak classifier on each view. Within a view, the distribution of examples evolves both with the ability of the dedicated classifier to deal with examples of the corresponding features space, and with the ability of classifiers in other views to process the same examples within their own description spaces. Hence, the principle is to slightly remove the hard examples from the learning space of one view, while their weights get higher in the other views. This way, we expect that examples are urged to be processed by the most appropriate views, when possible. At the end of the iterative learning process, a final classifier is computed by a weighted combination of selected weak classifiers. This paper provides the Mumbo algorithm in a multiclass and multiview setting, based on recent theoretical advances in boosting. The boosting properties of Mumbo are proved, as well as some results on its generalization capabilities. Several experimental results are reported which point out that complementary views may actually cooperate under some assumptions.

Introduction

In many application domains of machine learning, such as bioinformatics or multimedia indexing, data may be described by several sources or views [START_REF] Masulli | Natural computing methods in bioinformatics: A survey[END_REF], [START_REF] Mansoorizadeh | Multimodal information fusion application to human emotion recognition from face and speech[END_REF]. When facing a classification or a regression task, the use of these views might be of great interest, since each view is supposed to carry some information that the other views would not embed. Fortunately, there exist many methods to select the most informative sources, or set of features, that either best discriminate data concepts or best describe one concept among others [START_REF] Culp | On multi-view learning with additive models[END_REF] [START_REF] Sridharan | An information theoretic framework for multi-view learning[END_REF].

Most of these selective methods are statistically founded, which means that they tend to disregard localized -isolated -information although it could be useful to compensate the lack of performance on some (group of) learning examples. Indeed, real-life data descriptions are often noisy. When the noise rate of a set of feature descriptions reaches a threshold, which depends on the problem, no learning algorithm has been proved, neither theoretically nor empirically, to be able to overcome the noise disruption on the generalization capabilities of classifiers. Yet, multiview learning approaches should enable some localized failures of classifiers trained on one view, to be compensated by -or subordinated to -the abilities of classifiers on the other views.

Up to now, several approaches of multiview learning have been developed, most of them in the semi-supervised setting. The first of them was the well-known Co-Training algorithm [START_REF] Avrim | Combining labeled and unlabeled data with co-training[END_REF], which was based on far too much restrictive assumptions [START_REF] Christoudias | Co-training with noisy perceptual observations[END_REF]. Other semi-supervised multiview algorithms have then been developed and theoretically founded, all of them promoting the agreement between views [START_REF] Sindhwani | An rkhs for multi-view learning and manifold coregularization[END_REF] [8] [START_REF] Ambrym | Complexity versus agreement for many views: Coregularization for multi-view semi-supervised learning[END_REF], except for the semi-supervised boosting approach presented in [START_REF] Saffari | Robust multi-view boosting with priors[END_REF]. No compelling application on real-life problems has promoted these approaches yet, although many problems would actually need a multiview learning process.

In addition, in the supervised setting, leveraging the performances of classifiers learned on different views has mainly been performed through fusion-based methods, either early or late fusion [START_REF] Kludas | Information fusion in multimedia information retrieval[END_REF] [START_REF] Wozniak | Some remarks on chosen methods of classifier fusion based on weighted voting[END_REF]. Early fusion consists in grouping (selected) descriptions of the different views into a large vector, and then to learn a classifier on this resulting view. On the opposite, late fusion allows one classifier per view to be learned, while the final classifier is a combination of them. Usually, late fusion performs better than early fusion. Yet, none of them leads to good performances when the views are of unbalanced informative content, for weaker views tend to reduce the final performances. An empirical comparison of these methods applied on multimedia problems is presented in [START_REF] Snoek | Early versus late fusion in semantic video analysis[END_REF].

Whatever the fusion-based approach is, it relies on a weighted combination of classifiers (or selected descriptions), where classifiers are learned independently. One drawback of these methods is that the classifier learned on one view does not communicate its failures to the other views. Besides, views must be independent in order for the combined classifier to be most accurate.

Yet, we think it could be interesting that, when the classifier on a view fails on a region of examples in the instance space, it could entrust the other views with the classification of these examples. One of the major difficulties is then to delimit the concerned subareas of the instance space, without loss of generalization capabilities. Instead of precisely locating these subareas, we propose an algorithm based on boosting [START_REF] Freund | Experiments with a new boosting algorithm[END_REF] [START_REF] Mukherjee | A theory of multiclass boosting[END_REF] whose principle is to slightly remove the hard examples from the learning space of one view, while their weights get higher in the other views. This way, we expect that examples are processed by the most appropriate views.

In order to implement this principle, we designed Mumbo as a boosting-like algorithm which works from different views on data. Each view is associated with a weak learner, and one distribution per view is maintained (section 2). The distributions are updated at each iteration in such a way that views communicate the ones to the others their capability of processing learning examples. Hence, not only the distribution update in one view takes into account the performances of that view in classifying the learning examples, but it also embeds the performances of the other views on these examples. The properties of Mumbo are discussed and proved in section 3: both empirical and generalization errors are bounded. In order to warrant the boosting properties, and the generalization error bound, we define a global distribution of examples that reflects the overall behaviour of the algorithm within a given hypothesis space. In section 4, we present experimental results of Mumbo on synthetic data, which confirm that Mumbo is a boosting algorithm, better than other basic fusion approaches. Before concluding, we discuss this approach with other methods, and give some clues to improve Mumbo (section 5).

The Mumbo Algorithm

Principles and assumptions

Mumbo is a multiview boosting algorithm: each example of the learning sample S is represented by several independent sets of features. Each one of these representations is called a view. Eventually, these views are used to train models, which are then used to classify other examples. Even though these models are learned on different representations of the same examples, they are by no means equal performance wise. Classifiers learned on some views perform better than those learned on other views, due to the noise in the data and/or views, or the lack of information, etc., which may be different from one representation space to another. In other words, we may define the strength of a view as the possibility to learn a good classifier on that view. At the same time, the weakness of a view may reflect the impossibility of learning a good classifier from the instance space defined by this view.

More formally, let S be a sample of n tagged examples chosen according to some distribution D over X × Y , where X is some instance space and Y is some class space. Let V be a view, H be the space of all the hypothesis that we can learn on V and h be the best classifier that we can learn on this space. Finally, let ρ be the error of random guessing over S and σ V ≤ ρ be the lower bound of the error of h on S. We define the notion of weak and strong view as follows :

V is called a strong view if σ V is near 0 and V is called a weak view if γ V = ρ -σ V is near 0.
Mumbo has been designed in order to learn a classifier in a multiview setting, where views are supposed to be of different strengths. More specifically, we suppose that among the views, there exists one strong major view V , and several weaker minor views

v 1 , • • • , v z .
In our setting, γ V is supposed to be greater than γ v1 , • • • , γ vz . For example, in speech recognition, three usual views for describing a speech (or dialog) to be classified, are known to be of unequal strength [START_REF] Goldwater | Which words are hard to recognize? prosodic, lexical, and disfluency factors that increase speech recognition error rates[END_REF]. The major view is the lexical analysis of a speech (syntactic trees, for example); other minor views may be the prosodic information, and syntactic information. Although the major view allows to learn rather good classifiers, researchers in speech recognition still use minor views for learning in order to compensate the failures of the major view in case of noise disruptions [START_REF] Hasegawa-Johnson | Simultaneous recognition of words and prosody in the boston university radio speech corpus[END_REF].

As pointed out in the introduction, the basic principle of Mumbo is to encourage each view v to focus on the examples that are hard to process in other views, and easy to process in v. Hence, it assumes that if one representation space does not embed information on one (set of) examples, part of that information can be provided by other representation spaces.

Framework and notations

In this paper, we present Mumbo within the framework defined by [START_REF] Mukherjee | A theory of multiclass boosting[END_REF], where basically γ denotes the edge of a classifier with regards to random. We use the following typings:

matrices are denoted by bold capital letters like C; element of row i and column j in matrix C is denoted C(i, j), and C(i) is the row i of C. -M • M' denotes the Frobenius inner product of two matrices.

the indicator function is denoted by 1[•], and the cartesian product is denoted by

X 1 × X 2 .
Let S = {(x 1 , y 1 ), ..., (x n , y n )} be the learning sample, where x i ∈ X, and y i ∈ Y is the class of the example x i . The set of classes is Y = {1, ..., k}. The set of features X is made up of different subsets: X = X 1 × ... × X m . Each subset represents a view, as in [START_REF] Janodet | Boosting classifiers built from different subsets of features[END_REF]. Then, the representation of example x i within view m is written x i,m 1 .

In this paper we use the definition of weak classifier as defined in [START_REF] Mukherjee | A theory of multiclass boosting[END_REF], that is a classifier whose performance on a cost matrix is better than that of some edge-over-random γ baseline.

Definition 1 (edge-over-random baseline and cost matrix, by Mukherjee et al. [15]).

The edge-over-random baseline B ∈ B eor γ ⊆ R n×k , where B eor γ is the space of edge-over-random baselines, is γ more likely to predict the correct label rather than an incorrect one on every example i: ∀l = y i , B(i, y i ) ≥ B(i, l) + γ, with equality holding for some l.

The edge-over-random cost matrix C puts the least cost on the correct label, i.e. the rows of the cost matrix come from the set

{ c ∈ R k : ∀l, c(y i ) ≤ c(l) } .
Definition 2 (edge-over-random baseline in the Mumbo setting). The edge-over-random baseline used in this paper is a cost matrix U γ defined as follows:

U γ (i, l) = (1-γ)/k+γ if l = y i and U γ (i, l) = (1 -γ)/k if l = y i .
The 1 ht,m matrix is the prediction matrix defined as

1 ht,m (i, l) = 1 if h t,m (i) = l and 1 ht,m (i, l) = 0 if h t,m (i) = l.
Definition 3 (edge-condition, by Mukherjee et al. [START_REF] Mukherjee | A theory of multiclass boosting[END_REF]). Let C ⊆ R n×k and matrix B ∈ B eor γ , an eor-baseline; we say that a weak classifier h satisfies the edge condition if

C • 1 h ≤ C • B
In the case of binary classification, the ith row of the baseline

U γ is ( 1 2 (1 -γ), 1 2 (1 + γ)) if the label of example i is +1, and ( 1 2 (1 + γ), 1 2 (1 -γ)) if the label of example i is -1. A given classifier h satisfies the edge condition if ∑ i C(i, h(i)) ≤ ∑ i { ( 1 2 -γ 2 )C(i, y i ) + ( 1 2 + γ 2 )C(i, y i ) }
and [START_REF] Mukherjee | A theory of multiclass boosting[END_REF] shows that this condition is equivalent to the usual weak learning condition for binary classification.

The core of Mumbo

Mumbo (algorithm 1) is an attempt to promote the collaboration between major and minor views, in order to enhance the performances of classifiers usually learned only on the major view.

It is a boosting algorithm theoretically founded on the framework presented in [START_REF] Mukherjee | A theory of multiclass boosting[END_REF]. Y is not limited to {-1, +1}, since we are in the multiclass setting, based on the theoretical approach of [START_REF] Mukherjee | A theory of multiclass boosting[END_REF], whose one of the main ideas is to replace the weights of the examples with a cost matrix. We use the same idea here: C is a cost matrix so that C(i, l) is the cost of assigning the label l to the example i. Since we deal with more than one view, we use one cost matrix C j per view j, and a global cost matrix C G . Thus m + 1 cost matrices are maintained.

Mumbo runs for T rounds: at each round t, a weak learner is trained on each view v, which returns m weak classifiers h t,m . These weak classifiers must satisfy the weak learning condition given in definition 3. For each h t,j , we compute a parameter α t,j that measures its importance depending on the edge of the h t,j on the cost matrix C t,j (c.f. algorithm 1).

As stated before, one of the main ideas of Mumbo is to have some sort of collaboration between the different views. This idea is implemented in two different parts of this algorithm: first during the update of the m cost matrices, and second when choosing the classifier h t selected at round t in the final combination.

The update of each cost matrix depends on all the classifiers. The i th line, corresponding to the example x i in the matrix of the view j, is updated only if the classifier learned on this view classifies correctly x i OR if all the m -1 other weak classifiers misclassify it. Intuitively this means that a view gives up on the hardest examples and lets the other views handle them. In the scenario of one major and several minor views, this allows the minor views to focus on the hardest examples of the major view.

Algorithm 1 Mumbo: MUltiModal BOosting

Initialize (∀i ∈ {1, • • • , n}, ∀j ∈ {1, • • • , m}, ∀l ∈ {1, • • • , k}): f0,j(i, l) = 0 C0,G(i, l) = C0,j(i, l) = { 1 if yi = l -(k -1) if yi = l
where C0,G is the global cost matrix

for t = 1 to T do Train W L using Ct-1,1, ..., Ct-1,m for j = 1 to m do
Get ht,j satisfying the edge condition on B, and compute edge δt,j on Ct-1,j, and αt,j = 1 2 ln

1+δ t,j 1-δ t,j end for Update cost matrices (for each view j = 1 • • • m): Ct,j(i, l) =    exp(ft,j(i, l) -ft,j(i, yi)) if l = yi - k ∑ p=1;p =y i exp(ft,j(i, p) -ft,j(i, yi)) if l = yi where ft,j(i, l) = t ∑ z=1 1[hz,j(i) = l]αz,jdz,j(i) and dz,j(i) = { 1 if hz,j(i) = yi or ∃q ∈ {1, ..., m}, hz,q(i) = yi 0 else Choose ht = argmax h t,j
(edge ht,j on Ct,G) and δt = {edge of ht on Ct,G}

Compute αt = 1 2 ln 1+δ t 1-δt Update Ct,G : Ct,G(i, l) =    exp(ft,G(i, l) -ft,G(i, yi)) if l = yi - k ∑ j =y i exp(ft,G(i, j) -ft,G(i, yi)) if l = yi where ft,G(i, l) = t ∑ z=1 1[hz,m(i) = l]αz,m

end for

Output final hypothesis :

H(x) = argmax l∈1,...,k fT (x, l), where fT (i, l) = T ∑ t=1 1[ht(i) = l]αt,m
In the last part of each round t, Mumbo chooses the classifier h t among the m that minimizes the error on the global cost matrix. The confidence α t is computed for h t , based on its edge on the global cost matrix. Finally, the global cost matrix is updated, in a similar way that in the adaptive case of the OS algorithm [START_REF] Mukherjee | A theory of multiclass boosting[END_REF].

The final hypothesis H is a weighted vote of the T selected weak classifiers h t , and α t is the weight assigned to h t .

Properties of Mumbo

In this section, we present two properties of the Mumbo algorithm that together ensure it is a boosting algorithm. We first show that the update rules for the cost matrix of each view, as presented in the previous section, actually reduce the training error on this view. We then prove that the criterion for choosing the unique h t at each step t, and eventually the update rule, allows Mumbo to be a safe boosting algorithm: the training error decreases with rounds. The most important property, a bound on the generalization error of Mumbo, is proved.

Bounding the training error on each view

One property of Mumbo is that we can bound the empirical error made by the final classifiers of one view, when views are considered independently. We give a formal proof of this property, then we define a way of computing α t,m in each round t for each view m in order to prove the decreasing of the empirical error of the final combination of weak classifiers.

Theorem 1 is an adaptation to Mumbo of the lemma presented in the supplement of [START_REF] Mukherjee | A theory of multiclass boosting[END_REF].

∑ i=1 ∑ l =yi exp(f t,m (i, l) -f t,m (i, y i )),
at most a factor

τ t,m = 1 - 1 2 (exp(α t,m ) -exp(-α t,m ))δ t,m + 1 2 (exp(α t,m ) + exp(-α t,m ) -2)
of the loss before choosing (α t,m ), where δ t,m is the edge of

h t,m , δ t,m = C t,m • U γm -C t,m • 1 ht,m .

Proof.

Let S + be the set of the examples correctly classified by h t,m , S -the set of the examples misclassified by all the m classifiers returned by WL, and S -+ the set of the examples misclassified by h t,m and correctly classified by at least one of the other h t,j , j = m.

In order to simplify the reading of the proof, we introduce the quantities:

L t,m (i) = ∑ l =yi exp(f t,m (i, l) -f t,m (i, y i )), and ζ t,m (i, l) = f t,m (i, l) -f t,m (i, y i ).
Using the edge condition we have :

C t,m • 1 ht,m ≤ C t,m • U δt,m (1)
The left and right sides of equation 1 can be rewritten as:

C t,m • 1 ht,m = - ∑ i∈S+ L t-1,m (i) + ∑ i∈S- exp(ζ t-1,m (i, h t,m (x i ))) + ∑ i∈S-+ exp(ζ t-1,m (i, h t,m (x i ))) C t,m • U δt,m = ∑ N i=1 ( -L t-1,m (i)( 1-δt,m k + δ t,m ) + L t-1,m (i)( 1-δt,m k ) ) = -δ t,m ∑ i L t-1,m (i)
So, using the edge condition 1 we obtain:

-

∑ i∈S+ L t-1,m (i)+ ∑ i∈S- exp(ζ t-1,m (i, h t,m (x i )))+ ∑ i∈S-+ exp(ζ t-1,m (i, h t,m (x i ))) ≤ -δ t,m ∑ i∈S L t-1,m (i) hence: ∑ i∈S+ L t-1,m (i) - ∑ i∈S-∪S-+ exp(ζ t-1,m (i, h t,m (x i ))) ≥ δ t,m ∑ i∈S L t-1,m (i) (2)
In order to compute the drop in loss after choosing h t,m with weight α t,m , let us consider three cases:

1. For i ∈ S + : We have f t,m (i, l) -f t,m (i, y i ) = f t-1,m (i, l) -(f t-1,m (i, y i ) + α t,m
), then:

∆ + = ∑ i∈S+ -L t,m (i) - ∑ i∈S+ -L t-1,m (i) = ∑ i∈S+ -exp(-α t,m )L t-1,m (i) - ∑ i∈S+ -L t-1,m (i) = (1 -exp(-α t,m )) ∑ i∈S+ L t-1,m (i)
2. For i ∈ S -:

∆ -= ∑ i∈S- exp(f t,m (i, h t,m (i)) -f t,m (i, y i )) - ∑ i∈S- exp(f t,m (i -1, h t,m (i)) -f t-1,m (i, y i )) = ∑ i∈S- exp(f t-1,m (i, h t,m (i)) + α t,m -f t-1,m (i, y i )) - ∑ i∈S- exp(f t-1,m (i, h t,m (i)) -f t-1,m (i, y i )) = (exp(α t,m ) -1) ∑ i∈S- exp(f t-1,m (i, h t,m (i)) -f t-1,m (i, y i )) = (exp(α t,m ) -1) ∑ i∈S- exp(ζ t-1,m (i, h t,m (x i )))
3. For i ∈ S -+ :

∆ -+ = ∑ i∈S- exp(f t,m (i, h t,m (i)) -f t,m (i, y i )) - ∑ i∈S- exp(f t-1,m (i, h t,m (i)) -f t-1,m (i, y i )) = ∑ i∈S- exp(ζ t,m (i, h t,m (x i ))) - ∑ i∈S- exp(ζ t-1,m (i, h t,m (x i )))
= 0 since the value of f t,m does not change for these examples So, the drop in loss

∆ = ∆ + -∆ --∆ -+ is: = (1 -exp(-α t,m )) ∑ i∈S+ L t-1,m (i) -(exp(α t,m ) -1) ∑ i∈S- exp(ζ t-1,m (i, h t,m (i))) = ( exp(αt,m)-exp(-αt,m) 2 ) ( ∑ i∈S+ L t-1,m (i) - ∑ i∈S- exp(ζ t-1,m (i, h t,m (i)))
)

- ( exp(αt,m)+exp(-αt,m)-2 2 ) ( ∑ i∈S+ L t-1,m (i) + ∑ i∈S- exp(ζ t-1,m (i, h t,m (i))) ) ≥ ( exp(αt,m)-exp(-αt,m) 2 ) ( ∑ i∈S+ L t-1,m (i) - ∑ i∈S-∪S-+ exp(ζ t-1,m (i, h t,m (i))) ) - ( exp(αt,m)+exp(-αt,m)-2 2 ) ( ∑ i∈S+ L t-1,m (i) + ∑ i∈S-∪S-+ exp(ζ t-1,m (i, h t,m (i)))
)

Using the result we obtained in equation 2 and the fact that exp(ζ t-1,m (i, h t,m (i))) ≤ L t-1,m (i), we can give a lower bound of the loss drop:

∆ ≥ ( exp(αt,m)-exp(-αt,m) 2 ) δ t,m ∑ i L t-1,m (i) - ( exp(αt,m)+exp(-αt,m)-2 2 ) ( ∑ i∈S+ L t-1,m (i) + ∑ i∈S-∪S-+ L t-1,m (i) ) ≥ ( exp(αt,m)-exp(-αt,m) 2 ) δ t,m ∑ i L t-1,m (i) - ( exp(αt,m)+exp(-αt,m)-2 2 ) ∑ i L t-1,m (i) ≥ ( exp(αt,m)-exp(-αt,m) 2 δ t,m - exp(αt,m)+exp(-αt,m)-2 2 ) ∑ i L t-1,m (i)
Hence the loss 1 -∆ at round t is at most a factor 1 -

T,m ≤ (k -1) T ∏ t=1 √ 1 -δ t,m ≤ (k -1) exp { - 1 2 T ∑ t=1 δ 2 t,m } (3)
This result shows that Mumbo is a boosting algorithm even when the selected weak classifier always comes from the same view m for all steps. This might occur when the major view is far better than minor views for all training examples.

Bounding the whole empirical error

At each step t of the algorithm 1, one classifier is selected among m weak classifiers, if m is the number of views, that is, the space of weak hypothesis H in this case is {h t,1 , • • • , h t,m }. This space is a particular case of the space of hypothesis used by the OS algorithm, thus we obtain the same bound on the empirical error as the OS algorithm, that is :

T ≤ (k -1) T ∏ t=1 √ 1 -δ t,m ≤ (k -1) exp { - 1 2 T ∑ t=1 δ 2 t,m } (4)
In practice, one may observe that the edges of the classifiers at step t are all negative. In such a case, since each weak classifier of view v is trained on a subset of the learning samples randomly drawn from the current distribution of v, iterating the learning step until γ v is positive allows the algorithm to fulfill the conditions.

Results in generalization

We show here that the generalization error of the final hypothesis learned by Mumbo after T iterations can be bound, and this bound converges towards 0 with the number of iterations.

The generalization error of a classifier is defined as the probability to misclassify any new example. For multiclass algorithms such as AdaBoost.MR, [START_REF] Schapire | Boosting the margin: A new explanation for the effectiveness of voting methods[END_REF] shows that the generalization error of the final hypothesis can be bound and that it is related to the margins of the learning examples. We thus first recall the definitions of the bound on the generalization error, then we extend existing results to Mumbo.

Generalization Error for Multiclass Problems

The final hypothesis of Mumbo is a multi class classifier, thus its output space can be defined as Y = {1, 2, ..., k}. In this section, the weak classifiers h ∈ H are defined as mappings from X × Y to {0, 1}, where X is some description space. The label y is predicted as a potential label for x i if h(x, y) = 1. Note that these classifiers are equivalent to 1[h t (x) = l], the weak classifiers described in the algorithm.

Let C denote the convex hull of H, that is :

C = { f : (x, y) → ∑ h∈H α h h(x, y)|α h ≥ 0 and ∑ h α h = 1 }
For a given example x and a label y, a classifier f in C predicts y as the class of x if argmax l∈Y f (x, l) = y. The margin of an example is then defined as :

margin(f, x, y) = f (x, y) -max l =y f (x, l)
The function f misclassifies an example x if the margin given by f on the couple (x, y) is negative or zero.

Using the previous definitions,

P D [margin(f, x, y) ≤ 0] ≤ P S [margin(f, x, y) ≤ θ]+O ( 1 √ n + ( log(nk) log(|H|) θ 2 + log(1/δ) ) 1/2 )
More generally, for finite or infinite H with VC-dimension d, the following bound holds as well, assuming that n ≤ d ≤ 1 :

P D [margin(f, x, y) ≤ 0] ≤ P S [margin(f, x, y) ≤ θ]+O ( 1 √ n + ( d log 2 (nk/d) θ 2 + log(1/δ) ) 1/2 )
In theorem 2, the term P D [margin(f, x, y) ≤ 0] is the generalization error of the function f . The term P S [margin(f, x, y) ≤ θ] is the empirical margin error of f on the sample S, that is, the proportion of examples of S which are misclassified, or which are correctly classified but with a margin smaller than θ. In the following section, we use θ (f, S) instead of P S [margin(f, x, y) ≤ θ].

The second term in the theorem is a complexity penalization cost.

Mumbo Theorem 2 holds for every voting method using multiclass classifiers as weak classifiers; it thus also holds for Mumbo since his final hypothesis is H T (x) = argmax l∈1,2,...,k f T (x, l), where :

f T (x, l) = ( T ∑ t=1 h t (x, l)α t ) / T ∑ t=1 α t
The weak classifier h t chosen at each iteration is selected from a set of classifiers {h t,1 , ..., h t,m }. These classifiers are selected from potentially different spaces of hypothesis, namely H 1 , ..., H m . Thus the space of hypothesis H from which h t is selected is the union of H 1 , ..., H m . We deduce by the definition of the VC-dimension [START_REF] Vapnik | Statistical Learning Theory[END_REF] that d H = min{d H1 , ..., d Hm }.

We still have to prove that the generalization error decreases with the number of iterations. To do so, it is sufficient to prove that the empirical margin error decreases, since the term

O ( 1 √ n + ( d log 2 (nk/d) θ 2 + log(1/δ) ) 1/2
) is a constant.

We start with showing that we can find a bound for θ (f T , S).

Lemma 1 The empirical margin error of Mumbo after T iterations is bounded by:

θ (f T , S) ≤ (k -1) n ( T ∏ t=1 (1 + δ t ) 1+θ 2 (1 -δ t ) 1-θ 2 )
Proof.

Let l = argmax y =y f (x, y ). For readability reasons, we may write

∑ t instead of T ∑ t=1
.

By the definition of the margin and f , we get :

margin(f, x, y) = f (x, y) -f (x, l) = ∑ t h t (x, y)α t ∑ t α t - ∑ t h t (x, l)α t ∑ t α t Hence, margin(f, x, y) < θ ⇔ ∑ t ht(x,y)αt ∑ t αt - ∑ t ht(x,l)αt ∑ t αt ≤ θ ⇔ θ ∑ t α t - ( ∑ t h t (x, y)α t - ∑ t h t (x, l)α t ) ≥ 0 Let A i = - ( ∑ t α t h t (x i , y) - ∑ t α t h t (x i , l) ) and B = θ ∑ t α t . We deduce that P[margin(f, x i , y) ≤ θ] = 1 ⇔ A i + B ≥ that is, exp(A i + B) ≥ P[margin(f, x, y) ≤ θ]. Thus, θ (f T , S) ≤ 1 n n ∑ i=1 exp(A i ) exp(B). θ (f T , S) ≤ 1 n n ∑ i=1 exp(A i ) exp(B) ≤ 1 n n ∑ i=1 exp ( -( ∑ t α t h t (x i , y) - ∑ t α t h t (x i , l)) ) exp(θ ∑ t α t ) ≤ 1 n n ∑ i=1 exp (-(f T (x i , y) -f T (x i , l))) exp(θ ∑ t α t ) ≤ 1 n n ∑ i=1 ∑ y =y exp (f T (x i , y ) -f T (x i , y)) exp(θ ∑ t α t )
Using the bound on the empirical error, we deduce :

θ (f T , S) ≤ 1 n exp(θ ∑ t α t )(k -1) ∏ t √ 1 -δ 2 t ≤ 1 n exp(θ ∑ t 1 2 ln( 1+δt 1-δt ))(k -1) ∏ t √ 1 -δ 2 t ≤ 1 n ∏ t ( 1+δt 1-δt ) θ 2 (k -1) ∏ t √ 1 -δ 2 t ≤ (k-1) n ( ∏ t (1 + δ t ) 1+θ 2 (1 -δ t ) 1-θ 2 )
The lemma 1 gives a bound on the empirical margin error. As it was shown in [START_REF] Schapire | Boosting the margin: A new explanation for the effectiveness of voting methods[END_REF]

, if θ < δ t /2, then (1 + δ t ) 1+θ 2 (1 -δ t ) 1-θ 2 < 1.
We thus finally claim that the generalization error decreases with the number of iterations: Theorem 3. Let θ > 0 be a fixed margin, then the empirical margin error θ (f T , S) converges towards 0 with the number of iterations, if the edge of the weak hypothesis selected at each iteration is > 2θ.

Theorem 3 and the bound given in theorem 2 together prove that the generalization error of the final hypothesis of Mumbo decreases with the number of iterations. Indeed, the second term of the bound in theorem 2 is a constant, since all the parameters, including d H , are constant in a given problem, and theorem 3 proves that the first term of the bound decreases with the number of iterations.

Experiments on Mumbo

In order to empirically validate and illustrate this approach of multiview learning with boosting, we mainly used synthetic data that obey the underlying assumptions of Mumbo. After explaining the used protocol, this section presents and discusses the results of experiments. in each view, the distributions of classes may overlap: some examples are likely to belong to both classes 3 . -In each view, some examples are generated using a uniform distribution, the same for both classes. Let η be the rate of such a description noise (η M is the noise rate of the major view, while η m is the noise rate of minor views).

One major view is generated. The two minor views are generated with η m = 3-2ηM 4 in such a way that half of the noisy examples in view M are likely to be sane in minor views. Figure 1 pictures an example of a learning sample with n = |S| = 20 examples per class.

We can associate the disruption amount (distribution overlap and noise on descriptions) with the edge-over-random capabilities of weak-classifiers. The more disruption we have in a view, the more γ v is low on that view. Such a sample generation process was designed in order to fit the assumptions that lead to the design of Mumbo: views are rather weak, and learning a classifier on the whole sample needs a cooperation between learners on each view, because information may be distributed among views.

Processing experiments Each weak classifier on view v is obtained by training a linear SVM on a subsample of examples randomly drawn from the current distribution (cost matrix) of v. We check that each weak classifier trained on the view v complies with the definition of weak classifiers in the theoretical scheme of [START_REF] Mukherjee | A theory of multiclass boosting[END_REF], using B=U. Results are the mean of 10 experiments: one experiment is made up of (1) the generation of learning and test samples, (2) the learning process, and (3) the evaluation process.

As said in the introduction, Mumbo was designed as an alternative way to fuse classifiers. We thus compare it with two basic methods of fusion, and with Adaboost: (1) late fusion SVM: one RBF SVM is trained on each view, and the final decision is a margin-weighted combination of their results; (2) early fusion SVM: descriptions of each example are concatenated, then a RBF SVM is trained on the single resulting view; and (3) early fusion Adaboost: descriptions of each example are concatenated, then Adaboost is trained on the single resulting view, with a RBF SVM on a subsample of examples as the weak learner.

Classifiers performances are computed using a testing sample drawn from the same setting that generated the learning sample, but twice bigger.

Results

We present here two kinds of results: an illustration of the behaviour of Mumbo, and a comparison of Mumbo with basic fusion approaches. The bad results of Adaboost are not surprising. Indeed, it processes examples on only one view that concatenates the smaller views. Since data was generated such that half of the disrupted examples on the major view are not disrupted in the minor views, the concatenation of descriptions leads to about 75% of noisy data. Adaboost is well-known to be sensitive to the noise, so one cannot expect better results, despite the true convergence of its empirical error. In addition, which is not reported here, we observed that, whatever η M is (always under 0.5), weak classifiers on minor views are selected in some rounds, in addition to the weak classifiers of the major view which are the most often selected. First rounds tend to only select the classifiers of the major view, then the minor views are alternatively selected with the major view. Besides, this behaviour can be observed on the first rounds on figure 2. It empirically shows that Mumbo actually encourages views to cooperate.

Comparison with other approaches Table 1 compares Mumbo with basic early and late fusion approaches, with various values of η M and various sizes of S. Late SVM is the best fusion approach with this type of data, which is not surprising since data is partially noisy (either in description or because distributions overlaps). Yet Mumbo is better for it processes the cooperation among views, leading each view to focus on the examples disrupted in other views. However, the learning time of Mumbo is T times longer than the learning time of Late SVM. The collaboration slightly improves the results when the major view is disrupted. This is quite obvious with smaller learning samples (when |S|=15 or 30).

Discussion

As expected theoretically, the boosting usual behaviour is observed throughout the experiments, and the results of Mumbo are very good on synthetic data. These results validate the relevance of the Mumbo algorithm when cooperation among views is mandatory for obtaining a strong classifier. Results on empirical and generalization bounds of section 3 are also observed.

In further works, we should test Mumbo on UCI benchmarks. However, these benchmarks are not designed for multiview learning. We plan to select relevant views on these benchmarks (one major and several minor) using PCA or Canonical Component Analysis tools.

Related works and discussion

Related works

So far, in the supervised setting, there is no multiview machine learning algorithm that considers the representation spaces as complementary. Early and late fusion-based approaches are only empirical ways to process the whole useful information available on samples.

The Multiple Kernel Learning (MKL) approaches [START_REF] Gert | Learning the kernel matrix with semidefinite programming[END_REF], which may be used to process multiview samples, is then a costing way to rank the views. But yet, MKL does not promote the cooperation between views: it is much like a way to select the strongest views.

The closest approaches to Mumbo are co-training [START_REF] Avrim | Combining labeled and unlabeled data with co-training[END_REF] and 2-boost [START_REF] Janodet | Boosting classifiers built from different subsets of features[END_REF]. The former is a multiview approach in the semi-supervised setting, where views iteratively cooperate for producing classifiers that converge to the same final hypothesis. The latter is a multiview boosting algorithm. However, Mumbo is different from co-training, first because it works in the supervised setting, and second because it does not assume that the classifiers actually must agree on the same examples. Indeed, Mumbo exploits the disagreements between views. Mumbo is thus closer to 2-Boost, although the motivations are not the same. 2-Boost is designed for dealing with one specific weak learner per view, in order to manage homogeneous views. Then, 2-Boost maintains only one global distribution of examples, whereas Mumbo maintains as many distributions as views in order to process cooperation.

Mumbo is an algorithm that may be categorized as an ensemble of classifiers, for the final classifier is a combination of other classifiers. In the literature, it was proved that without diversity between combined classifiers, the resulting classifier can not be better than the best of the combined classifiers. Many measures of diversity have then been studied so far [START_REF] Ludmila | Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy[END_REF]. We think that the hypothesis underlying Mumbo promote such a diversity. In that sense, we aim at obtaining some theoretical results between some diversity measures and classification accuracy of Mumbo.

Discussion and improvements

In algorithm 1, the function d z (•) indicates whether the update of the cost matrix is possible or not. It is a discrete 0 -1 function, which allows the update of the cost matrix only when some conditions are met (section 2.3). However, such an update rule might be too drastic for promoting the collaboration between views. We think that smoothing it, by changing its range to [0, 1], could improve the efficiency of the cooperation. Hence, in addition of having one or several views, each specialized in some parts of the description space, the minor views could be used more efficiently to increase the accuracy of the final predictions.

One of the main ideas of Mumbo is to enhance as much collaboration as possible between the views. We believe that it is possible to achieve a better cooperation also by changing the decision rule for h t . Indeed, in algorithm 1, h t is the best classifier among the m chosen classifiers at round t, i.e. the one that guarantees the best edge on the general cost matrix G t,m . Many other ways to choose h t could be studied, namely a combination of a subset of weak classifiers, or the choice between the weak classifier of the major view and a combination of the classifiers on minor views, etc. Hence, many alternate selections deserve to be studied, both theoretically (for example, in the PAC-Bayes framework [START_REF] Germain | Pac-bayesian learning of linear classifiers[END_REF]) and empirically.

The most urging work on Mumbo is its study on benchmark and real data. In some domains, such as image indexing, the views are quite natural: there exists dozens of image descriptors, either global or local, that could be considered as complementary views (texture vs. color, etc.). In many other domains, though, we must select the views according to the hypothesis underlying Mumbo (one major still weak view, and many minor views). We wish to adapt statistical tools for view selection, such as Principal Component Analysis as the simplest one.

Conclusion and future works

Mumbo is a boosting-like algorithm in the setting of multiview learning, where views are of different strenghts with regard to a classification task. The idea underlying Mumbo is to promote the cooperation between stronger and weaker views. To implement this idea, the originality of Mumbo is to maintain one distribution of examples per view, and to proceed to distribution updates that allow some views to focus on examples that are hard to classify in other views.

Mumbo is proved to be a boosting algorithm, within the new theoretical framework of [START_REF] Mukherjee | A theory of multiclass boosting[END_REF]: the empirical error decreases with iterations, globally and within each view. Then, the generalization error of Mumbo is proved to be bounded. Finally, the experimental results on dedicated synthetic data give credits to the relevance of Mumbo for encouraging the cooperation among complementary views.

For now, Mumbo is a first attempt to tackle the problem of unbalanced views on data, and we expect to improve it both theoretically and through experiments on benchmarks and real data.

  Given-S = {(x1, y1), ..., (xn, yn)} where xi ∈ X1 × X2 × ... × Xm, yi ∈ {1, ..., k} m weak learning algorithms W L -T the number of iterations a baseline B (edge-over-random prior baseline)

4. 1

 1 Protocols Data generation Data is generated within 3 views, and clustered in two classes {-1, +1}. In each view, the descriptions of examples are vectors of real values. Examples of each class y in view v are generated along a gaussian distribution G[m y,v , σ y,v ]. However, in order to generate weak views, two types of noise disrupt the sample:

Fig. 1 .

 1 Fig. 1. Each example of the learning sample is represented under three views: the major view is on top, with ηM = 0.38; other views are minor (bottom), with ηm = 0.56. The ovals picture the parameters of the examples distribution within each class. The same example is pointed out in each view, in order to illustrate the distribution of information among views.

Fig. 2 .

 2 Fig. 2. Empirical and test errors of Mumbo (top), and Mumbo vs. Adaboost early fusion

Theorem 1 (bounding the empirical error in view

  m). For a given view m, suppose the cost matrix C t,m is chosen as in the algorithm 1, and the returned classifier h t,m satisfies the edge condition for the baseline U γm and cost matrix C t,m , i.e. C t,m • 1 ht,m ≤ C t,m • U γm .

Then choosing a weight α t,m > 0 for h t,m makes the error t,m = n

  1 2 (exp(α t,m ) -exp(-α t,m ))δ t,m + 1 2 (exp(α t,m ) + exp(-α t,m ) -2) of the loss in round t -1.We proved that, in each view, the training error (cost) decreases. Based on theorem 1, and tuning α t,m to1 2 ln , we get the following bound on the empirical error of the classifier H m obtained by the weighted combination of weak classifiers learned in view m after T iterations:

	1+δt,m
	1-δt,m

  Let D be a distribution over X × Y , and let S be a sample of n examples chosen independently at random according to D. Assume that the base-classifier 2 space H is finite, and let δ > 0. Then with probability at least 1 -δ over the random choice of the training set S, every function f ∈ C satisfies the following bound for all θ > 0 :

Schapire et al. give a proof of theorem 2 [19]: Theorem 2 (Schapire et al., [19]).

Table 1 .

 1 Comparison of Mumbo with early fusion and late fusion (base classifiers RBF SVM). Note that results of Adaboost are given after 200 iterations (like Mumbo): raw results show that Adaboost obtains slightly better results after about 50 iterations, then tends to over-fit.

	|S| = 80, ηM	0.5 0.38 0.25 0.12 0	|S| = 120, ηM	0.5 0.38 0.25 0.12 0
	Early+SVM	0.390 0.410 0.437 0.396 0.389	Early+SVM	0.367 0.382 0.396 0.389 0.343
	SVM+Late	0.246 0.229 0.263 0.254 0.232	SVM+Late	0.198 0.225 0.240 0.208 0.279
	Early+Adaboost 0.415 0.420 0.403 0.364 0.358	Early+Adaboost 0.425 0.415 0.466 0.411 0.389
	Mumbo	0.148 0.152 0.168 0.174 0.164	Mumbo	0.02 0.036 0.012 0.026 0.020

When possible, we simplify xi,m to xi in the scope of view m.

Note : the base-classifiers are referred to as weak classifiers in our paper

This work is supported by the French agency ANR, project DECODA, contract no 2009-CORD-005-01, and the French business clusters Cap Digital and SCS.


Dr Iain A Lang Phd

Ms Ruth R Kipping
email: ruth.kipping@bristol.ac.uk

Dr Russell Jago

Prof Debbie A Lawlor

Variation in childhood and adolescent obesity prevalence defined by international and country-specific criteria in England and the US Running head: Childhood obesity prevalence in England and US

Keywords: Adolescent, Body Weight, Child, Epidemiologic Measurements, Obesity, Overweight, Public health

Objective: To compare childhood obesity prevalence in England and the US using different criteria. Methods: Participants included 2 to 17 year olds in the Health Survey for England (HSE, n=33,563) and the US National Health and Nutrition Examination Survey (NHANES, n=14,540) 1999 through 2006. Mean BMI and prevalence of obesity were compared using the UK 1990, US 2000 Centers for Disease Control and International Obesity Task Force (IOTF) criteria.

Results: English children at ages 2 through 5 had a higher mean BMI than US children (mean difference (English minus US) =0.41kg/m 2 , 95% CI 0.31 to 0.52). At ages >8 mean BMI was lower in England (for ages 8 through 11 mean difference= -1.00kg/m 2 , 95% CI -1.26 to -0.75; for ages 12 through 17 mean difference= -1.37kg/m 2 , 95% CI -1.59 to -1.14). The IOTF criteria produced the lowest estimates of obesity prevalence. The 2000 CDC criteria produced the highest estimates in younger children and the UK 1990 criteria produced the highest in adolescents. Children aged 2-5 in England had higher prevalence of obesity than those in the US when using the 2000 CDC and UK 1990 criteria. US adolescents had the highest prevalence of obesity by age group using each of the three criteria.

Conclusion:

The 2000 CDC and UK 1990 criteria give a higher prevalence of obesity in England than the US at ages 2 through 5 but at ages >8 the reverse is true. Estimates of childhood obesity prevalence rely on the criteria used, which has implications for surveillance and clinical practice.

Introduction

Since 1970 the greatest increases in obesity in school children have been in North America and Western Europe [START_REF] Wang | Worldwide trends in childhood overweight and obesity[END_REF]. International comparisons are important for documenting and identifying country-level differences that might be used to suggest causal mechanisms [START_REF] Cole | Establishing a standard definition for child overweight and obesity worldwide: international survey[END_REF]. However, direct international comparisons are currently difficult because existing studies use different methods to define childhood obesity [START_REF] Wang | Worldwide trends in childhood overweight and obesity[END_REF].

The International Obesity Taskforce (IOTF) has developed international criteria for overweight and obesity by gender from 2 through 18 years of age, derived using childhood BMI data from six countries. The criteria provide gender-specific values that were equivalent to BMI of 25 and 30 kg/m 2 and provide a means to compare childhood overweight and obese prevalence internationally [START_REF] Cole | Establishing a standard definition for child overweight and obesity worldwide: international survey[END_REF].

To our knowledge, only one study has made comparisons of childhood obesity between countries using different BMI reference criteria. In that study childhood obesity prevalence was compared between the China, Russia and the US using IOTF, WHO weight for height z scores for <10 year olds and WHO BMI percentiles for >10 year olds, and the US percentiles based on NHANES I (Must et al. 1991a;Must et al. 1991b). Obesity prevalence and between country comparisons varied depending upon which criteria were used [START_REF] Wang | A comparison of international references for the assessment of child and adolescent overweight and obesity in different populations[END_REF].

In this paper we compared mean BMI data and prevalence of overweight and obesity for children and adolescents (aged 2 through 17) from large cross-sectional surveys in England and the US using three criteria: (1) UK 1990 reference charts for BMI from the Child Growth Foundation; [START_REF]Difference is England percentage minus US[END_REF] the US 2000 Centre for Disease Control and Prevention (CDC) BMI-for-age charts; and (3) IOTF. Our analysis had two aims: first, to compare the prevalence of overweight and obesity in England and the US by age and gender, because the US is leading the global epidemic of obesity and we were interested in whether childhood obesity prevalence in England (as representative of a Northern European country) had reached US levels. Second, to present these comparisons using the two country-specific methods of defining obesity.

Methods

Data for these analyses came from two nationally representative cross-sectional surveys.

For England, data were from repeated waves of the Health Survey for England (HSE) (http://www.dh.gov.uk/en/Publicationsandstatistics/PublishedSurvey/HealthSurveyForEngl and/index.htm). Ethical approval for the HSE was obtained from all relevant Multi-Centre and Local Research Ethics Committees. Across the years included here, the number of participants aged 2 through 17 varied from 2,160 (in 2000) to 8,114 (in 2002). Weighting was applied to take account of the under-representation of children in households with more than two children and the clustered, stratified multistage sample design (Sproston K 2006;Sproston K 2003;Sproston K 2004). From each study we used data on participants aged 2 through 17 for whom complete height and weight data were available from years 1999 through 2006. In HSE, 38,936 participants were eligible for inclusion in this age range and of these 33,563 (86.2%) had usable heights and weights. In NHANES, 15,500 participants were eligible for inclusion and of these 14,540 (93.8%) had usable heights and weights. Characteristics of the participants from the two surveys are shown in table 1.

Measurement of height and weight

In the HSE, height and weight were measured in participants' homes by trained researchers. Height was measured using a portable stadiometer. Informants were asked to remove their shoes. One measurement was taken, with the participant stretching to the maximum height and the head aligned in the Frankfort horizontal plane. Height was recorded to the nearest millimeter. Weight was measured using Soehnle, Seca and Tanita electronic scales with a digital display. Participants were asked to remove their shoes and any bulky clothing. Weight was recorded to the nearest 100g.

In NHANES, height and weight were measured in the body measurement room of the NHANES Mobile Examination Centre by trained health technicians. Height was measured using a fixed stadiometer. One measurement was taken, with participants instructed to breathe in and stand as tall as possible and the head aligned in the Frankfort horizontal plane. Height was recorded to the nearest millimeter. Weight was measured using a Toledo electronic scale with a digital display. Participants were asked to wear only underwear, disposable paper gowns, and foam slippers. Weight was measured in pounds and automatically converted to kilograms, recorded to the nearest 100g.

Growth charts

The UK 1990 age-related BMI reference curves cover the age range from birth through 23 years, and the curves are presented as nine centiles in the format of Cole (0.4 th , 3 rd , 10 th , 25 th , 50 th , 75 th , 90 th , 97 th and 99.6 th centiles) [START_REF] Cole | Body mass index reference curves for the UK, 1990[END_REF]. The reference sample of children was based on data from 11 surveys on gender, age, height, and weight collected between 1978 and 1990 for 15,636 boys and 14,899 girls, aged from 33 weeks through 23 years. Exact age was calculated from the dates of birth and the dates of the measurements for all participants. Summary centile curves were fitted to the data using Cole's LMS (lambda mu sigma) method and penalised likelihood [START_REF] Cole | Body mass index reference curves for the UK, 1990[END_REF].

The US 2000 CDC BMI-for-age charts for ages 2 through 20 years were developed with data from five national health examination surveys between 1963 and 1994 and limited supplemental data. Smoothed percentile curves were developed in two stages: first, selected empirical percentiles were smoothed with a variety of parametric and nonparametric procedures; in the second stage, parameters were created to obtain the final curves, additional percentiles, and z-scores [START_REF] Kuczmarski | CDC growth charts for the United States: Methods and development[END_REF].

The IOTF criteria for obesity were based on international survey data from six large nationally representative cross-sectional growth studies in Brazil, Great Britain, Hong Kong, the Netherlands, Singapore, and the United States from birth through 25 years of age [START_REF] Cole | Establishing a standard definition for child overweight and obesity worldwide: international survey[END_REF]. For each of the surveys, centile curves were drawn that at age 18 passed through the widely used criteria of 25 and 30 kg/m 2 for adult overweight and obesity. The resulting curves were averaged to provide age-and genderspecific criteria from 2 through 18 years.

Statistical analyses

We estimated the prevalence of overweight and obesity in children and adolescents according to the three sets of criteria from the growth curves described above. Using the UK1990 and 2000 CDC BMI-for-age reference charts for BMI we classified as overweight/obese those who had an age-gender-specific BMI above the 85 th percentile, and classified as obese those who had a BMI above the 95 th percentile. Using the IOTF classification, we used participants' defined age-gender specific overweight and obesity criteria. This meant that for each of the three methods, each participant was graded as being of recommended weight, overweight/obese, or obese. Where growth charts referred to months rather than years of age we used the nearest mid-year figures. Using the appropriate weighting for each study, as described above, we estimated the proportion of participants in each country who were above the overweight and obesity criteria from the three growth charts. In addition, we compared the trends in mean BMI and prevalence of obesity using the IOTF criteria for two year periods across the eight years for all children combined. All analyses were conducted using Stata SE 10.1 (College Station, Texas).

Results

Mean BMI

Mean BMI for children and adolescents in England and the US, by age in years and with 95% confidence intervals, are shown in Figure 1 and Table 1. As expected, the distribution of BMI is positively skewed for all age groups and both countries. At younger ages English children had higher mean BMI than their peers in the US but this pattern was reversed in older children. Specifically, higher mean BMI was found in English compared to US children at ages 2 through 5 (mean difference England minus US =0.41kg/m 2 , 95% CI 0.31 to 0.52) but lower mean BMI in English compared to US children/adolescents from ages 8 through 17: at ages 8 through 11 the mean difference was -1.00kg/m 2 (95% CI -1.26 to -0.75) and at ages 12 through 17 the mean difference was -1.37kg/m 2 (95% CI -1.59 to -1.14). At ages 5 through 7 the mean BMI was similar in the two countries (mean difference=0.09kg/m 2 , 95% CI -0.07 to 0.25).

There were gender differences in the ages at which there were the greatest differences in BMI between English and US children. In girls the biggest differences in BMI were at age 4, when English girls had a mean BMI 0.42kg/m 2 (95% CI 0.25 to 0.60) higher than US girls, and at age 13, when English girls had a mean BMI -1.62kg/m 2 (95% CI -1.88 to -1.36) lower than US girls. In boys, the biggest differences were at age 2, when English boys had a mean BMI 0.64kg/m 2 (95% CI 0.43 to 0.84) higher than US boys, and at age 16, when English boys had a mean BMI -2.40kg/m 2 (95% CI -2.69 to -2.12) lower than US boys.

Prevalence of overweight and obesity

Tables 2 and3 show the prevalence of overweight/obesity and obesity for England and the US in age-and gender-specific groups according to each of the three criteria. Using the 2000 CDC criteria, the prevalence of obesity in 2-through 5-year-olds (girls and boys combined) reflected the pattern for mean BMI: prevalence of obesity in English children was 14.2% (95% CI 13.1 to 15.3) and in US children in this age group was 11.9 (95% CI 10.2 to 13.6; mean difference = 2.3%, 95% CI -0.3 to 4.3). The mean difference was similar for the UK 1990 criteria (mean difference = 2.0%, 95% CI 0.0 to 3.9). There was little between-country difference in the prevalence of obesity in this age group in either gender using the IOTF criteria (Table 3).

Estimates of obesity for older children were consistent with patterns of mean BMI. Whilst the actual estimates of obesity prevalence varied considerably by the criteria used in both countries, the mean difference in prevalence was similar for each criteria. For example, at ages 12 through 17 the prevalence of obesity was markedly lower in English than US adolescents using the 2000 CDC criteria for obesity (difference = -7.4%, 95% CI -5.8 to -9.0), using the IOTF criteria (difference = -6.6%, 95% CI -4.7 to-8.5) and using the UK 1990 criteria (difference = -8.3%, 95% CI -10.0 to -6.5). US adolescents (aged 12 through 17) had the highest prevalence of obesity by age group and by country using each of the three criteria, with the highest estimate being 26.4% using the UK 1990 criteria.

Within country and gender groups, patterns by age differed according to the criteria used.

For example, applied to English boys the three criteria each suggested a different pattern by age: 2000 CDC criteria indicated a higher prevalence of obesity in 2-through 5-yearolds (15.8%, 95% CI 14.2 to 17.3) than in 12-through 17-year-olds (9.0%, 95% CI 8.1 to 9.9) but the UK 1990 criteria indicated the opposite: a lower prevalence of obesity in 2through 5-year-olds (14.7%, 95% CI 13.2 to 16.2) than in 12-through 17-year-olds (17.7%, 95% CI 16.5 to 19.0). For the same groups, IOTF criteria showed almost no difference between the 2-through 5-year-olds and the 12-through 17-year-olds, with estimated prevalences of obesity of 6.0% (95% CI 5.0 to 7.1) and 6.1% (95% CI 5.3 to 6.9), respectively. The same pattern of differences, according to the different criteria, was evident for obesity in English girls. In US boys and girls all three criteria identified increasing prevalence of obesity with age but the relative proportions were different: in US girls, for example, the 2000 CDC criteria identified 11.7% (95% CI 9.5 to 13.8) of 2through 5-year-olds and 16.0% (95% CI 14.1 to 17.9) of 12-through 17-year-olds as obese whereas the UK1990 criteria identified a slightly lower percentage of 2-through 5year-olds (10.9%, 95% CI 8.9 to 12.9) but a higher percentage of 12-through 17-year-olds (24.3%, 95% CI 22.0 to 26.6) as obese.

Figures 2 and3 show how the criteria for obesity vary by age and gender. These illustrate why we see the differences in patterns by age, gender and country as described above.

For example, in boys the IOTF criteria are higher than either of the two country specific criteria at all ages, resulting in lower prevalences of obesity. In contrast, for girls the IOTF criteria is similar to that of the UK1990 growth curve 95 th percentile from ages 2 through 7, higher than both the country specific criteria until age 15, and has criteria similar to the 2000 CDC curve 95 th percentile (but not the UK1990 curve) at ages 16 and 17.

We analysed the mean BMI and obesity prevalence (using only the IOTF criteria) for the age groups 2-5, 6-11, 12-19, by two year periods to check if there were between country differences in trends across time in BMI and obesity prevalence (Table 4

web-reference).

There is no evidence of a difference between the countries in trends overtime. However, the numbers within each time period strata are small. There is also no evidence of a difference between consecutive two year periods for the prevalence of obesity or the mean BMI.

Discussion

Mean BMI differed between the countries by age, with English children having a higher mean BMI than their US peers at ages 2 through 5 but lower mean BMI at ages 8 and above. For public health the prevalence of obesity is particularly important but because population distributions of BMI are positively skewed, the patterns of mean BMI do not necessarily reflect patterns of obesity. We found US adolescents (ages 12 through 17) had the highest prevalence of obesity of all age groups and in either country by all three criteria (UK 1990, 2000 CDC, andIOTF).

A key finding of our analyses is that there were marked differences in obesity prevalence in each country and by age and gender when applying each of the three different criteria.

However, general differences in the prevalence between England and the US were consistent irrespective of which criteria were used, except at younger ages where for all three criteria we found English children aged 2-5 to have a higher prevalence than US children, which was least marked for the IOTF criteria.

These findings have important implications for public health surveillance, clinical practice, and research into childhood obesity. For example, the higher prevalence of obesity in young children in England compared to the US might suggest efforts to prevent childhood obesity in England should focus on surveillance and intervention research in pre-school children. Furthermore, using IOTF criteria the overall prevalence of childhood obesity in both England and the US in all age groups was considerably lower than using either of the country-specific criteria, which is in keeping with studies which have found that the IOTF criteria have poor sensitivity for obesity (Reilly et al. 2000;[START_REF] Reilly | Accuracy of simple clinical and epidemiological definitions of childhood obesity: systematic review and evidence appraisal[END_REF][START_REF] Zimmermann | Detection of overweight and obesity in a national sample of 6-12-y-old Swiss children: accuracy and validity of reference values for body mass index from the US Centers for Disease Control and Prevention and the International Obesity Task Force[END_REF]). Using the IOTF criteria one would be likely to estimate the magnitude of the childhood obesity epidemic as far smaller than when using the country-specific analyses.

Strengths and weaknesses of the study

A key strength of this analysis is the use of data collected as part of nationally representative surveys with standardized methods. We have used three established methods of classifying children and adolescents as overweight/obese and obese.

International comparisons of the prevalence of obesity have relied on published studies from single countries using whatever classification system and years of coverage were used in the initial publication [START_REF] Wang | Worldwide trends in childhood overweight and obesity[END_REF]. Our findings suggest relying on country-specific classifications is likely to present problems in terms of making meaningful comparisons between as well as within countries.

A limitation of this study is that the prevalence estimates by age and gender are pooled over an 8-year period, during which time the prevalence of obesity increased and more recently has appeared to plateau (though longer term trends are required before it can be concluded that the epidemic has reached its peak in these countries) [START_REF] Ogden | High body mass index for age among US children and adolescents, 2003-2006[END_REF]The Information Centre. 2008). The relatively small numbers of obese children/adolescents in each age group by year make it impossible to draw conclusions about annual differences between countries, however there appeared to be no difference between the two countries in trends over time when we examined the data in two-year periods. The mean and median age of the 2-5 year old English children was slightly older than the US children; this is unlikely to explain the differences seen in mean BMI because older children in the 2-5 year age group have decreasing BMI. We have not attempted to distinguish how the prevalence of obesity differs between the two countries in relation to these ethnicity and socioeconomic profiles because the two surveys assess these factors differently.

Two studies suggest all three criteria for defining childhood obesity used here have similar and high specificity but that the IOTF criteria may have lower sensitivity when compared to a gold standard of high body fat (Reilly, Dorosty, Emmett, & The ALSPAC Study Team 2000;[START_REF] Reilly | Accuracy of simple clinical and epidemiological definitions of childhood obesity: systematic review and evidence appraisal[END_REF][START_REF] Zimmermann | Detection of overweight and obesity in a national sample of 6-12-y-old Swiss children: accuracy and validity of reference values for body mass index from the US Centers for Disease Control and Prevention and the International Obesity Task Force[END_REF]. It could be argued that the best method for defining childhood obesity should be that which best predicts adverse obesity related short-and long-term outcomes. We therefore recommend that the IOTF and one or more country-specific criteria are used and compared in research studies until it is clear which provides the best method for identifying children at greatest adverse consequences from obesity. US adolescents aged 12 through 17 had the highest prevalence of obesity by age group compared to England using each of the three criteria assessed. In contrast, English children aged 2 through 5 had higher mean BMI and higher prevalence of obesity. It is unclear what factors drive these between country differences as there is currently a lack of high-quality studies exploring country-level differences in risk factors for obesity and examining how these explain the observed differences. Differences in the age of adiposity rebound and onset of puberty could account for some of the differences in mean BMI and obesity; both have been documented to occur earlier in US compared to European populations [START_REF] Euling | Role of Environmental Factors in the Timing of Puberty[END_REF][START_REF] Wang | Epidemiology of childhood obesity -methodological aspects and guidelines: what is new?[END_REF][START_REF] Wyshak | Evidence for a Secular Trend in Age of Menarche[END_REF]. However, it has not been possible to examine this further using this data.

Our results demonstrate very marked differences in the prevalence of childhood obesity by age, gender and country when different, established methods for defining childhood obesity are applied to the data. For public health surveillance, clinical practice, research, and to permit meaningful international comparisons in prevalence of childhood obesity, it would be ideal to identify a gold-standard method of defining childhood obesity and to apply this method universally. Until there is clear evidence for adopting one method to apply to all data both IOTF and one or more country-specific criteria should be reported in future country-level surveillance systems and research. This would make international comparisons feasible and contribute to our understanding of the best method for defining childhood obesity, whilst preventing selective use of different criteria for political or publication reasons.

  US data were from repeated waves of the National Health and Nutrition Examination Study (NHANES). NHANES was approved by the National Centers for Health Statistics institutional review board, and all participants provided written informed consent. To account for the clustered, stratified multistage sample design, complex sampling, weighted estimates of population parameters were computed (National Center for Health Statistics 2006).

Figure 2 .

 2 Figure 2.

Figure 3 .

 3 Figure 3.

Table 1 . Characteristics of participants in English and US surveys

 1 Figure 1. Mean BMI in English and US children and adolescents for 1999 through 2006 by age, boys and girls combined, with 95% Confidence Intervals Figure 2. BMI thresholds for defining obesity by gender and age, boys aged 2 through 17 Figure 3. BMI thresholds for defining obesity by gender and age, girls aged 2

	Characteristics	Distribution	England (n=33,563)	US (n=14,540)
	2-5y children		Male	Female	Male	Female
		Mean (SD)	3.50 (1.12)	3.50 (1.11)	3.31 (1.14)	3.33 (1.16)
	Age	Median (IQR)	4.00 (3)	3.50 (1)	3.00 (2)	3.00 (2)
		Skewness	-0.011	0.016	0.233	0.180
		Mean (SD)	17.99 (3.98)	17.41 (3.96)	17.43 (4.54)	16.82 (4.04)
	Weight	Median (IQR)	17.6 (4.8)	16.9 (4.8)	16.5 (5.0)	16.2 (5.1)
		Skewness	2.097	1.432	1.970	1.307
		Mean (SD)	103.39 (9.43)	102.2 (9.64)	102.05 (9.83)	101.11 (9.73)
	Height	Median (IQR)	104.0 (14.1)	102.4 (14.5)	101.20 (15.5)	101.0 (15.5)
		Skewness	-0.142	0.093	0.257	0.151
		Mean (SD)	16.75 (2.52)	16.56 (2.19)	16.53 (2.05)	16.29 (1.97)
	BMI	Median (IQR)	16.45 (1.96)	16.3 (2.03)	16.19 (1.93)	15.93 (2.11)
		Skewness	6.568	3.310	2.401	1.519
	6-11y children
		Mean (SD)	8.50 (1.71)	8.52 (1.71)	8.48 (1.70)	8.50 (1.72)
	Age	Median (IQR)	9.00 (3.00)	8.00 (3.00)	8.00 (3.00)	8.00 (3.00)
		Skewness	-0.012	0.004	0.014	0.009
		Mean (SD)	32.20 (9.68)	32.82 (10.45)	34.74 (12.64)	35.36 (13.55)
	Weight	Median (IQR)	30.2 (11.8)	30.6 (13.1)	31.7 (14.7)	31.8 (17.4)
		Skewness	1.238	1.181	1.605	1.315
		Mean (SD)	133.67 (11.61)	133.55 (12.17)	134.75 (11.76)	134.98 (13.08)
	Height	Median (IQR)	133.5 (17.0)	133.1 (17.7)	134.40 (17.0)	134.00 (20.1)
		Skewness	0.048	0.146	0.190	0.214
		Mean (SD)	17.67 (3.07)	18.01 (3.36)	18.63 (4.30)	18.80 (4.36)
	BMI	Median (IQR)	16.9 (3.3)	17.2 (3.9)	17.31 (4.70)	17.49 (5.19)
		Skewness	1.544	1.953	1.800	1.390
	12-17y adolescents
		Mean (SD)	14.24 (1.66)	14.40 (1.68)	14.53 (1.73)	14.45 (1.70)
	Age	Median (IQR)	14.00 (3.00)	14.00 (3.00)	15.00 (3.00)	14.00 (3.0)
		Skewness	0.186	0.088	-0.040	0.062
		Mean (SD)	58.80 (15.38)	56.74 (12.80)	66.16 (20.44)	61.01 (16.82)
	Weight	Median (IQR)	57.6 (19.5)	55.1 (15.1)	63.1 (22.9)	57.1 (18.1)
		Skewness	0.799	0.934	1.240	1.501
		Mean (SD)	166.01 (11.51)	160.01 (7.27)	168.10 (10.82)	159.97 (7.10)
	Height	Median (IQR)	167.2 (16.9)	160.2 (9.5)	169.2 (14.5)	160.1 (9.3)
		Skewness	-0.296	-0.301	-0.425	-0.052
		Mean (SD)	21.008(3.98)	22.05 (4.27)	23.12 (5.80)	23.73 (5.88)
	BMI	Median (IQR)	20.3 (4.6)	21.22 (4.99)	21.75 (6.67)	22.14 (6.71)
		Skewness	1.270	1.194	1.592	1.488

Obesity is defined as having age-gender specific Body Mass Index (BMI) above the 95 th centile in the

CDC/ UK 1990 growth charts or above the criteria for obesity in the IOTF classification. 2 Difference is England percentage minus US percentage, i.e. positive values indicate English figure higher than US, negative values indicate lower English figure

Acknowledgements:

No funder played any role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Sources

of support for the work: no specific funding was received for the work. IAL is funded by NIHR, the UK National Institute for Health Research, as part of PenCLAHRC, the Peninsula Collaboration for Leadership in Applied Health Research and Care. RRK is supported by the South West Public Health Training Scheme. DAL is funded by the Higher Education Funding Council for England. DAL has grants from the UK Medical 2 Research Council, the Wellcome Trust and the British Heart Foundation, although this work was not directly funded by any of these. RJ is funded from a Career Development Fellowship supported by the National Institute for Health Research. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.

Conflict of interest statement: All authors declare that they have no competing interests.

Supplementary information is available at The European Journal of Clinical Nutrition's website. 11.9 (10.2 to 13.6) 2.3 (-0.3 to 4.3) 13.2 (12.2 to 14.3) 11.3 (9.5 to 13.0) 2.0 ( 0.0 to 3.9) 6.1 (5.3 to 6.9) 5.6 (4.6 to 6.6) 0.5 (-2.4