Rachel Nadif

PhD Marta Rava
email: marta.rava@inserm.fr

Brigitte Decoster

Hélène Huyvaert

Nicole Le Moual

Jean Bousquet

Valérie Siroux

Raphaëlle Varraso

Isabelle Pin

Farid Zerimech

Régis Matran

Exhaled NO, nitrite/nitrate levels, allergy, rhinitis and asthma in the EGEA study

Keywords: asthma, allergic sensitization, rhinitis, adults, NO metabolism, exhaled breath condensate, total nitrite/nitrate, exhaled fraction of NO

Background: Although interest in biomarkers in the nitrate-nitrite-NO pathway has recently increased, associations between nitrite (NO 2 -), nitrate (NO 3 -) and asthma, allergic sensitization and rhinitis remain unclear. Objective: To evaluate the associations between NO 2 -/NO 3 -and exhaled fraction of nitric oxide (FE NO ) levels with asthma, allergic sensitization and rhinitis. Methods: Plasma and exhaled breath condensate (EBC) NO 2 -/NO 3 -and FE NO levels were measured in 523 adults of the French Epidemiological study on Genetics and Environment of Asthma. Allergic sensitization was defined by a positive skin prick test for at least one aeroallergen. Subjects were classified as non-sensitized, sensitized and as having allergic rhinitis. Results: Plasma NO 2 -/NO 3 -level was unrelated to any disease phenotypes. EBC NO 2 -/NO 3 level was unrelated to any asthma phenotypes. EBC NO 2 -/NO 3 -and FE NO levels were correlated in sensitized subjects only (r=0.21±0.10, p=0.01). EBC NO 2 -/NO 3 -and FE NO levels were higher in sensitized than in non-sensitized subjects (adjusted GM (95%CI): 2.36 (1.96; 2.84) vs.1.72 (1.38; 2.14) µmol/mg proteins, p=0.008; and 18.3 (16.7; 20.0) vs.14.8 (13.3; 16.5) ppb, p=0.0006 respectively), with gradual relationships from sensitized subjects to those with allergic rhinitis (p<0.0001).

Conclusion:

Results suggest that EBC NO 2 -/NO 3 -and FeNO levels may be considered as biological markers of intensity of allergic sensitization and rhinitis.

Introduction

The interest in measuring biological markers in exhaled breath condensate (EBC) in epidemiological studies on respiratory diseases has increased in the last years. Among the pathways involved in the pathophysiology of asthma, the metabolism of nitric oxide (NO) also called the nitrate-nitrite-NO pathway has taken a growing place in this research field [START_REF] Lundberg | The biological role of nitrate and nitrite: the times they are a-changin[END_REF].

The NO metabolism is complex, and both NO measured by the exhaled fraction of NO (FE NO ) and NO-related compounds such as nitrites (NO 2 -) and nitrates (NO 3 -) are relevant biological markers that may help to better understand the patho-physiology of asthma and allergy [START_REF] Lundberg | The nitrate-nitrite-nitric oxide pathway in physiology and therapeutics[END_REF].

FE NO is the most studied one, and it is commonly considered as a non-invasive indirect marker of airway inflammation [START_REF] Szefler | Asthma outcomes: biomarkers[END_REF]. Both epidemiological and clinical studies in adults showed increased level of FE NO in children and adults with asthma, and positive associations between FE NO and allergic sensitization are consistent over the studies, regardless of rhinitis or asthma [START_REF] Dweik | An official ATS clinical practice guideline: interpretation of exhaled nitric oxide levels (FENO) for clinical applications[END_REF]. Studies on associations between NO 2 -and NO 3 -levels with asthma, allergy or rhinitis have led to more conflicting results both in adults and children [START_REF] Chérot-Kornobis | Analysis of nitrogen oxides (NOx) in the exhaled breath condensate (EBC) of subjects with asthma as a complement to exhaled nitric oxide (FeNO) measurements: a crosssectional study[END_REF][START_REF] Chladkova | Validation of nitrite and nitrate measurements in exhaled breath condensate[END_REF][START_REF] Corradi | Nitrate in exhaled breath condensate of patients with different airway diseases[END_REF][START_REF] Dressel | Independent information of nonspecific biomarkers in exhaled breath condensate[END_REF][START_REF] Ganas | Total nitrite/nitrate in expired breath condensate of patients with asthma[END_REF][START_REF] Dweik | NO chemical events in the human airway during the immediate and late antigen-induced asthmatic response[END_REF][START_REF] Gratziou | Breath markers of oxidative stress and airway inflammation in Seasonal Allergic Rhinitis[END_REF][START_REF] Malinovschi | Exhaled breath condensate nitrates, but not nitrites or FENO, relate to asthma control[END_REF][START_REF] Nguyen | Assaying all of the nitrogen oxides in breath modifies the interpretation of exhaled nitric oxide[END_REF][START_REF] Ojoo | Exhaled breath condensate pH and exhaled nitric oxide in allergic asthma and in cystic fibrosis[END_REF][START_REF] Rihák | Nitrite in exhaled breath condensate as a marker of nitrossative stress in the airways of patients with asthma, COPD, and idiopathic pulmonary fibrosis[END_REF][START_REF] Robroeks | Exhaled nitric oxide and biomarkers in exhaled breath condensate indicate the presence, severity and control of childhood asthma[END_REF][START_REF] Ueno | Inflammatory markers in exhaled breath condensate from patients with asthma[END_REF]. Until now, none of these studies has simultaneously performed measurements of EBC NO 2 -/NO 3 -and FE NO levels in the same subjects. Recently, in a large number of adults from the French Epidemiological study on Genetics and Environment of Asthma (EGEA), EBC NO 2 -/NO 3 and FE NO levels were found to be correlated in subjects without asthma [START_REF] Bouzigon | Associations between Nitric Oxide Synthase Genes and Exhaled NO-Related Phenotypes according to Asthma Status[END_REF].

Nitric oxide has different functions and roles in pathophysiology, which may be better explained by considering its compartmentalized production [START_REF] Villanueva | Subcellular and cellular locations of nitric oxide synthase isoforms as determinants of health and disease[END_REF]. In this study, we compared the association between total NO 2 -/NO 3 -levels measured in two compartments (plasma and exhaled breath condensate) and FE NO levels with asthma, allergic sensitization and rhinitis among 523 adults from the EGEA study. We hypothesized that the associations will be different depending on the compartments, biomarkers and outcomes studied.

Methods

Study design

Data used for the analyses were collected in the framework of the 12-year follow-up of EGEA. EGEA is a French cohort study based on an initial group of asthma cases and their first-degree relatives, and controls (first survey, n=2047) [20]. The protocol and descriptive characteristics have been described previously [START_REF] Kauffmann | EGEA (Epidemiological study on the Genetics and Environment of Asthma, bronchial hyperresponsiveness and atopy)--design issues. EGEA Co-operative Group[END_REF][START_REF] Kauffmann | Epidemiological study of the genetics and environment of asthma, bronchial hyperresponsiveness, and atopy: phenotype issues[END_REF]. A follow-up of the initial cohort was conducted between 2003 and 2007 [START_REF] Nadif | Passive and active smoking and exhaled nitric oxide levels according to asthma and atopy in adults[END_REF]. Among the alive cohort (n=2002), 92% (n=1845) completed a short self-administered questionnaire and among them 1601 had a complete examination. All subjects responded to a questionnaire based on international standardized tools to diagnose asthma and to determine respiratory and allergic symptoms, treatments, and environmental exposures. The present cross-sectional analysis includes those who were adults at the second survey (≥ 16 years old, n=1570 adults) with available data on asthma, current rhinitis, allergic sensitization, and available measurements of exhaled breath condensate NO 2 -/NO 3 -and FE NO levels (n=523). Subjects included in the analyses were younger, reported more often ever asthma and current rhinitis, and had higher levels of NO 2 -/NO 3 -and Immunoglobulin E (IgE) than those not included in the analyses (n=1047). The two groups were similar for sex, smoking, current asthma status, allergic sensitization, lung function tests, and eosinophil (EOS) count (see Table 1 in supplementary data).

Ethical approval was obtained from the relevant institutional review board committees (Cochin Port-Royal Hospital and Necker-Enfants Malades Hospital, Paris). Written informed consent was signed by all participants.

Respiratory phenotypes

Subjects with ever asthma were defined by a positive answer to either: "Have you ever had attacks of breathlessness at rest with wheezing?", or "Have you ever had asthma attacks?", or if they were recruited as asthmatic cases at the first survey.

Allergic sensitization was defined by a positive skin prick test (SPT+) with a mean wheal diameter 3mm than the negative control for at least one of 12 aeroallergens (indoor: cat, Dermatophagoides pteronyssinus, Blattela germanica, outdoor: olive, birch, Parieteria judaica, timothy grass, Cupressus and ragweed pollen, and molds: Aspergillus, Cladosporium herbarum, Alternaria tenuis). Subjects were classified as sensitized if they have one or more SPT+. Current rhinitis was defined by a positive answer to one of the two questions: "Have you ever had rhinitis?" or " Have you ever had hay fever?" and a positive answer to "have you had sneezing problems or a runny nose in the past 12 months?" Allergic rhinitis was defined as having both current rhinitis and one or more SPT+. Subjects were also classified in three groups as non sensitized (no SPT+), sensitized only (having one or more SPT+ and no current rhinitis) and as having allergic rhinitis (one or more SPT+ and current rhinitis).

Eosinophilia was defined as eosinophil count ≥5%. Details on other phenotypes are given in supplementary data.

Biological phenotypes

Exhaled breath condensate (EBC) was collected with an RTube™ according a standardized method. Briefly, the RTube (TM) was rinsed with deionized water and dried thoroughly.

Participants breathed orally at tidal volumes into a mouthpiece attached to a cold condenser (-20°C). They were seated comfortably with a headrest. All headrests and back seats were tilted slightly to avoid any saliva contamination during breathing maneuvers (see supplementary data for more details).

Total nitrite-nitrate (NO 2 -/NO 3 -) levels were measured in plasma and EBC as previously described [START_REF] Rava | Plasma and exhaled breath condensate nitrite-nitrate level in relation to environmental exposures in adults in the EGEA study[END_REF]. All measurements were done in duplicate. Analytical intra-run imprecision was below 3%. Measurements with a coefficient of variation >15% and extreme outliers (n=7) were excluded from the analyses (see supplementary data for more details).

Measurements of FE NO were realized before other pulmonary function tests according to ATS/ERS recommendations (see supplementary data). The measurement was performed only in 3 of the 5 centers involved in the EGEA study, which explained in a large part the attrition on numbers of subjects included in the analysis compared to the total number. FE NO level was measured at 50mL/s flow rate as previously described [START_REF] Nadif | Passive and active smoking and exhaled nitric oxide levels according to asthma and atopy in adults[END_REF].

Statistical Methods

Joint distribution of asthma, SPT+ and current rhinitis was shown with a Venn diagram (Figure 1). Total plasma and EBC NO 2 -/NO 3 -and FE NO levels were log10-transformed as a result of their skewed distribution.

In the same study, we previously reported that plasma NO 2 -/NO 3 -level was increased with leafy vegetable consumption and decreased in smokers and with storage time, that EBC NO 2 -/NO 3 -level was decreased in smokers and with exposure to ambient ozone concentration [START_REF] Rava | Plasma and exhaled breath condensate nitrite-nitrate level in relation to environmental exposures in adults in the EGEA study[END_REF], and that FE NO level was associated with season of examination [START_REF] Nadif | Passive and active smoking and exhaled nitric oxide levels according to asthma and atopy in adults[END_REF].

Furthermore, storage time and season of examination varied with centre. Therefore estimates were adjusted for 1) age, sex, smoking, leafy vegetable consumption and centre for plasma NO 2 -/NO 3 -, 2) age, sex, smoking, ambient ozone concentration and centre for EBC NO 2 -/NO 3 -, and 3) age, sex, height, smoking and centre for FE NO . Since the ratio of higher oxides of nitrogen (HiNOx including NO 2 -and NO 3 -) to NO was reported be more informative than each measurement alone by Nguyen et al. [START_REF] Nguyen | Assaying all of the nitrogen oxides in breath modifies the interpretation of exhaled nitric oxide[END_REF], the (NO 2 -+ NO 3 -)/NO ratio (NOx/NO ratio) has also been studied.

As inhaled corticosteroids (ICS) use can decrease FE NO levels, and as NO 2 -/NO 3 -and FE NO are biological markers involved in the same pathway, association between ICS use and EBC NO 2 -/NO 3 -level was studied. Since increased body mass index (BMI)/obesity has been associated with lower FeNO level, estimates were also adjusted for BMI as a sensitivity analyses.

Associations between total NO 2 -/NO 3 -levels, FE NO levels, and the NOx ratio (NO 2 -+ NO 3 -)/NO and asthma phenotypes, allergic sensitization and current rhinitis were estimated with linear regression models. Parameter estimates were assessed by using generalized estimating equations, with an exchangeable working correlation to account for the potential clustering within families (SAS MIXED procedure). The level of statistical significance was set at α=0.05. Two-sided P values were reported for all association estimates. All analyses were conducted using SAS software, version 9.3 (SAS Institute, Inc., Cary, NC, USA).

Results

The characteristics of the 523 adults according to their asthma status are summarized in Table 1. As expected, subjects with asthma had significantly higher eosinophilia, lower FEV 1 % predicted, more often bronchial hyper-responsiveness (BHR), SPT+, and reported more often current rhinitis than subjects without asthma. After adjustment for age, sex and smoking, the following associations between asthma and eosinophilia (odds ratio (OR) 3.12, 95% confidence interval (CI, 1.72-5.66), FEV 1 % predicted (mean ± SD: 107.7 ± 10.5 vs. 97.5 ± 11.0), BHR (OR, 4.18; 2.66-6.57), SPT+ (OR, 5.07; 3.32-7.72) and current rhinitis (OR, 4.62;

3.11-6.85) were confirmed (all P<0.0005). EOS count, IgE and FE NO levels were significantly higher in subjects with asthma than in those without (all P<0.0001). EBC NO 2 -/NO 3 -level was unrelated to ICS use (data not shown, P value=0.5).

Pairwise association between EBC NO 2 -/NO 3 -, FE NO levels and blood eosinophil counts

EBC NO 2 -/NO 3 -levels were unrelated with EOS, whereas FE NO levels were positively associated with EOS in all subjects, both in non-sensitized and sensitized subjects (Table 2).

In sensitized subjects, EBC NO 2 -/NO

Plasma and EBC NO 2 -/NO 3 -, FE NO levels and asthma and asthma-related phenotypes

Both plasma and EBC NO 2 -/NO 3 -levels were unrelated to ever asthma, current asthma, symptomatic score, and asthma control (data not shown, all P values>0.3). Furthermore, plasma NO 2 -/NO 3 -level was unrelated to allergic sensitization and current rhinitis.

Both plasma and EBC NO 2 -/NO 3 -levels were unrelated to eosinophilic asthma nor to age at asthma onset (Table 3). As expected, a positive and significant association was observed between FE NO level and eosinophilic asthma (Table 3) but no other significant association was observed.

EBC NO 2 -/NO 3 -and FE NO levels, NOx/NO ratio and allergic sensitization

A positive association at borderline significance was observed between EBC NO 2 -/NO 3 -level and SPT+ (see Table 4). In a model adjusted for covariates, including asthma, EBC NO 2 -/NO 3 -level was positively and significantly associated with SPT+, and a positive association at borderline significance was observed with current rhinitis. Furthermore, positive and gradual increases in EBC NO 2 -/NO 3 -level were observed with SPTQ (see Figure 1 in supplementary data), and when subjects were classified in the following groups: no SPT+, SPT+ only, and both SPT+ and current rhinitis (see Table 4 and Figure 2). The median EBC NO 2 -/NO 3 -value in our population was 2.25 µmol/mg proteins (range 1.12 to 4.98, Table 1). Stratification according to this median value showed positive and significant associations between subjects above the median value and 1) allergic sensitization (OR, 1.92; Similarly to EBC NO 2 -/NO 3 -, FE NO level was also positively related to allergic sensitization expressed as SPT+ (Table 4), and gradually increased with SPTQ (see Figure online for details), and with allergic sensitization and current rhinitis. FE NO was also significantly and positively associated with current rhinitis. The associations between FE NO levels and SPT+, and current rhinitis were confirmed when BMI instead of height was added as covariate in the models (data not shown).

No significant association were observed between the (NO 2 -+ NO 3 -)/NO ratio (NOx/NO ratio) and SPT+, current rhinitis or both (Table 4 and Figure 2). When analysing SPT+ to indoor, outdoor or molds allergens separately, EBC NO 2 -/NO 3 -levels showed positive and significant associations with sensitization to molds allergens, and FE NO levels were positively associated to indoor allergens (Table 4).

Discussion

The present study conducted on a large sample of adults with a precise phenotypic characterization shows for the first time the similarities and differences for the associations of both FE NO and exhaled breath condensate NO 2 -/NO 3 -levels with asthma, allergic sensitization and rhinitis. Results showed higher EBC NO 2 -/NO 3 -and FE NO levels in subjects with allergic sensitization, with current rhinitis, and in particular when both are present. Only FE NO levels were found to be higher with asthma. EBC NO 2 -/NO 3 -and FE NO levels were positively associated in sensitized subjects only, and EBC NO 2 -/NO 3 -levels were found to be associated with allergic sensitization in subjects with higher FE NO levels only.

The selection of the 523 subjects included in the present analyses was driven first by the random availability of the FE NO measurements in three of the five participating centres [START_REF] Nadif | Passive and active smoking and exhaled nitric oxide levels according to asthma and atopy in adults[END_REF],

and secondly by the availability of the other variables of interest. Definition of asthma case is very precise in our study since asthmatic cases were recruited in chest clinics, and a procedure was set up to include true asthmatics, leading to a very limited risk of false positives.

Prevalence of bronchial hyper-responsiveness, measured by a methacholine challenge test was quite high in subjects without asthma. A possible explanation is that part of the subjects without asthma are first degree relatives of asthma cases. Nevertheless, this result is consistent with the relatively considerable number of asymptomatic subjects with BHR reported in cross-sectional epidemiologic studies, ranging from 19.3 to 62.4%. Subjects included in the analyses had higher EBC NO 2 -/NO 3 -and FE NO levels than non-selected subjects. Other limitations of the present study were those commonly related to crosssectional analyses of the data.

We reported no association between NO 2 -/NO 3 -level measured in plasma and any disease phenotypes. We previously reported that plasma and EBC NO 2 -/NO 3 -levels were not correlated [START_REF] Bouzigon | Associations between Nitric Oxide Synthase Genes and Exhaled NO-Related Phenotypes according to Asthma Status[END_REF]. The metabolism of NO is complex, and the production of NO 2 -/NO 3 -in plasma differs from that in EBC due to their compartmentalization. In plasma NO 2 -/NO 3 production derives from several sources, such as bacteria, enzymatic production and dietary sources [START_REF] Lundberg | NO-synthase independent NO generation in mammals[END_REF]. In EBC ionized NO 3 -and NO 2 -(not volatile) may arise from NO after reaction with oxygen [START_REF] Hunt | Exhaled breath condensate: an overview[END_REF] or from activated immune cells present in the lining fluid of the lungs [START_REF] Fitzpatrick | Levels of nitric oxide oxidation products are increased in the epithelial lining fluid of children with persistent asthma[END_REF].

Overall, the specificities of the NO metabolism in plasma and in EBC may partly explain the lack of association with any clinical phenotypes in plasma. Our results are consistent with the hypotheses of Villanueva and Giulivi [START_REF] Villanueva | Subcellular and cellular locations of nitric oxide synthase isoforms as determinants of health and disease[END_REF], for whom the compartmentalized production of NO better explains its different functions and roles in pathophysiology.

No association was found between total NO 2 -/NO 3 -level in EBC and asthma phenotypes, as previously reported in other studies [START_REF] Dressel | Independent information of nonspecific biomarkers in exhaled breath condensate[END_REF][START_REF] Ojoo | Exhaled breath condensate pH and exhaled nitric oxide in allergic asthma and in cystic fibrosis[END_REF]. Contrary to our results, other studies have reported total NO 2 -/NO 3 -level in EBC to be elevated in subjects with asthma as compared to healthy non-smoking subjects [START_REF] Ueno | Inflammatory markers in exhaled breath condensate from patients with asthma[END_REF], healthy non-atopic controls [START_REF] Chérot-Kornobis | Analysis of nitrogen oxides (NOx) in the exhaled breath condensate (EBC) of subjects with asthma as a complement to exhaled nitric oxide (FeNO) measurements: a crosssectional study[END_REF], or controls [START_REF] Ganas | Total nitrite/nitrate in expired breath condensate of patients with asthma[END_REF]. These conflicting results may be due to the very small number of subjects included in these studies, the various methodologies used for measuring NO 2 -and NO 3 -levels, the choice of the reference group for comparisons and other differences such as those related to phenotypes definition. Beside, information regarding allergic sensitization was not available or subjects were defined as asthmatics if they had both asthma and allergy, suggesting that the increase in NO 2 -/NO 3 level could be more related to allergy than to asthma. Furthermore, none of these previous studies have expressed the NO 2 -/NO 3 -level divided by the amount of proteins. As reported by Gessner and Wirtz [START_REF] Gessner | Interleukins and other proteins[END_REF], the measurement of total protein in EBC is important to confirm that protein and peptide markers are comparable between studies. They should always be performed in addition to specific markers investigated, and we previously found that NO 2 -/NO 3 -level in EBC was positively related to protein concentration in our study [START_REF] Nadif | Total Nitrate/Nitrite Levels In Plasma And Exhaled Breath Condensate: Associations With Age And Smoking According To Asthma Among 1159 Adults From The EGEA Study[END_REF].

As reported in the literature [START_REF] Dweik | An official ATS clinical practice guideline: interpretation of exhaled nitric oxide levels (FENO) for clinical applications[END_REF][START_REF] Condensate | ATS Workshop Proceedings: Exhaled nitric oxide and nitric oxide oxidative metabolism in exhaled breath condensate: Executive summary[END_REF], positive associations between FE NO level and asthma, allergic sensitization, and current rhinitis were found in this study. To our knowledge, our study reported for the first time similarities and differences for the associations of both FE NO and exhaled breath condensate NO 2 -/NO 3 -levels with asthma, allergic sensitization and current rhinitis. We found positive associations between EBC NO 2 -/NO 3 -and FE NO levels in sensitized subjects, and between EBC NO 2 -/NO 3 -levels with allergic sensitization in subjects with higher FE NO levels. Consistently, we found that both EBC NO 2 -/NO 3 -and FE NO levels increased with allergic sensitization, with the number of SPT+, and that gradual relationships were observed between sensitized subjects only and those with both allergic sensitization and rhinitis. An immediate practical utility could not be inferred from the results obtained in the framework of this epidemiologic study; but taken together, our results suggest that EBC NO 2 -/NO 3 -and FE NO levels may be considered as biological markers of intensity of allergic sensitization and rhinitis. Longitudinal studies are also needed to better understand the role of these biomarkers, in line with the idea that part of the "allergic march" involves oxidative and nitrosative processes.

By considering together EBC NO 2 -/NO 3 -and FE NO levels rather than each alone, our results provided complementary interesting information. To go further, we also studied the association between the NOx(NO 2 -+ NO 3 -)/NO ratio and allergic sensitization.

Unfortunately, this ratio was not more informative than considering the measurement of EBC NO 2 -/NO 3 -alone. Contrary to our results, a ratio including also S-nitrosothiols (NO 2 -+ NO 3 -+ S-nitrosothiols)/NO was found to better evaluate inflammation in a case-control study on asthma [START_REF] Nguyen | Assaying all of the nitrogen oxides in breath modifies the interpretation of exhaled nitric oxide[END_REF] than the measurement of each oxide of nitrogen alone. This discrepancy in the results may be partly explained by the lack of measurement of S-nitrosothiols in our study, by the fact that we studied allergic sensitization rather than inflammation, and/or by differences in study designs.

Overall, even if our results need to be replicated, they may suggest a role of the nitrate-nitrite-NO pathway in allergic sensitization. We suggest that exposure to allergens results in uptake and proceeding by dendritic cells inducing the development of Th2 cells in sensitized individuals. Recent evidence indicates that airway epithelium also plays an important role in the allergic airway response by the release of IL-25, IL-33 and TSLP which activate dendritic cells, basophils, eosinophils and Th2 cells [START_REF] Nadif | The role of eosinophils and basophils in allergic diseases considering genetic findings[END_REF][START_REF] Diamant | Biomarkers in asthma and allergic rhinitis[END_REF]. TSLP, IL-25 and IL-33 promote eosinophilia in airway mucosa by inducing IL-5 production. Eosinophilic airway inflammation may increase the NO concentration and subsequently produces the formation of NO 2 -, NO 3 -and reactive nitrogen species in EBC.

The results reported in this study highlight the complexity of NO metabolism. Initially considered completely inert, it is now apparent that nitrate and nitrite are physiologically recycled in blood and tissues to form NO and other bioactive nitrogen oxides [START_REF] Lundberg | The nitrate-nitrite-nitric oxide pathway in physiology and therapeutics[END_REF]. They may be viewed as storage pools for NO-like bioactivity, thereby complementing the NO synthase (NOS)-dependent pathway. NO and related compounds are produced by a wide variety of residential and inflammatory cells in the respiratory tract [START_REF] Sugiura | Nitrative stress in inflammatory lung diseases[END_REF]. In response to allergens, both dendritic cells (DCs) and airway epithelial cells are stimulated, and release various cytokines which activate DCs, basophils, mast cells, eosinophils and Th2 cells, leading to eosinophil activation and proliferation [START_REF] Blanchard | Biology of the eosinophil[END_REF]. We previously reported that FE NO level was positively associated with blood eosinophil counts [START_REF] Bouzigon | Associations between Nitric Oxide Synthase Genes and Exhaled NO-Related Phenotypes according to Asthma Status[END_REF], and there are in vitro evidences that human blood eosinophils produce NO and participate in the regulation of the NO pool in pulmonary tissues [START_REF] Pozo | Eosinophils transcribe and translate messenger RNA for inducible nitric oxide synthase[END_REF][START_REF] Iijima | Nitric oxide and protein nitration are eosinophil dependent in allergen-challenged mice[END_REF]. Moreover, NO modulates the Th1/Th2 balance by favoring Th2 response and IL-5 production and thus recruiting eosinophils into the airways. Nevertheless, even if EBC NO 2 -/NO 3 -level can be viewed as a potential biological marker of allergy in our study, its specific role remains unknown, and mechanistic studies are required. As suggested through the results of the present study, and as reported by Erzurum et al. [START_REF] Erzurum | Biomarkers in asthma: a real hope to better manage asthma[END_REF], the complexity of the nitrate-nitrite-NO pathway provide evidence that more targeted biological markers are needed to put them into a global scheme that help us to identify a type of response or phenotype for a given patient, requiring the integration of multiple factors in a system biology approach.

Further studies are also warranted to better investigate the associations we observed in this epidemiological study, and the potential for a practical utility of our findings.

In conclusion, we report for the first time in a large epidemiological study that both total NO 2 -/NO 3 -and FE NO levels in exhaled breath condensate are associated with allergic sensitization and rhinitis. The role of the nitrate-nitrite-NO pathway in the "allergic march" need to be further investigated in longitudinal studies. However, contrary to what has been shown with FENO, we did not find an association of this biomarker with clinical phenotypes of asthma. Studying both exhaled fraction of NO and EBC NO 2 -/NO 3 -may be helpful for disentangle the associations between NO metabolism and asthma, allergic sensitization and rhinitis. Data on current rhinitis was missing for four participants without allergic sensitization and asthma, one participant with asthma and one participant with allergic sensitization (n=517). Regression coefficients (Beta) and 95%CI for associations between Fe NO , total NO

Figure 1 .

 1 Figure 1. Concordance of ever asthma, allergic sensitization (SPT+) and current rhinitis

Figure 2 .

 2 Figure 2. Associations between Fe NO , total NO 2 -/NO 3 -and (NO 2 -+ NO 3 -)/NO ratio levels

Figure 1 33Fig 2

 12 Figure 1

  3-was positively associated with FE NO levels.

	The median FE NO value in the population was 15.6 ppb (range 2.4 to 99.0 ppb). Stratification
	according to this median value showed positive and significant association between EBC
	NO 2	-/NO 3	-level and allergic sensitization in subjects above the median only (2.66 (2.06-
	3.43) vs. 1.64 (1.18-2.28), P=0.01 and 2.03 (1.52-2.71) vs. 1.76 (1.30-2.38), P=0.4 in subjects
	above and below the median respectively).

Table 1 .

 1 NO ratio and allergic sensitization, current rhinitis and both, estimated through GEE linear regression methods, and adjusted for covariates: age, sex, smoking, ambient ozone concentration, asthma and centre for NO 2 -/NO 3 -level; age, sex, smoking, height, asthma and centre for Fe NO level; age, sex, smoking, asthma and centre for (NO 2 Characteristics of subjects according to asthma status

	2	-/NO 3 -

Table 2 .

 2 Pair-wise association of EBC NO 2 -NO 3 , FeNO levels and eosinophil count in all subjects, and according to allergic sensitization

			EBC NO 2	-/NO 3	-level, µmol/mg proteins a	Fe NO level, ppb b
		N	Estimate	SD	p-value N Estimate	SD	p-value
	All subjects
	Eosinophils, cells/mm 3	492		0.14	0.09	0.10 505	0.25	0.04	<0.0001
	Fe NO level, ppb	510		0.14	0.08	0.10
	Non-sensitized subjects
	Eosinophils, cells/mm 3	204		0.15	0.13	0.3 208	0.15	0.05	0.01
	Fe NO level, ppb	215		-0.17	0.15	0.3
	Sensitized subjects
	Eosinophils, cells/mm 3	288		0.15	0.11	0.2 297	0.31	0.06	<0.0001
	Fe NO level, ppb	295		0.21	0.10	0.04
	Estimates are adjusted for

a : age, sex, smoking, ambient ozone concentration, asthma and centre; b : age, sex, smoking, height, asthma and centre (GEE linear regression methods).

Acknowledgments

The authors thank all those who participated in the study and in the various aspects of the examinations and all those who supervised the study in all centers. The authors are grateful to the three CIC-Inserm units of Necker, Grenoble and Marseille which supported the study and where subjects were examined. They are also grateful to the three biobanks in Lille (CIC-Inserm), Evry (Centre National de Genotypage) and Annemasse (Etablissement francais du sang) where biological samples are stored. They are indebted to all the individuals who participated, without whom the study would not have been possible.

EGEA cooperative group

Coordination: V Siroux (epidemiology, PI since 2013); F Demenais (genetics); I Pin (clinical aspects); R Nadif (biology); F Kauffmann (PI 1992-2012). Respiratory epidemiology: Inserm U 700, Paris: M Korobaeff (Egea1), F Neukirch (Egea1); Inserm U 707, Paris: I Annesi-Maesano (Egea1-2) ; Inserm CESP/U 1018, Villejuif: F Kauffmann, N Le Moual, R Nadif, MP Oryszczyn (Egea1-2), R Varraso ; Inserm U 823, Grenoble: V Siroux. Genetics: Inserm U 393, Paris: J Feingold ; Inserm U 946, Paris: E Bouzigon, F Demenais, MH Dizier ; CNG, Evry: I Gut (now CNAG, Barcelona, Spain), M Lathrop (now Univ McGill, Montreal, Canada). Clinical centers: Grenoble: I Pin, C Pison; Lyon: D Ecochard (Egea1), F Gormand, Y Pacheco ; Marseille: D Charpin (Egea1), D Vervloet (Egea1-2) ; Montpellier: J Bousquet ; Paris Cochin: A Lockhart (Egea1), R Matran (now in Lille) ; Paris Necker: E Paty (Egea1-2), P Scheinmann (Egea1-2) ; Paris-Trousseau: A Grimfeld (Egea1-2), J Just. Funding Research funded in part by PHRC-Paris, PHRC-Grenoble, ANR 05-SEST-020-02/05-9-97, ANR-06-CEBS, ANR-CES-2009, Region Nord Pas-de-Calais, Merck Sharp & Dohme (MSD), and the GA2LEN project, Global Allergy and Asthma European Network.

Data and quality management: Inserm ex-U155 (Egea1): J Hochez ; Inserm CESP/U 1018, Villejuif: N Le Moual ; Inserm ex-U780: C Ravault (Egea1-2) ; Inserm ex-U794: N Chateigner (Egea1-2) ; Grenoble: J Quentin-Ferran (Egea1-2).


Jessica Lebenberg
email: jessica.lebenberg@gmail.com

Alain Lalande

Member, IEEE Patrick Clarysse

Member, IEEE Irène Buvat

Christopher Casta

Alexandre Cochet

Constantin Constantinides

Jean Cousty

Alain De Cesare

Stéphanie Jehan-Besson

Constantin Constantinidès

Muriel Lefort

Laurent Najman

Elodie Roullot

Laurent Sarry

Christophe Tilmant

Member, IEEE Frédérique Frouin

Mireille Garreau

Improved estimation of the left ventricular

Keywords: Multiple segmentations, Regression Without Truth, STAPLE, Cardiac image analysis

I. INTRODUCTION

C Ardiac Magnetic Resonance Imaging (cMRI) is used more and more frequently in clinical routine to study simultaneously the cardiac anatomy and function. A series of clinical parameters can be deduced from the acquired scans in cMRI. Among these parameters, the left ventricular ejection fraction (LVEF) remains a major prognostic index for coronary artery diseases assessment. The correct estimation of this parameter requires the accurate measurement of both enddiastolic and end-systolic volumes. Although MRI makes these measurements possible with a high precision (generally from a series of short-axis cine-MR images), the segmentation of the left ventricle (LV) is still a contemporary issue [START_REF] Petitjean | A review of segmentation methods in short axis cardiac MR images[END_REF] due to the colossal amount of data that are acquired in a single examination. For clinical routine, semi-automated algorithms that are proposed by commercial image post-processing software are largely used. For retrospective studies, research studies, or large database studies, automated segmentation algorithms are preferentially used in order to avoid the labor intensive and time consuming manual segmentation task and reduce the intra-and inter-operator variabilities [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF]. To assess the performance of these automated segmentation algorithms, the common approach consists in comparing the contours resulting from the automated segmentation with the ones obtained by one or several experts who are known to often outperform automated methods [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF].

When visually comparing segmentation results obtained by different automated methods as in [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF], the respective performance of two methods depends on the data: when a first segmentation method provides more accurate contours than a second automated method on a specific database, the second algorithm might actually be more relevant for a subdatabase or, at least, for some particular MR examinations. Therefore, it is reasonable to hypothesize that there might be an advantage in combining several automated segmentation methods to overcome the specific limitations of each one.

To combine segmentation approaches, different algorithms have been proposed [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF], [START_REF] Warfield | Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation[END_REF], [START_REF] Chen | Evaluation of multiple-atlas-based strategies for segmentation of the thyroid gland in head and neck ct images for imrt[END_REF], [START_REF] Robitaille | Label fusion strategy selection[END_REF], [START_REF] Jehan-Besson | A mutual reference shape based on information theory[END_REF]. The Simultaneous Truth and Performance Level Estimation (STAPLE) algorithm [START_REF] Warfield | Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation[END_REF] is very popular and highly cited. Furthermore, the associated software is freely available for academic purposes upon written request. For these reasons, we evaluated the performance of STAPLE. To objectively assess the segmentation accuracy, criteria based on estimated contours and associated image classification are often used. These include various metrics allowing to compare boundaries at a local level such as distances between contours, overlap criteria like the Dice coefficient [START_REF] Dice | Measures of the amount of ecologic association between species[END_REF], or the sensitivity, the specificity, the predictive negative value and the predictive positive value criteria computed by the STAPLE algorithm. All these criteria assume that there is a "gold standard" segmentation, at least implicitly. Furthermore, these criteria are partly correlated and are also directly related to the optimization process involved in STAPLE. To avoid these limitations, we rather focused our evaluation on the clinical task and evaluated the accuracy of the estimated LVEF parameter. Indeed, this parameter value is the clinically relevant end result of the left cavity segmentation process.

To evaluate the interest of the STAPLE algorithm for combining segmentation results, we applied it to a cardiac cine MRI database including LV segmentation obtained from eight independent segmentation approaches: five resulted from five different automated image processing approaches, and three volume contours were drawn by three different experts. All possible combinations of the five automated methods (all five methods, four among the five methods, and three among the five methods) were tested against results provided by the three experts, using the LVEF values as the clinical parameter of interest. The evaluation was first carried out using a supervised approach, assuming a gold standard was available, and then using an unsupervised approach, the extended Regression Without Truth (eRWT) [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF] to rank all segmentation methods as a function of their performance..

Our study presents some similarities with [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF]: both used a public cardiac cMRI database (although not the same) for which contours were delineated by experts and algorithms. In our case, the selected database included controls and patients with different cardiac pathologies. In [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF], only cMRI acquired on patients were included. Furthermore, both studies used STAPLE to combine different contours, but they differ in their approach. Indeed, [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF] proposed to use STAPLE to define a gold standard segmentation based on two fully-automated algorithms and three semi-automated algorithms requiring manual input, while the present study focuses on improving the accuracy of segmentation algorithms by combining them with STAPLE to get a precision similar to the one achieved by experts i.e. make it acceptable for clinical routine. Since this study is based on previous work that requested manual delineations on a specific database, we run the tests on the public database previously used in our studies.

This paper is organized as follows. Section II presents the database that was used in our study, the segmentation methods that were compared and combined, and the different combinations that were tested. Section III describes the statistical supervised evaluation and the unsupervised evaluation used to characterize the performance of the different combinations of segmentation methods. Section IV presents the results and the ranking of each segmentation method. These results are discussed in section V.

II. DATABASE AND SEGMENTATION METHODS

A. Database

This work uses the public database provided by Sunnybrook Health Sciences Center [START_REF] Radau | Evaluation framework for algorithms segmenting short axis cardiac MRI[END_REF]. This cardiac database was first distributed to the participants in the Cardiac MR Left Ventricular Segmentation Grand Challenge (MICCAI 2009). It includes images from forty-five subjects who were divided into four subgroups: healthy individuals (CTRL, n=9), patients with hypertrophic cardiomyopathy (HYP, n=12), patients with heart failure without ischemia (HF-NI, n=12) and patients with heart failure due to ischemia (HF-I, n=12). For each examination, about ten short axis slices covering the LV were acquired using a breath-hold, retrospective ECG-gated cine-MRI sequence (twenty cardiac phases per slice, thickness = 8 mm, FOV = 320 mm, acquisition matrix 256 × 256 with a 1.5T MR scanner (GE Healthcare)).

We focused here on the left ventricular ejection fraction (LVEF) estimate. LVEF was calculated conventionally as the ratio between the stroke volume and the end-systolic volume. The end-diastolic and end-systolic volumes were measured from the endocardial border that was delineated on each selected image. MR slices corresponding to the end-systolic and end-diastolic phases were indicated to the participants in the Challenge, so as to avoid any variability due to the choice of these temporal phases.

B. Segmentation approaches

Eight independent estimates of the LVEF were obtained from three manual contouring methods (M 1-M 3) provided by three independent experts from two different laboratories and from five algorithms (M 4-M 8). The five algorithms described respectively in [START_REF] Constantinides | Semi-automated cardiac segmentation on cine magnetic resonance images using GVF-Snake deformable models[END_REF], [START_REF] Schaerer | A dynamic elastic model for segmentation and tracking of the heart in MR image sequences[END_REF], [START_REF] Cousty | Segmentation of 4D cardiac MRI: Automated method based on spatio-temporal watershed cuts[END_REF], [START_REF] Lalande | Left ventricular ejection fraction calculation from automatically selected and processed diastolic and systolic frames in short-axis cine-MRI[END_REF], [START_REF] Constantinides | Fully automated segmentation of the left ventricle applied to cine mr images: description and results on a database of 45 subjects[END_REF] 1 use different segmentation strategies and various user's interactions. Endocardial borders were obtained on the end-diastolic and end-systolic phases with all methods (methods M 5 and M 6 provided contours for all cardiac phases). All methods but M 5 included the papillary muscles in the LV cavity. Method M 4 was the least automated one; method M 8 was fully automated.

Using each segmentation method, the mean LVEF value and its associated standard deviation were calculated for each of the four subgroups of subjects. More than 99% of these values ranged from 0.05 to 0.85. The twenty-four patients of the studied database with heart failure (HF-NI and HF-I) had a reduced LVEF that was considered as pathological (≤0.45).

C. Combination of the segmentation approaches

1) Method: Several segmentation results were combined using the Simultaneous Truth and Performance Level Estimation (STAPLE) algorithm developped by Warfield et al. [START_REF] Warfield | Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation[END_REF]. This method was implemented using the version 1.5.2 of CRKit, which is the software provided by Warfield's team.

The STAPLE framework is based on an Expectation Maximization (EM) algorithm [START_REF] Dempster | Maximum Likelihood from Incomplete Data via the EM Algorithm[END_REF], [START_REF] Mclachlan | The EM Algorithm and Extensions[END_REF]. It uses several segmentation results and calculates simultaneously a probabilistic estimate of a representative segmentation result and a performance level of each delineation included in the calculation. This performance level is provided by the computation of the sensitivity and the specificity indexes between each input segmentation and the segmentation result. The process is iterated until a stable solution is reached. Here, the STAPLE algorithm was run using the default parameters that were proposed by its authors. The binary version was used since only two classes were considered: the left ventricle and the remaining structures outside the left ventricle. Provided results did not depend on the size of the background (the region of interest surrounding the left cavity in our application) as mentioned in [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF]. Furthermore, the STAPLE algorithm was applied in 2D, for each slice separately. The resulting contours were stacked to get a 3D segmentation result.

2) Application: The STAPLE algorithm was applied to several combinations of segmentation results obtained from the five automated methods previously described:

• a STAPLE segmentation M S45678 was created from the five automated methods.

• STAPLE was used to combine all five combinations of four automated methods. For instance, the resulting segmentation was denoted M S4567 when methods M 4, M 5, M 6 and M 7 were involved in the algorithm.

• STAPLE was also applied to each combination of three automated methods among the five available (10 combinations). The result was denoted M S456 when methods M 4, M 5 and M 6 were involved in the algorithm. Using each STAPLE segmentation result, the mean LVEF value and associated standard deviation were calculated for each of the four subgroups of subjects.

III. STATISTICAL EVALUATIONS A. Supervised evaluation

There is no perfect reference when three experts have delineated contours on real data for which the gold standard segmentation is unknown [START_REF] Chalana | A methodology for evaluation of boundary detection algorithms on medical images[END_REF]. We could have used STAPLE to define a consensus as proposed for instance in [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF]. In order to be independent of STAPLE for the evaluation, we rather used M 2 as the reference method (M ref ) for the supervised evaluation step. Indeed, it was shown in [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF] that method M 2 performed the best and that the LVEF obtained by the three experts were more accurate than any of the five automated methods that were tested. The supervised evaluation was based on the LVEF estimation and on the computation of the bias β and its associated standard deviation (s) of each segmentation method M k with respect to the reference M 2, (k representing either one of the original methods or one of the sixteen STAPLE combinations described in II-C2).

B. Unsupervised evaluation using eRWT 1) Theory: The eRWT approach, an extension of the Regression Without Truth [START_REF] Kupinski | Comparing cardiac ejection fraction estimation algorithms without a gold standard[END_REF], [START_REF] Hoppin | Objective comparison of quantitative imaging modalities without the use of a gold standard[END_REF], [START_REF] Kupinski | Estimation in medical imaging without a gold standard[END_REF], aims at comparing and ranking different methods which estimate a specific biomarker such as the LVEF, the true value Θ p of the biomarker being unknown. Considering P samples (denoted by p, ranging from 1 to P ) and K segmentation methods (denoted by M k, ranging from 1 to K), each segmentation method M k yields an estimate θ pk of the biomarker for sample p.

The eRWT approach assumes a parametric relationship between the true value Θ p and its estimate θ pk based on three hypotheses: H1: The statistical distribution of the true value Θ p on the whole database has a finite support. H2: The estimate θ pk is linearly related to the true value (equation (1)). The error term ε pk is normally distributed with zero mean and standard deviation σ k . The a k and b k parameters are specific to each method M k and independent of sample p:

θ pk = a k Θ p + b k + ε pk . (1)
H3: The error terms ε pk for each method M k are statistically independent. Regards to H1, a Beta distribution Beta(µ, ν) was chosen for LVEF [START_REF] Kupinski | Comparing cardiac ejection fraction estimation algorithms without a gold standard[END_REF]. Besides, given all these assumptions, the probability of the estimated values θ pk given the linear model parameters and the true value Θ p can be expressed and the log-likelihood can be written as a function of a k , b k , σ k and the probability distribution of Θ p .

The maximization of this log-likelihood does not require the numerical values of the true LVEF, but only a model of its statistical distribution (pr (Θ p )); it leads to the estimates of the linear model parameters for each method (a k , b k and σ k ).

The numerical implementation uses an optimization function implemented in MATLAB (R2012a, The Mathworks, Inc.). The figure of merit F Mk chosen to rank the methods M k is defined as the expected value of the square error between the true value of the parameter Θ p and its estimated value by a given method (equation (2)) [START_REF] Soret | Accuracy of partial volume effect correction in clinical molecular imaging of dopamine transporter using SPECT[END_REF].

F Mk = E (Θ -a k Θ -b k -ε k ) 2 . ( 2
)
If the statistical distribution of Θ p is a Beta distribution, it can be expressed analytically by equation [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF].

F Mk = (a k -1) 2 µ (µ + 1) (µ + ν) (µ + ν + 1) + 2 (a k -1) b k µ µ + ν + b 2 k + σ 2 k . (3)
To set the shape parameters of the Beta distribution (µ and ν), we started from the values chosen in [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF] (µ = 4 and ν = 5) and refined these initial values so as to minimize the sum of the K figures of merit. Final values of the µ and ν parameters were set to 2.85 and 3.40 respectively. These slight modifications of the Beta distribution compared to that used in [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF] did not yield substantial changes in the ranking of the methods, as already shown in [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF].

The final ranking of methods was based on a bootstrap process [START_REF] Efron | An Introduction to the Bootstrap[END_REF] running on the database of P values θ pk generating N (N = 1000) θ p n k values. From each drawing n, P values p n were drawn from the 45 samples. From these θ p n k values, the K figures of merit F n Mk were computed using the previously described optimization. The non-parametric Kruskal-Wallis test [START_REF] Kruskal | Use of Ranks in One-Criterion Variance Analysis[END_REF] was applied to the N × K values of F n Mk to test the equality of the median among the K methods. When it was not equal, each pair of methods was tested, using a Bonferroni correction with a Type I error equal to 5% [START_REF] Miller | Simultaneous Statistical Inference[END_REF] to determine the significantly different pairs.

2) Experiments: The eRWT approach was first performed to rank the eight segmentation methods (M 1 -M 8 described in II-B). This first ranking confirmed that the reference method chosen for the supervised evaluation (i.e. M 2) ranked first, as previously established in [START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF], despite the update of method M 8 and the new values of the Beta distribution parameters. The unsupervised eRWT approach was then systematically applied to the eight methods M 1 -M 8 and to one of the STAPLE results described in II-C2 to rank each segmentation combination, M Si, among the eight initial segmentation methods. 1 and2 show the endocardial contours obtained using the eight segmentation approaches M 1 -M 8 and using three different STAPLE combinations, superimposed on an end-diastolic image. These two figures correspond to two different cases: one patient (SC-HF-01) and one control (SC-N-05). In these two examples, the LV contour was correctly delineated by STAPLE whereas it was over-delineated when using M 6 and M 8 (Figure 1) or under-delineated by M 5 and M 7 (Figure 2).

IV. RESULTS

A. Combination of the segmentation approaches 1) Superimposition of contours resulting from different segmentation methods on cMRI: Figures

2) Estimation of LVEF values for each method:

The mean LVEF values and their standard deviations estimated for each subgroup of subjects are displayed in Table I for each initial segmentation method (M 1 -M 8) and each M Si method. II presents the figures of merit computed using the eRWT approach when the eight initial segmentation methods (M 1-M 8) were compared. These scores confirmed that M 2 could be chosen as the reference method for the supervised evaluation.

B. Supervised evaluation 1) Choice of the reference method: Table

2) Bias in LVEF estimated values: Figure 3 shows the results obtained for the supervised evaluation. Each bias β with respect to the M 2 result is represented with its associated standard deviation (error bars corresponding to ±1.96s). This figure shows that expert delineations M 1 and M 3 give the closest results to M 2, with M 3 showing less variability than M 1. When comparing the five automated methods (M 4-M 8), M 4 yields the closest result to M 2 with a bias near 0, and the smallest s. Although all semi-automated methods have slightly greater variability than the inter-expert variability, several STAPLE combinations are within the inter-expert variability, with six combinations presenting smaller variability than M 1. Method M S456 was the one presenting the smallest variability [β ± 1.96s] among all M Si.

To easily compare results obtained for M Si with those obtained with M 4 (the best automated method according to the supervised evaluation), the variability obtained with M 4 is represented by a red colored box in Figure 3. Among the sixteen tested M Si methods (using or not M 4 to create the STAPLE segmentation result), ten were within the range [β ± 1.96s] obtained with M 4. The six remaining M Si had a higher bias (in absolute value) than the one obtained with M 4, but three of them (M S4567, M S5678 and M S678) had a lower s than M 4. M S578 had a higher s than M 4, but lower than the s obtained by the four methods used to create the STAPLE segmentation result. Finally, whereas M S4578 had a s 10% higher than the one obtained with M 4, M S457 had a standard deviation s only 1% higher than the one obtained with M 4. (a) (b)

(a) (b) (f) (e) (d) (c) (g) (h) (i) (j) (k)
(f) (e) (d) (c) (g) (h) (i) (j) (k)

C. Unsupervised comparison of segmentation methods

Table III presents the ranking of the eight initial segmentation methods and of each STAPLE method M Si. Among the sixteen comparisons, method M Si was at a ranking similar to the experts in 14 cases (green M S in the table). The best rank was reached by M S456 (rank equal to 2). Method M S578 was ranked like M 4 (rank equal to 4, blue M S in the table), this rank being worse than the experts ranks but better than the individual methods used to create the combination. These results demonstrate that the LVEF parameters were more accurately estimated using this combination of segmentation methods than with any of the segmentation methods used in the combination. The worst rank observed for an M Si approach was obtained for M S4578 with a rank equal to 5 (red M S in the table), worse than M 4 used to provide the STAPLE segmentation result. For this test, F M1 and F M4 were equal to 0.004, F MS4578 was equal to 0.005, and F M8 was equal to 0.007. So, even if M S4578 was at the fifth position, its figure of merit was close to the scores obtained with methods M 1 and M 4. Thus in this case, LVEF parameters estimated using M Si show a clear improvement compared to LVEF estimated using M 5, M 7 and M 8.

V. DISCUSSION

A. Use of STAPLE to combine LV segmentations

The aim of this work was to evaluate the efficiency of the STAPLE algorithm [START_REF] Warfield | Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation[END_REF] to estimate a clinical biomarker, the LVEF, from a segmentation resulting from the combination of different independent segmentation algorithms. To demonstrate it, a collection of segmentation applied to the MICCAI 2009 cardiac MRI database was used. For the forty-five cases of this database eight segmentation methods were available, including delineations provided by three independent experts, and five delineations obtained using five up-to-date automated LV segmentation algorithms. As the LVEF is a primordial biomarker used in clinical routine, all evaluation tests were performed by focusing on this parameter. The database had the advantage of including a large variety of cardiac diseases (with normal or reduced LVEF) and control subjects. The computation of the mean LVEF value and associated standard deviations for each subgroup showed that values were homogeneous for each subgroup of subjects, whatever the segmentation method used for the LVEF calculation. These first results confirmed that all segmentation methods provided coherent estimates for each subgroup of subjects.

The STAPLE algorithm has already been proposed to define a reference method from different expert segmentations [START_REF] Warfield | Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation[END_REF], [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF]. In the present study, our goal was not to define a consensus between "experts", but rather to determine whether some combinations of different independent automated segmentation methods could yield a segmentation as reliable as that of an expert, keeping in mind that each automated method is slightly less powerful than expert delineation. In other words, could a combination of different automated segmentation results yield better results than the ones from each individual method? The question was challenging since several evaluation studies [START_REF] Robitaille | Label fusion strategy selection[END_REF], [START_REF] Suinesiaputra | A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images[END_REF] already showed that the STAPLE output strongly depends on the number and on the quality of the inputs used to create the combined segmentation. However, assuming that the automated methods incorporate different strategies, we tested whether their combined use could actually help in improving segmentation results on a whole database. All possible combinations of three, four and five automated segmentations were systematically tested.

To assess the segmentation results, a visual inspection of the contours of all STAPLE segmentation results superimposed onto the MR images was first performed. This visual assessment showed that in most cases, the STAPLE algorithm was able to correct, in every slice, too loose or too tight delineations obtained from automated methods. Supervised and unsupervised statistical evaluations were then performed to assess the results obtained using STAPLE combinations of automated methods.

B. Supervised evaluation

The main idea of the supervised evaluation was to compare the LVEF values estimated by all methods (including the M Si methods) with the values computed by a "reference" method. This "reference" method was the M 2 method as it yields the best figure of merit when using the eRWT approach on the eight initial methods. The comparison of LVEF values was based on the bias (β) and its associated standard deviation (s) obtained when computing LVEF values using each individual segmentation method compared to the M 2 results.

Results showed that M 4 was the closest automated method to M 2 with a low bias and the smallest standard deviation. Ten among sixteen M Si were closer to the reference method M 2 than M 4 and less variable than this latter method. Furthermore, except for M S4578, all combinations resulting from the STAPLE algorithm show more accurate or at least equivalent results compared to the methods being involved in the combination. It can be concluded that the STAPLE algorithm provided segmentation results that yielded more accurate or equivalent LVEF results compared to the automated segmentation methods from which the STAPLE combination was based.

The practical comparison of the different segmentation combinations using STAPLE also shows interesting results. For instance, the combination of only three automated segmentation methods can provide a LVEF estimate as accurate as the one provided by an expert. Furthermore the bias related to each M Si method is correlated with the sum of the biases observed in the initial methods used in the combination (r = 0.736).

We also observed a systematic reduction of the standard deviation s when combining different methods using STAPLE, compared to the standard deviation of each individual method used in the STAPLE combination. This decrease in standard deviation did not necessarily depend on the number of methods used for the combination.

C. Ranking provided by the eRWT approach

The eRWT approach ranked the expert delineation M 2 first, and more generally, the three expert delineations in the top three. The semi-automated method M 4 was ranked as the best automated method to estimate LVEF. To evaluate the STAPLE segmentation results (M Si) without using strong a priori on the truth, the eRWT approach was systematically applied to the eight original methods and to an M Si method. In most cases, M Si ranked similarly to the expert delineations (M 3 and M 1). This means that the STAPLE algorithm based on several automated methods provided similar results to those obtained by experts. In one case (M S578), the rank of the STAPLE method was less than those of experts but was still better than those of the three methods STAPLE was based on. This suggests that the LVEF parameters were once again better estimated using the combination of segmentation methods than using any of each initial segmentation method used in STAPLE. Finally, in only one instance (M S4578), M Si was ranked after one of the four methods (M 4) used in the combination. However, the figures of merit showed that LVEF parameters estimated using M Si were better than those estimated using three of four methods involved in the combination (M 5, M 7 and M 8). Furthermore, results obtained with (M S4578) were very close to those obtained with M 4. Overall, all experiments showed that the clinical parameters calculated for a specific database were better estimated when combining segmentation methods with STAPLE than when using one of the initial methods entering the STAPLE combination.

Last, both supervised and unsupervised statistical approaches led to very similar conclusions. Indeed, both approaches showed that the most accurate LVEF was obtained when combining M 4, M 5, and M 6. Furthermore, both approaches showed that the poorest results were obtained when combining M 4, M 5, M 7 and M 8. This a posteriori consistency between conclusions suggests that the use of the unsupervised eRWT approach was relevant in our context and that the different hypotheses underlying the eRWT approach proved to be realistic. The major interest of eRWT is to provide a ranking of different estimation methods based on only few a priori hypotheses.

D. Future directions

The statistical tools that were used for this study could also be used to compare the STAPLE algorithm with other algorithms that have been developed to define representative contours (for instance, the ones described in [START_REF] Robitaille | Label fusion strategy selection[END_REF], [START_REF] Jehan-Besson | A mutual reference shape based on information theory[END_REF]). This could help identify the most efficient algorithm to combine contours. However, this would require testing the statistical independency of σ k in the eRWT model (equation (1)) when comparing different methods of combination based on the same initial methods.

Due to the difficulty in getting one or multiple expert delineations for clinical segmentation problems, the combined use of different independent algorithms could yield a valuable alternative. Of course, the combination process requires some computing resources, which depend on the segmentation methods involved in the combination and on the method used for combining them (here STAPLE) but it guarantees reproducible results and manual delineation is no longer needed. Due to the quality of results demonstrated by this study, which shows a clear improvement in LVEF estimates using the STAPLE combinations compared to the initial automated segmentation algorithms, it becomes feasible to use automated segmentation algorithms and get stable and reliable results.

Finally our approach was applied to LV segmentation. To extend to other organs, additional tests should be realized.

VI. CONCLUSION

This work aimed at determining whether combining different segmentation results using the STAPLE algorithm could yield a final segmentation as reliable as that of an expert. This approach was tested in the framework of the estimation of left ventricular ejection fraction on the MICCAI 2009 cardiac cine MRI database. Both supervised and unsupervised evaluations showed that in most cases, the LVEF were better estimated using the STAPLE approach than using individually the segmentation methods used to create the STAPLE result. Moreover, the STAPLE segmentation results provided, in most cases, similar estimates to the ones obtained based on manual delineations performed by an expert. The results show that combining different independent automated segmentation methods using the STAPLE approach yielded segmentations that were as accurate as those provided by expert delineating the left ventricular cavities.

Fig. 1 .

 1 Fig. 1. Basal cine MRI slice at end-diastole with superimposed contours of the LV (green line) as obtained using the eight segmentation methods included in the study (M 1 to M 8 represented from (a) to (h) respectively) and using the three different combinations of the STAPLE algorithm (M S45678 (i), M S456 (j) and M S4578 (k)).

Fig. 2 .

 2 Fig. 2. Median cine MRI slice at end-diastole with superimposed contours of the LV (green line) as obtained using the eight segmentation methods included in the study (M 1 to M 8 represented from (a) to (h) respectively) and using the three different combinations of the STAPLE algorithm (M S45678 (i), M S456 (j) and M S4578 (k)).

Fig. 3 .

 3 Fig. 3. Supervised evaluation: Computation of the LVEF bias β of each method with respect to values obtained with M 2 and its associated standard deviation (error bars corresponding to ±1.96s). The red box represents results obtained for M 4, the automated method whose results are closest to the M 2 results for this evaluation.

Please note that the method M 8 described in[START_REF] Constantinides | Fully automated segmentation of the left ventricle applied to cine mr images: description and results on a database of 45 subjects[END_REF] is an update of the method[START_REF] Constantinides | Automated heart localization for the segmentation of the ventricular cavities on cine magnetic resonance images[END_REF] evaluated in[START_REF] Lebenberg | Nonsupervised ranking of different segmentation approaches: application to the estimation of the left ventricular ejection fraction from cardiac cine MRI sequences[END_REF].

ACKNOWLEDGMENTS

The authors thank Béranger Browaeys for the optimization step of the Beta distribution parameters. This work was performed in the framework of the French MedIEval (Medical Image segmentation Evaluation) working group. The authors gratefully acknowledge the GdR 2647 Stic-Santé for its support to the MedIEval action. The research concerning the STAPLE algorithm was supported in part by NIH R01 RR021885 from the National Center for Research Resources, and by an award from the Neuroscience Blueprint I/C through R01 EB008015 from the National Institute of Biomedical Imaging and Bioengineering.