Gaëlle Pédrono

Rodolphe Thiébaut

Ahmadou Alioum

Philippe Lesprit

Bernard Fritzell

Yves Lévy

Geneviève Chêne
email: genevieve.chene@isped.u-bordeaux2.fr

Prof Geneviève Chêne

A NEW ENDPOINT DEFINITION IMPROVING CLINICAL RELEVANCE AND STATISTICAL POWER IN VACCINE TRIALS

Keywords: Vaccine, endpoint, proportional odds model, HIV infection, statistical power, random effect

Objective: Endpoints used for the evaluation of immunogenicity in vaccine trials are often the proportion of individuals with immune response or geometric means of antibody concentrations for each serotype. When a vaccine includes several types of the same species, we illustrate how an endpoint combining all responses may improve clinical relevance and statistical power.

Study design and settings: The motivating example was the ANRS-114 PNEUMOVAC trial where the effect of two vaccine strategies against Streptococcus pneumoniae was assessed in adults infected by the Human Immunodeficiency Virus. The power associated with several endpoints was calculated in the example and in simulations. A new endpoint based on four ordered levels is formulated and analysed by using a proportional odds model.

Results and conclusion:

The analysis of this new endpoint led to an odds-ratio allowing detection of improvement and detriment. In the simulation study, this endpoint was associated with the largest statistical power by increasing the amount of information used as compared to usual endpoints. We recommend this new endpoint formulation in the formal development of a new vaccination regimen, whenever applicable.

INTRODUCTION

The efficacy and safety of vaccines are assessed by several traditional parameters [1,2]: incidence of the disease(s) to be prevented, immune response (antibody response and immunocompetent cells), reactogenicity (local and systemic reactions). Phase II trials are usually built to test meaningful differences in the distribution of immunogenicity endpoints.

Antibody response to the target pathogen often defines the primary immunogenicity endpoint, antibody concentrations being used as surrogate endpoints for a clinically relevant endpoint [1]. Some vaccines such as pneumococcal polysaccharide vaccines yield a response to several serotypes and are called multivalent vaccines. The immunogenicity evaluation of such vaccines must take into account the response to any serotype, all of them participating in the definition of the primary endpoint. In general, the comparison between groups involves the distribution of serotype specific antibody concentrations for each serotype using geometric means [3,4,5,6,7,8,9]. The P-value is then adjusted for Bonferroni or Family Discovery Rate (FDR)controlling methods [1,10]. Another approach consists in comparing the percentage of responders to a minimum number (k) of serotypes defined a priori. First, the response to each serotype according to a given threshold is defined and then the percentage of responders to k serotypes is compared between groups [4,7,11].

Nevertheless, those endpoints are not fully satisfactory. Indeed, the former may be difficult to interpret, when results are discordant across serotypes. The latter, although clinically relevant, leads to an obvious loss of information. Furthermore, it is difficult to define the threshold for the number of responses to consider. When several serotypes are used to define immunogenicity and if they have the same relative importance, we propose to use a categorical endpoint based on the number of responses obtained and the mode of analysis using a proportional odds model [12].

Our report aims at showing the statistical and clinical relevance of this approach; firstly through the motivating example and secondly based on a simulation study.

METHODS

Study design (Figure 1)

The ANRS 114-Pneumovac trial was a phase II randomized trial comparing two vaccine strategies against pneumococcal infection among adult patients infected by the Human Immunodeficiency Virus (VIH) [13]. The rationale was that HIV-infected patients are more susceptible to infections caused by common bacterial pathogens such as S. pneumoniae and therefore are in need of effective preventive strategies against this infection. In this trial, 212 patients were enrolled, 106 in each arm. The tested vaccine strategy consisted in a vaccination by heptavalent pneumococcal protein conjugate vaccine: Prevenar  (PCV) at baseline (W0) and by 23-valent pneumococcal polysaccharide vaccine: Pneumo23  (PPV) at week 4 (W4). The currently recommended strategy was used in the control arm and consisted in a vaccination by PPV at W4. The 7 serotypes of PCV (4, 6B, 9V, 14, 18C, 19F, 23F) are included in PPV and are the most frequently encountered serotypes in Western countries. Antibody concentrations of each common serotype were measured at W0 and W8 for each patient. Antibody response at W8 was compared between the 2 arms (Figure 1).

Statistical methods

Direct comparison of antibody concentrations

The distribution of antibody concentrations is usually not gaussian [3,4,5]. Therefore, antibody concentrations were log-transformed and geometric means were calculated. T-test was used to compare distributions and FDR-controlling method to control for multiplicity of tests [14]. Response to each serotype was defined as two-fold increase between W0 and W8 and titre >1 µg/ml measured at 8 weeks.

Binary endpoints

Three binary endpoints were considered: (i) response to all 7 serotypes (versus non response to at least one of them), (ii) response to at least 5 serotypes among 7 (versus response to less than 4 serotypes) and (iii) response to at least 3 serotypes among 7 (versus response to less than 2 serotypes). The comparison of responder rates according to treatment group was performed with Fisher exact test.

New endpoint formulation (figure 2)

The new formulation is based on: i) the identification of a response to each serotype, ii) the number of serotypes with response. These two variables were combined into four ordered levels: 0 response, response to 1 or 2 serotypes, response to 3 or 4 serotypes or response to 5 to 7 serotypes (Figure 2). This categorization was considered a priori the most relevant by clinicians and immunologists. All serotypes have the same importance and contribute in the same way in the definition of the endpoint. This endpoint is categorical but also ordered because the higher the category, the better the response. First, Chi-2 and trend Chi-2 were used for crude analysis of this endpoint.

Secondly, the Proportional Odds Model (POM) [12] was used to analyse the new endpoint formulated. It is a generalization of a logistic model, the response variable is categorical and ordered (Y=1,2,…,J) instead of binary. A definition of this model is provided in appendix.

When the only explanatory variable is the treatment group, the global odds ratio β θ e = can be interpreted as the odds of being in the higher category above a given category when someone is vaccinated by the tested strategy compared to the control group. The comparison between the two groups can be performed by testing θ = 1 using a likelihood ratio statistics or a Wald test. This is testing an absence of difference between the groups in the distribution of the patients according to each category.

The POM assumption is that the effect of any explanatory variable is the same whatever the level of grouping of the response. In other words, the odds ratio between no response at all versus at least one response should not be different from the odds-ratio between 2 responses or less versus 3 responses or more, etc. estimated using maximum likelihood methods with standard softwares. In SAS (SAS ® software v.9.1, SAS Institute, Cary, NC, USA), we used proc LOGISTIC for the former and proc NLMIXED for the latter. Code for fitting random effects POM model is provided in Appendix.

Sample size and power were calculated using Kolassa's solution in N'Query software [18].

Simulations

Simulations were performed in order to calculate the power associated with each endpoint according to different scenarii. True parameters were defined according to the distribution of antibody concentrations in the ANRS 114-Pneumovac trial. Antibody concentrations were assumed to follow a log-normal distribution. For a given patient and a given serotype, antibody concentrations were correlated between different time of measurements and for a given patient, antibody concentrations of different serotypes were also correlated. We used the following four steps approach to simulate such data:

1) Y 0 ~ MVN (M 0 , C) (MVN for multivariate normal)
Y 0 is the multivariate vector of the logarithm of the titre of the seven serotypes.

M 0 (7x1) and C (7x7) were estimated from a multivariate linear mixed model using real data presented in the results section. They represent vector of means and variance-covariance matrix, respectively. R(7x7) is the correlation matrix .

2) For the reference group, the difference of antibody concentrations from baseline was generated according to a normal distribution: Y 8 -Y 0 ~ N (p 1 , σ p ) where p 1 and σ p were based on estimations from multivariate linear mixed models fitted on data from the Pneumovac trial.

For the test group : Y 8 -Y 0l ~ N (p 2 , σ p ) where p 2 = p 1 + β 2 with β 2 being the treatment effect

3) Test group effect was estimated using the different methods presented above 4) Steps 1 to 3 were repeated N=1000 times out of 1000. Simulations were repeated for different values of β 2 (treatment effect) including the one estimated using the real data set.

RESULTS

Application with ANRS 114 Pneumovac data (table 1)

Geometric Means (GMT) were calculated for each serotype (table 1). GMT were generally higher in those vaccinated with the tested strategy. These differences were statistically significant for 2 of them (out of 7) after correction for multiple testing. The statistical power associated to each serotype varied from 35% (serotype 6B) up to 99% (serotype 18C). The power was therefore not homogeneous among serotypes and rather low (<60%) for 4 serotypes among 7. We used a chi-2 and a chi-2 for trend to test the difference of response between the two groups, p-value was respectively 0.0373 and 0.0015.

We then considered the binary endpoint: response to all serotypes versus non response to at least one of them. The proportion of patients with successful response was significantly different between groups according to this endpoint. The odds-Ratio (OR)=2.80 (95% Confidence Interval [CI] 1.22-6.43, p=0.0123), meaning that the tested vaccinal strategy was associated with a better immunogenicity. The statistical power of the test for the null hypothesis OR=1 was 68%.

With an endpoint defined as being responder for at least 5 serotypes versus less than 5, the OR was 2.23 (95% CI 1.28-3.89, p=0.0044) and the power was 77%. The odds to be good responder (more than 5 serotypes) was therefore better for the tested vaccine strategy.

Finally, the global OR estimated with the proportional odds model for the previously defined ordered categories was 2.12 (95% CI 1.26-3.56, p=0.0047). The power was 81%. Patients in the tested vaccine strategy group showed a higher number of responses than the control group.

Morever, the proportional odds assumption was satisfied (test of non homogeneity p = 0.82). In this trial, the categorical endpoint was the most powerful as compared to other endpoints considered.

Simulations (table 2)

Estimated values for simulation parameters were: [2.0728, 1.2022, 1.3784, 1.6299, 1.4080, 0.9547, 1.2380] [0.3520, 0.6219, 0.5427, 0.6075, 0.6277, 0.5527, 0.8066] The statistical power for the comparison of geometric means of antibody concentrations between the two groups increased with the treatment effect (β t 2 ), as expected (table 2).

M 0 = [-2.
σ p =
Furthermore, the power was quite heterogeneous according to the serotype considered (eg from 26.0 to 42.3% when β t 2 = D).

The statistical power was always higher with other outcomes considered (binary or ordered categories). Among binary outcomes, the binary endpoint using a threshold at 5 serotypes led to the best statistical power compared to a threshold set at 7 serotypes (13.2% vs 8.6% for β t 2 = B; 85.0% vs 75.9% for β t 2 = D for instance). Although the outcome based on ordered categories led to the best power, the differences with the binary endpoint using a threshold at 5 serotypes were tiny (around 3%), when the effect of the treatment (β 2 ) was identical according to each serotype. With a treatment effect differing from one serotype to another (very low effect for 5 serotypes and high effect for 2 serotypes, scenario F) the binary endpoint using a threshold at 5

serotypes was substantially less powerful to discriminate poor responders (2 responses for instance) than the new endpoint formulation (54.9.6% vs 63.2%, respectively). Furthermore, a null effect scenario (β t 2 = 0) was tested and the type I error was slightly higher for the POM model (α=0.06 as compared as α=0.05 for "> 5 reponses" binary endpoint).

correlation structures between serotypes (data not shown). Whatever the correlation structure (no correlation, slight correlation, high correlation), the power of this new endpoint formulation was better as compared to the other endpoints.

DISCUSSION

The new endpoint formulation based on ordered categories was more powerful than current endpoints used for the evaluation of a multivalent vaccine efficacy. This was even more true when the response was not homogeneous among all serotypes. With binary endpoints, the power was not satisfactory even if the test was significant in our specific study. This loss of power might be explained by a loss of information due to dichotomization. For an ordered categorical variable, a chi-2 test for trend seemed to be the most powerful test. However, the use of such a test does not yield a measurement of association, does not allow adjustment for confounders and is conditional to the existence of a linear trend of responses among categories.

The main advantage of the proposed strategy is to allow evaluating the effect of the intervention on all serotypes estimating only one parameter (the odds ratio) and performing one test. Hence, it avoids any correction for multiple testing and the main result is easy to interpret.

Indeed, it had practical relevance for clinicians. Firstly, all serotypes contributed in the same way to the construction of the endpoint. The 7 serotypes of interest were those contained in the PCV vaccine since they were the most frequent causes of pneumococcal infections. Therefore, there was no rationale to give more consideration to a serotype than another. Secondly, results can be summarized using a single parameter, i.e a global OR estimated by the POM model, and presented in a simple contingency table. Such a summary contributes to improve the understanding of results by practitioners and makes it easier to remind them and to disseminate them among those who will use the vaccines.

Another advantage is that the implementation is fairly easy and convenient using standard statistical software. It is possible to accommodate for General Linear Model and to adjust for fixed or random effects. Indeed, the model can be extended by including random effects to capture heterogeneity between centers, or between individuals with repeated measurements. In our study, we included testing for such random effects (SAS code in Appendix) and did not find significant heterogeneity among them regarding vaccine response.

A major assumption for the use a POM is the homogeneity of odds ratios among the different categories. This assumption holded in our specific example. Table 3 shows two examples illustrating extreme fictive situations when the assumption holds or not. In the first example, the vaccine strategy is efficient for all serotypes; in that case, the assumption should be satisfied.

This translates into quite similar partial odds ratios. In the second example, the new vaccine strategy generates especially very good responders and the referent strategy more likely generates medium responders (example 2). The first strategy could be completely efficient (for the whole serotypes) or not at all. The second strategy could generate responses to almost 3 or 4 serotypes, leading to many responders in medium categories and few in the lowest and the strongest. According to considered cutpoints, one would differently conclude about superiority between vaccine strategies. If the cutpoint is "at least 1 response" versus none, referent strategy would be better, if the cutpoint 5-7 responses is retained, the tested strategy would be better. In this example, the criterion of definite judgment would not be relevant because partial odds-ratio would be different (p<0.0001), so proportional odds assumption would not be satisfied. It is especially important to check this proportionality assumption because it is not obvious to intuitively anticipate the homogeneity of partial odds-ratio. Indeed, a vaccine strategy with an effect that varies from one serotype to another may stick to partial odds homogeneity (see Table 2, scenarios A and F).

When the assumption is not satisfied, other models are available such as a continuous ratio model or a partial odds ratio model [16,17,21]. This latter model allows odds to be partially subgroup. To be able to use this model the number of categories should be large enough (at least 5 categories). A recent paper [15] provides SAS code for other multinomial model, which can be useful if the proportional odds assumption is not satisfied and more sophisticated model is needed. Another paper [19] suggests solutions to analyse ordinal endpoint when the proportional odds assumption does not hold. However those alternative models estimate additional parameters which leads to a decrease of statistical power to detect an effect.

Another limitation (also shared with binary endpoints) is that the threshold for antibody responses should be defined a priori and according to a relevant biological and clinical rationale. In the litterature, definition of antibody response is varying according to the increase considered (e.g. 2-or 3-fold increase of antibody concentrations) [5,23], to a threshold of the titre (e.g. titre greater than 1 µg/ml) [3] or the combination of both: 2-fold increase and titre>1

µg/ml at W8 [11]. This threshold seemed to be the most relevant because it took into account not only the increase of titre between W0 and W8 but also a minimal threshold to reach response level. Indeed, a 2-fold increase from 0.01 to 0.02 µg/ml, could not be considered as a response. In the same way, a patient could present a titre > 1 µg/ml at W8 but also at W0 and be considered as a responder even if the titre was stable. That is why the endpoint definition chosen in the present study seemed to be appropriate.

In some studies [5,6] an alternative approach was used to analyze efficacy of multivalent vaccine trial. Geometric means of different serotypes were compared using mixed MANOVA (Multivariate analysis of variance). This type of model assumes independence of errors across observations. A correlation between dependant variables is modeled through a multivariate normal distribution. For the comparison of group effect, the null hypothesis is that global effect is null for all serotypes. The alternative hypothesis is that there is at least one serotype for which the group effect is significant. Thus, this model is not discriminant enough and does not really answer our initial question of whether the global immunologic efficacy is better in the test group compared to the reference group. Moreover, the fit of those distributions might need complex assumptions such as mixture of lognormal and joint distributions in the context of antibody measurement with left-censuring issues due to undetectable values [3,20].

Composite endpoints are frequently used as primary endpoint in randomized trials because it may increase the statistical power and thus the efficiency of the trial and may facilitate interpretation [22]. However, the main rationale for using binomial composite endpoint would be the ability to include a range of potentially important outcomes in the composite like in ANRS 114 PNEUMOVAC trial. The use of categorization in vaccine area is often necessary, yet, the construction of such a categorical endpoint is not always suitable, depending on the vaccine strategy, vaccine composition, importance of every serotype. The proposed formulation might also be useful in other situations than vaccine evaluation, as far as a categorization of endpoint seems to be relevant.

CONCLUSION

Guidelines for the formulaton and analysis of primary endpoint in multivalent vaccine trials. We propose a formulation based on a combination of the responses to serotypes and the number of response achieved that is able to achieve a higher statistical power as compared to commonly used endpoints such as means of antibody concentrations and has better clinical relevance.

This formulation could also be used in other contexts, whenever the outcome can be divided in few ordered categories formed by units of the same relative importance.

Tables/Figures legends

Table 1. Results of several endpoint formulations in the ANRS 114 Pneumovac trial, measurements of association, P-value and associated statistical power.

Table 2. Probability of rejecting the null hypothesis at 0.05 level according to possible endpoints with simulated data. N=200 subjects, 1000 simulations Table 3. Two examples of responses distributions : example 1 where proportional odds assumption holds and example 2 where the assumption does not hold

INTRODUCTION

The efficacy and safety of vaccines are assessed by several traditional parameters [1,2]:

incidence of the disease(s) to be prevented, immune response (antibody response and immunocompetent cells), reactogenicity (local and systemic reactions). Phase II trials are usually built to test meaningful differences in the distribution of immunogenicity endpoints.

Antibody response to the target pathogen often defines the primary immunogenicity endpoint, antibody concentrations being used as surrogate endpoints for a clinically relevant endpoint [1].

Some vaccines such as pneumococcal polysaccharide vaccines yield a response to several serotypes and are called multivalent vaccines. The immunogenicity evaluation of such vaccines must take into account the response to any serotype, all of them participating in the definition of the primary endpoint. In general, the comparison between groups involves the distribution of serotype specific antibody concentrations for each serotype using geometric means [3,4,5,6,7,8,9]. The P-value is then adjusted for Bonferroni or Family Discovery Rate (FDR)controlling methods [1,10]. Another approach consists in comparing the percentage of responders to a minimum number (k) of serotypes defined a priori. First, the response to each serotype according to a given threshold is defined and then the percentage of responders to k serotypes is compared between groups [4,7,11].

Nevertheless, those endpoints are not fully satisfactory. Indeed, the former may be difficult to interpret, when results are discordant across serotypes. The latter, although clinically relevant, leads to an obvious loss of information. Furthermore, it is difficult to define the threshold for the number of responses to consider. When several serotypes are used to define immunogenicity and if they have the same relative importance, we propose to use a categorical endpoint based on the number of responses obtained and the mode of analysis using a proportional odds model [12].

Our report aims at showing the statistical and clinical relevance of this approach; firstly through the motivating example and secondly based on a simulation study.

METHODS

Study design (Figure 1)

The W8 was compared between the 2 arms (Figure 1).

Statistical methods

Direct comparison of antibody concentrations

The distribution of antibody concentrations is usually not gaussian [3,4,5]. Therefore, antibody concentrations were log-transformed and geometric means were calculated. T-test was used to compare distributions and FDR-controlling method to control for multiplicity of tests [14]. Response to each serotype was defined as two-fold increase between W0 and W8 and titre >1 µg/ml measured at 8 weeks.

Binary endpoints

Three binary endpoints were considered: (i) response to all 7 serotypes (versus non response to at least one of them), (ii) response to at least 5 serotypes among 7 (versus response to less than 4 serotypes) and (iii) response to at least 3 serotypes among 7 (versus response to less than 2 serotypes). The comparison of responder rates according to treatment group was performed with Fisher exact test.

New endpoint formulation (figure 2)

The new formulation is based on: i) the identification of a response to each serotype, ii) the number of serotypes with response. These two variables were combined into four ordered levels: 0 response, response to 1 or 2 serotypes, response to 3 or 4 serotypes or response to 5 to 7 serotypes (Figure 2). This categorization was considered a priori the most relevant by clinicians and immunologists. All serotypes have the same importance and contribute in the same way in the definition of the endpoint. This endpoint is categorical but also ordered because the higher the category, the better the response. First, Chi-2 and trend Chi-2 were used for crude analysis of this endpoint.

Secondly, the Proportional Odds Model (POM) [12] was used to analyse the new endpoint formulated. It is a generalization of a logistic model, the response variable is categorical and ordered (Y=1,2,…,J) instead of binary. A definition of this model is provided in appendix.

When the only explanatory variable is the treatment group, the global odds ratio β θ e = can be interpreted as the odds of being in the higher category above a given category when someone is vaccinated by the tested strategy compared to the control group. The comparison between the two groups can be performed by testing θ = 1 using a likelihood ratio statistics or a Wald test. This is testing an absence of difference between the groups in the distribution of the patients according to each category.

The POM assumption is that the effect of any explanatory variable is the same whatever the level of grouping of the response. In other words, the odds ratio between no response at all versus at least one response should not be different from the odds-ratio between 2 responses or less versus 3 responses or more, etc. estimated using maximum likelihood methods with standard softwares. In SAS (SAS ® software v.9.1, SAS Institute, Cary, NC, USA), we used proc LOGISTIC for the former and proc NLMIXED for the latter. Code for fitting random effects POM model is provided in Appendix.

Sample size and power were calculated using Kolassa's solution in N'Query software [18].

Simulations

Simulations were performed in order to calculate the power associated with each endpoint according to different scenarii. True parameters were defined according to the distribution of antibody concentrations in the ANRS 114-Pneumovac trial. Antibody concentrations were assumed to follow a log-normal distribution. For a given patient and a given serotype, antibody concentrations were correlated between different time of measurements and for a given patient, antibody concentrations of different serotypes were also correlated. We used the following four steps approach to simulate such data:

1) Y 0 ~ MVN (M 0 , C) (MVN for multivariate normal)

Y 0 is the multivariate vector of the logarithm of the titre of the seven serotypes.

M 0 (7x1) and C (7x7) were estimated from a multivariate linear mixed model using real data presented in the results section. They represent vector of means and variance-covariance matrix, respectively. R(7x7) is the correlation matrix .

2) For the reference group, the difference of antibody concentrations from baseline was generated according to a normal distribution: Y 8 -Y 0 ~ N (p 1 , σ p ) where p 1 and σ p were based on estimations from multivariate linear mixed models fitted on data from the Pneumovac trial.

For the test group : Y 8 -Y 0l ~ N (p 2 , σ p ) where p 2 = p 1 + β 2 with β 2 being the treatment effect

3) Test group effect was estimated using the different methods presented above 4) Steps 1 to 3 were repeated N=1000 times out of 1000. Simulations were repeated for different values of β 2 (treatment effect) including the one estimated using the real data set.

RESULTS

Application with ANRS 114 Pneumovac data (table 1)

Geometric Means (GMT) were calculated for each serotype (table 1). GMT were generally higher in those vaccinated with the tested strategy. These differences were statistically significant for 2 of them (out of 7) after correction for multiple testing. The statistical power associated to each serotype varied from 35% (serotype 6B) up to 99% (serotype 18C). The power was therefore not homogeneous among serotypes and rather low (<60%) for 4 serotypes among 7. We used a chi-2 and a chi-2 for trend to test the difference of response between the two groups, p-value was respectively 0.0373 and 0.0015.

We then considered the binary endpoint: response to all serotypes versus non response to at least one of them. The proportion of patients with successful response was significantly different between groups according to this endpoint. The odds-Ratio (OR)=2.80 (95% Confidence Interval [CI] 1.22-6.43, p=0.0123), meaning that the tested vaccinal strategy was associated with a better immunogenicity. The statistical power of the test for the null hypothesis OR=1 was 68%.

With an endpoint defined as being responder for at least 5 serotypes versus less than 5, the OR was 2.23 (95% CI 1.28-3.89, p=0.0044) and the power was 77%. The odds to be good responder (more than 5 serotypes) was therefore better for the tested vaccine strategy.

Finally, the global OR estimated with the proportional odds model for the previously defined ordered categories was 2.12 (95% CI 1.26-3.56, p=0.0047). The power was 81%. Patients in the tested vaccine strategy group showed a higher number of responses than the control group.

Morever, the proportional odds assumption was satisfied (test of non homogeneity p = 0.82). In this trial, the categorical endpoint was the most powerful as compared to other endpoints considered.

Simulations (table 2)

Estimated values for simulation parameters were: [2.0728, 1.2022, 1.3784, 1.6299, 1.4080, 0.9547, 1.2380] [0.3520, 0.6219, 0.5427, 0.6075, 0.6277, 0.5527, 0.8066] The statistical power for the comparison of geometric means of antibody concentrations between the two groups increased with the treatment effect (β t 2 ), as expected (table 2).

M 0 = [-2.
σ p =
Furthermore, the power was quite heterogeneous according to the serotype considered (eg from 26.0 to 42.3% when β t 2 = D).

The statistical power was always higher with other outcomes considered (binary or ordered categories). Among binary outcomes, the binary endpoint using a threshold at 5 serotypes led to the best statistical power compared to a threshold set at 7 serotypes (13.2% vs 8.6% for β t 2 = B; 85.0% vs 75.9% for β t 2 = D for instance). Although the outcome based on ordered categories led to the best power, the differences with the binary endpoint using a threshold at 5 serotypes were tiny (around 3%), when the effect of the treatment (β 2 ) was identical according to each serotype. With a treatment effect differing from one serotype to another (very low effect for 5 serotypes and high effect for 2 serotypes, scenario F) the binary endpoint using a threshold at 5

serotypes was substantially less powerful to discriminate poor responders (2 responses for instance) than the new endpoint formulation (54.9.6% vs 63.2%, respectively). Furthermore, a null effect scenario (β t 2 = 0) was tested and the type I error was slightly higher for the POM model (α=0.06 as compared as α=0.05 for "> 5 reponses" binary endpoint).

correlation structures between serotypes (data not shown). Whatever the correlation structure (no correlation, slight correlation, high correlation), the power of this new endpoint formulation was better as compared to the other endpoints.

DISCUSSION

The new endpoint formulation based on ordered categories was more powerful than current endpoints used for the evaluation of a multivalent vaccine efficacy. This was even more true when the response was not homogeneous among all serotypes. With binary endpoints, the power was not satisfactory even if the test was significant in our specific study. This loss of power might be explained by a loss of information due to dichotomization. For an ordered categorical variable, a chi-2 test for trend seemed to be the most powerful test. However, the use of such a test does not yield a measurement of association, does not allow adjustment for confounders and is conditional to the existence of a linear trend of responses among categories.

The main advantage of the proposed strategy is to allow evaluating the effect of the intervention on all serotypes estimating only one parameter (the odds ratio) and performing one test. Hence, it avoids any correction for multiple testing and the main result is easy to interpret.

Indeed, it had practical relevance for clinicians. Firstly, all serotypes contributed in the same way to the construction of the endpoint. The 7 serotypes of interest were those contained in the PCV vaccine since they were the most frequent causes of pneumococcal infections. Therefore, there was no rationale to give more consideration to a serotype than another. Secondly, results can be summarized using a single parameter, i.e a global OR estimated by the POM model, and presented in a simple contingency table. Such a summary contributes to improve the understanding of results by practitioners and makes it easier to remind them and to disseminate them among those who will use the vaccines.

Another advantage is that the implementation is fairly easy and convenient using standard statistical software. It is possible to accommodate for General Linear Model and to adjust for fixed or random effects. Indeed, the model can be extended by including random effects to capture heterogeneity between centers, or between individuals with repeated measurements. In our study, we included testing for such random effects (SAS code in Appendix) and did not find significant heterogeneity among them regarding vaccine response.

A major assumption for the use a POM is the homogeneity of odds ratios among the different categories. This assumption holded in our specific example. Table 3 shows two examples illustrating extreme fictive situations when the assumption holds or not. In the first example, the vaccine strategy is efficient for all serotypes; in that case, the assumption should be satisfied.

This translates into quite similar partial odds ratios. In the second example, the new vaccine strategy generates especially very good responders and the referent strategy more likely generates medium responders (example 2). The first strategy could be completely efficient (for the whole serotypes) or not at all. The second strategy could generate responses to almost 3 or 4 serotypes, leading to many responders in medium categories and few in the lowest and the strongest. According to considered cutpoints, one would differently conclude about superiority between vaccine strategies. If the cutpoint is "at least 1 response" versus none, referent strategy would be better, if the cutpoint 5-7 responses is retained, the tested strategy would be better. In this example, the criterion of definite judgment would not be relevant because partial odds-ratio would be different (p<0.0001), so proportional odds assumption would not be satisfied. It is especially important to check this proportionality assumption because it is not obvious to intuitively anticipate the homogeneity of partial odds-ratio. Indeed, a vaccine strategy with an effect that varies from one serotype to another may stick to partial odds homogeneity (see Table 2,scenarios A and F).

When the assumption is not satisfied, other models are available such as a continuous ratio model or a partial odds ratio model [16,17,21]. This latter model allows odds to be partially subgroup. To be able to use this model the number of categories should be large enough (at least 5 categories). A recent paper [15] provides SAS code for other multinomial model, which can be useful if the proportional odds assumption is not satisfied and more sophisticated model is needed. Another paper [19] suggests solutions to analyse ordinal endpoint when the proportional odds assumption does not hold. However those alternative models estimate additional parameters which leads to a decrease of statistical power to detect an effect.

Another limitation (also shared with binary endpoints) is that the threshold for antibody responses should be defined a priori and according to a relevant biological and clinical rationale. In the litterature, definition of antibody response is varying according to the increase considered (e.g. 2-or 3-fold increase of antibody concentrations) [5,23], to a threshold of the titre (e.g. titre greater than 1 µg/ml) [3] or the combination of both: 2-fold increase and titre>1

µg/ml at W8 [11]. This threshold seemed to be the most relevant because it took into account not only the increase of titre between W0 and W8 but also a minimal threshold to reach response level. Indeed, a 2-fold increase from 0.01 to 0.02 µg/ml, could not be considered as a response. In the same way, a patient could present a titre > 1 µg/ml at W8 but also at W0 and be considered as a responder even if the titre was stable. That is why the endpoint definition chosen in the present study seemed to be appropriate.

In some studies [5,6] an alternative approach was used to analyze efficacy of multivalent vaccine trial. Geometric means of different serotypes were compared using mixed MANOVA (Multivariate analysis of variance). This type of model assumes independence of errors across observations. A correlation between dependant variables is modeled through a multivariate normal distribution. For the comparison of group effect, the null hypothesis is that global effect is null for all serotypes. The alternative hypothesis is that there is at least one serotype for which the group effect is significant. Thus, this model is not discriminant enough and does not really answer our initial question of whether the global immunologic efficacy is better in the test group compared to the reference group. Moreover, the fit of those distributions might need complex assumptions such as mixture of lognormal and joint distributions in the context of antibody measurement with left-censuring issues due to undetectable values [3,20].

Composite endpoints are frequently used as primary endpoint in randomized trials because it may increase the statistical power and thus the efficiency of the trial and may facilitate interpretation [22]. However, the main rationale for using binomial composite endpoint would be the ability to include a range of potentially important outcomes in the composite like in ANRS 114 PNEUMOVAC trial. The use of categorization in vaccine area is often necessary, yet, the construction of such a categorical endpoint is not always suitable, depending on the vaccine strategy, vaccine composition, importance of every serotype. The proposed formulation might also be useful in other situations than vaccine evaluation, as far as a categorization of endpoint seems to be relevant.

CONCLUSION

Guidelines for the formulaton and analysis of primary endpoint in multivalent vaccine trials. We propose a formulation based on a combination of the responses to serotypes and the number of response achieved that is able to achieve a higher statistical power as compared to commonly used endpoints such as means of antibody concentrations and has better clinical relevance.

This formulation could also be used in other contexts, whenever the outcome can be divided in few ordered categories formed by units of the same relative importance.

Tables/Figures legends (Armstrong & Sloan 1989, Greenland 1994), or McCullagh's grouped continuous model (Greenwood & Farewell, 1988 [30]. It is a generalization of a logistic model, the response variable is categorical and ordered (Y=1,2,…,J) instead of binary. The model can be written as follows:

  i j i i i X j Y P j Y P j Y P                ) ( ) ( log ) ( logit
for i=1,..,n and j=1,…,J-1 where p[(Y i > j)] is the probability of a subject i = 1,…n to present a response in a category greater than j ,  j is the log-odds of being above category j when the vector of covariate X i is equal to 0,  the q-vector of parameters to be estimated and X i a (q x n) matrix of covariates including the treatment group (

0 1  i X
for subject i in the control group and 1 for subject i in the tested strategy). When the only explanatory variable is the treatment group, the global odds ratio   e  can be interpreted as the odds of being in the higher category above a given category when someone is vaccinated by the tested strategy compared to the control group.

The comparison between the two groups can be performed by testing  = 1 using a likelihood ratio statistics or a Wald test. This is testing an absence of difference between the groups in the distribution of the patients according to each category.

With our endpoint, three logits can be calculated, one for each of the following categories: >1 vs 0 response, >3 vs <3 responses, >5 vs <5 responses. The POM assumption is that the parameter  does not vary according to the categories, i.e. the model assumes that effects of covariates are the same for each cutpoint. This is also called 'equal slopes assumptions ' [30].

Hence, the odds is proportional to the difference between the categories of X and  is the Marked Revision constant of proportionality. Because  is the same whatever j, the underlying hypothesis is that the effect of any explanatory variable is the same whatever the level of grouping of the response. In other words, the POM is invariant under collapsibility.

An extension of the model including random effects to take into account correlation between subunit (in case of a clinical trial performed in k=1,… K centres, for instance) was written as follows:  is a random effect reflecting a centre-specific correction to the baseline risk

  k ik j ik u X j Y P        ) ( logit for

Figure 1 .

 1 Figure 1. ANRS 114 Pneumovac trial design.

Figure 2 .

 2 Figure 2. New endpoint definition: 1) response for each serotype: 2 fold-increase and a titer >

  ANRS 114-Pneumovac trial was a phase II randomized trial comparing two vaccine strategies against pneumococcal infection among adult patients infected by the Human Immunodeficiency Virus (VIH)[13]. The rationale was that HIV-infected patients are more susceptible to infections caused by common bacterial pathogens such as S. pneumoniae and therefore are in need of effective preventive strategies against this infection. In this trial, 212 patients were enrolled, 106 in each arm. The tested vaccine strategy consisted in a vaccination by heptavalent pneumococcal protein conjugate vaccine: Prevenar  (PCV) at baseline (W0) and by 23-valent pneumococcal polysaccharide vaccine: Pneumo23  (PPV) at week 4 (W4). The currently recommended strategy was used in the control arm and consisted in a vaccination by PPV at W4. The 7 serotypes of PCV(4, 6B, 9V, 14, 18C, 19F, 23F) are included in PPV and are the most frequently encountered serotypes in Western countries. Antibody concentrations of each common serotype were measured at W0 and W8 for each patient. Antibody response at

  Figure 2.

  k=1,…,K, i=1,..,n k and j=1,…,

Table 1 .

 1 Results of several endpoint formulations in the ANRS 114 Pneumovac trial, measurements of association, P-value and associated statistical power.

Table 2 .

 2 Probability of rejecting the null hypothesis at 0.05 level according to possible endpoints with simulated data. N=200 subjects, 1000 simulations

Table 3 .

 3 Two examples of responses distributions : example 1 where proportional odds assumption holds and example 2 where the assumption does not hold Figure 1. ANRS 114 Pneumovac trial design.

	Figure 2. New endpoint definition: 1) response for each serotype: 2 fold-increase and a titer >
	1mg/ml at W8 2) whole response: number of serotypes with response. Example of a patient
	who had response to 4 different serotypes.

Table 1 :

 1 Results of several endpoint formulations in the ANRS 114 Pneumovac trial, measurements of association, P-value and associated statistical power.

	Endpoint	Vaccine Strategy	Measurement	p-value Statistical
			Referent n=103	Test n=104	of association		Power (%)
	Serotypes of PCV	Geometric mean antibody
			titers (g/ml)
	4		1.0	1.5	= 0.5	0.052*	55
	6B		1.6	2.2	= 0.6	0.112*	35
	9V		1.9	2.7	= 0.8	0.052*	57
	14		5.9	9.2	= 3.3	0.052*	51
	18C		1.6	3.1	= 1.5	0.002*	99
	19F		2.0	2.9	= 0.9	0.052*	86
	23F		0.9	1.9	= 1.0	0.005*	88
	Chi-2					0.0373
	Trend Chi-2					0.0015
	Response to all serotypes	9 (9)	22 (21)	OR=2.80	0.0123	68
	n(%)
	> 5 serotypes		41 (40)	62 (60)	OR=2.23	0.0044	77
	n(%)
	> 3 serotypes		69 (66)	82 (79)	OR=1.84	0.0549	48
	n(%)
	Responders in ordered
	categories	n(%)
	0 serotype		10 (10)	5 (5)			81
	1-2		24 (23)	17 (16)
	3-4		28 (27)	20 (19)
	5-7		41 (40)	62 (60)	OR = 2.12	0.0047
	* correction for multiple testing (FDR), significant level is 0.05. Abbreviations: =difference of geometric mean,
	OR=Odds-Ratio

Table ( s)

 (

Table 2 :

 2 Probability of rejecting the null hypothesis at 0.05 level according to possible endpoints with simulated data. N=200 subjects, 1000 simulations

	Power (%)	 2 = 		t 2 = B		t 2 = C		t 2 = D		t 2 = E		t 2 = F		t	= G
	Serotypes of
	PCV
	4	63		1		17		39		60		5
	6B	27		2		17		33		54		5
	9V	42		2		22		42		65		4
	14	12		2		16		29		48		39
	18C	89		2		20		42		64		53
	19F	42		3		19		40		61		56
	23F	85		2		12		26		42		3
	Chi-2	94		7		55		82		97		40
	Trend Chi-2	99		15		84		96		99		74
	Response to all	91		9		53		76		91		32
	serotypes
	> 5 serotypes	95		13		67		85		96		55
	> 3 serotypes	64		7		35		52		69		32
	Responders in	96		14		72		89		98		63
	ordered categories
	 = [0.55, 0.34, 0.38, 0.20, 0.70, 0.38, 0.86] estimations from ANRS 114 PNEUMOVAC trial data
	B = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1],
	C = [0.3, 0.3, 0.3, 0.3, 0.3, 0.3, 0.

Table ( s)

 (

Table 3 :

 3 Two examples of responses distributions : example 1 where proportional odds assumption holds and example 2 where the assumption does not hold

			Categories of responses
		0	1-2	3-4	5-7
	Example 1
	Test : PCV+PPV (N=100)	5	15	20	60
	Referent : PPV (N=100)	10	25	25	40
	Partial odds-ratio	OR 1 = 2,11	OR 2 = 2,15		OR 3 = 2,25
	p-value for proportional odds assumption		p = 0,9863
	Global odds ratio IC95%		OR = 2,21 [1,30-3,76]
	Example 2
	Test : PCV+PPV (N=100)	40	10	10	40
	Referent : PPV (N=100)	10	50	30	10
	Partial odds-ratio	OR 1 = 0,17	OR 2 = 1,50		OR 3 = 6,00
	p-value for proportional odds assumption		p < 0,0001
	Global odds ratio (p-value)		OR = 1,07 [0,65 -1,76]

Table ( s)

 (

	Figure(s)
	Figure 1.
					PCV PCV	PPV PPV	Measurement of primary endpoint
	Test	: PCV + PPV
		n=106	w-4	w0 w0	w4 w4	w8 w8 w8	w24 w24	w48 w48	w72 w72	w96 w96
	Referent	: PPV			PPV PPV
		n=106	Measurement of antibody titers

Dr. D. Goldblatt from ICH for the processing of antibody concentrations. We thank Pr. F. Mentré and Pr. P. Morlat members of the Data and Safety Monitoring board of the study. We thank N.Sarrazin, O.Danet, J.Collin who monitored the study and maintained the database.

Source of funding. This study was performed with the financial support of the Agence Nationale de Recherches sur le Sida et les Hépatites Virales (ANRS).

Conflict of interest statement

ABSTRACT

Objective: Endpoints used for the evaluation of immunogenicity in vaccine trials are often the proportion of individuals with immune response or geometric means of antibody concentrations for each serotype. When a vaccine includes several types of the same species, we illustrate how an endpoint combining all responses may improve clinical relevance and statistical power.

Study design and settings: The motivating example was the ANRS-114 PNEUMOVAC trial where the effect of two vaccine strategies against Streptococcus pneumoniae was assessed in adults infected by the Human Immunodeficiency Virus. The power associated with several endpoints was calculated in the example and in simulations. A new endpoint based on four ordered levels is formulated and analysed by using a proportional odds model.

Results and conclusion:

The analysis of this new endpoint led to an odds-ratio allowing detection of improvement and detriment. In the simulation study, this endpoint was associated with the largest statistical power by increasing the amount of information used as compared to usual endpoints. We recommend this new endpoint formulation in the formal development of a new vaccination regimen, whenever applicable. (Greenwood & Farewell, 1988 [30]. It is a generalization of a logistic model, the response variable is categorical and ordered (Y=1,2,…,J) instead of binary. The model can be written as follows:

for i=1,..,n and j=1,…,J-1 where p[(Y i > j)] is the probability of a subject i = 1,…n to present a response in a category greater than j ,  j is the log-odds of being above category j when the vector of covariate X i is equal to 0,  the q-vector of parameters to be estimated and X i a (q x n) matrix of covariates including the treatment group (

for subject i in the control group and 1 for subject i in the tested strategy). When the only explanatory variable is the treatment group, the global odds ratio   e  can be interpreted as the odds of being in the higher category above a given category when someone is vaccinated by the tested strategy compared to the control group.

The comparison between the two groups can be performed by testing  = 1 using a likelihood ratio statistics or a Wald test. This is testing an absence of difference between the groups in the distribution of the patients according to each category.

With our endpoint, three logits can be calculated, one for each of the following categories: >1 vs 0 response, >3 vs <3 responses, >5 vs <5 responses. The POM assumption is that the parameter  does not vary according to the categories, i.e. the model assumes that effects of covariates are the same for each cutpoint. This is also called 'equal slopes assumptions ' [30].

Hence, the odds is proportional to the difference between the categories of X and  is the Appendix constant of proportionality. Because  is the same whatever j, the underlying hypothesis is that the effect of any explanatory variable is the same whatever the level of grouping of the response. In other words, the POM is invariant under collapsibility.

An extension of the model including random effects to take into account correlation between subunit (in case of a clinical trial performed in k=1,… K centres, for instance) was written as follows:

 is a random effect reflecting a centre-specific correction to the baseline risk


Clément Bessouet

Sylvain Lemettre
email: sylvain.lemettre@universite-paris-saclay.fr

Charlotte Kutyla

Alain Bosseboeuf

Philippe Coste

Thierry Sauvage

Hélène Lecoq

Olivier Wendling

Aurélien Bellamy

Piyush Jagtap

Stéphanie Escoubas

Christophe Guichet

Olivier Thomas

Johan Moulin

Electrical and ion beam analyses of Yttrium and Yttrium-Titanium getter thin films oxidation

Keywords: Yttrium alloys, oxidation, getter, thin films, low-temperature bonding, MEMS packaging

I. INTRODUCTION

Vacuum is mandatory for the optimal operation of many kinds of microelectromechanical systems (MEMS) such as resonant sensors, bolometers or RF switches. In industry, vacuum packaging is more and more realized by wafer-level packaging (WLP) processes. To maintain the required vacuum level during the 10 to 20 years of the MEMS lifetime, hermetic sealing is combined with the integration of a getter film. A key parameter is the bonding temperature, because it drives outgassing inside the cavity and the activation of the getter film [START_REF] Hilton | Wafer-Level Vacuum Packaging of Smart Sensors[END_REF] .

Getter films presenting a low activation temperature are object of numerous studies since about 1998, especially by the CERN [START_REF] Benvenuti | Nonevaporable getter films for ultrahigh vacuum applications[END_REF] . The lowest activation temperatures reported are 133 °C and 150 °C for Pd/Ti 3 and Ti-Zr-Hf-V [START_REF] Malyshev | Pumping properties of Ti-Zr-Hf-V nonevaporable getter coating[END_REF] films respectively. However the use of these getters for MEMS packaging was never reported and is then restricted to ultra-high vacuum (UHV) systems like particle accelerators. This must come from the great difference in specifications between UHV systems and MEMS packaging due to a different gaseous environment during activation [START_REF] Bosseboeuf | Effect of Environment on Activation and Sorption of Getter Alloys and Multilayers for Hybrid Wafer-level Vacuum Packaging[END_REF] .

Getter films used in MEMS packaging are typically multilayers or alloys of highly reactive transition metals (Ti, Zr, V, Co…), eventually doped with rare earths, and eventually covered with a thin passivating overlayer and/or deposited on a sublayer [START_REF] Bosseboeuf | Effect of Environment on Activation and Sorption of Getter Alloys and Multilayers for Hybrid Wafer-level Vacuum Packaging[END_REF] .

Since 2010, highly hermetic sealing of MEMS-sized cavity at wafer-scale was demonstrated with low bonding temperatures (200-300 °C) and duration times (≤1 hour), for example by transient liquid phase bonding with Au-In [START_REF] Welch | Vacuum and Hermetic Packaging of MEMS using Solder[END_REF] or Cu-Sn [START_REF] Hilton | Wafer-Level Vacuum Packaging of Microbolometer-base Infrared Imagers[END_REF] or by thermocompression bonding with Cu-Cu [START_REF] Wang | Wafer-Level Vacuum Packaging Enabled by Plastic Deformation and Low-Temperature Welding of Copper[END_REF][START_REF] Al Farisi | Low-temperature hermetic thermo-compression bonding using electroplated copper sealing[END_REF] or Au-Al [START_REF] Wang | Wafer-Level Vacuum Sealing by Transfer Bonding of Silicon Caps for Small Footprint and Ultra-Thin MEMS Packages[END_REF] . However, there is still no direct proof of a getter thin film compatible with these low temperatures bonding processes in the literature. The lowest bonding temperature reported for a functional WLP process with a getter film is 300 °C with a 30 min bonding time [START_REF] Chagnon | Encapsulation hermétique de microbolomètres pour caméras infrarouges[END_REF] , but the getter material was not specified.

In this paper, we present preliminary results concerning a new generation of getter films containing yttrium. Contrary to its close neighbors in the periodic table Ti, Zr, or V, yttrium has received little attention as a getter material, despite it is an excellent nonreversible getter for hydrogen [START_REF] Huiberts | Synthesis of yttriumtrihydride films for ex-situ measurements[END_REF] and a material with a high oxygen diffusivity [START_REF] Bosseboeuf | Effect of Environment on Activation and Sorption of Getter Alloys and Multilayers for Hybrid Wafer-level Vacuum Packaging[END_REF] .

Furthermore, it has been shown that it can reduce, as a doping element, the activation temperature of a Zr-Co sintered getter [START_REF] Fattahzadeh | Effect of milling and adding yttrium on sorption characteristics of Zr-Co based nanostructure chemical getter[END_REF] .

In this work we investigated single metal yttrium and binary alloy Y-Ti thin films activation under a gaseous environment with high partial pressures of O2 and H2O. Titanium was chosen to be combined with yttrium because of its excellent gettering properties [START_REF] Tenchine | NEG thin films for under controlled atmosphere MEMS packaging[END_REF] and because it is complementary to yttrium in terms of oxygen diffusivity and solubility: titanium has a low oxygen diffusivity and a high oxygen solubility, while yttrium has a high oxygen diffusivity and a low solubility [START_REF] Bosseboeuf | Effect of Environment on Activation and Sorption of Getter Alloys and Multilayers for Hybrid Wafer-level Vacuum Packaging[END_REF] . Y-Ti thin films have been also scarcely studied [START_REF] Li | Thickness and component distributions of yttrium-titanium alloy films in electronbeam physical vapor deposition[END_REF] .

II. EXPERIMENTAL

200 nm-thick yttrium, titanium and Y-Ti films were deposited by using UHV ee-beam co-evaporation at room temperature on 100 mm silicon wafers. Titanium and Yttrium grains with a purity of 99.99 % and 99.9 % respectively were evaporated from alumina coated conical crucibles.. The background pressure before deposition was lower than 1.0 x 10 -8 mbar. The pressure during evaporation did not exceed 4.0 x 10 -8 mbar and was almost exclusively composed of H2 gas, as measured by residual gas analysis. Y-Ti alloys were obtained by varying the deposition rate ratio of yttrium to titanium, with a maximum total deposition rate in the 0.5-1 nm/s. The unloading of the wafer after deposition was realized by introducing nitrogen in the loadlock chamber. A 200 nm-thick yttrium film capped in situ with a 10 nm-thick gold layer was also deposited on silicon with the same evaporation set-upThe wafer was kept under UHV between the deposition of the Y and Au layers.

To simulate the activation process with a high partial pressure of oxidizing gases, yttrium and Y-Ti samples were annealed in a furnace under 99.9999% pure Ar with traces of oxidizing gases, at 1 atm, for 1 h, and at temperatures ranging from 225 °C to 400 °C.

The argon flow during annealing was adjusted to 2 000 sccm. For each annealing temperature, all samples of different compositions were annealed at the same time. The effective gas purity at the sample level could not be assessed but it contains at least a few ppm traces of oxidizing species corresponding to a partial pressure larger than 10 -3 mbar. Sheet resistance of the films was measured by the 4 point probe method at room temperature. The Temperature Coefficient of Resistance (TCR) was determined by using the method described by S. Lemettre et al. [START_REF] Lemettre | In-situ electrical characterization of co-evaporated Zr-Ti, Zr-V and Zr-Co thin getter films during thermal activation[END_REF] .

Surface microstructure of as-deposited and annealed films was observed by SEM.

The mean grain size of the film was estimated by using the mean linear intercept method in accordance with ASTM E112 standard. The microstructure and phase identification were carried out using an X-ray diffractometer (Philips-Panalytical) with Cu K source.

The samples were analyzed in standard Bragg-Brentano geometry by varying the angle 2 from 25° to 45. The diffracted peaks were identified using reference data from powder samples.

Ion Beam Analyses (IBA) techniques were used to determine the atomic composition of the films. First, Rutherford Backscattering Spectroscopy (RBS) analyses were performed with a beam of [START_REF] Malyshev | Pumping properties of Ti-Zr-Hf-V nonevaporable getter coating[END_REF] He at 2 MeV to control the metallic elements composition and their in-depth homogeneity in the films. The detector is a 25 mm² PIPS located at a 166° angle with the beam. A better quantification of the light elements such as oxygen was obtained using Nuclear Reaction Analysis (NRA) performed with a deuteron beam at 900 keV. The oxygen content is measured by the [START_REF] Lemettre | In-situ electrical characterization of co-evaporated Zr-Ti, Zr-V and Zr-Co thin getter films during thermal activation[END_REF] O(d,α0) [START_REF] Tenchine | NEG thin films for under controlled atmosphere MEMS packaging[END_REF] N nuclear reaction at 166°. Combination of these techniques and their analysis by SIMNRA software [START_REF] Mayer | SIMNRA, a Simulation Program for the Analysis of NRA, RBS and ERDA[END_REF] with SigmaCalc cross section [START_REF] Gurbich | SigmaCalc recent development and present status of the evaluated crosssections for IBA[END_REF] allows determining the in-depth concentration profiles of oxygen and metals in the films.

Finally, sorption of activated pure yttrium and Y-Ti films was characterized inside sealed glass ampoules. Twenty silicon square samples with a getter film deposited on them corresponding to a total getter surface area of 102 mm² were inserted inside a glass ampoule. Four ampoules were prepared, each one filled with a getter film having a different composition. The getter films inserted in the ampoules had been previously exposed during 10 months to air at room temperature. A fifth ampoule without any sample was also prepared as a reference. Then the glass ampoules were connected to a pumping station. Once a 10 -6 mbar internal pressure was reached, the ampoules were sealed off by a localized heating of the ampoule end. The five ampoules were then annealed during 1 h at 250 °C. Once cooled down to ambient temperature, the residual gas composition inside the ampoules was measured by Oneida Research Services by the Internal Vapor Analysis (IVA) method based gas analysis by mass spectrometry

III. RESULTS AND DISCUSSION

A. Single metal yttrium

Exposure of a getter to air is unavoidable in a MEMS packaging process flow.

Curzon [START_REF] Curzon | Thin film studies of yttrium, yttrium hydrides and yttrium oxide[END_REF] stated that evaporated yttrium exposed to ambient air passivates, but the measurement was limited to few minutes. In the same way, Burnham [START_REF] Burnham | Oxidation kinetics for thin rare-earth metal films[END_REF] measured only 5 nm of yttrium oxide after 2 months. Conversely, Reichl [START_REF] Reichl | An investigation of air-grown yttrium oxide and experimental determination of the sputtering yield and the inelastic mean free path[END_REF] measurthed a growth of an Y2O3 oxide layer during one year. Finally, Huiberts [START_REF] Huiberts | Synthesis of yttriumtrihydride films for ex-situ measurements[END_REF] demonstrated that oxygen diffuses deeply in an yttrium covered with an insufficiently thick Pd layer: oxygen was measured down to 175 nm in the bulk of the film, but after an unknown period of time. These contradictory observations indicate that oxidation of evaporated yttrium films is highly sensitive to the deposition conditions. Recent observations by Moldarev 22 on sputtered Y-O-H films concluded that the native oxide layer is not self-passivating for oxygen.

A continuous measurement of the sheet resistance was performed on 190-250 nm yttrium films after deposition by e-beam-evaporation under UHV for the following three different conditions: i) in ambient air; ii) exposed to ambient air for less than 10 min, then kept under vacuum (<10 -7 mbar) for 46 days, and then exposed again to air; and iii) in ambient air after in situ deposition of an additional 10 nm-thick gold capping layer (see Figure 1).

When exposed to air, the sheet resistance of an yttrium film rapidly increased during the first three days, then slowly afterwards. After 800 days of storage in ambient air, it increased by 70 %. For the film stored under vacuum, the sheet resistance increased only by a few percent during measurement over 46 days. Upon air exposure afterwhile, its sheet resistance rapidly increases and extrapolation at very long storage durations shows that the steady-state final value is the same as for the sample measured in ambient air. Thus the sheet resistance increase is clearly related to air exposure. By considering a parallel-resistance model for multilayer films [START_REF] Loudjani | Models of Electric Transport in Bilayer Ni/Cu And in a Tri-layer Ni/Pd/Cu Thin Films[END_REF] , we can attribute the increase of resistance to the continuous oxidation of the conductive metallic layer leading to an insulating oxide layer. This is in coherence with the oxide growth during ambient air exposure measured by Reichl [START_REF] Reichl | An investigation of air-grown yttrium oxide and experimental determination of the sputtering yield and the inelastic mean free path[END_REF] (see Figure 1).

The dependence of sheet resistance with air oxidation was also confirmed by the sheet resistance measurement of the yttrium film passivated with Au cap layer: the sheet resistance was stable over 60 days because conversely to yttrium native oxide, gold prevents yttrium air oxidation. Only a small increase is observed the first 4 days, for a reason to that could not be determined. at/cm² which corresponds to a thickness of 12±-2.5 nm if we consider a density of bulk Y2O3 equal to 5.01 g/cm [START_REF] Miyazawa | Oxygen-free palladium/titanium coating, a novel nonevaporable getter coating with an activation temperature of 133 °C[END_REF] and apacking density of 0.91 (value measured for evaporated Y2O3 [START_REF] Bezuidenhout | The optical properties of evaporated Y2O3 films[END_REF] ). THis thickness value is close to the value of 11 nm measured by Reichl 21 after 13 days of storage in air.

Oxygen atoms are detected not only at the surface but also in depth, with a concentration decreasing with depth. Because of the very low partial pressures of H2O and COx oxidizing species during deposition (lower than 10 -8 mbar), these oxygen atoms are essentially arising from air exposure rather than from contamination during evaporation. The oxygen profile after 370 days of air exposure shows that in depth oxidation was not avoided by the presence of Y2O3 at the surface. The Oxygen/Yttrium atomic ratio integrated in the film depth increased from 0.27 to 0.37, meanwhile the sheet resistance increased by 10 %.

Hence, our results are in opposition with a surface passivation after few minutes as observed by Curzon [START_REF] Curzon | Thin film studies of yttrium, yttrium hydrides and yttrium oxide[END_REF] or with a very limited growth found by Burnham [START_REF] Burnham | Oxidation kinetics for thin rare-earth metal films[END_REF] . However they are in agreement with the in-depth oxidation through grain boundaries observed by Huiberts [START_REF] Huiberts | Synthesis of yttriumtrihydride films for ex-situ measurements[END_REF] and with the results of Moldarev [START_REF] Moldarev | Composition of photochromic oxygen-containing yttrium hydride films[END_REF] on Y-O-H films. The oxide peak is also absent in the Au/Y analysis, showing an oxidation below the detection limit of the capped yttrium film, in agreement with the sheet resistance measurements. However, an oxide peak appeared and became more intense after a long period of ambient storage (> 1 year) and upon annealing of the films at higher temperatures. The oxide phase was identified as cubic Y2O3 (a = b = c = 10.39 Å). The shift in the peak positions of (111) Y2O3 and (002) Y after annealing is also evident from XRD measurements, which may be due to stress induced by oxidation. This will be investigated in more details in a future work.

A Scherrer analysis of the peak widths yields crystallite sizes of 35 nm for yttrium, and 50 nm for the yttrium oxide layer in the out-of-plane direction. Figure 4 After annealing, all the samples became insulating, even at the lowest processing temperature of 225°C. IBA analyses revealed an oxygen content increasing from 1 x 10 17 at.cm -2 to 7 x 10 17 at.cm -2 after annealing at 300°C. In addition, profiles show that the sample annealed at 250°C exhibits an Y2O3 oxide layer thickness of around 150 nm, which is largely enough to make the film highly insulating. For an annealing temperature above 300 °C, the 200 nm-thick Y film is oxidized over all its thickness. All results in this section show the high affinity of yttrium for oxygen at low temperatures (≤300 °C). [START_REF] Lemettre | In-situ electrical characterization of co-evaporated Zr-Ti, Zr-V and Zr-Co thin getter films during thermal activation[END_REF][START_REF] Gantmakher | Mooij Rule and Weak Localization[END_REF] . The TCR of Y-Ti films are lower than the ones of single metals titanium and yttrium, and the lowest TCR are obtained with compositions close to equal content in Ti and Y, as expected by the absence of definite intermetallic compounds or solid solution in the Y-Ti system [START_REF] Murray | The Ti-Y System, Phase Diagrams of Binary Titanium Alloys[END_REF] . However, the resistivity does not follow the same trend, as it is more sensitive to impurity scattering than to crystalline disorder. The Y-Ti films have a much larger resistivity than the single metal titanium film, but similar to the single metal yttrium one. The Y10Ti90 film has the highest resistivity. The XRD analysis confirmed the absence of a Y-Ti solid solution, as predicted by the phase diagram [START_REF] Murray | The Ti-Y System, Phase Diagrams of Binary Titanium Alloys[END_REF] (see Figure 9). Ti and Y crystallites were detected in Y-Ti films with high content of Ti and Y respectively. An out-of-plane grain size between 5 and 10 nm was deduced from the Scherrer law, which is lower than the one (35 nm) measured for the single metal yttrium film. The resulting high density of grain boundaries in the Y-Ti films contributes to their high resistivity when compared to titanium.

B. Y-Ti alloys

FIG. 9. XRD analysis of non-annealed Y-Ti films

The sheet resistance evolution of an Y56Ti44 film stored at ambient air is compared to the ones of yttrium and titanium single metal films in Figure 10. The sheet resistance of the titanium film stays nearly constant, as expected for this material known to selfpassivate in air with a stable titanium dioxide [START_REF] Tenchine | NEG thin films for under controlled atmosphere MEMS packaging[END_REF] . The sheet resistance of the Y56Ti44 film increases over time, but at a much lower rate than single metal yttrium.

The sheet resistances of titanium, yttrium and various Y-Ti alloyed films were measured after 280 days of exposure to air (Figure 11). The highest sheet resistance increase is obtained for the single metal yttrium film, and the lowest (almost null) increase for single metal titanium films. Concerning Y-Ti films, higher is the yttrium content, larger is the increase of its resistance. As the alloyed film is composed of both yttrium and titanium grains and because it is thermodynamically favorable (the enthalpy of formation of Y2O3 of -1905 kJ/mol is lower to the one of TiO of -519 kJ/mol and TiO2 of -966 kJ/mol 5 ), it is supposed that a preferential oxidation of yttrium is at the origin of this behavior. The oxygen and metals profiles of the same samples are shown in Figure 12. The alloy composition is in depth homogeneous in the bulk of the films. The oxygen profile of the Y10Ti90 film confirms the in-depth oxidation, which may be due to its less compact structure. For other samples, the surface oxygen content (25%) in Y68Ti32 is higher than the one (16%) of Y33Ti67 , and the oxygen atoms diffused deeper in Y68Ti32 than in Y33Ti67. Thus, it confirms that oxidation, like the sheet resistance, could be is related to the Y/Ti ratio in the film: Yttrium-rich films have a higher absorption of oxygen in ambient air. However, for these films, oxygen content and diffusion depth are much lower than in single metal yttrium (cf Figure 5). Hence, compared to single metal yttrium, this demonstrates that incorporation of titanium in yttrium improves the resistance to room temperature oxidation of the alloy. For all films, the sheet resistance (see Figure 13) increases with the annealing temperature, at a rate that rises with the yttrium content. This trend is explained by a raise in oxygen content, as deduced by IBA analyses (see Figure 14), thus to a decrease of the metallic volume in the film. Single metal yttrium, Y68Ti32 and Y33Ti67 films become insulating after annealing at 225 °C, 275 °C and 350 °C respectively, while Y10Ti90 and Ti films remain conductive even after 400 °C annealing. So alloyed films show an intermediate behaviour between the ones of single metal yttrium (which oxidizes at low temperature) and titanium (which oxidizes at high temperature). This behaviour may originate from the dual phase structure (yttrium + titanium) in the Y-Ti alloys.

The oxygen profiles of annealed Y33Ti67 and Y68Ti32 films, (Figure 15) show that they are oxidized along their whole thickness, even after an annealing at only 225 °C. At 250 °C, the decrease in the slope of the oxygen content is much lower than the one measured in single metal yttrium (Figure 5). This demonstrates a higher oxygen diffusivity in the alloyed films. This is likely related to their higher density of grain boundaries.

The consequence is that Y33Ti67 and Y68Ti32 films exhibit higher sorption property for oxygen at low temperature than single metal yttrium: if we quantify sorption by the equivalent number of sorbed O2 molecules, Y68Ti32 and Y33Ti67 films sorbed 0.5 µmol.cm -2 of O2 after 1 h annealing at 250 °C, while yttrium sorbed 0.2 µmol.cm -2 , and n oxygen sorption for titanium was below the detection limit. Titanium already proved its getter properties for vacuum packaging at bonding temperature higher than 300 °C27 . The comparison of the yttrium-based films with the titanium film shows their interest as getter material for packaging at low temperature. Y-Ti thin film with yttrium content comprised between 30 % and 70 % appear as the most appropriate candidate for a getter presenting a low temperature activation, as it combines high oxygen sorption and a good resistance to surface thermal oxidation. Indeed, as demonstrated, a lower content in yttrium would give a lower sorption of oxygen at low activation temperature, while a higher content would give a lower resistance to atmospheric oxidation.

C. Getter performances in vacuum-sealed conditions

Three different getter films exposed 10 months to ambient air were packaged under vacuum in glass ampoules and activateded at 250 °C. Table 1 reports the residual gas composition measured inside the ampoules according to their content. The IVA does not measure the exact quantity of gas trapped in a ampoule, so the pressures are given in arbitrary units. O2 partial pressure (arbitrary unit) 2 x 10 -10 0 0 0

By comparing the H2O and O2 partial pressures of the ampoules with getters with the ones of the ampoule without getter, the sorption performance of the getters in vacuumsealed conditions is emphasized. The Y5Ti95 film did not sorb water vapour but only the traces of O2, demonstrating an almost negligible activation at 250 °C, as expected from our previous measurements for Y-Ti films with low yttrium content (<30 %). The pure yttrium film sorbed all the O2 and 60 % of the H2O outgassed by the glass walls, while the Y69Ti31 film sorbed all the O2 and more than 99 % of the H2O. This demonstrates that these two getters were activated at 250 °C, and confirm the higher sorption performance of Y-Ti film over Y.

IV. SUMMARY AND CONCLUSIONS

Yttrium, titanium and Y-Ti thin films were elaborated by co-evaporation under UHV. Y-Ti films exhibit nanometric crystallites size (5-10 nm), smaller than the one of single metal yttrium (35 nm). This leads to a very high grain boundary density in the films, which is a favorable microstructure for low temperature oxygen diffusion, and thus gettering properties. The yttrium film is not passivated after exposure to ambient air and instead continuously oxidizes over a period as long as one year. A stronger resistance to air oxidation at room temperature was proved for Y-Ti film, depending of the yttrium content in the alloy. Y-Ti film with an yttrium content higher than 30 % showed large sorption in oxygen after annealing at temperature as low as 250 °C: it was estimated that the equivalent of 0.5 µmol.cm -2 of O2 were absorbed by a 200 nm-thick Y68Ti32 film, while Y absorbed 0.2 µmol.cm -2 , and no oxygen sorption was detected for Ti. This makes

FIG. 1 .

 1 FIG. 1. Sheet resistance monitoring over time of three yttrium films stored at ambient air or under vacuum (start of the measurement: <10 min after exposure to air).

Figure 2

 2 Figure 2 displays the oxygen profile in yttrium measured by NRA after 14 and

FIG. 2 .

 2 FIG. 2. Oxygen profile in a 200 nm-thick yttrium film after two different times of exposure to air.

  FIG.3. XRD analysis of evaporated yttrium films after different exposures to oxygen.

Figure 5

 5 Figure 5 shows the results of electrical measurements and RBS-NRA analyses

Figure 6

 6 Figure 6 displays the resistivity and TCR of Y-Ti films as function of Y content ,

FIG. 6 .

 6 FIG. 6. Resistivity and TCR of as-deposited 200 nm-thick Y-Ti films 40 days after deposition.

FIG. 10 .

 10 FIG. 10. Sheet resistance monitoring over time of Y, Ti and Y56Ti44 films stored at ambient air (start of the measurement: <10 min after exposure to air).

FIG. 12 .

 12 FIG. 12. Oxygen and metal profiles of Y10Ti90 (a), Y33Ti67 (b), and Y68Ti32(c) films after 280 days of exposure to ambient air.

FIG. 13 .

 13 FIG. 13. Sheet resistance of Y, Ti and Y-Ti films after 1 h annealing in argon.

TABLE I .

 I H2O and O2 partial pressures measured by IVA in glass ampoule containing 102 mm² of Y-Ti getter film after 250 °C-1 h annealing

	Ampoule content No getter	Y5Ti95	Y69Ti31	Y
	H2O partial pressure (arbitrary unit) 1 x 10 -6	1 x 10 -6 6 x 10 -9 3 x 10 -7

YxTi100-x with 30 < x < 70 suitable candidates for packaging MEMS under vacuum with low bonding temperature.

ACKNOWLEDGEMENTS

This work was partially funded by the "Agence Nationale de la Recherche" (project ANR-19-CE08-0011) and by the French RENATECH network. The authors wish to thank Abdelhanin Aassime, Laetitia Leroy, Fabien Bayle, Cédric Villebasse for their work allowing a good operation of deposition, annealing and characterization set-ups in C2N clean room facilities.

DATA AVAILIBILITY

The data that support the findings of this study are available from the corresponding author upon reasonable request.