Ophélie Couperier
email: ophelie.couperier@ensae.fr.

Jérémy Leymarie
email: jeremy.leymarie@univie.ac.at.we

Denisa Banulescu-Radu

Ansgar Belke

Sylvain Benoit

Christophe Boucher

Massimiliano Caporin

Christian Francq

Jean-Charles Garibal

Sullivan Hué

Christophe Hurlin

Sébastien Laurent

Yang Lu

Richard Luger

Štefan Lyócsa

Daniel Platte

Mathias Pohl

Luca Riccetti

Olivier Scail- Let

Sessi Tokpavi

Jean-Michel Zakoïan

We

Keywords: Banking regulation, Financial risk management, Forecast evaluation, Hypothesis testing, Tail risk. JEL classification: C12, C52, G18, G28, G32

This article proposes an original approach to backtest Expected Shortfall (ES) exploiting the definition of ES as a function of Value-at-Risk (VaR). Our methodology examines jointly the validity of the VaR forecasts along the tail distribution of the risk model, and encompasses the Basel Committee recommendation of verifying quantiles at risk levels 97.5%, and 99%. We introduce four easy-to-use backtests in which we regress the ex-post losses on the VaR forecasts in a multi-quantile regression model, and test the resulting parameter estimates. Monte-Carlo simulations show that our tests are powerful to detect various model misspecifications. We apply our backtests on S&P500 returns over the period 2007-2012. Our tests identify misleading ES forecasts in this period of financial turmoil. Empirical results also show that the detection abilities are higher when the evaluation procedure involves more than two quantiles, which should accordingly be taken into account in the current regulatory guidelines.

Introduction

In response to the market failures revealed by the global 2007-2008 financial crisis, the Basel Committee on Banking Supervision (BCBS) has adopted the Basel III accords to improve the banking sector's ability to absorb shocks arising from financial and economic stress (BCBS, 2010). Among the number of fundamental reforms that must be implemented until January 1st, 2022 (BCBS, 2019), the BCBS has substituted Value-at-Risk (VaR) by Expected Shortfall (ES) for the calculation of market risk capital requirements. Expected Shortfall, also referred to as Conditional VaR (CVaR) or Tail VaR (TVaR), measures the expected loss incurred on an asset portfolio given that the loss exceeds VaR. That is, if L t is the (integrable) ex-post loss on a portfolio at time t, Ω t-1 is the information at time t -1, and Q Lt (.) is the quantile function of L t , the τ -level ES and VaR are given by

ES t (τ ) = E [L t | L t ≥ V aR t (τ ) ; Ω t-1 ] ,
V aR t (τ ) = Q Lt (τ ; Ω t-1 ) .

As an alternative tail risk measure, ES offers a number of appealing properties that overcomes the deficiencies of the more-familiar VaR. In particular, ES is coherent meaning that it satisfies the properties of monotonicity, sub-additivity, homogeneity, and translational invariance (see [START_REF] Artzner | Coherent measures of risk[END_REF][START_REF] Acerbi | On the coherence of expected shortfall[END_REF]. Furthermore, ES gives information about the expected size of the loss given that a loss bigger than VaR is experienced, while VaR only captures the likelihood of an incurred loss, and tells us nothing about tail sensitivity. In its revised standards for market risk, the BCBS emphasizes the important role of ES in place of VaR "to ensure a more prudent capture of "tail risk" and capital adequacy during periods of significant financial market stress" (BCBS, 2016, page 1).

Although ES is considered as the new standard for risk management and regulatory requirements, there are still outstanding questions about the modeling of ES (see e.g., De Roon and Karehnke, 2017; [START_REF] Patton | Dynamic semiparametric models for expected shortfall (and value-at-risk)[END_REF][START_REF] Taylor | Forecasting value at risk and expected shortfall using a semiparametric approach based on the asymmetric laplace distribution[END_REF][START_REF] Hautsch | Multivariate dynamic intensity peaks-over-threshold models[END_REF], and the validation of the ES forecasts, or backtesting. [START_REF] Jorion | Value at risk: the new benchmark for managing financial risk[END_REF] defines backtesting as a formal statistical framework that consists of verifying if actual losses are in line with projected losses. Because ES is unobservable, its evaluation cannot be done conventionally as a direct comparison of the observed value with its forecast, and thus generally relies on the elicitability property. A risk measure is elicitable if there exists a loss function such that the solution of minimizing the expected loss is the risk measure itself. However, it has been established that, in contrast to VaR, ES does not meet the general property of elicitability [START_REF] Gneiting | Making and evaluating point forecasts[END_REF], but satisfies narrower properties such as conditional elicitability [START_REF] Emmer | What is the best risk measure in practice? a comparison of standard measures[END_REF], or joint elicitability with VaR [START_REF] Acerbi | Backtesting expected shortfall[END_REF][START_REF] Fissler | Higher order elicitability and osband's principle[END_REF], making its evaluation trickier than VaR in practice. Several contributions are tied to these properties, and provide backtests by making explicit reference of the ES forecasts in the testing procedure [START_REF] Mcneil | Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach[END_REF][START_REF] Acerbi | Backtesting expected shortfall[END_REF][START_REF] Nolde | Elicitability and backtesting: Perspectives for banking regulation[END_REF][START_REF] Bayer | Regression based expected shortfall backtesting[END_REF].

To circumvent the lack of elicitability of ES, several alternative testing strategies have been proposed in the literature. Following the recent classification of [START_REF] Kratz | Multinomial var backtests: A simple implicit approach to backtesting expected shortfall[END_REF], these backtests enter the category of implicit backtests, as they focus on the tail distribution characteristics of the model rather than directly on ES. They generally exploit the fact that ES can be expressed as a function of VaR, which itself is elicitable. Assume the law of L t is continuous. The definition of a conditional probability and a change of variable yield a useful representation of ES in terms of VaR

ES t (τ ) = 1 1 -τ 1 τ V aR t (u) du.
(1)

Using this connection, [START_REF] Costanzino | Backtesting general spectral risk measures with application to expected shortfall[END_REF] derive a coverage backtest for spectral risk measures such as ES in the spirit of the traditional VaR coverage backtests. [START_REF] Du | Backtesting expected shortfall: Accounting for tail risk[END_REF] define a cumulative violation process for ES that generalizes the violation process for VaR and propose two backtests of ES. Starting with the same process, Löser et al.

(2019) develop a backtest of ES that is theoretically valid in finite out-of-sample size and that can be easily extended to a multivariate setting. [START_REF] Costanzino | A simple traffic light approach to backtesting expected shortfall[END_REF] provide a Trafic Light backtest for ES which extends the so-called Traffic Light backtest for VaR.

More largely, several additional techniques have been proposed to assess the whole return distribution encompassing ES as a special case [START_REF] Berkowitz | Testing density forecasts, with applications to risk management[END_REF][START_REF] Kerkhof | Backtesting for risk-based regulatory capital[END_REF][START_REF] Wong | Backtesting trading risk of commercial banks using expected shortfall[END_REF]. See the survey of [START_REF] Argyropoulos | A survey on risk forecast evaluation[END_REF] for more details.

This article reviews the relation between ES and VaR to offer an original way for backtesting ES that has the advantage of examining a limited number of VaRs in the tail loss distribution. Indeed, the definition of a Riemann sum gives a handy approximation of ES,

ES t (τ ) ≈ 1 p p j=1 V aR t (u j ) ,
where the risk level u j is defined by u j = τ +(j -1) 1-τ p for j = 1, 2, . . . , p. This representation suggests that p quantiles with appropriate risk levels would be convenient to assess the performance of an ES model. In other words, an estimate/forecast of ES t (τ ) issued from a given model could be considered valid if the sequence of V aR t (u j ) estimates/forecasts issued from the same model is itself valid. This testing strategy is fully consistent with the general recommendation of financial supervisors, indicating that "Backtesting requirements [for ES] are based on comparing each desk's 1-day static value-at-risk measure [...] at both the 97.5th percentile and the 99th percentile" (BCBS, 2016, page 57).

The main contribution of this article is to propose an original backtesting methodology to ES based on the theory of multi-quantile regression. We develop a multivariate framework, focusing on multi-quantile regression, to jointly assess VaR at multiple levels in the tail distribution of the risk model. The method extends the seminal idea of [START_REF] Gaglianone | Evaluating value-at-risk models via quantile regression[END_REF] to evaluate the validity of a single VaR relying on a single quantile regression.

Our procedure has many advantages. First, it encompasses the regulatory standards that consist of verifying the validity of two given quantiles. Second, it offers flexibility since the risk manager or the supervisor may select both the number of risk levels and their magnitude depending on the objective in mind (regulatory guidelines, ES statistical approximation, etc.). Third, it enters the category of regression-based backtests and complements the existing literature on regression-based risk forecast evaluation (see e.g., [START_REF] Engle | Caviar: Conditional autoregressive value at risk by regression quantiles[END_REF][START_REF] Christoffersen | Elements of financial risk management[END_REF][START_REF] Bayer | Regression based expected shortfall backtesting[END_REF]. It also represents an alternative to the multiple VaR exceptions backtests (see e.g., [START_REF] Colletaz | The risk map: A new tool for validating risk models[END_REF][START_REF] Kratz | Multinomial var backtests: A simple implicit approach to backtesting expected shortfall[END_REF].

Finally, the procedure may be useful for systemic risk measures that involve VaR and ES estimation (see e.g, [START_REF] Benoit | Implied risk exposures[END_REF][START_REF] Tarashev | Risk attribution using the shapley value: Methodology and policy applications[END_REF][START_REF] Van Oordt | Systemic risk and bank business models[END_REF].

Formally, we show that the parameters of the multi-quantile regression model have specific properties under the hypothesis of valid ES forecasts. We propose four backtests which correspond to various linear restrictions on these parameters. These restrictions are implications of a Mincer-Zarnowitz representation [START_REF] Mincer | The evaluation of economic forecasts[END_REF]. Then, we test the resulting parameter restrictions using Wald-type inference. We apply QML estimation for the quantile regression parameters and we implement a pairs bootstrap algorithm [START_REF] Freedman | Bootstrapping regression models[END_REF] to correct the finite sample size distortions of our backtests. Finally, we introduce a procedure deduced from our regression framework to adjust the invalid risk forecasts.

Several Monte Carlo experiments are reported and an empirical application with the S&P500 series is conducted. Our backtests deliver good performances to flag misleading ES risk forecasts. We also find that the use of asymptotic critical values is prone to substantial size distortions, while the implementation of bootstrap critical values provides satisfactory size performances regardless of the sample size. The latter should hence be preferred when asymptotic theory does not apply conveniently.

Our empirical results suggest an update of the current regulatory guidelines. First, we

show that the BCBS recommendation of assessing quantiles at risk levels 97.5% and 99% is not always sufficient to identify misspecified ES models. The use of additional quantiles is recommended to improve the soundness of the decision. Second, our results suggest to limit the number p of quantiles in small samples (with typically p ≤ 6) and to consider higher values if the historical sample covers longer periods. Finally, we show numerically that our approximation of ES as a combination of several VaRs is close to its theoretical counterpart, which strongly supports its implementation in a risk management viewpoint.

The rest of the paper is structured as follows. In Section 2, we introduce the multiquantile regression framework. Section 3 describes the null hypotheses, the test statistics, their asymptotic properties, and the implementation of the bootstrap critical values. Section 4 studies the finite sample properties of our backtests from a set of Monte Carlo experiments.

Section 5 applies our methodology to the S&P500 index and introduce a procedure to adjust the imperfect ES risk forecasts. We conclude the paper in Section 6.

Multi-quantile regression framework

This section describes our proposed multi-quantile regression approach. In the first part, we discuss the usefulness of approximating ES via a finite sum of VaRs. In a second part, we describe the multi-quantile regression model that we employ in our testing strategy. The last part is devoted to the description of the estimation method and the asymptotic theory.

ES as an approximation of VaRs

Our backtesting procedure exploits the relationship between VaR and ES. We suppose that ES can be approximated as an average of VaRs. This assertion stems from the representation of ES as the limit of a Riemann sum when the partition becomes infinitely fine.

ES t (τ ) ≈ 1 p p j=1 V aR t (u j ) , ( 2
)
where risk levels u j , j = 1, 2, . . . , p, satisfy u j = τ + (j -1) 1-τ p , and p denotes the number of subdivisions taken in the definite integral.

Our approximation of ES averages VaRs in the upper tail distribution of the risk model.

The number of quantiles involved in the sum is given by p and characterizes the approximation accuracy. In particular, p = 1 involves a single VaR at coverage level τ , while increasing p to infinity leads Equation (2) to converge to the theoretical ES. As we rely on a Riemann sum, the approximation assigns equal weights 1/p to each element in the sum, and the risk levels u j , j = 1, 2, . . . , p, are determined so that the interval is equally partitioned between the two boundaries τ and 1. Several alternatives for the approximation of a definite integral are available. Here, we rely on a Riemann sum for its simplicity and ease of implementation.

We show how to derive the above formula in Appendix A.

In practice, p may be chosen small as the interval of the definite integral is restricted to the extreme upper tail distribution. For instance, [START_REF] Gouriéroux | Converting tail-var to var: An econometric study[END_REF]

Multi-quantile regression model

In the sequel, we consider an asset or a portfolio, and denote by L t the corresponding loss observed at time t, for t = 1, 2, . . . , T . In addition, we denote by Ω t-1 the information set available at time t -1, with (L t-1 , L t-2 , . . .) ⊆ Ω t-1 . Formally, the Ω t-1 conditional VaR at level u j of the L t distribution is the quantity V aR t (u j ) such that

Pr (L t ≥ V aR t (u j ) |Ω t-1 ) = u j . (3)
A VaR model is said to be correctly specified (at coverage level u j ) as soon as Equation ( 3) holds for all t. In practice, VaR forecasts are assessed through the evaluation of this simple equality. Given the ES approximation introduced in Definition 1, this equality may arguably be adapted for the assessment of ES models. The chief insight is to evaluate Equation (3) for a number p of risk levels as set out in Definition 1. Accordingly, one should conclude to the appropriateness of a given ES model as soon as the sequence V aR t (u j ), t = 1, 2, . . . , T , issued by the ES model satisfies Equation (3) jointly for j = 1, 2, . . . , p.

We refer to the original idea of [START_REF] Gaglianone | Evaluating value-at-risk models via quantile regression[END_REF] who derive a backtest of VaR at a single coverage level, introducing VaR as a regressor of a quantile regression model. We generalize their approach for the assessment of multiple VaRs. To do so, we regress the expost losses {L t , t = 1, 2, . . . , T } on the p VaR forecasts {V aR t (u j ) , t = 1, 2, . . . , T } j=1,2,...,p in a multi-quantile regression model as follows:

L t = β 0 (u j ) + β 1 (u j ) V aR t (u j ) + j,t ∀ j = 1, 2, . . . , p, (4)
where β 0 (u j ), and β 1 (u j ), respectively, denote the intercept and the slope parameters at level u j , and where j,t is the error term at risk level u j and time t, such that the u j -th conditional quantile of j,t satisfies Q j,t (u j ; Ω t-1 ) = 0. This specification could be interpreted as a multi-quantile regression version of [START_REF] Koenker | Inference on the quantile regression process[END_REF] and is related to the multiquantile CaViAR model (MQ-CaViAR) of [START_REF] White | Var for var: Measuring tail dependence using multivariate regression quantiles[END_REF] which allows a joint modeling of multiple conditional VaRs. Given the multi-quantile regression model of Equation ( 4), the u j -th conditional quantile of L t is defined as

Q Lt (u j ; Ω t-1 ) = β 0 (u j ) + β 1 (u j ) V aR t (u j ) ∀ j = 1, 2, . . . , p. ( 5
)
This equation is central for our backtesting methodology as it establishes a direct link between the VaR forecasts (issued from the external ES model), with the true unknown conditional quantile (issued from the ex-post observed losses). Our procedure consists in verifying if there exists a perfect match between V aR t (u j ) and Q Lt (u j ; Ω t-1 ). Consistently with [START_REF] Gaglianone | Evaluating value-at-risk models via quantile regression[END_REF], we rely on the regression parameters, and test if the intercept parameter β 0 (u j ), and the slope parameter β 1 (u j ), are respectively equal to zero, and one, for j = 1, 2, . . . , p. For these values, and given Definition 1, the risk model is accepted as a valid proxy of the true unknown data generating process to deliver the ES forecasts.

Parameter estimation and asymptotic properties

Our backtesting procedure requires to consistently estimate the parameters β 0 (u j ), and β 1 (u j ), for j = 1, 2, . . . , p. Under the hypothesis that a sequence of VaR is valid, coefficients satisfy β 0 (u j ) = 0, and β 1 (u j ) = 1, for j = 1, 2, . . . , p. In what follows, we denote by β (u j ) = (β 0 (u j ) , β 1 (u j )) the vector of parameters for the u j -th quantile index, and we write β = β (u 1 ) , β (u 2 ) , . . . , β (u p ) the stacked vector of 2p coefficients. We assume that the sequence {u j , j = 1, 2, . . . , p} is ordered in the sense that u 1 < u 2 < . . . < u p < 1.

In order to estimate β, we consider a QML estimator dedicated to multi-quantile regres-sion, given by

β = arg min β∈R 2p T -1 T t=1   p j=1 ρ u j (L t -β 0 (u j ) -β 1 (u j ) V aR t (u j ))   ,
where ρ u j (x) = xψ u j (x) is the standard "check function", and

ψ u j (x) = u j -1 (x ≤ 0)
is the usual quantile step function. Under suitable regularity conditions, it is shown that this estimator is consistent and asymptotically normally distributed. The conditions are described in Appendix B and a discussion is provided on how these assumptions are fulfilled.

Under Assumptions A0-A2 in Appendix B, the asymptotic distribution of the QML estimator is given by

√ T β -β d → N (0, Σ) ,
where Σ denotes the asymptotic covariance matrix which takes the form of a Huber (1967) sandwich. Its expression is given by Σ

= A -1 V A -1 , with V = E [η t η t ], η t = p j=1 ∇Q Lt (u j ; Ω t-1 ) ψ u j ( j,t ), A = p j=1 E [f j,t (0) ∇Q Lt (u j ; Ω t-1 ) ∇ Q Lt (u j ; Ω t-1 )],
where ∇Q Lt (u j ; Ω t-1 ) denotes the 2p gradient vector differentiated with respect to β, j,t = L t -Q Lt (u j ; Ω t-1 ), and f j,t (0) denotes the pdf of j,t evaluated at zero. In Appendix C, we provide a consistent estimator Σ of Σ that will be used to compute our test statistics.

Finally, Appendix D provides a discussion on the rate of convergence and interplay of p and T when T tends to infinity. Under this asymptotic framework, it is shown that p is increasing with T . We consider a simple illustration assuming p takes the form of a power function. Under this assumption, T needs to diverge faster than p in order to maintain the asymptotic theory. This condition is not importantly restrictive but suggests the existence of an (asymptotic) upper limit for p which depends on the sample size. Section 4 provides several Monte-Carlo experiments with various values for p and T to give guidelines on how to choose these parameters jointly in finite samples. In Section 5, we suggest an avenue of research using extremal conditional quantile models [START_REF] Chernozhukov | Extremal quantile regression[END_REF][START_REF] Chernozhukov | Inference for extremal conditional quantile models, with an application to market and birthweight risks[END_REF]) that allows examining deeper the tail distribution of ES risk models.

Backtesting ES

In this section, we present our backtests for ES. Our procedures assess whether the parameters β 0 (u j ) and β 1 (u j ) coincide with their expected values for risk levels u j , j = 1, 2, . . . , p.

To this end, we propose four backtests that analyze various settings on the regression coefficients. In the sequel, we introduce the null hypotheses, the test statistics, and establish their asymptotic properties. Finally, we discuss the use of finite sample critical values and provide a bootstrap algorithm when the asymptotic theory does not apply conveniently.

The backtests

Formally, our goal is to test β 0 (u j ) = 0, and β 1 (u j ) = 1, for j = 1, 2, . . . , p. As highlighted by [START_REF] Gaglianone | Evaluating value-at-risk models via quantile regression[END_REF] for a unique quantile regression, the aforementioned set of restrictions retains a [START_REF] Mincer | The evaluation of economic forecasts[END_REF] interpretation for each quantile regression in (4). Here, we propose to test various implications of these coefficient restrictions by taking into consideration four distinct null hypotheses based on a reduced number of constraints.

Many backtests test implications of a more general hypothesis. In this context, [START_REF] Du | Backtesting expected shortfall: Accounting for tail risk[END_REF] assess two implications for the martingale difference sequence of their cumulative violation process. [START_REF] Mcneil | Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach[END_REF] and [START_REF] Nolde | Elicitability and backtesting: Perspectives for banking regulation[END_REF] propose to test the zero mean hypothesis of their residuals which more largely behave as white noise.

Definition 2 (Null hypotheses). Denote by J 1 , J 2 , I, and S, the four backtests. The corresponding null hypotheses H 0,J 1 , H 0,J 2 , H 0,I , H 0,S , are defined as follows:

H 0,J 1 : p j=1 (β 0 (u j ) + β 1 (u j )) = p, ( 6
)
H 0,J 2 : p j=1 β 0 (u j ) = 0, and, p j=1 β 1 (u j ) = p, ( 7
)
H 0,I :

p j=1 β 0 (u j ) = 0, (8)
H 0,S :

p j=1 β 1 (u j ) = p, ( 9
)
where notations J 1 and J 2 indicate the "joint" backtests, and where I and S refer to the "intercept" backtest and to the "slope" backtest, respectively.

Equations ( 6)-( 9) of Definition 2 gives the null hypotheses H 0,J 1 , H 0,J 2 , H 0,I , H 0,S . They are devised to assess various implications that the regression coefficients should satisfy when the ES risk forecasts are valid. The coefficients are summed across risk levels u j , j = 1, 2, . . . , p. This aggregation substantially reduces the number of constraints. H 0,J 2 is hence characterized by two constraints, and H 0,J 1 , H 0,I , H 0,S involve a single constraint.

Our null hypotheses analyze various settings on the regression coefficients. The null of the joint backtests, H 0,J 1 and H 0,J 2 , look at the expected value of both the intercept and slope parameters β 0 (u j ) and β 1 (u j ) for j = 1, 2, . . . , p. H 0,J 1 sums the two types of coefficient together, while H 0,J 2 sums the coefficients separately depending on whether they are slope parameters or intercept parameters. Finally, the null hypotheses of the intercept backtest and the slope backtest, H 0,I and H 0,S , focus solely on one of the two parameter components. H 0,I is built to examine the intercept parameters β 0 (u j ), j = 1, 2, . . . , p, and H 0,S is devoted to the analysis of the slope parameters β 1 (u j ), j = 1, 2, . . . , p. These additional null hypotheses complement the joint backtests to identify the nature of the misspecification. If the joint hypotheses are rejected, separate tests for these two types of measurement error should be considered. They are inspired by the prediction-realization framework of [START_REF] Mincer | The evaluation of economic forecasts[END_REF]. When H 0,I is rejected, the intercept parameters, β 0 (u j ) , j = 1, 2, . . . , p, do not sum to 0, and hence, the average of VaR risk forecasts either underestimate or overestimate the true quantiles, if the sign of the sum is positive or negative, respectively.

The rejection of H 0,S indicates that the sum of the slope parameters β 1 (u j ) , j = 1, 2, . . . , p, does not equal p, which highlights correlation between the forecasting errors and the quantile series.

Definition 3 (Wald-test statistics). Let us denote by W ∈ {J 1 , J 2 , I, S} the generic notation for the test statistic, and consider the classical formulation of a Wald-type test such as H 0,W :

R W β = q W .
The general expression of the test statistics is given by

W = T R W β -q W R W ΣR W -1 R W β -q W , ( 10
)
where T is the out-of-sample size, and Σ denotes a consistent estimator of the asymptotic covariance matrix.

To assess our null hypotheses we consider Wald-type inference. Equation ( 10) of Definition 3 gives the general expression of the test statistics. According to our notations, substituting W by J 1 , J 2 , I, and S, yields the four test statistics. For ease of presentation, the null hypotheses are now presented in a classical formulation, such that H 0,W : R W β = q W . Given the null hypotheses of Definition 2, the quantities R W and q W are as follows: R J 1 = ι p ⊗ 1 1 ,

q J 1 = p, R J 2 = ι p ⊗ I 2 , q J 2 = 0 p , R I = ι p ⊗ 1 0 , q I = 0, R S = ι p ⊗ 0 1 , q S = p,
where ι p is a p-row unit vector, and I 2 denotes the identity matrix of size 2.

Proposition 1 (Chi-squared distribution). Consider the multi-quantile regression model in Equation ( 4), Assumptions A0-A3 in Appendix B, and the null hypotheses of Definition 2, the test statistics J 1 , I, and S, converge to a chi-squared distribution with 1 degree of freedom, and the test statistic J 2 converges to a chi-squared distribution with 2 degrees of freedom.

Proposition 1 gives the asymptotic distribution of the Wald statistics J 1 , J 2 , I, S under their respective null hypotheses H 0,J 1 , H 0,J 2 , H 0,I , H 0,S . As a result of coefficients' aggregation, the asymptotic distributions are based on a small and fixed number of degrees of freedom no matter how p is chosen. Thus, the four backtests have unchanged critical values whatever the number of quantiles considered in the ES approximation. Note that we provide in Appendix E the proof for consistency of the tests under fixed untrue hypothesis.

Finite sample inference

Our four backtests are asymptotically chi-squared distributed and we can employ them if the asymptotic conditions are fulfill for realistic sample sizes. However, in the case of ES assessment, the focus is on the extreme tail distribution, that is for risk levels above the regulatory coverage level, i.e. τ = 0.975. This may induce scarce information and affect the inference when the sample size is not large enough. Furthermore, our asymptotic framework implicitly assumes that (1 -u p )T diverges to infinity, where u p denotes the highest level of the multi-quantile regression. [START_REF] Chernozhukov | Extremal quantile regression[END_REF] and [START_REF] Chernozhukov | Inference for extremal conditional quantile models, with an application to market and birthweight risks[END_REF] provide a refinement of this assumption based on the extreme value theory allowing

(1 -u p )T → k < ∞.
However, to date this literature has only considered single conditional extremal quantile models and it is not obvious how the results for the single quantile models extend to multi-quantile models. To overcome these typical deficiencies, we implement a bootstrap procedure to adjust the critical values of our test statistics in finite samples.

In the following, we propose a pairs bootstrap algorithm [START_REF] Freedman | Bootstrapping regression models[END_REF] in order to correct the finite sample size distortions of our backtests. This is a fully non-parametric procedure that can be applied to a very wide range of models, including quantile regression model [START_REF] Koenker | Handbook of quantile regression[END_REF]. This approach consists in resampling the data, keeping the dependent and independent variables together in pairs. The procedure is valid for any sample sizes T , and large levels u j , j = 1, 2, . . . , p, and ideally applies in our case when the constraints of the null hypothesis are linear in the parameters. The algorithm is as follows:

1. Estimate β and Σ on the original data {L t , V aR t (u j )} j=1,2,...,p , t = 1, 2, . . . , T , to obtain β and Σ, and compute the unconstrained test statistic W given by

W = T R W β -q W R W ΣR W -1 R W β -q W .
2. Build a bootstrap sample by drawing with replacement T pairs of observations from the original data {L t , V aR t (u j )} j=1,2,...,p , t = 1, 2, . . . , T .

3. Estimate the model on the bootstrap sample, to obtain β b and Σ b , and compute the bootstrapped test statistic W b under the null hypothesis as follows:

W b = T R W β b -R W β R W Σ b R W -1 R W β b -R W β .
4. Repeat B -1 times steps 2 and 3, to obtain the bootstrap statistics

W b , b = 1, 2, . . . , B.
Two remarks should be made about the algorithm. First, when we use the pairs bootstrap we cannot impose the null hypothesis on the bootstrap data generating process since imposing restrictions on β is unfeasible. To overcome this issue, we calculate the bootstrap statistics by considering the difference R W β -R W β rather than R W β -q. Since the estimate of β from the bootstrap samples should, on average, be equal to β, at least asymptotically, the null hypothesis tested by W b becomes "true" for the pairs bootstrap data generating process. Second, the critical value c α is obtained as the α-quantile of the bootstrap statistics

W b , b = 1, 2, . . . , B.
The decision rule is as follows. If the original test statistic W is greater than the α-level bootstrapped critical value c α , we conclude to the rejection of the null hypothesis. In addition, we compute the p-value of the test as

P = B -1 B b=1 1 W b > W .

Simulation study

In this section, we provide Monte Carlo simulations to illustrate the finite sample properties (empirical size and power) of our four backtests. The simulation study is performed on 5000 replications, and we consider sample sizes T = 250, 500, 1000, 2500. The results associated with the bootstrap critical values are based on B = 1000 bootstrap samples. Finally, the backtests are computed with τ = 0.975 that is the current banking regulation coverage level.

Beyond the traditional size and power analysis, a second important objective of this section is to characterize the influence of the number p of quantiles used to assess the ES forecasts. We aim at examining whether an ES backtest based on a large number of quantiles may provide better performances than a backtest based on a small number of quantiles, as it is recommended by the current BCBS guidelines. For that, we consider different choices for the number of risk levels, namely p = 1, 2, 4, 6, 8, 10, 12. The p risk levels u 1 , u 2 , . . . , u p are computed in accordance with Definition 1. Notice that p = 1 coincides with the VaR backtest at level τ of [START_REF] Gaglianone | Evaluating value-at-risk models via quantile regression[END_REF]. With p = 2 risk levels, our backtests are in accordance with the number of quantiles of the regulatory guidances. Finally, the case p = 4 corresponds to the framework considered by [START_REF] Emmer | What is the best risk measure in practice? a comparison of standard measures[END_REF].

The correct data generating process is given by the AR(1)-GARCH(1,1) specification with Student innovations. This class of model is some of the most widely used for capturing variance dynamics in daily asset returns (see e.g., [START_REF] Berkowitz | How accurate are value-at-risk models at commercial banks[END_REF][START_REF] Berkowitz | Evaluating value-at-risk models with desk-level data[END_REF][START_REF] Du | Backtesting expected shortfall: Accounting for tail risk[END_REF][START_REF] Löser | New backtests for unconditional coverage of expected shortfall[END_REF]. The ex-post portfolio loss L t , t = 1, 2, . . . T , is given by

L t = δ 0 + δ 1 L t-1 + t , t = σ t η t , η t ∼ t v , σ 2 t = γ 0 + γ 1 2 t-1 + γ 2 σ 2 t-1 , ( 11
)
where t v denotes the Student's t distribution with v degrees of freedom. Given the model in Equation ( 11), the true ES and VaR at coverage level τ are given by

ES t (τ ) = δ 0 + δ 1 L t-1 + σ t m (τ ) , ( 12
)
V aR t (τ ) = δ 0 + δ 1 L t-1 + σ t F -1 v (τ ) , ( 13
) with m (τ ) = E [η t |η t ≥ F -1 v (τ )]
, and where F -1 v (τ ) denotes the τ -quantile of the Student distribution with v degrees of freedom. As a robustness check of the above model, Appendix F provides simulation results for the simple case of a GARCH(1,1) model that excludes the conditional mean component with L t = t where t is as in Equation ( 11).

Both models are calibrated using the opposite of the daily log-returns of the S&P500 index over the period from January 2, 2013 to December 29, 2017, with δ 0 , δ 1 , γ 0 , γ 1 , γ 2 , v = (-0.085, -0.093, 0.034, 0.214, 0.748, 5) and ( γ 0 , γ 1 , γ 2 , v) = (0.034, 0.197, 0.763, 5), respectively for the AR(1)-GARCH(1,1) model and the GARCH(1,1) model. Finally to investigate the power, we consider several misspecified alternatives for L t :

A 1 : AR(1)-GARCH(1,1) model with underestimated conditional variances: L t is as Equation ( 11), with

σ 2 t = γ 0 + γ 1 2 t-1 + γ 2 σ 2 t-1 × (1 -κ)
, where κ = 0.25, 0.50, 0.75, respectively.

A 2 : GARCH in mean model:

L t = κ × σ 2 t + t , t = σ t η t , σ 2 t = γ 0 + γ 1 2 t-1 + γ 2 σ 2 t-1 , η t ∼ t v ,
where κ = +2.5, -2.5, respectively.

A 3 : AR(1)-GARCH(1,1) model with mixed normal innovations: L t satifies Equation ( 11), with η t ∼ 0.5X + + 0.5X -/ √ 10, where X + ∼ N (3, 1) and X -∼ N (-3, 1) .

A 4 : 12-month historical simulation model : VaR and ES are given by their empirical counterparts from the 250 previous trading days such that V aR t (τ ) = percentile {L t-i } 250 i=1 , 100τ , and

ES t (τ ) = 1 250 i=1 1 (L t-i ≥V aR t-i (τ )) 250 i=1 L t-i × 1 (L t-i ≥V aR t-i (τ )).
In A 1 , the conditional variance of the series σ t is alternately underestimated of 25%, 50%, and 75% to examine whether our tests are able to detect an underestimation of ES stemming from a misleading appreciation of volatility. In A 2 , the misspecification occurs To reply about the interrelationship between p and T , we note that the size of the four bootstrap-based backtests slightly deteriorates for p > 6 with T = 250 and T = 500 revealing that the tests are sensitive to the choice of p in small samples. In details, the slope backtest is the most affected by these distortions, while the J 1 backtest is well-sized most of the time.

On the contrary, for larger sample sizes, typically T = 1000 and T = 2500, these distortions are negligible. Our recommendation is hence to restrict the number p of quantiles when applying the tests in small samples, with typically p ≤ 6, and to consider higher values if the historical sample covers longer periods.

To provide robustness check of these results, Figure 7 in Appendix F reports empirical sizes when the data generating process is given by a GARCH(1,1) model. We observe the same findings as those provided with the AR(1)-GARCH(1,1) model. The asymptotic tests are largely oversized, while the bootstrap tests are close to the nominal size of 5% for all reported sample sizes and risk levels. Finally, there is also an asymptotic refinement of the empirical sizes as T increases for both asymptotic and bootstrap tests. natives A 1 , A 2 , A 3 , A 4 , and we verify that there is a general improvement of powers as the sample size T increases (from row 1 to row 4), suggesting that these tests are consistent for these alternatives. For instance, with T = 500, and p = 4, the test statistic J 1 identifies the misleading scenario A 3 in 49.3% of times, while it reaches 98.1% of times with T = 2500.

Second, the joint test statistics, J 1 and J 2 , generally deliver higher power performances compared to the intercept and slope test statistics I and S. This finding comes from the definition of the joint null hypotheses that focus on both intercept and slope coefficients and are thus more conservative than the null of the intercept and slope backtests. In details for the two joint tests, we find that J 1 performs generally better to detect A 1 and A 4 , while J 2 more often identifies A 2 and A 3 , which suggests complementarity between the two joint backtests.

Although the intercept and slope backtests exhibit lower power performances, they provide useful informations on the type of misspecification. In details, the slope backtest performs better in alternatives A 1 and A 3 , while the intercept backtest is superior for alternative A 4 .

Thus, A 1 and A 3 mainly affect the expected value of the slope parameters meaning that the errors are correlated and proportional to the true quantiles. In contrast, alternative A 4 induces distortions in the expected value of the intercept coefficients suggesting that the origin of errors is more global as they are not related to the true quantiles.

Third, we observe that the selection of number p of risk levels is difficult to link with the rejection frequencies in alternatives A 1 , A 2 , A 3 , since reported powers are slightly affected by p in general. This finding is explained by the nature of these alternatives for which the misspecification is relatively uniform along the tail, and does not require many levels. On the contrary, in alternative A 4 , an increase of p is beneficial for detecting the misleading one-year historical simulation method as power is unequivocally increasing with p, especially when T is large. This is due to the fact that, for this alternative, the error made along the tail is more irregular and requires the use of additional levels. In the same spirit as in the Portmanteau tests when selecting an optimal number of lags to flag serial correlation [START_REF] Ljung | On a measure of lack of fit in time series models[END_REF][START_REF] Box | Distribution of residual autocorrelations in autoregressiveintegrated moving average time series models[END_REF], it follows that considering successive values p = 1, 2, . . . , p max with p max = 12, as illustrated above, may offer useful information to improve the statistical decision.

Finally, we provide a robustness check of the powers with the GARCH(1,1) model (see Figures 8 and9 in Appendix F). The rejection frequencies are very close to those associated with the AR(1)-GARCH(1,1) model. Consequently, the decision whether to introduce or not a conditional mean in the risk model does not affect the power performances.

Empirical application

In this section, we apply our backtests to the daily returns of the S&P500 index. In addition, we provide a method for the adjustment of imperfect risk forecasts relying on our backtesting framework. In the sequel, we set τ = 0.975 to coincide with the regulatory ES coverage level. The probability levels u j , j = 1, 2, . . . , p, are calculated accordingly with Definition 1. In addition, we consider the risk levels suggested by the BCBS, i.e. u 1 = 0.975, and u 2 = 0.990, respectively. Finally, for comparison purposes and to provide useful backtesting recommendations, we consider several values p = 1, 2, 4, 6, 8, 10, 12.

Data

We consider the daily adjusted closing prices of the S&P500 index over the period January

FIGURE 3 ABOUT HERE

The S&P500 series is depicted in Figure 3 with the three aforementioned sub-periods.

The in-sample period (1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007) is weakly volatile, while the out-of-sample crisis periods (2007-2009 and 2007-2012) display more severe levels of volatility, with several extreme events. Figure 3 also provides some descriptive statistics. The variance and the average expost losses are higher in the out-of-sample periods than in the in-sample period, especially for the period 2007-2009. In addition, the series is right-skewed and has a kurtosis excess.

To predict the ES risk measure, we fit an AR(1)-GARCH(1,1) model with Student innovations, as defined in (11), using the S&P500 daily losses of the in-sample period. The ES and VaR forecasts are defined as in Equations ( 12) and ( 13), respectively. The set of unknown parameters is estimated by maximum likelihood. We obtain the following coefficient estimates δ 0 , δ 1 , γ 0 , γ 1 , γ 2 , v = {-0.057, -0.032, 0.007, 0.060, 0.936, 9}. As a robustness check, we also fit a GARCH(1,1) model on the same period as defined in the simulation study and for which we obtain the following estimates { γ 0 , γ 1 , γ 2 , v} = {0.007, 0.059, 0.937, 9}.

Empirical results

We start by evaluating the relevancy of the ES approximation of Definition 1, consisting in averaging several quantiles in the tail of the risk model. To do so, we compare the approximation considering p = 1, 2, 4, 6, 8, 10, 12 quantiles, with what we refer to as "exact ES". The latter corresponds to an ES which is computed via an exact method of calculation.

The technique relies on simulations and is described in Appendix G.

FIGURE 4 ABOUT HERE

Figure 4 reports the in-sample ES estimates obtained with the approximation and the exact calculation method. Two remarks should be made here. First, the ES risk forecasts issued from the approximation and the exact method strongly correlate regardless of the value p. The approximation performs very well to capture the ex-post losses information. Second, we observe that the approximation is substantially improved when p slightly increases and coincides almost completely with the exact ES using six (or more) quantiles.

Because the approximation is obtained by combining VaRs, our finding is in accordance with several papers. [START_REF] Gouriéroux | Converting tail-var to var: An econometric study[END_REF] study the relationship between VaR and ES and show that they are related through their risk levels by some link function. [START_REF] Daníelsson | Why risk is so hard to measure[END_REF] argue that the two measures of risk are related by a small constant and are conceptually equally informative. This similarity also comes from the structure of the model used to compute the risk measure. For instance, VaR and ES issued by an AR(1)-GARCH(1,1) model have common conditional mean and variance across risk levels implying that these risk measures are closely related (see Equations ( 12) and ( 13)). Finally, Figure 10 in Appendix H displays the same results using a GARCH(1,1) model. Removing the conditional mean component does not affect the approximation accuracy as the two computation methods match almost perfectly for p ≥ 6. For its ease of implementation and accuracy, the approximation is appealing to compute and evaluate the performance of ES risk forecasts. are now experienced at a 1% significance level and even for p > 6, as opposed to panel A.

This highlights the underlying link between p and T as panel B uses T = 1384 observations enabling the use of additional quantiles. Table 3 of Appendix H displays the p-values of the backtests when applying a GARCH(1,1) model. The results are similar. Note however, for p = 1, that the p-values are generally higher with the GARCH(1,1) model than for the AR(1)-GARCH(1,1) model. For instance, the p-value of J 1 in panel B is 0.056 with the AR(1)-GARCH(1,1) model, while it reaches 0.199 with the GARCH(1,1) model. For that model, additional quantiles are needed to increase the rejection capabilities of the tests.

In sum, we should be cautious in using a single quantile to assess the tail distribution of the risk model. Such procedures may lead market practitioners to select a model that generates mistaken ex-post forecasts. Furthermore, the results issued from the regulatory guidelines are contrasted. Two risk levels are not always enough to provide a sound conclusion about the correctness of the ES forecasts. We recommend the use of additional risk levels beyond the regulatory coverage level τ = 0.975 to improve the reliability of the decision.

Table 2 displays the coefficient estimates of the multi-quantile regression of Equation ( 4) for p = 6 risk levels, to help understand the reasons that explain the rejections of the ES forecasts. Panels A and B provide the results for periods 2007-2009 and 2007-2012, respectively. It must be recalled that, if the risk model is correctly specified, the intercept coefficient β 0 and the slope coefficient β 1 take values zero and one, respectively. We observe in both panels that the coefficients β 0 are overestimated for all the risk levels u 1 , u 2 , . . . , u 6 , while the coefficient β 1 is overestimated for the first level u 1 , and it becomes underestimated for all the remaining risk levels u 2 , u 3 , . . . , u 6 . The average errors of β 0 and β 1 are respectively equal to 0.84 and -0.13 in panel A, and 0.66 and -0.10 in panel B, indicating that the magnitude of errors is more important in panel A than in panel B, and that the intercept coefficients are more affected than the slope coefficients. Finally, we observe that the distortion of the regression coefficients with respect to their expected values is more pronounced for the highest risk levels suggesting that the errors are more severe far in the tail.

Furthermore, we provide in Table 2 one by one inference on the regression parameters with the pairs bootstrap algorithm. The results are depicted with the symbol " * " and are discussed at a 5% significance level. We observe that the intercept parameters are statistically not equal to zero for the intermediary levels u 3 and u 4 in panel A, and the additional u 5 risk level is also significantly different from zero in panel B. For the slope coefficients, the u 4 and u 6 order quantiles are statistically different from one in panel A, and only the level u 4 is misspecified in panel B. In addition, we report joint inference, i.e. looking at both the intercept and slope coefficients. The results are provided in the row labeled as "joint"

(bottom of the panels). Similarly to the previous findings, we find that the intermediary, and highest order quantiles u 3 , u 4 and u 6 are misleading in panel A, whereas in panel B, all the quantiles are misspecified (except for the highest, presumably because the coefficients have large standard errors), meaning that the entire tail distribution is incorrectly estimated.

We conclude the study by suggesting an avenue of research to backtesting tail risk measures under the extreme value (EV) theory. As pointed out in Section 3.2, our baseline regression implicitly assumes that (1 -u p )T → ∞ when T → ∞. [START_REF] Chernozhukov | Extremal quantile regression[END_REF] shows that EV laws apply under the condition that (1

-u p )T → k < ∞ when T → ∞.
We consider the extremal conditional quantile model of [START_REF] Chernozhukov | Inference for extremal conditional quantile models, with an application to market and birthweight risks[END_REF] which is valid under that extremal condition. The procedure allows testing individual restrictions while assuming the above EV theory. In the same spirit, [START_REF] Bee | Realized extreme quantile: A joint model for conditional quantiles and measures of volatility with evt refinements[END_REF] have proposed a new conditional quantile model with EV theory refinements that overcomes the issue of unstable inference in the tails where data are sparse. The results are depicted in Table 2 with the symbol "•". Overall, we find similar results between EV theory and pairs bootstrap. Rejection of the null is mostly experienced at the same levels in panel A and panel B. However, we observe that the procedure of [START_REF] Chernozhukov | Inference for extremal conditional quantile models, with an application to market and birthweight risks[END_REF] is generally more powerful than pairs bootstrap at the highest risk levels. For instance, the expected value of the intercept parameters β 0 (u j ) in panel A is rejected at level 1% for j = 3, 4, 5, 6 with the EV procedure, while the pairs bootstrap rejects the null at larger levels (5% or 10%). This robustness check greatly illustrates the superiority of EV theory when applied to multi-quantile regression models and tail risks. We get overall the same results with the GARCH(1,1) model (see Table 4 in Appendix H).

Adjusted ES forecasts

In this section, we devise an adjustment for the ES risk forecasts deduced from our backtests.

Our routine accounts for both misspecification and estimation uncertainty, without having to change the misspecified risk model. The procedure serves at identifying whether the model overestimates or underestimates the true unknown ES, by comparing the initial forecast with its adjusted counterpart, which may be useful for risk managers and regulatory agencies.

The correction of imperfect risk forecasts is not new in the financial literature. (2019) recently apply the same strategy to adjust imperfect ES forecasts. The typical method consists of modifying the coverage level τ of the risk measure to meet the null hypothesis of valid risk forecasts. Here, the originality stems from the fact that we use a regression-based method to correct the ex-ante forecasts, while available techniques are based on the concept of violation [START_REF] Christoffersen | Evaluating interval forecasts[END_REF][START_REF] Kupiec | Techniques for verifying the accuracy of risk measurement models[END_REF] . This allows adjusting the risk forecasts by application of a regression model, without having to rescale the coverage level τ .

For ease of notation, we assume the parameters of the multi-quantile regression to be known. Formally, the adjusted VaR forecast at level u j , and time t, is defined as the ex-ante prediction of the multi-quantile regression model, namely Q Lt (u j ; Ω t-1 ). In view of Equation ( 5), the initial imperfect VaR forecast is subsequently weighted by the regression parameters β 0 (u j ) and β 1 (u j ), which provides an adjustment corresponding to the global bias caused by misspecification and estimation uncertainty. The adjusted ES forecast at coverage level τ and time t is derived from the ES approximation as follows:

ES * t (τ ) = 1 p p j=1 Q Lt (u j ; Ω t-1 ) .
The adjusted ES forecasts are robust to model risk, as they meet the desirable properties on the regression coefficients. Indeed, if we compute the backtesting procedure with the sequence {Q Lt (u j ; Ω t-1 )} p j=1 instead of the initial misleading {V aR t (u j )} p j=1 , the parameters would exactly coincide with the expected values under the null hypothesis, i.e. β 0 (u j ) = 0, and β 1 (u j ) = 1, for the risk levels u 1 , u 2 , . . . , u p . [START_REF] De Nicolò | Forecasting tail risks[END_REF] show that AR-type models often underestimate tail risks for horizons up to one year ahead, while there is probably a room for improvement using factoraugmented quantile models. Note also that the skewness of the daily losses is positive in our sample (see Figure 3), and it would be useful to account for asymmetry considering an asymmetric power distribution as proposed by [START_REF] Komunjer | Asymmetric power distribution: Theory and applications to risk measurement[END_REF]. Using the AR(1)-GARCH(1,1), we observe that the underestimation is more pronounced for the smallest predictions, the error being more severe when the risk forecasts are originally small. Finally, the ES forecasts are slightly overestimated when the variance of the series is larger, suggesting that the risk model may overestimate the true volatility in turbulent financial times. This is due to the volatility persistence in the GARCH component. Our findings are robust to (1) the use of a simple GARCH(1,1) model, (2) the use of the two BCBS regulatory levels, and (3) the extended period 2007-2012 (see Figures 11, 12, 13, and14 in Appendix H).

Conclusion

The financial crisis of 2007-2008 and its aftermath has led to a reassessment of riskmanagement practices and financial market regulation through the Basel III accords (BCBS, 2010). Among the number of fundamental reforms for the market risk, the BCBS has adopted ES in place of VaR as the new standard for risk management. One of the major obstacle to its implementation was the deficit of simple tools for the evaluation of its forecasts. This article introduces four easy-to-use regression-based backtests of ES. Our econometric approach consists in regressing the ex-post losses on the VaRs forecasts in a multi-quantile regression model, and then, testing the resulting parameter estimates using Wald-type inference.

Several simulation studies are provided. We find that the use of asymptotic critical values may lead to important size distortions if the sample size is not large enough. We propose a pairs bootstrap algorithm to correct these small-sample biases [START_REF] Freedman | Bootstrapping regression models[END_REF] and show that our regression-based tests are reasonably sized within this bootstrap framework. We consider several misleading alternatives in line with the existing literature on risk assessment [START_REF] Gaglianone | Evaluating value-at-risk models via quantile regression[END_REF][START_REF] Du | Backtesting expected shortfall: Accounting for tail risk[END_REF][START_REF] Bayer | Regression based expected shortfall backtesting[END_REF]Kratz et al., 2018, etc.). Our methodology detects misspecifications in all considered simulation experiments. In particular, they identify the most frequent inaccuracies in risk modeling, namely mean, variance, tail, and dynamic misspecifications.

We apply our tests to the S&P500 index over the period 2007-2012. During this period of financial turmoil, our backtests clearly reject the validity of the ES forecasts based on a AR(1)-GARCH(1,1) and a GARCH(1,1) model. We also highlight the importance of choosing a sufficient number of quantiles to assess ES. The use of one or two quantiles is inadvisable as they are not always enough to identify improper risk forecasts. On the contrary, four or more quantiles (until an optimal number) deliver much more sound decisions, suggesting an update of the regulatory guidelines to the evaluation of more than two quantiles.

FIGURES

u p = 1 -T , (15)
where the nonstochastic positive sequence T = (1 -τ )/p satisfies T → 0 when T → ∞.

Equation ( 15) is a common representation to extremal quantile regression [START_REF] Chernozhukov | Extremal quantile regression[END_REF][START_REF] Chernozhukov | Inference for extremal conditional quantile models, with an application to market and birthweight risks[END_REF]. Given the definition of T and since τ is a constant parameter, it follows that p is increasing with T . To illustrate this point, assume p takes the form of a power function,

p = T γ , ( 16
)
with γ > 0. Next, we consider a QML estimator which implicitly assumes that T (1-u p ) → ∞ as T goes to infinity. [START_REF] Chernozhukov | Extremal quantile regression[END_REF] and [START_REF] Chernozhukov | Inference for extremal conditional quantile models, with an application to market and birthweight risks[END_REF] relate this condition to an intermediate order quantile regression. Our goal is to identify a suitable rate of convergence of p and T which ensures the above condition. We have

T T → ∞. (17)
Then, combining Equations ( 16) and ( 17), we get

(1 -τ )T 1-γ → ∞. (18)
Equation ( 18) is only satisfied when γ < 1. Looking at Equation ( 16), this condition implies that T needs to diverge faster than p to guarantee asymptotic theory.

F -Robustness checks of the simulation study

G -Exact calculation method of ES

This section describes the methodology for the exact computation of ES forecasts at coverage level τ . Several techniques are available in practice. As the distribution of the innovations is parametric, we rely on Monte Carlo simulations. For ease of notation, we assume parameters to be known while in practice we use estimated parameters. The algorithm is as follows:

1. Randomly draw S pseudo standardized innovations {η s t } S s=1 from the Student distribution, with degrees of freedom v. We set the number S = 100000 in the empirical application.

2. Compute the ES at time t of the standardized innovation η t as the Monte Carlo

  FIGURES 1-2 ABOUT HERE

Figure 2

 2 Figure2reports the empirical powers (size-corrected) associated with our seven alter-

  For a sake of clarity, we only report the p-values obtained with the bootstrap critical values and the results are discussed at 5% significance level. Panel A provides the results over the sample 2007-2009. The test statistic J 1 leads to reject the validity of the ES predictions regardless of the number p of quantiles (except for p = 8 where the rejection occurs at a 10% significance level). Interestingly, we observe that the larger p, the smaller the p-value until p = 6, indicating that the rejections are more severe when the number of risk levels increases until an optimal number p. It supports the existence of an upper limit for p relative to the sample size as T is relatively small (T = 504), and p should not be chosen too large. The test statistic J 2 displays higher p-values in general. The backtest based on a single VaR no longer rejects the validity of the ES predictions, and the p-value based on the regulatory levels of the BCBS is close to 5%, making the decision rule more unclear for those number of risk levels. Finally, given the p-values of the test statistic I for p = 2, 4, 6, 12, we tend to reject the expected value on the intercept coefficients, and there is a global bias in the quantile estimates issued by the ES model. On the contrary, the statistic S leads to the conclusion that the slope parameters are as expected under the null hypothesis, and thus, the magnitude of errors is not related to the true quantiles. Panel B contains the p-values for the period 2007-2012. Overall, we obtain similar results, but the rejections are more severe in this enlarged sample. Rejections of J 1

FIGUREFigure 5

 5 FIGURE 5 ABOUT HERE

Figure 2 :

 2 Figure 2: Empirical power of the tests at 5% significance level (AR(1)-GARCH(1,1) model, bootstrap critical values)

Figure 3 :

 3 Figure 3: S&P500 daily losses (%), and descriptive statistics

Figure 4 :Figure 5 :

 45 Figure 4: In-sample ES estimates issued from the approximation and the exact calculation method (AR(1)-GARCH(1,1) model)

Figure 6 :

 6 Figure 6: Empirical power of the tests at 5% significance level (AR(1)-GARCH(1,1) model, asymptotic critical values)

Figure 7 :

 7 Figure 7: Empirical size of the tests at 5% significance level (GARCH(1,1) model)

Figure 8 :

 8 Figure 8: Empirical power of the tests at 5% significance level (GARCH(1,1) model, bootstrap critical values)

Figure 9 :

 9 Figure 9: Empirical power of the tests at 5% significance level (GARCH(1,1) model, asymptotic critical values)

u 1

 1 = 0.975, u 2 = 0.990. Reported p-values are obtained using bootstrap critical values. Panel A gives the results for the period 2007-2009 and Panel B provides results for the period 2007-2012.

  respectively, and are obtained with the pairs bootstrap algorithm. • , •• , and ••• , indicate statistical significance at the same levels and are obtained with the procedure of Chernozhukov and Fernández-Val (2011). Panel A gives estimation results for the period 2007-2009 and Panel B provides estimation results for the period 2007-2012.

Figure 12 :

 12 Figure 12: ES forecasts and adjusted ES forecasts over the period 2007-2009 (GARCH(1,1) model)

Table 1 :

 1 p-values of the backtesting tests (AR(1)-GARCH(1,1) model)

	p	J 1 (b)	J 2 (b)	I (b)	S (b)
			Panel A. 2007-2009
	1	0.035	0.051	0.125	0.949
	2	0.014	0.041	0.038	0.200
	4	0.009	0.040	0.023	0.103
	6	0.009	0.038	0.021	0.123
	8	0.099	0.049	0.154	0.564
	10	0.029	0.061	0.053	0.432
	12	0.023	0.052	0.038	0.223
	2 (regulatory levels) 0.024	0.047	0.053	0.351
			Panel B. 2007-2012
	1	0.056	0.040	0.176	0.554
	2	0.004	0.013	0.014	0.215
	4	0.002	0.004	0.003	0.096
	6	0.004	0.005	0.009	0.196
	8	0.008	0.008	0.041	0.538
	10	0.007	0.010	0.021	0.410
	12	0.004	0.006	0.008	0.245
	2 (regulatory levels) 0.006	0.012	0.032	0.448

Note: p-values of the four backtests computed with p = 1, 2, 4, 6, 8, 10, 12 risk levels successively, and the two regulatory levels u 1 = 0.975, u 2 = 0.990. Reported p-values are obtained using bootstrap critical values. Panel A gives the results for the period 2007-2009 and Panel B provides results for the period 2007-2012.

Table 1

 1

reports the p-values of the backtests.

Table 2 :

 2 QML coefficient estimates (p = 6, AR(1)-GARCH(1,1) model)

		u 1	u 2	u 3		u 4	u 5	u 6
				Panel A. 2007-2009
	β 0	0.661	0.696	0.808 •••	0.846 •••	0.965 •••	1.076 •••
		(0.295)	(0.296)	(0.227)		(0.240)	(0.429)	(0.265)
	β 1	1.005	0.953	0.911		0.847 •••	0.804	0.689 •••
		(0.093)	(0.088)	(0.056)		(0.053)	(0.142)	(0.042)
	joint
				Panel B. 2007-2012
	β 0	0.376	0.510	0.692 •••	0.808 •••	0.777 •••	0.784
		(0.200)	(0.182)	(0.195)		(0.186)	(0.284)	(0.611)
	β 1	1.031	0.974	0.902		0.851	0.826	0.787
		(0.073)	(0.067)	(0.065)		(0.050)	(0.107)	(0.232)
	joint
	Note: Standard errors are reported in parentheses. , , and	indicate statistical significance at the 10%, 5% and 1% level,
	respectively, and are obtained with the pairs bootstrap algorithm.

• , •• , and

  ••• , indicate statistical significance at the same levels and are obtained with the procedure of Chernozhukov and Fernández-Val (2011). Panel A gives estimation results for the period 2007-2009 and Panel B provides estimation results for the period 2007-2012.

  [START_REF] Gouriéroux | Estimation-adjusted var[END_REF] adjust VaR forecasts affected by estimation uncertainty. Boucher et al. (2014) adjust imperfect VaR forecasts based on backtesting frameworks and Lazar and Zhang

Table 3 :

 3 p-values of the backtesting tests (GARCH(1,1) model) Note: p-values of the four backtests computed with p = 1, 2, 4, 6, 8, 10, 12 risk levels successively, and the two regulatory levels

	p	J 1 (b)	J 2 (b)	I (b)	S (b)
			Panel A. 2007-2009
	1	0.060	0.082	0.141	0.853
	2	0.027	0.054	0.045	0.233
	4	0.014	0.048	0.027	0.106
	6	0.016	0.046	0.026	0.135
	8	0.153	0.053	0.229	0.673
	10	0.036	0.067	0.052	0.448
	12	0.026	0.058	0.042	0.223
	2 (regulatory levels) 0.036	0.064	0.085	0.406
			Panel B. 2007-2012
	1	0.199	0.076	0.401	0.839
	2	0.011	0.029	0.050	0.458
	4	0.005	0.012	0.015	0.196
	6	0.008	0.011	0.036	0.317
	8	0.029	0.015	0.102	0.563
	10	0.010	0.020	0.021	0.360
	12	0.012	0.020	0.043	0.415
	2 (regulatory levels) 0.005	0.015	0.014	0.230

Table 4 :

 4 QML coefficient estimates (p = 6, GARCH(1,1) model) Note: Standard errors are reported in parentheses. , , and indicate statistical significance at the 10%, 5% and 1% level,

		u 1	u 2	u 3	u 4	u 5	u 6
				Panel A. 2007-2009
	β 0	0.600	0.683	0.769	0.811 •••	0.972 •••	1.065 •••
		(0.307)	(0.298)	(0.264)	(0.257)	(0.446)	(0.266)
	β 1	1.011	0.955	0.917	0.853	0.804	0.692 •••
		(0.093)	(0.089)	(0.059)	(0.055)	(0.143)	(0.043)
	joint
				Panel B. 2007-2012
	β 0	0.338	0.388 •••	0.601 •••	0.743 •••	0.753 •••	0.603
		(0.303)	(0.198)	(0.197)	(0.189)	(0.293)	(0.799)
	β 1	1.025	0.987	0.911	0.860	0.829	0.832
		(0.122)	(0.069)	(0.066)	(0.056)	(0.109)	(0.308)
	joint

* This work has received the 2019 best paper award of the German Finance Association (DGF).

Appendix A -Application of a finite Riemann sum to ES

In the sequel, we show how to derive the approximation of ES suggested in Definition 1.

Consider the following improper Riemann integral, b a f (t)dt, (14) where f (.) is given by the increasing function 1 1-τ V aR t (.) and where a and b are respectively τ and 1 so that the above expression is identical to the ES defined in Equation ( 1). Definition of a Riemann sum yields a useful approximation of Equation ( 14),

where p is the number of subdivisions or quantiles taken in the definite integral to approximate ES. Replacing a, b, and f (.), by their corresponding quantities leads,

This verifies the ES formula of Definition 1 where risk levels u j are given by τ + (j -1) 1-τ p .

B -Assumptions

This section introduces the assumptions needed to establish the asymptotic normality and the consistency of the QML estimator and to ensure the validity of Proposition 1.

Assumption A0: {L t , V aR t (u j )} p j=1 is a stationary and ergodic process and measurable with respect to Ω t-1 .

Assumption A1: L t has conditional (on Ω t-1 ) distribution function F t , with continuous and positive density f t at conditional quantile Q Lt (u; Ω t-1 ) = F -1 t (u|Ω t-1 ) for all u ∈ (0, 1). sup|Q Lt (u j ; Ω t-1 ) |, then we have E [D 0,t ] < ∞.

Assumption A3:

The matrices A = p j=1 E [f j,t (0) ∇Q Lt (u j ; Ω t-1 ) ∇ Q Lt (u j ; Ω t-1 )] and

Assumption A0 is standard in modeling financial times series. It is broadly accepted that asset prices are integrated at order one, so that financial returns are stationary. This data assumption is hence satisfied. Assumption A1 allows for nonidentical distributions as we enable L t to be conditional on an unknown information set Ω t-1 . Assumption A2 imposes moment conditions, and in particular ensures finite expectation for L t . This is satisfied by the vast majority of financial time series models, including stationary and invertible ARMA processes, GARCH processes, etc. Assumption A3 is standard in Wald-type inference to ensure that the variance-covariance matrix Σ is positive definite. Furthermore, Assumptions A0 through A2 are standard in QML estimation (e.g., [START_REF] White | Estimation, inference and specification analysis[END_REF], and are also widely used in the literature on quantile regression models (e.g., [START_REF] Koenker | Goodness of fit and related inference processes for quantile regression[END_REF][START_REF] Koenker | Inference on the quantile regression process[END_REF]. They are of great importance to establish consistency and to apply the central limit theorem of White (2001, theorem 5.24) based on the method proposed by Huber (1967).

C -Consistent variance-covariance matrix estimation

In what follows, we provide a consistent estimator of the variance-covariance matrix Σ.

The methodology is derived from [START_REF] White | Var for var: Measuring tail dependence using multivariate regression quantiles[END_REF]. A consistent estimate of Σ can be obtained from the decomposition of the Huber (1967) sandwich form and is thus given by Σ = A -1 V A -1 . In the sequel, we provide consistent estimators A and V . To obtain V , we apply a simple plug-in estimator as follows:

where η t is given by its estimated counterpart

The estimation of A is trickier because it requires to consistently estimate f j,t (0), namely the density of the error term j,t given Ω t-1 evaluated at zero. Because the function is unknown, we follow [START_REF] Powell | Least absolute deviations estimation for the censored regression model[END_REF] and use a non parametric estimator. The method was implemented by [START_REF] Engle | Caviar: Conditional autoregressive value at risk by regression quantiles[END_REF] for instance to estimate the variance-covariance matrix of a set of coefficients issued from the so-called CaViaR model. Then, A is given by

D -On the rate of convergence and interplay of p and T

Let us consider the highest risk level u p issued from the sequence u j , j = 1, 2, . . . , p. Given Definition 1, we have

The null hypothesis of the proposed Wald-type test can be written as H 0,W : R W β -q W = 0, against the two-sided alternative H 1,W : R W β -q W = 0. The continuous mapping theorem

Rearranging the term T in the test statistic and using the continuous mapping theorem leads

and therefore we get lim

is the fractile of order 1 -α of the chi-square distribution with d W degrees of freedom, and where α is the significance level of the test.

average of the simulated innovations such that m (τ ) =

, where F -1 v (τ ) is the τ -quantile of the innovation distribution and is obtained as F -1 v (τ ) = percentile {η s t } S s=1 , 100τ .

3. Compute the ES at time t as ES t (τ ) = δ 0 + δ 1 L t-1 + σ t m (τ ) .

H -Robustness checks of the empirical application

ES forecasts

Adjusted ES forecasts