L Six

S Ieng

J Saunier
email: julien.saunier@ifsttar.fr

Z Guessoum
email: zahia.guessoum@lip6.fr

Understanding simulated driver behavior using hysteresis loops

Keywords: Modeling, Control and Optimization of Transportation Systems, Hysteresis, Simulation

Modeling techniques evolved a lot during the last half century. Acceleration models used for microscopic traffic simulation are heterogeneous. For instance, differential equation based models are different from agent based ones. Comparison of those models is a difficult task because studies are often conducted based on a suitable and particular theory, usually not applicable to all models. So we propose to use the hysteresis loops to understand and compare models regardless of the used modeling technique. We show how this study can reflect important properties of the models in order to compare them.

INTRODUCTION

There is a wide range of models available to simulate and study the properties of a traffic flow. Usual classification [START_REF] Hoogendoorn | State-of-the-art of vehicular traffic flow modelling[END_REF]) recognizes two main families of models: macroscopic and microscopic. The macroscopic models allow to study the evolution of macroscopic order variables of a traffic such as the density of vehicles, the average speed and the output rate. The microscopic models study individual entities composing the traffic: the vehicles. Those models deal with microscopic variables such as individual position and speed. The interaction among individual entities results in a consistent and hopefully realistic traffic flow. Two aspects are considered in microscopic models: acceleration and lane changing. In this paper we only consider acceleration models, used to determine the acceleration of a particular vehicle according to the vehicles in front. Those models are usually referred to as "car-following" models.

During the last 50 years, modeling and simulation techniques have significantly evolved, resulting in a heterogeneous variety of acceleration models [START_REF] Brackstone | Car-following: a historical review[END_REF]). The first models in the car-following area were based on dynamical equations (see for instance [START_REF] Gazis | Nonlinear follow-the-leader models of traffic flow[END_REF]). Those equations have been then enriched to integrate physical values used by drivers in their decision making process (such as [START_REF] Gipps | A behavioural car-following model for computer simulation[END_REF]). Artificial intelligence (AI) and multi-agent systems (MAS) allowed to develop even more realistic models based on a wide variety of theories and techniques such as fuzzy logic theory [START_REF] Kikuchi | Car-following model based on fuzzy inference system[END_REF]), constraint satisfaction [START_REF] Doniec | Multi-agent coordination and anticipation model to design a road traffic simulation tool[END_REF]) or game theory [START_REF] Champion | A game-based, multi-agent coordination mechanism-application to road traffic and driving simulations[END_REF]).

Even if those models are used to study the same phenomenon (the vehicle's acceleration) and its consequences at a macroscopic level (resulting flow, emergence of stop & go waves. . . ), the variety of tools used makes the study of those models difficult. Finding the most suitable model for a given situation is then a challenging task. Benchmarking has been conducted in [START_REF] Brockfeld | Toward benchmarking of microscopic traffic flow models[END_REF] to determine the most realistic resulting flow. However this approach does not give any information on how individual entities actually behave and how they can influence the macroscopic properties of the traffic.

In this paper we propose to use the hysteresis phenomenon to study and understand car-following models while being independent of the modeling techniques. The hysteresis loop can be used to understand the dynamics of a model, reflect its anticipation capabilities and study the propagation of a perturbation along a platoon of vehicles.

Section 2 presents the models we selected for this study, then Section 3 presents the hysteresis phenomenon and its application to study driver behavior. Section 4 describes the experimental protocol and Section 5 presents the results we obtained. Finally, Section 6 discusses the results and gives some further work perspectives.

MODELS

We selected three car-following models for this study. Each model determines at a given point of time the vehicle's acceleration. The first model is described with a simple dynamic equation, the second introduces physical values involved in the decision making process of drivers and the third is an agent based model. Figure 1 summarizes naming conventions used across this paper.

Optimal Velocity (OV)

The OV model, proposed by [START_REF] Bando | Structure stability of congestion in traffic dynamics[END_REF], is based on the following principle : for each situation there the n th vehicle follows the (n -1) th vehicle. The front bumper's position of the n th vehicle is called x n , its velocity ẋn , and its acceleration ẍn . We call l n the length of the n th vehicle. The gap available in front of the vehicle is

∆x n = x n-1 -l n-1 -x n .
is an optimal speed to adopt. Any deviation from this optimal speed causes the vehicle to adopt an acceleration proportional to the difference between the optimum and the actual speed.

This behavior is expressed in the following equation:

ẍn = a n (V (∆x n ) -ẋn ) (1)
Where V (∆x) is the optimal velocity for a given situation represented by the available gap, a n is a parameter of the model influencing how strong the reaction is.

The optimal velocity function used in this paper is inspired from the one proposed in [START_REF] Bando | Structure stability of congestion in traffic dynamics[END_REF]. The function maintains the main properties of the one previously used but introduces a parameter v n 0 to specify the desired speed of the vehicle :

V (∆x) = tanh 2∆x v n 0 -2 + tanh(2) * v n 0 2 (2)

Intelligent Driver Model (IDM)

The IDM model, proposed by [START_REF] Treiber | Microscopic simulation of congested traffic[END_REF], is designed as the aggregation of two behaviors : the first one leads the vehicle to accelerate until it reaches its desired speed, the second one forces it to decelerate to maintain a safe gap ahead.

The model is described as follow :

ẍn = a n 1 - ẋn v n 0 δ - s * n ( ẋn , ∆v n ) ∆x 2 (3)
Where a n is the maximum acceleration of the n th vehicle, v n 0 its desired speed, δ the acceleration exponent and ∆v n is the approaching rate of the leading vehicle given by ∆v n = ẋn -ẋn-1 . s * n is a function that calculates the desired gap to maintain for a particular (speed / approaching rate) couple :

s * n ( ẋn , ∆v n ) = s n 0 + s n 1 ẋn v n 0 + T n ẋn + ẋn ∆v n 2 √ a n b n (4)
Where T n is the safe time headway, b n represents the conformable deceleration, and s n 0 and s n 1 are jam distance parameters.

Equation 3 reflects the two sub-behaviors of IDM :

a n 1 -( ẋn /v n 0 )
δ is the acceleration part on a free road and -a n (s * n ( ẋn , ∆v n ) /∆x) 2 represents the part influenced by other vehicles applying a constraint and forcing the vehicle to decelerate to maintain the situation safe.

Archisim

The Archisim model, presented in [START_REF] Espié | Microscopic traffic simulation and driver behaviour modelling: the archisim project[END_REF], is based on in-depth studies conducted by driving psychologists and developed with MAS techniques. The behavior of an Archisim vehicle is influenced not only by the leading vehicle, but by potentially all the surrounding vehicles. During each time step, the vehicle receives a list of its acquaintances (vehicles in perception range). From this list, it determines which one of the vehicles down the road causes the biggest constraint (causes the biggest deceleration). The constraint is then classified as short term or long term depending on the context and a corresponding strategy is adopted as proposed in [START_REF] Espié | Archisim: a behavioral multi-actors traffic simulation model for the study of a traffic system including its aspects[END_REF]. A short term constraint corresponds to a situation where the constraint is known to reduce with no particular action of the driver.

The short term strategy consists in maintaining short minimal bumper to bumper time given by the S bbt parameter. The bumper to bumper time is defined as the time necessary for the front bumper of the follower vehicle to reach the position of the rear bumper of the leader vehicle. Thus, we can deduce a minimal gap to maintain given by the expression ∆x min = ẋn * S bbt .

If the vehicle chooses the long term strategy, the minimal safe bumper to bumper time is given by E bbt = α n S bbt + (1-α n )L bbt with E bbt the effective bumper to bumper time targeted, L bbt the longest bumper to bumper time and α n a vehicle specific parameter.

THE HYSTERESIS LOOP

The hysteresis, from Greek husteros meaning "coming after", is the lag of a reaction compared to its cause. It was first introduced by the physicist James Alfred Ewing as follows :

When there are two quantities M and N , such that cyclic variations of N cause cyclic variation of M, then if the changes of M lag behind those of N , we may say that there is hysteresis in the relation of M and N .

Figure 2 shows the graphical representation of the hysteresis phenomenon. The loop shows the values taken by the reaction as a function of the effect. The reaction shows different values for the same effect when this effect changes.

In this paper, we suppose that for each stable value of the effect there is one stable value of the reaction and it is unique. When the system is in a situation where both effect and reaction are stable, the system is told to be in an equilibrium state. In traffic theory, [START_REF] Bando | Structure stability of congestion in traffic dynamics[END_REF] observed that when stop and go waves appear, it is possible to plot on a graph the speed of individual vehicles as a function of the gap available in front of them and obtain an hysteresis loop. Other authors did notice this property in [START_REF] Jiang | Full velocity difference model for a car-following theory[END_REF]; [START_REF] Helbing | Generalized force model of traffic dynamics[END_REF]; [START_REF] Bando | Analysis of optimal velocity model with explicit delay[END_REF]; [START_REF] Davis | Modifications of the optimal velocity traffic model to include delay due to driver reaction time[END_REF] and discussed the influence of model parameters on the loop's size and shape.

We present here an interpretation of the hysteresis loop where the cause is a constraint applied to a vehicle -a vehicle present in front of it, given by the gap available -and the reaction is the behavior of the vehicle given by its speed. In this interpretation, the equilibrium states correspond to desired states the models will eventually converge to. The equilibrium states are also considered as optimal because they represent the state where a vehicle can, given a particular gap, maximize its speed while staying safe. Any point below the equilibrium state is considered as under constrained and any point above is over constrained.

SIMULATION PROTOCOL

The three selected models are tested in the same environment. We selected the Archisim platform [START_REF] Espié | Microscopic traffic simulation and driver behaviour modelling: the archisim project[END_REF] to conduct this experiment because it allows us to:

• create a repeatable scenario,

• record simulation traces (individual vehicle trajectories), • select or define the model used to guide the vehicle in the simulation.

This platform also provides the reference implementation for the Archisim model so we only had to add the two remaining models through two new kind of vehicle agents.

The scenario developed in this study involves a platoon of vehicles we want to study led by a controlled vehicle on a straight one lane free road. The speed and acceleration of the leader vehicle are controlled by the modeler. This allows to apply a known and controlled constraint on the platoon to study its reactions. For each studied model, the scenario is instantiated with the same leader vehicle, assuring a comparable constraint.

During a simulation run, the leader's speed varies to study the reaction of the platoon following it. The leader's speed changes from 4 to 22 m.s -1 and vice-versa. Each speed variation, of an intensity of 3 m.s -2 , occurs every 2 minutes. The follower can thus adapt to the new constraint and eventually join a new equilibrium state before the leader changes its speed again.

Inside the platoon, all the studied vehicles are given the same set of parameters. The desired speed is the same for all the three models: 25 m.s -1 . The model-specific parameters are set to the values originally proposed by the authors. The parameters for IDM are from [START_REF] Treiber | Microscopic simulation of congested traffic[END_REF], those for OV are from [START_REF] Bando | Structure stability of congestion in traffic dynamics[END_REF] and those for Archisim are the default values provided by the authors. Table 1 summarizes the parameters used.

RESULTS AND HYSTERESIS LOOP'S INTERPRETATION

In this section we present the equilibrium states and simulation results of each model.

Equilibriums

To analytically identify equilibriums, we have to find points of the (gap / speed) plan where the acceleration adopted by the vehicle is null. For each model we have to solve the equation A M (∆x, ẋ) = 0 where A M represents the acceleration given by a particular model M.

The calculation for OV is straightforward. Considering the optimal velocity function proposed in Equation 2, the equilibrium line is given by :

V (∆x n ) = v 0 tahn 2∆xn v n 0 -2 + tanh(2) 2 (5)
For the IDM model, we have to consider that the approaching rate is null : ∆ ẋn = 0. This hypothesis was also used by [START_REF] Treiber | Microscopic simulation of congested traffic[END_REF] to determine stable flow. The equilibrium line is then given by : ∆x n ( ẋn ) =

s n 0 + s n 1 ẋn v n 0 + T n ẋn 1 -ẋn v n 0 δ (6)
For Archisim, we consider that an equilibrium state is reached when the bumper to bumper time is the one given by the long term strategy. Indeed the short term strategy cannot be, given its definition, stable over time. Also, once the vehicle reaches its desired speed, it will not continue to accelerate. As a consequence, the equilibrium line for the Archisim model can be expressed as : ẋn

(∆x n ) = M in ∆x n αS bbt + (1 -α) L bbt , v n 0 (7)
The same results for equilibrium states have been approached experimentally by setting stable flows at different speeds and recording the behavior adopted by vehicles.

Figure 3 summarizes the three theoretical equilibriums. It can provides important elements of comparison because gaps and speeds have immediate implication on macroscopic variables such as flow density and speed. If all vehicles are in stable state, the flow is stable with the same speed as individual vehicles. Also, with the approximation of vehicles length, the flow density is the reverse of individual gap head. Some authors discussed the impact of different microscopic parameters on macroscopic variable [START_REF] Treiber | Microscopic simulation of congested traffic[END_REF]).

Considering the parameters we used in this study, for every speed adopted by the leader, the Archisim model heads for smaller gaps than the other models, implying higher density. For small speeds (less than about 12m.s -1 ), the OV model goes for bigger gaps than IDM. On the opposite, for high speeds, the IDM model adopts very large gaps compared to the two others models, implying lower concentration. Finally, we can deduce from those results that a IDM platoon can not travel at desired speed because the equilibrium line never reaches it.

Those results do not provide any information about the dynamic behavior of the models and how they actually adapt to a constraint variation.

Delay and anticipation

We focus on the results obtained after using the scenario described in Section 4. The objective is to be able to study how a model operates the transition from one equilibrium state to another when the constraint applied to the vehicle changes. In this section, we only consider the hysteresis loop of the vehicle immediately following the leader. For each observed loop, we first propose an interpretation and then link it to the knowledge we have about the models to validate that the hysteresis loop reflects the behavior. ) ∆x (m) Fig. 4. Reaction of an OV vehicle to a constraint varying between 4m.s -1 and 22m.s -1 equilibrium line, and the other half above. This shape is similar to usual hysteresis loops seen in general literature. The part below corresponds to the growth of the available gap during the acceleration phase. During this phase, for all the given gaps, the associated velocity is below the optimal one. The constraint applied to the vehicle is therefore lower than the optimal one, letting the vehicle free to accelerate in order to join the equilibrium line. The deceleration phase (above the equilibrium) has analogous properties and shows that the vehicle only decelerates when it is in an over constrained situation. This information perfectly matches the description made of the model. It is purely reactive and only engages reaction when a variation from optimal state is detected. the reduction of the gap before it happens and reduces its speed early. This clearly shows the ability of the model to anticipate the constraint's variation.

Those results reflect the initial description of the model given in Section 2. We mentioned the fact that IDM is an aggregation of two sub-behaviors. One manages the acceleration, the other manages the deceleration and takes into account the approaching rate of the leading vehicle. Knowledge of the approaching rate allows the model to anticipate the future reduction of the available gap and explains the anticipation capabilities exhibited by this model.

Finally, we study the hysteresis loop for the Archisim model (see Figure 6). This loop has distinctive properties compared to the two previous ones. It has an "8" shape : at some point the trace of the acceleration phase is above the deceleration phase and at some point it is below. First, we consider the deceleration phase given by the straight line on the loop. It exactly matches equilibrium line meaning that there is no hysteresis when decelerating. The model manages to maintain an optimal reaction during this phase. The acceleration phase is slightly more complex because the vehicle is, at the beginning of the acceleration phase, above the equilibrium line and, at the end, under the equilibrium line. This loop shows that, at least for the beginning of the acceleration, the follower chooses to be in an over constrained situation.

Once again, those observations match the original description of the model. During the acceleration phase, the leader vehicle is considered as a short term constraint because it goes faster and is accelerating. Those two properties imply that the available gap will continue to grow, making the constraint weak. The position of the acceleration phase above the equilibrium line is considered as an expression of the anticipation capabilities of the model, exactly like the fact that the deceleration phase of IDM being below the equilibrium reflects anticipation.

Propagation of a perturbation

The previous part was about the interpretation of the hysteresis phenomenon on a single vehicle. We now consider a platoon of 30 vehicles and study the propagation of the hysteresis phenomenon in this context. For this study we introduce a notion of pseudo distance between the hysteresis loop and the equilibrium line. This distance is defined as the maximum difference between the equilibrium speed and the velocity used by the vehicle during the simulation.

Figure 7 represents the evolution of the hysteresis loop along a platoon of OV vehicles. In this figure, the first vehicle's loop is in the center and as we get closer to the tail of the platoon, the loop area gets bigger. At the end of the platoon, the loops seem to reach a final state comparable to an attractor. Those readings are confirmed by the second part of Figure 7 that shows an augmentation of the distance between equilibrium and the hysteresis loops on the 10 first vehicles. After this first phase, the distance gets stable around 6.8m/s. Figure 7 also shows that the first vehicle has a speed variation comparable to the one applied by the modeler from 4 to 22m/s. At the end of the platoon, the speed variation has changed to become a variation from 0.9 to 22.4m/s which is a clear augmentation. Finally, even if the perturbation is tending to something different from what we modeled, we can still observe singular points at speed 4 and 22m/s. This corresponds to the stable flow maintained by the platoon for a long time compared to the phase transitions.

Those results are coherent with the ones already exposed in literature. First, the emergence of a stable hysteresis loop after some time, independent of initial perturbation, was originally observed by [START_REF] Bando | Structure stability of congestion in traffic dynamics[END_REF]. This is a limit behavior observable during stop and go waves. This property was used in [START_REF] Nakanishi | Solvable optimal velocity models and asymptotic trajectory[END_REF] simplify the calculation. The author supposed that the behavior of a given vehicle at time t is the same as the one of its predecessor shortly before. Analogous properties are used in other models such as [START_REF] Newell | A simplified theory of kinematic waves in highway traffic, part i: General theory[END_REF] and present interesting results. Even if the hysteresis loop validates this approach (only after few vehicles to let the perturbation get stable), it cannot give any information about temporality: it does not give information on the time elapsed between the reactions of the two vehicles.

Figure 8 represents the propagation of a perturbation across a platoon of IDM vehicles. To reduce overlaps on the graphs, top figure only represents the hysteresis loop for vehicle 10, given the fact that all loops starting from vehicle number 2 share similar properties. It shows a "8" shape, but has to be distinguished from the one obtained in Figure 6 because the rotations are in opposite directions. It shows the deceleration phase above the acceleration phase for small gaps, and the opposite for large gaps. We can observe that the deceleration phase is really similar to the one observed in Figure 4. However, the acceleration phase seems stronger here because the leader is also an IDM vehicle, with similar acceleration capabilities, leading to a higher grade. This shape has to be linked to the results obtained by [START_REF] Zhang | A mathematical theory of traffic hysteresis[END_REF] at microscopic level, who studied similar "8" shapes on the (density / speed) plan.

Middle part of Figure 8 shows that along the platoon, the pseudo distance between the hysteresis loop of the vehicles and the equilibrium tends to reduce unlike what is observed in Figure 7. Also, we notice that considering our scenario, each vehicle in the platoon reaches a lower maximal speed compared to its predecessor as shown in the bottom part of Figure 8. This reflects a really slow convergence to the final equilibrium considering that in a 2 minutes phase, the last vehicle does not have time to reach "full" speed. This propagation pattern has opposite properties compared with OV showed in Figure 7.

Figure 9 shows the evolution of the propagation on an Archisim platoon. All the loops starting from the position number 2 present a triangular shape on the bottom left part. Even if the overall shape is not the same with the first vehicle, they share this property of exhibiting anticipation for early stages of acceleration. All the triangles share really similar shapes with a scaling factor between them. The reduction of the triangle size is interpreted as follow : it corresponds to the acceleration phase where each vehicle can perceive that the platoon is accelerating. During this phase each vehicle considers that the maximal constraint perceived is temporary and as a consequence accepts a small gap head. When the acceleration of the leader stops, each follower vehicle begins to consider it as a long term constraint and as a consequence tries to reach the equilibrium line. This transition corresponds to the horizontal lines. During the acceleration of the platoon, each vehicle is moving slightly faster than the one behind it, so when the leader stops accelerating each vehicle has its own speed, smaller than its predecessors. This explains why from one vehicle to its follower, the size of the triangle gets smaller. Bottom part of Figure 9 confirms it and shows that after 20 vehicles, the hysteresis loops match the equilibrium line, reflecting the disappearance of hysteresis in those vehicle's behavior. Basically when the leader stops its acceleration, the 20 th hasn't started its own.

It is really important at this point to notice the timeindependent nature of the hysteresis loop. During this scenario we know that all the vehicles react to the same stimuli happening at a particular point in time. However a simple reading of the evolution of hysteresis loops is not enough to reach this conclusion and we need to add additional knowledge to get a complete understanding of the situation, such as adding a temporal dimension.

CONCLUSION AND PERSPECTIVES

In this paper we used the hysteresis loop to understand and compare behaviors produced by different models. Studying the hysteresis phenomenon is an natural approach to evaluate a reaction as a function of the stimulus causing it. It is thus useful to study the reaction and the adaptation of an agent, in a dynamic environment. We showed that study of hysteresis loops allows to identify key properties of a behavior such as delay in reaction, anticipation capabilities and limit behavior in a platoon through comparison between equilibrium states and actual behavior. The results presented are consistent with the ones obtained by original authors of the models, but are obtained independently of the modeling technique used.

For each studied model, we determined the equilibrium state using analytical and simulation studies. Both approaches lead to the same information so the experimental approach is adequate to study those models as black boxes. However, there is no guaranty that this is sufficient for any model. As an example, stochastic models can accept surfaces to reflect states that have a high probability of being stable. Simple protocol will then not be enough to experimentally determine what are the desired states, which is a mandatory information to conduct the analysis. Determination of those equilibrium states on any "blackbox" model is still an open issue that needs to be addressed to extend the validity of our approach.

Fig. 1 .

 1 Fig. 1. Naming conventions used in model description :the n th vehicle follows the (n -1) th vehicle. The front bumper's position of the n th vehicle is called x n , its velocity ẋn , and its acceleration ẍn . We call l n the length of the n th vehicle. The gap available in front of the vehicle is ∆x n = x n-1 -l n-1 -x n .

  Fig. 2. Hysteresis phenomenon showing the lag of the reaction compared to the effect.

Fig. 3 .

 3 Fig. 3. Equilibrium states for the three studied models.

Figure 4 -

 4 Figure 4 -representing the reaction of an OV vehicle -has one half of the hysteresis loop staying below the

Fig. 5 .Figure 5 Fig. 6 .

 556 Fig. 5. Reaction of a IDM vehicle to a constraint varying between 4m.s -1 and 22m.s -1Figure5represents the adaptation of the IDM model on the same constraint's variation. The overall shape of the hysteresis phenomenon drawn is still a loop. Moreover, when the follower's speed matches the leader's one (at 4 and 22 m.s -1 ), the vehicle is in an equilibrium state. However, there is a clear distinction to make with the OV model : the loop is nearly always below the equilibrium line. This position of the loop shows that during almost all the speed transitions, the IDM model maintains the vehicle in an under constrained situation. This reveals two properties of the model. First during the acceleration phase, the IDM vehicle's reaction lags compared to the constraint variation, exactly like a OV vehicle does. Second, during the deceleration phase, the vehicle manages to anticipate

Fig. 7 .

 7 Fig. 7. Study of a platoon of 30 vehicles using the Optimal Velocity model. Top figure shows hysteresis for the vehicle number 1, 2, 6, 10, 14, 18, 22 and 26 in the platoon and bottom figure shows hysteresis to equilibrium pseudo distance evolution along the platoon

  Fig. 8. Evolution of the hysteresis loop across a platoon of 30 IDM vehicles

Fig. 9 .

 9 Fig. 9. Study of a platoon of 30 vehicles using the Archisim model.

Table 1 .

 1 Parameters used during this study

	Model	Parameter	Value
	OV	a	1.0
	IDM	T	1.2s
	IDM	a	0.8m.s -2
	IDM	b	1.25m.s -2
	IDM	s 0	1m
	IDM	s 1	10m
	IDM	δ	3
	Archisim	S bbt	0.7s
	Archisim	L bbt	2.5s
	Archisim	α	0.5
	*	v 0	25m.s -1


Pierre Alquier
email: pierre.alquier@ucd.ie

Xiaoyin Li
email: xiaoyin.li@u-cergy.fr

Olivier Wintenberger
email: wintenberger@ceremade.dauphine.fr

Olivier Catoni

Paul Doukhan

Pascal Massart

Gilles Stoltz

Prediction of time series by statistical learning: general losses and fast rates

Keywords: Primary 62M20; secondary 68T05, 62M10 Statistical learning theory, Time series prediction, PAC-Bayesian bounds, weak-dependence, mixing, oracle inequalities, fast rates, GDP Forecasting

. In particular, we generalize the results of [DT08] on sparse regression estimation to the case of autoregression. * We deeply thank Matthieu Cornec (INSEE) for useful discussions, and for providing the data with detailed explanations.

Introduction

Time series forecasting is a fundamental subject in the mathematical statistics literature. The parametric approach contains a wide range of models associated with efficient estimation and prediction methods, see e.g. [START_REF] Hamilton | Time series analysis[END_REF]. Classical parametric models include linear processes such as ARMA models [START_REF] Brockwell | Time series: Theory and methods (2nd edition)[END_REF]. More recently, non-linear processes such as stochastic volatility and ARCH models received a lot of attention in financial applications -see, e.g., the seminal paper by Nobel prize winner [START_REF] Engle | Autoregressive conditional heteroscedasticity with estimates of variance of united kingdom inflation[END_REF], and [START_REF] Francq | Garch models: Structure, statistical inference and financial applications[END_REF] for a more recent introduction. However, parametric assumptions rarely hold on data. Assuming that the data satisfy a model can biased the prediction and underevaluate the risks, see among others the the polemical but highly informative discussion in [START_REF] Taleb | Black swans and the domains of statistics[END_REF].

In the last few years, several universal approaches emerged from various fields such as non-parametric statistics, machine learning, computer science and game theory. These approaches share some common features: the aim is to build a procedure that predicts the time series as well as the best predictor in a given set of initial predictors Θ, without any parametric assumption on the distribution of the observed time series. However, the set of predictors can be inspired by different parametric or non-parametric statistical models. We can distinguish two classes in these approaches, with different quantification of the objective, and different terminologies:

• in the "prediction of individual sequences" approach, predictors are usually called "experts". The objective is online prediction: at each date t, a prediction of the future realization x t+1 is based on the previous observations x 1 , ..., x t , the objective being to minimize the cumulative prediction loss. See for example [START_REF] Cesa-Bianchi | Prediction, learning, and games[END_REF][START_REF] Stoltz | Agrégation séquentielle de prédicteurs : méthodologie générale et applications à la prévision de la qualité de l'air et à celle de la consommation électrique[END_REF] for an introduction. • in the statistical learning approach, the given predictors are sometimes referred as "models" or "concepts". The batch setting is more classical in this approach. A prediction procedure is built on a complete sample X 1 , ..., X n . The performance of the procedure is compared on the expected loss, called the risk, with the best predictor, called the "oracle". The environment is not deterministic and some hypotheses like mixing or weak dependence are required: see [Mei00, MM98, AW12].

In both settings, one is usually able to predict a time series as well as the best model or expert, up to an error term that decreases with the number of observations n. This type of results is referred in statistical theory as oracle inequalities. In other words, one builds on the basis of the observations a predictor θ such that

R( θ) ≤ inf θ∈Θ R(θ) + ∆(n, Θ) (1.1)
where R(θ) is a measure of the prediction risk of the predictor θ ∈ Θ. In general, the remainder term is of the order ∆(n, Θ) ∼ c(Θ)/n in both approaches, where c(Θ) measures the complexity of Θ. See, e.g., [START_REF] Cesa-Bianchi | Prediction, learning, and games[END_REF] for the "individual sequences" approach; for the "statistical learning approach" the rate c(Θ)/n is reached in [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF] with the absolute loss function and under a weak dependence assumption. Different procedures are used to reach these rates. Let us mention the empirical risk minimization [START_REF] Vapnik | The nature of statistical learning theory[END_REF] and aggregation procedures with exponential weights, usually referred as EWA [START_REF] Dalalyan | Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity[END_REF][START_REF] Gerchinovitz | Sparsity regret bounds for individual sequences in online linear regression[END_REF] or Gibbs estimator [START_REF] Catoni | Statistical learning theory and stochastic optimization[END_REF][START_REF] Catoni | PAC-Bayesian supervised classification (the thermodynamics of statistical learning[END_REF] in the batch approach, linked to the weighted majority algorithm of the online approach [START_REF] Littlestone | The weighted majority algorithm[END_REF], see also [START_REF] Vovk | Aggregating strategies[END_REF]. Note that results from the "individual sequences" approach can sometimes be extended to the batch setting, see e.g. [START_REF] Gerchinovitz | Sparsity regret bounds for individual sequences in online linear regression[END_REF] for the iid case, and [START_REF] Agarwal | The generalization ability of online algorithms for dependent data[END_REF][START_REF] Duchi | Ergodic mirror descent[END_REF] for mixing time series.

In this paper, we extend the results of [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF] to the case of a general loss function. Another improvement with respect to [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF] is to study both the ERM and the Gibbs estimator under various hypotheses. We achieve here inequalities of the form of (1.1) that hold with large probability (1ε for any arbitratily small confidence level ε > 0) with ∆(n, Θ) ∼ c(Θ)/n. We assume to do so that the observations are taken from a bounded stationary process (X t ) (see [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF] however for some possible extensions to unbounded observations). We also assume weak dependence conditions on the process process (X t ). Then we prove that the fast rate ∆(n, Θ) ∼ c(Θ)/n can be reached for some loss functions including the quadratic loss. Note that [START_REF] Meir | Nonparametric time series prediction through adaptive model selection[END_REF][START_REF] Modha | Memory-universal prediction of stationary random processes[END_REF] deal with the quadratic loss, their rate can be better than c(Θ)/n but cannot reach c(Θ)/n.

Our main results are based on PAC-Bayesian oracle inequalities. The PAC-Bayesian point of view emerged in statistical learning in supervised classification using the 0/1-loss, see the seminal papers [START_REF] Shawe-Taylor | A PAC analysis of a bayes estimator[END_REF][START_REF] Mcallester | PAC-Bayesian model averaging[END_REF]. These results were then extended to general loss functions and more accurate bounds were given, see for example [Cat04, Cat07, Alq08, Aud10, AL11, SLCB+12, DS12]. In PAC-Bayesian inequalities the complexity term c(Θ) is defined thanks to a prior distribution on the set Θ.

The paper is organized as follows: Section 2 provides notations used in the whole paper. We give a definition of the Gibbs estimator and of the ERM in Section 3. The main hypotheses necessary to prove theoretical results on these estimators are provided in Section 4. We give examples of inequalities of the form (1.1) for classical set of predictors Θ in Section 5. When possible, we also prove some results on the ERM in these settings. These results only require a general weak-dependence type assumption on the time series to forecast. We then study fast rates under a stronger φ-mixing assumptions of [START_REF] Ibragimov | Some limit theorems for stationary processes[END_REF] in Section 6. Note that the φ-mixing setting coincides with the one of [START_REF] Agarwal | The generalization ability of online algorithms for dependent data[END_REF][START_REF] Duchi | Ergodic mirror descent[END_REF] when (X t ) is stationary. In particular, we are able to generalize the results of [START_REF] Dalalyan | Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity[END_REF][START_REF] Gerchinovitz | Sparsity regret bounds for individual sequences in online linear regression[END_REF][START_REF] Alquier | PAC-Bayesian bounds for sparse regression estimation with exponential weights[END_REF] on sparse regression estimation to the case of autoregression. In Section 7 we provide an application to French GDP forecasting. A short simulation study is provided in Section 8. Finally, the proofs of all the theorems are given in Appendices A and B.

Notations

Let X 1 , . . . , X n denote the observations at time t ∈ {1, . . . , n} of a time series X = (X t ) t∈Z defined on (Ω, A, P). We assume that this series is stationary and take values in R p equipped with the Euclidean norm • . We fix an integer k, that might depend on n, k = k(n), and assume that family of predictors is available:

f θ : (R p ) k → R p , θ ∈ Θ .
For any parameter θ and any time t, f θ (X t-1 , . . . , X t-k ) is the prediction of X t returned by the predictor θ when given (X t-1 , . . . , X t-k ). For the sake of shortness, we use the notation:

Xθ t = f θ (X t-1 , . . . , X t-k
). We assume that θ → f θ is a linear function. Let us fix a loss function ℓ that measures a distance between the forecast and the actual realization of the series. Assumptions on ℓ will be given in Section 4.

Definition 1. For any θ ∈ Θ we define the prediction risk as

R (θ) = E ℓ Xθ t , X t (R(θ)
does not depend on t thanks to the stationarity assumption).

Using the statistics terminology, note that we may want to include parametric set of predictors as well as non-parametric ones (i.e. respectively finite dimensional and infinite dimensional Θ). Let us mention classical parametric and non-parametric families of predictors:

Example 1. Define the set of linear autoregressive predictors as

f θ (X t-1 , . . . , X t-k ) = θ 0 + k j=1 θ j X t-j for θ = (θ 0 , θ 1 , . . . , θ k ) ∈ Θ ⊂ R k+1 .
In order to deal with non-parametric settings, we will also use a modelselection type notation: Θ = ∪ M j=1 Θ j . Example 2. Consider non-parametric auto-regressive predictors

f θ (X t-1 , . . . , X t-k ) = j i=1 θ i ϕ i (X t-1 , . . . , X t-k ) where θ = (θ 1 , . . . , θ j ) ∈ Θ j ⊂ R j and (ϕ i ) ∞
i=0 is a dictionnary of functions (R p ) k → R p (e.g. Fourier basis, wavelets, splines...).

ERM and Gibbs estimator

The estimators

As the objective is to minimize the risk R(•), we use the empirical risk r n (•) as an estimator of R(•).

Definition 2. For any θ ∈ Θ, r n (θ) = 1 n-k n i=k+1 ℓ Xθ i , X i .
Definition 3 (ERM estimator [START_REF] Vapnik | The nature of statistical learning theory[END_REF]). We define the Empirical Risk Minimizer estimator (ERM) by

θERM ∈ arg min θ∈Θ r n (θ).
Let T be a σ-algebra on Θ and M 1 + (Θ) denote the set of all probability measures on (Θ, T ). The Gibbs estimator depends on a fixed probability measure π ∈ M 1 + (Θ) called the prior that will be involved when measuring the complexity of Θ.

Definition 4 (Gibbs estimator or EWA). Define the Gibbs estimator with inverse temperature λ > 0 as θλ = Θ θ ρλ (dθ), where ρλ (dθ) = e -λrn(θ) π(dθ) e -λrn(θ ′ ) π(dθ ′ ) .

The choice of π and λ in practice is discussed in Section 5.

Overview of the results

Our results assert that the risk of the ERM or Gibbs estimator is close to inf θ R(θ) up to a remainder term ∆(n, Θ) called the rate of convergence. For the sake of simplicity, let θ ∈ Θ be such that

R(θ) = inf θ R(θ).
If θ does not exist, it is replaced by an approximative minimizer θ α satisfying R(θ α ) ≤ inf θ R(θ) + α where α is negligible w.r.t. ∆(n, Θ) (e.g. α < 1/n 2 ). We want to prove that the ERM satisfies, for any ε > 0,

P R θERM ≤ R(θ) + ∆(n, Θ, ε) ≥ 1 -ε (3.1)
where ∆(n, Θ, ε) → 0 as n → ∞. We also want to prove that and that the Gibbs estimator satisfies, for any ε > 0,

P R θλ ≤ R(θ) + ∆(n, λ, π, ε) ≥ 1 -ε (3.2)
where ∆(n, λ, π, ε) → 0 as n → ∞ for some λ = λ(n). To obtain such results called oracle inequalities, we require some assumptions discussed in the next section.

Main assumptions

We prove oracle inequalities under assumptions of two different types. On the one hand, assumptions LipLoss(K) and Lip(L) hold respectively on the loss function ℓ and the set of predictors Θ. In some extent, we choose the loss function and the predictors, so these assumptions can always be satisfied. Assumption Margin(K) also holds on ℓ.

On the other hand, assumptions Bound(B), WeakDep(C), PhiMix(C) hold on the dependence and boundedness of the time series. In practice, we cannot know whether these assumptions are satisfied on data. However, remark that these assumptions are not parametric and are satisfied for many classical models, see [START_REF] Doukhan | Mixing[END_REF][START_REF] Dedecker | Weak dependence, examples and applications[END_REF].

Assumption LipLoss(K), K > 0: the loss function ℓ is given by ℓ(x, x ′ ) = g(xx ′ ) for some convex K-Lipschitz function g such that g(0) = 0 and g ≥ 0.

Example 3. A classical example in statistics is given by ℓ(x, x ′ ) = xx ′ , see [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF]. It satisfies LipLoss(K) with K = 1. In [START_REF] Modha | Memory-universal prediction of stationary random processes[END_REF][START_REF] Meir | Nonparametric time series prediction through adaptive model selection[END_REF], the loss function used is the quadratic loss ℓ(x, x ′ ) = xx ′ 2 . It satisfies LipLoss(4B) for time series bounded by a constant B > 0.

Example 4. The class of quantile loss functions introduced in [KB78] is given by

ℓ τ (x, y) = τ (x -y) , if x -y > 0 -(1 -τ ) (x -y) , otherwise
where τ ∈ (0, 1) and x, y ∈ R. The risk minimizer of t → E(ℓ τ (Vt)) is the quantile of order τ of the random variable V . Choosing this loss function one can deal with rare events and build confidence intervals, see [START_REF] Koenker | Quantile regression[END_REF][START_REF] Belloni | L1-penalized quantile regression in high-dimensional sparse models[END_REF][START_REF] Biau | Sequential quantile prediction of time series[END_REF]. In this case, LipLoss(K) is satisfied with K = max(τ, 1τ ) ≤ 1.

Assumption Lip(L), L > 0: for any θ ∈ Θ there are coefficients a j (θ) for 1 ≤ j ≤ k such that, for any x 1 , ..., x k and y 1 , ..., y k ,

f θ (x 1 , . . . , x k ) -f θ (y 1 , . . . , y k ) ≤ k j=1 a j (θ) x j -y j , with k j=1 a j (θ) ≤ L.
Assumption Bound(B), B > 0: we assume that X 0 ≤ B almost surely.

Remark that under Assumptions LipLoss(K), Lip(L) and Bound(B), the empirical risk is a bounded random variable. Such a condition is required in the approach of individual sequences. We assume it here for simplicity but it is possible to extend the slow rates oracles inequalities to unbounded cases see [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF].

Assumption WeakDep(C) is about the θ ∞,n (1)-weak dependence coefficients of [START_REF] Rio | Ingalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes[END_REF][START_REF] Dedecker | Weak dependence, examples and applications[END_REF]. Definition 5. For any k > 0, define the θ ∞,k (1)-weak dependence coefficients of a bounded stationary sequence (X t ) by the relation

θ ∞,k (1) := sup f ∈Λ k 1 ,0<j1<•••<j k E [f (X j1 , . . . , X j ℓ )|X t , t ≤ 0] -E [f (X j1 , . . . , X j ℓ )] ∞ where Λ k 1 is the set of 1-Lipshitz functions of k variables Λ k 1 = f : (R p ) k → R, |f (u 1 , . . . , u k ) -f (u ′ 1 , . . . , u ′ k )| k j=1 u j -u ′ j ≤ 1 .
The sequence (θ ∞,k (1)) k>0 is non decreasing with k. The idea is that as soon as X k behaves"almost independently" from X 0 , X -1 , ... then θ ∞,k (1)θ ∞,k-1 (1) becomes negligible. Actually, it is known that for many classical models of stationary time series, the sequence is upper bounded, see [START_REF] Dedecker | Weak dependence, examples and applications[END_REF] for details. Assumption WeakDep(C), C > 0: θ ∞,k (1) ≤ C for any k > 0.

Example 5. Examples of processes satisfying WeakDep(C) are provided in [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF][START_REF] Dedecker | Weak dependence, examples and applications[END_REF]. It includes Bernoulli shifts X t = H(ξ t , ξ t-1 , . . . ) where the ξ t are iid, ξ 0 ≤ b and H satisfies a Lipschitz condition:

H(v 1 , v 2 , ...) -H(v ′ 1 , v ′ 2 , ...) ≤ ∞ j=0 a j v j -v ′ j with ∞ j=0 ja j < ∞.
Then (X t ) is bounded by B = H(0, 0, ...) + bC and satisfies WeakDep(C) with C = ∞ j=0 ja j . In particular, solutions of linear ARMA models with bounded innovations satisfy WeakDep(C).

In order to prove the fast rates oracle inequalities, a more restrictive dependence condition is assumed. It holds on the uniform mixing coefficients introduced by [START_REF] Ibragimov | Some limit theorems for stationary processes[END_REF]. Definition 6. The φ-mixing coefficients of the stationary sequence (X t ) with distribution P are defined as

φ r = sup (A,B)∈ σ(Xt,t≤0)×σ(Xt,t≥r) |P(B/A) -P(B)|. Assumption PhiMix(C ′ ), C ′ > 0: 1 + ∞ r=1 √ φ r ≤ C ′ .
This assumption appears to be more restrictive than WeakDep(C) for bounded time series: (This result is not stated in [START_REF] Rio | Ingalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes[END_REF] but it is a direct consequence of the last inequality in the proof of Corollaire 1, p. 907 in [START_REF] Rio | Ingalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes[END_REF]).

Proposition 1 ([Rio00]).
Finally, for fast rates oracle inequalities, an additional assumption on the loss function ℓ is required. In the iid case, such a condition is also required. It is called Margin assumption, e.g. in [START_REF] Mammen | Smooth discrimination analysis[END_REF][START_REF] Alquier | PAC-Bayesian bounds for randomized empirical risk minimizers[END_REF], or Bernstein hypothesis, [START_REF] Lecué | Interplay between concentration, complexity and geometry in learning theory with applications to high dimensional data analysis[END_REF].

Assumption Margin(K), K > 0:

E ℓ X q+1 , f θ (X q , ..., X 1 ) -ℓ X q+1 , f θ (X q , ..., X 1 ) 2 ≤ K R(θ) -R(θ) .
As assumptions Margin(K) and PhiMix(C) are used only to obtain fast rates, we give postpone examples to Section 6.

Slow rates oracle inequalities

In this section, we give oracle inequalities (3.1) and/or (3.2) with slow rates of convergence ∆(n, Θ) ∼ c(Θ)/n. The proof of these results are given in Section B. Note that the results concerning the Gibbs estimator are actually corollaries of a general result, Theorem 9, stated in Section A. We introduce the following notation for the sake of shortness. Definition 7. When Assumptions Bound(B), LipLoss(K), Lip(L) and WeakDep(C) are satisfied, we say that we are under the set of Assumption SlowRates(κ)

where κ = K(1 + L)(B + C)/ √ 2 .

Finite classes of predictors

Consider first the toy example where Θ is finite with |Θ| = M , M ≥ 1. In this case, the optimal rate in the iid case is known to be log(M )/n, see e.g. [START_REF] Vapnik | The nature of statistical learning theory[END_REF].

Theorem 1. Assume that |Θ| = M and that SlowRates(κ) is satisfied for κ > 0. Let π be the uniform probability distribution on Θ. Then the oracle inequality (3.2) is satisfied for any λ > 0, ε > 0 with

∆(n, λ, π, ε) = 2λκ 2 n (1 -k/n) 2 + 2 log (2M /ε) λ .
The choice of λ in practice in this toy example is already not trivial. The choice λ = log(M )n yields the oracle inequality:

R( θλ ) ≤ R(θ) + 2 log(M ) n κ 1 -k/n 2 + 2 log (2/ε) n log(M ) .
However, this choice is not optimal and one would like to choose λ as the minimizer of the upper bound

2λκ 2 n (1 -k/n) 2 +
2 log (M ) . λ

However κ = κ(K, L, B, C) and the constants B and C are, usually, unknown.

In this context we will prefer the ERM predictor that performs as well as the Gibbs estimator with optimal λ: Theorem 2. Assume that |Θ| = M and that SlowRates(κ) is satisfied for κ > 0. Then the oracle inequality (3.1) is satisfied for any ε > 0 with

∆(n, Θ, ε) = inf λ>0 2λκ 2 n (1 -k/n) 2 + 2 log (2M /ε) λ = 4κ 1 -k/n log (2M /ε) n .

Linear autoregressive predictors

We focus on the linear predictors given in Example 1.

Theorem 3. Consider the linear autoregressive model of AR(k) predictors

f θ (x t-1 , . . . , x t-k ) = θ 0 + k j=1 θ j x t-j with θ ∈ Θ = {θ ∈ R k+1 , θ ≤ L} such that Lip(L) is satisfied.
Assume that Assumptions Bound(B), LipLoss(K) and WeakDep(C) are satisfied. Let π be the uniform probability distribution on the extended parameter set {θ ∈ R k+1 , θ ≤ L + 1}. Then the oracle inequality (3.2) is satisfied for any λ > 0, ε > 0 with

∆(n, λ, π, ε) = 2λκ 2 n (1 -k/n) 2 + 2 (k + 1) log (KB∨K 2 B 2 )(L+1) √ eλ k+1 + log (2/ε) λ .
In theory, λ can be chosen of the order (k + 1)n to achieve the optimal rates (k + 1)/n up to a logarithmic factor. But the choice of the optimal λ in practice is still a problem. The ERM predictor still performs as well as the Gibbs predictor with optimal λ. Theorem 4. Under the assumptions of Theorem 3, the oracle inequality (3.1) is satisfied for any ε > 0 with

∆(n, Θ, ε) = inf λ≥2KB/(k+1)   2λκ 2 n (1 -k/n) 2 + (k + 1) log 2eKB(L+1)λ k+1 + 2 log (2/ε) λ   .
The additional constraint on λ does not depend on n. It is restrictive only when k + 1, the complexity of the autoregressive model, has the same order than n. For n sufficiently large and λ = ((1k/n)/κ) ((k + 1)n/2) satisfying the constraint λ ≥ 2KB/(k + 1) we obtain the oracle inequality

R( θERM ) ≤ R(θ) + 2(k + 1) n κ 1 -k/n log 2e 2 KB(R + 1) κ n k + 1 + 2 √ 2κ log (2/ε) (k + 1)n (1 -k/n) .
Theorems 3 and 4 are both direct consequences of the following results about general classes of predictors.

General parametric classes of predictors

We state a general result about finite-dimensional families of predictors. The complexity k + 1 of the autoregressive model is replaced by a more general measure of the dimension d(Θ, π). We also introduce some general measure D(Θ, π) of the diameter that will, for most compact models, be linked to the diameter of the model.

∀δ > 0, log 1 θ∈Θ 1{R(θ) -R(θ) < δ}π(dθ) ≤ d log D δ .
Then the oracle inequality (3.2) is satisfied for any λ > 0, ε > 0 with

∆(n, λ, π, ε) = 2λκ 2 n (1 -k/n) 2 + 2 d log (D √ eλ/d) + log (2/ε) λ .
A similar result holds for the ERM predictor under a more restrictive assumption on the structure of Θ, see Remark 1 below.

Theorem 6. Assume that 1. Θ = {θ ∈ R d : θ 1 ≤ D}, 2. Xθ1 1 -Xθ2 1 ≤ ψ. θ 1 -θ 2 1 a.s. for some ψ > 0 and all (θ 1 , θ 2 ) ∈ Θ 2 .
Assume also that Bound(B), LipLoss(K) and WeakDep(C) are satisfied and that Lip(L) holds on the extended model

Θ ′ = {θ ∈ R d : θ 1 ≤ D + 1}. Then the oracle inequality (3.1) is satisfied for any ε > 0 with ∆(n, Θ, ε) = inf λ≥2Kψ/d 2λκ 2 n (1 -k/n) 2 + d log (2eKψ(D + 1)λ/d) + 2 log (2/ε) λ .
This result yields to nearly optimal rates of convergence for the ERM predictors. Indeed, for n sufficiently large and λ = ((1k/n)/κ) (dn/2) ≥ 2Kψ/d we obtain the oracle inequality

R( θERM ) ≤ R(θ) + 2d n κ 1 -k/n log 2e 2 Kψ(D + 1) κ n d + 2 √ 2κ log (2/ε) √ dn (1 -k/n) .
Thus, the ERM procedure yields prediction that are close to the oracle with an optimal rate of convergence up to a logarithmic factor. Note that the context of Theorem 6 are less general than the one of Theorem 5:

Remark 1. Under the assumptions of Theorem 6 we have for any

θ ∈ Θ R(θ) -R(θ) = E g Xθ 1 -X 1 -g Xθ 1 -X 1 ≤ E K Xθ 1 -Xθ 1 ≤ Kψ θ -θ 1 .
Define π as the uniform distribution on Θ ′ = {θ ∈ R d : θ 1 ≤ D + 1}. We derive from simple computation the inequality

log 1 θ∈Θ 1{R(θ) -R(θ) < δ}π(dθ) ≤ log 1 θ∈Θ 1{ θ -θ 1 < δ Kψ }π(dθ)      = d log Kψ(D+1) δ when δ/Kψ ≤ 1 ≤ d log (Kψ(D + 1)) otherwise.
Thus, in any case, As a conclusion, for some predictors set with a non classical structure, the Gibbs estimator might be preferred to the ERM.

log 1 θ∈Θ 1{R(θ) -R(θ) < δ}π(dθ) ≤ d log (Kψ ∨ K 2 ψ 2 )(D +

Aggregation in the model-selection setting

Consider now several models of predictors Θ 1 , ..., Θ M and consider Θ = M i=1 Θ i (disjoint union). Our aim is to predict as well as the best predictors among all Θ j 's, but paying only the price for learning in the Θ j that contains the oracle. In order to get such a result, let us choose M priors π j on each models such that π j (Θ j ) = 1 for all j ∈ {1, ..., M }. Let π = M j=1 p j π j be a mixture of these priors with prior weights p j ≥ 0 satisfying M j=1 p j = 1. Denote

θ j ∈ arg min θ∈Θj R(θ)
the oracle of the model Θ j for any 1 ≤ j ≤ M . For any λ > 0, denote ρλ,j the Gibbs distribution on Θ j and θλ,j = Θj θ ρλ,j (dθ) the corresponding Gibbs estimator. A Gibbs predictor based on a model selection procedure satisfies an oracle inequality with slow rate of convergence: Theorem 7. Assume that:

1. Bound(B) is satisfied for some B > 0; 2. LipLoss(K) is satisfied for some K > 0; 3. WeakDep(C) is satisfied for some C > 0; 4. for any j ∈ {1, ..., M } we have (a) Lip(L j ) is satisfied by the model Θ j for some L j > 0, (b) there are constants d j = d(Θ j , π) and D j = c(Θ j , π j ) are such that

∀δ > 0, log 1 θ∈Θj 1{R(θ) -R(θ j ) < δ}π j (dθ) ≤ d j log D j δ Denote κ j = κ(K, L j , B, C) = K(1 + L j )(B + C)/ √
2 and define θ = θλ ĵ , ĵ where ĵ minimizes the function of

j Θj r n (θ)ρ λj ,j (dθ) + λ j κ j n(1 -k/n) 2 + K(ρ λj ,j , π j ) + log (2/(εp j )) λ j with λ j = arg min λ>0 2λκ 2 j n (1 -k/n) 2 + 2 d j log (D j eλ/d j ) + log (2/(εp j )) λ .
Then, with probability at least 1ε, the following oracle inequality holds

R( θ) ≤ inf 1≤j≤M R(θ j ) + 2 κ j 1 -k/n d j n log D j e 2 κ j n d j + log (2/(εp j )) nd j .
The proof is given in Appendix B. A similar result can be obtained if we replace the Gibbs predictor in each model by the ERM predictor in each model. The resulting procedure is known in the iid case under the name SRM (Structural Risk Minimization), see [START_REF] Vapnik | The nature of statistical learning theory[END_REF], or penalized risk minimization, [START_REF] Birgé | Gaussian model selection[END_REF]. However, as it was already the case for a fixed model, additional assumptions are required to deal with ERM predictors. In the model-selection context, the procedure to choose among all the ERM predictors also depends on the unknown κ j 's. Thus the model-selection procedure based on Gibbs predictors outperforms the one based on the ERM predictors.

Fast rates oracle inequalities

Discussion on the assumptions

In this section, we study conditions under which the rate 1/n can be achieved. These conditions are restrictive:

• now p = 1, i.e. the process (X t ) t∈Z is real-valued;

• the dependence condition WeakDep(C) is replaced by PhiMix(C); • we assume additionally Margin(K) for some K > 0.

Let us provide some examples of processes satisfying the uniform mixing assumption PhiMix(C). In the three following examples (ǫ t ) denotes an iid sequence (called the innovations).

Example 7 (AR(p) process). Consider the stationary solution (X t ) of an AR(p) model: ∀t ∈ Z, X t = p j=1 a j X t-j + ǫ t . Assume that (ǫ t ) is bounded with a distribution possessing an absolutely continuous component. If A(z) = p j=1 a j z j has no root inside the unit disk in C then (X t ) is a geometrically φ-mixing processe, see [START_REF] Athreya | Mixing properties of Harris chains and autoregressive processes[END_REF] and PhiMix(C) is satisfied for some C.

Example 8 (MA(p) process). Consider the stationary process (X t ) such that X t = p j=1 b j ǫ t-j for all t ∈ Z. By definition, the process (X t ) is stationary and φ-dependent -it is even p-dependent, in the sense that φ r = 0 for r > p. Thus PhiMix(C) is satisfied for some C > 0.

Example 9 (Non linear processes). For extensions of the AR(p) model of the form X t = F (X t-1 , . . . , X t-p ; ǫ t ), Φ-mixing coefficients can also be computed and satisfy PhiMix(C). See e.g. [START_REF] Meyn | Markov chains and stochastic stability[END_REF]. We now provide an example of predictive model satisfying all the assumptions required to obtain fast rates oracle inequalities, in particular Margin(K), when the loss function ℓ is quadratic, i.e. ℓ(x, x ′ ) = (xx ′ ) 2 : Example 10. Consider Example 2 where

f θ (X t-1 , . . . , X t-k ) = N i=1 θ i ϕ i (X t-1 , . . . , X t-k ),
for functions (ϕ i ) ∞ i=0 of (R p ) k to R p , and θ = (θ 1 , . . . , θ N ) ∈ R N . Assume the ϕ i upper bounded by 1 and Θ = {θ ∈ R N , θ 1 ≤ L} such that Lip(L). Moreover LipLoss(K) is satisfied with K = 2B. Assume that θ = arg min θ∈R N R(θ) ∈ Θ in order to have:

E X q+1 -f θ (X q , ..., X 1 ) 2 -X q+1 -f θ (X q , ..., X 1 ) 2 2 = E f θ (X q , ..., X 1 ) -f θ (X q , ..., X 1 ) 2 2X q+1 -f θ (X q , ..., X 1 ) -f θ (X q , ..., X 1 ) 2 ≤ E f θ (X q , ..., X 1 ) -f θ (X q , ..., X 1 ) 2 4B 2 (1 + R) 2 ≤ 4B 2 (1 + R) 2 R(θ) -R(θ) by Pythagorean theorem. Assumption Margin(K) is satisfied with K = 4B 2 (1 + D) 2 .
According to Theorem 8 below, the oracle inequality with fast rates holds as soon as Assumption PhiMix(C) is satisfied.

General result

We only give oracle inequalities for the Gibbs predictor in the model-selection setting. In the case of one single model, this result can be extended to the ERM predictor. For several models, the approach based on the ERM predictors requires a penalized risk minimization procedure as in the slow rates case. In the fast rates case, the Gibbs predictor itself directly have nice properties. Let Θ = M i=1 Θ i (disjoint union), choose π = M j=1 p j π j and denote θ j ∈ arg min θ∈Θj R(θ) as previously.

Theorem 8. Assume that:

1. Margin(K) and LipLoss(K) are satisfied for some K, K > 0; 2. Bound(B) is satisfied for some B > 0; 3. PhiMix(B) is satisfied for some C > 0; 4. Lip(L) is satisfied for some L > 0; 5. for any j ∈ {1, ..., M }, there exist d j = d(Θ j , π) and D j = D(Θ j , π j ) satisfying the relation

∀δ > 0, log 1 θ∈Θj 1{R(θ) -R(θ j ) < δ}π j (dθ) ≤ d j log D j δ .
Then for

λ = n -k 4kKLBC ∧ n -k 16kC the oracle inequality (3.2) for any ε > 0 with ∆(n, λ, π, ε) = 4 inf j    R(θ j ) -R(θ) + 4kC (4 ∨ KLB) d j log Dj e(n-k) 16kCdj + log 2 εpj n -k    .
Compare with the slow rates case, we don't have to optimize with respect to λ as the optimal order for λ is independent of j. In practice, the value of λ provided by Theorem 8 is too conservative. In the iid case, it is shown in [START_REF] Dalalyan | Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity[END_REF] that the value λ = n/(4σ 2 ), where σ 2 is the variance of the noise of the regression yields good results. In our simulations results, we will use λ = n/ v ar(X), where v ar(X) is the empirical variance of the observed time series.

Notice that for the index j 0 such that R(θ j0 ) = R(θ) we obtain:

R θλ ≤ R(θ) + 4kC (4 ∨ KLB) d j0 log (c j0 e(n -k)/(16kCd j0 )) + log (2/(εp j0 )) n -k .
So, the oracle inequality achieves the fast rate d j0 /n log (n/d j0 ) where j 0 is the model of the oracle. However, note that the choice j = j 0 does not necessarily reach the infimum in Theorem 8.

Let us compare the rates in Theorem 8 to the ones in [Mei00, MM98, AD11, DAJJ12]. In [START_REF] Meir | Nonparametric time series prediction through adaptive model selection[END_REF][START_REF] Modha | Memory-universal prediction of stationary random processes[END_REF], the optimal rate 1/n is never obtained. The paper [START_REF] Agarwal | The generalization ability of online algorithms for dependent data[END_REF] proves fast rates for online algorithms that are also computationally efficient, see also [START_REF] Duchi | Ergodic mirror descent[END_REF]. The fast rate 1/n is reached when the coefficients (φ r ) are geometrically decreasing. In other cases, the rate is slower. Note that we do not suffer such a restriction. The Gibbs estimator of Theorem 8 can also be computed efficiently thanks to MCMC procedures, see [START_REF] Alquier | PAC-Bayesian bounds for sparse regression estimation with exponential weights[END_REF][START_REF] Dalalyan | Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity[END_REF].

Corollary: sparse autoregression

Let the predictors be the linear autoregressive predictors Xθ p = p j=1 X p-j θ j .

For any J ⊂ {1, . . . , p}, define the model:

Θ J = {θ ∈ R p : θ 1 ≤ L and θ j = 0 ⇔ j ∈ J}.
Let us remark that we have the disjoint union Θ = J⊂{1,...,p} Θ J = {θ ∈ R p : θ 1 ≤ 1}. We choose π J as the uniform probability measure on Θ J and p j = 2 -|J|-1 p

|J|

Corollary 1. Assume that θ = arg min θ∈R N R(θ) ∈ Θ and PhiMix(C) is satisfied for some C > 0 as well as Bound(B). Then the oracle inequality (3.2) is satisfied for any ε > 0 with

∆(n, λ, π, ε) = 4 inf J R(θ J ) -R(θ) + cst. |J| log ((n -k)p/|J|) + log 2 ε n -k for some constant cst = cst(B, C, L).
This extends the results of [AL11, [START_REF] Dalalyan | Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity[END_REF][START_REF] Gerchinovitz | Sparsity regret bounds for individual sequences in online linear regression[END_REF] to the case of autoregression.

Proof. The proof follows the computations of Example 10 that we do not reproduce here: we check the conditions LipLoss(K) with K = 2B, Lip(L) and Margin(K) with K = 4B2 (1 + L) 2 . We can apply Theorem 8 with d J = |J| and D j = L.

Application to French GDP forecasting

Uncertainty in GDP forecasting

Every quarter t ≥ 1, the French national bureau of statistics, INSEE1 , publishes the growth rate of the French GDP (Gross Domestic Product). Since it involves a huge amount of data that take months to be collected and processed, the computation of the GDP growth rate log(GDP t /GDP t-1 ) takes a long time (two years). This means that at time t, the value log(GDP t /GDP t-1 ) is actually not known. However, a preliminary value of the growth rate is published 45 days only after the end of the current quarter t. This value is called a flash estimate and is the quantity that INSEE forecasters actually try to predict, at least in a first time. As we want to work under the same constraint as the INSEE, we will now focus on the prediction on the flash estimate and let ∆GDP t denote this quantity. To forecast at time t, we will use:

1. the past forecastings 2 ∆GDP j , 0 < j < t; 2. past climate indicators I j , 0 < j < t, based on business surveys.

Business surveys are questionnaires of about ten questions sent monthly to a representative panel of French companies (see [START_REF] Devilliers | Les enquêtes de conjoncture[END_REF] for more details). As a consequence, these surveys provide informations from the economic decision makers. Moreover, they are available each end of months and thus can be used to forecast the french GDP. INSEE publishes a composite indicator, the French business climate indicator that summarizes information of the whole business survey, see [START_REF] Clavel | A monthly indicator of the french business climate[END_REF][START_REF] Dubois | étalonnages à l'aide d'enquêtes de conjoncture: de nouvaux résultats[END_REF]. Following [START_REF] Cornec | Constructing a conditional gdp fan chart with an application to french business survey data[END_REF], let I t be the mean of the last three (monthly based) climate indicators available for each quarter t > 0 at the date of publication of ∆GDP t . All these values (GDP, climate indicator) are available from the INSEE website. Note that a similar approach is used in other countries, see e.g. [START_REF] Biau | Nonparametric forecasting of the manufacturing output growth with firm-level survey data[END_REF] on forecasting the European Union GDP growth thanks to EUROSTATS data.

In order to provide a quantification of the uncertainty of the forecasting, associated interval confidences are usually provided. The ASA and the NBER started using density forecasts in 1968, while the Central Bank of England and INSEE provide their prediction with a fan chart, see ee [START_REF] Diebold | Evaluating density forecasts of inflation: the survey of professional forecasters[END_REF][START_REF] Tay | Density forecasting: a survey[END_REF] for surveys on density forecasting and [START_REF] Britton | The inflation report projections: Understanding the fan chart[END_REF] for fan charts. However, the statistical methodology used is often crude and, until 2012, the fan charts provided by the INSEE was based on the homoscedasticity of the Gaussian forecasting errors, see [START_REF] Cornec | Constructing a conditional gdp fan chart with an application to french business survey data[END_REF][START_REF] Dowd | The inflation fan charts: An evaluation[END_REF]. However, empirical evidences are 1. the GDP forecasting is more uncertain in a period of crisis or recession; 2. the forecasting errors are not symmetrically distributed.

Application of Theorem 6 for the GDP forecasting

Define X t as the data observed at time t: X t = (∆GDP t , I t ) ′ ∈ R 2 . We use the quantile loss function (see Example 4 page 6) for some 0 < τ < 1 of the quantity of interested ∆GDP t :

ℓ τ ((∆GDP t , I t ), (∆ ′ GDP t , I ′ t )) = τ (∆GDP t -∆ ′ GDP t ) , if ∆GDP t -∆ ′ GDP t > 0 -(1 -τ ) (∆GDP t -∆ ′ GDP t ) , otherwise.
We use the family of forecasters proposed by [START_REF] Cornec | Constructing a conditional gdp fan chart with an application to french business survey data[END_REF] given by the relation

f θ (X t-1 , X t-2 ) = θ 0 + θ 1 ∆GDP t-1 + θ 2 I t-1 + θ 3 (I t-1 -I t-2 )|I t-1 -I t-2 | (7.1)
where θ = (θ 0 , θ 1 , θ 2 , θ 3 ) ∈ Θ(B). Fix D > 0 and

Θ = θ = (θ 0 , θ 1 , θ 2 , θ 3 ) ∈ R 4 , θ 1 = 3 i=0 |θ i | ≤ D . Let us denote R τ (θ) := E [ℓ τ (∆GDP t , f θ (X t-1 , X t-2
))] the risk of the forecaster f θ and let r τ n denote the associated empirical risk. We let θERM,τ denote the ERM with quantile loss ℓ τ : θERM,τ ∈ arg min θ∈Θ r τ n (θ).

We apply Theorem 6 as Lip(L) is satisfied Θ ′ with L = D+1 and LipLoss(K) with K = 1. If the observations are bounded, stationary such that WeakDep(C) holds for some C > 0, the assumptions of Theorem 6 are satisfied with ψ = B and d = 4: Corollary 2. Let us fix τ ∈ (0, 1). If the observations are bounded, stationary such that WeakDep(C) holds for some C > 0 then for any ε > 0 and n large enough, we have

P R τ ( θERM,τ ) ≤ inf θ∈Θ R τ (θ) + 2κ √ 2 √ n (1 -4/n) log 2e 2 B(D + 1) √ n κε ≥ 1 -ε.
In practice the choice of D has little importance as soon as D is large enough (only the theoretical bound is influenced). As a consequence we take D = 100 in our experiments.

Results

The results are shown in Figure 1 for forecasting corresponding to τ = 0.5. Figure 2 represents the confidence intervals of order 50%, i.e. τ = 0.25 and τ = 0.75 (left) and for confidence interval of order 90%, i.e. τ = 0.05 and τ = 0.95 (right). We report only the results for the period 2000-Q1 to 2011-Q3 (using the period 1988-Q1 to 1999-Q4 for learning). We denote θERM,τ [t] the estimator computed at time t -1, based on the observations X j , j < t. We report the online performance: mean abs. pred. error

= 1 n n t=1 ∆GDP t -f θERM,0.5 [t] (X t-1 , X t-2 ) mean quad. pred. error = 1 n n t=1 ∆GDP t -f θERM,0.5 [t] (X t-1 , X t-2 ) 2
and compare it to the INSEE performance, see Table 1. We also report the frequency that the GDPs fall above the predicted τ -quantiles for each τ , see Table 2. Note that this quantity should be close to τ .

Predictor

Mean absolute prediction error Mean quadratic prediction error θ ERM,0 2 Empirical frequencies of the event: GDP falls under the predicted τ -quantile.

The methodology fails to forecast the importance of the 2008 subprime crisis as it was the case for the INSEE forecaster, see [START_REF] Cornec | Constructing a conditional gdp fan chart with an application to french business survey data[END_REF]. However, it is interesting to note that the confidence interval is larger at that date: the forecast is less reliable, but thanks to our adaptive confidence interval, it would have been possible to know at that time that the prediction was not reliable. Another interesting point is to remark that the lower bound of the confidence intervals are varying over time while the upper bound is almost constant for τ = 0.95. It supports the idea of asymmetric forecasting errors. A parametric model with gaussian innovations would lead to underestimate the recessions risk.

Simulation study

In this section, we finally compare the ERM or Gibbs estimators to the Quasi Maximum Likelihood Estimator (QMLE) based method used by the R function ARMA [R D08]. The idea is not to claim any superiority of one method over another, it is rather to check that the ERM and Gibbs estimators can be safely used in various contexts as their performances are close to the standard QMLE even in the context where the series is generated from an ARMA model. It is also the opportunity to check the robustness of our estimators in case of misspecification.

Parametric family of predictors

Here, we compare the ERM to the QMLE. We draw simulations from an AR(1) models (8.1) and a non linear model (8.2):

X t = 0.5X t-1 + ε t (8.1) X t = 0.5 sin(X t-1 ) + ε t (8.2)
where ε t are iid innovations. We consider two cases of distributions for ε t : the uniform case, ε t ∼ U[-a, a], and the Gaussian case, ε t ∼ N (0, σ 2 ). Note that, in the first case, both models satisfy the assumptions of Theorem 8: there exists a stationary solutions (X t ) that is φ-mixing when the innovations are uniformly distributed and WeakDep(C) is satisfied for some C > 0. This paper does not provide any theoretical results for the Gaussian case as it is unbounded. However, we refer the reader to [START_REF] Alquier | Model selection for weakly dependent time series forecasting[END_REF] for truncations techniques that allows to deal with this case too. We fix σ = 0.4 and a = 0.70 such that V ar(ǫ t ) ≃ 0.16 in both cases. For each model, we simulate first a sequence of length n and then we predict X n using the observations (X 1 , . . . , X n-1 ). Each simulation is repeated 100 times and we report the mean quadratic prediction errors on the Table 3.

It is interesting to note that the ERM estimator with absolute loss performs better on model (8.1) while the ERM with quadratic loss performs slightly better on model (8.2). The difference tends be too small to be significative, however, the numerical results tends to indicate that both methods are robust to model mispecification. Also, both estimators seem to perform better than the R QMLE procedure when n = 100, but the differences tends to be less perceptible when n grows.

Sparse autoregression

To illustrate Corollary 1, we compare the Gibbs predictor to the model selection approach of the ARMA procedure in the R software. This procedure computes the QMLE estimator in each AR(p) model, 1 ≤ p ≤ q, and then selects the order p by Akaike's AIC criterion [START_REF] Akaike | Information theory and an extension of the maximum likelihood principle[END_REF]. The Gibbs estimator is computed using a Reversible Jump MCMC algorithm as in [START_REF] Alquier | PAC-Bayesian bounds for sparse regression estimation with exponential weights[END_REF]. The parameter λ is taken as λ = n/ v ar(X), the empirical variance of the observed time series. We draw the data according to the following models:

X t = 0.5X t-1 + 0.1X t-2 + ε t (8.3) X t = 0.6X t-4 + 0.1X t-8 + ε t (8.4) X t = cos(X t-1 ) sin(X t-2 ) + ε t (8.5)
where ε t are iid innovations. We still consider the uniform (ε t ∼ U[-a, a]) and the Gaussian (ε t ∼ N (0, σ 2 )) cases with σ = 0.4 and a = 0.70. We compare the Gibbs predictor performances to those of the estimator based on the AIC criterion and to the QMLE in the AR(q) model, so called "full model". For each model, we first simulate a time series of length 2n, use the observations 1 to n as a learning set and n + 1 to 2n as a test set, for n = 100 and n = 1000. Each simulation is repeated 20 times and we report in Table 4 the mean and the standard deviation of the empirical quadratic errors for each method and each model. Note that the Gibbs predictor performs better on Models (8.4) and (8.5) while the AIC predictor performs slightly better on Model (8.3). The difference tends to be negligible when n grows -this is coherent with the fact that we develop here a non-asymptotic theory. Note that the Gibbs predictor performs also well in the case of a Gaussian noise where the boundedness assumption is not satisfied.

Appendix A: A general PAC-Bayesian inequality

Theorems 1 and 5 are actually both corollaries of a more general result that we would like to state for the sake of completeness. This result is the analogous of the PAC-Bayesian bounds proved by Catoni in the case of iid data [START_REF] Catoni | PAC-Bayesian supervised classification (the thermodynamics of statistical learning[END_REF].

Theorem 9 (PAC-Bayesian Oracle Inequality for the Gibbs estimator). Let us assume that LowRates(κ) is satisfied for some κ > 0. Then, for any λ, ε > 0 we have

P R θλ ≤ inf ρ∈M 1 + (Θ) Rdρ + 2λκ 2 n (1 -k/n) 2 + 2K(ρ, π) + 2 log (2/ε) λ ≥ 1 -ε.
This result is proved in Appendix B, but we can now provide the proofs of Theorems 1 and 5. Proof of Theorem 1. We apply Theorem 9 for π = 1 M θ∈Θ δ θ and restrict the inf in the upper bound to Dirac masses ρ ∈ {δ θ , θ ∈ Θ}. We obtain K(ρ, π) = log M , and the upper bound for R( θλ ) becomes:

R θλ ≤ inf ρ∈{δ θ ,θ∈Θ} Rdρ + 2λκ 2 n (1 -k/n) 2 + 2 log (2M /ε) λ = inf θ∈Θ R(θ) + 2λκ 2 n (1 -k/n) 2 + 2 log (2M /ε) λ .
Proof of Theorem 5. An application of Theorem 9 yields that with probability at least 1ε

R( θλ ) ≤ inf ρ∈M 1 + (Θ) Rdρ + 2λκ 2 n (1 -k/n) 2 + 2K(ρ, π) + 2 log (2/ε) λ .
Let us estimate the upper bound at the probability distribution ρ δ defined as

dρ δ dπ (θ) = 1{R(θ) -R(θ) < δ} t∈Θ 1{R(t) -R(θ) < δ}π(dt)
.

Then we have:

R θλ ≤ inf δ>0 R(θ) + δ + 2λκ 2 n (1 -k/n) 2 + 2 -log t∈Θ 1{R(t) -inf Θ R < δ}π(dt) + log 2 ε λ .
Under the assumptions of Theorem 5 we have:

R θλ ≤ inf δ>0 R(θ) + δ + 2λκ 2 n (1 -k/n) 2 + 2 d log (D/δ) + log 2 ε λ .
The infimum is reached for δ = d/λ and we have:

R θλ ≤ R(θ) + 2λκ 2 n (1 -k/n) 2 + 2 d log (D √ eλ/d) + log 2 ε λ .
Appendix B: Proofs

B.1. Preliminaries

We will use Rio's inequality [START_REF] Rio | Ingalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes[END_REF] that is an extension of Hoeffding's inequality in a dependent context. For the sake of completeness, we provide here this result when the observations (X 1 , . . . , X n ) come from a stationary process (X t )

Lemma 1 (Rio [Rio00]). Let h be a function (R p ) n → R such that for all x 1 , ..., x n , y 1 , ..., y n ∈ R p , |h(x 1 , . . . , x n ) -h(y 1 , . . . , y n )| ≤ n i=1 x i -y i . (B.1)
Then, for any t > 0, we have

E (exp(t {E [h(X 1 , . . . , X n )] -h(X 1 , . . . , X n )})) ≤ exp t 2 n (B + θ ∞,n (1)) 2 2 .
Others exponential inequalities can be used to obtain PAC-Bounds in the context of time series: the inequalities in [START_REF] Doukhan | Mixing[END_REF][START_REF] Samson | Concentration of measure inequalities for markov chains and φ-mixing processes[END_REF] for mixing time series, and [DDL+07, Win10] under weakest "weak dependence" assumptions, [START_REF] Seldin | Pac-bayesian inequalities for martingales[END_REF] for martingales. Lemma 1 is very general and yields optimal low rates of convergence. For fast rates of convergence, we will use Samson's inequality that is an extension of Bernstein's inequality in a dependent context.

Lemma 2 (Samson [START_REF] Samson | Concentration of measure inequalities for markov chains and φ-mixing processes[END_REF]). Let N ≥ 1, (Z i ) i∈Z be a stationary process on R k and φ Z r denote its φ-mixing coefficients. For any measurable function f :

R k → [-M, M ], any 0 ≤ t ≤ 1/(M K 2 φ Z ), we have E(exp(t(S N (f ) -ES N (f )))) ≤ exp 8K φ Z N σ 2 (f )t 2 , where S N (f ) := N i=1 f (Z i ), K φ Z = 1 + N r=1 φ Z r and σ 2 (f ) = Var(f (Z i )).
Proof of Lemma 2. This result can be deduced easily from the proof of Theorem 3 of [START_REF] Samson | Concentration of measure inequalities for markov chains and φ-mixing processes[END_REF] which states a more general result on empirical processes. In page 457 of [START_REF] Samson | Concentration of measure inequalities for markov chains and φ-mixing processes[END_REF], replace the definition of f N (x 1 , . . . , x n ) by f N (x 1 , . . . , x n ) = n i=1 g(x i ) (following the notations of [START_REF] Samson | Concentration of measure inequalities for markov chains and φ-mixing processes[END_REF]). Then check that all the arguments of the proof remain valid, the claim of Lemma 2 is obtained page 460, line 7.

We also remind the variational formula of the Kullback divergence.

Lemma 3 (Donsker-Varadhan [START_REF] Donsker | Asymptotic evaluation of certain markov process expectations for large time. iii[END_REF] variational formula). For any π ∈ M 1 + (E), for any measurable upper-bounded function h : E → R we have:

exp(h)dπ = exp sup ρ∈M 1 + (E) hdρ -K(ρ, π) . (B.2)
Moreover, the supremum with respect to ρ in the right-hand side is reached for the Gibbs measure π{h} defined by π{h}(dx) = e h(x) π(dx)/π[exp(h)].

Actually, it seems that in the case of discrete probabilities, this result was already known by Kullback (Problem 8.28 of Chapter 2 in [START_REF] Kullback | Information theory and statistics[END_REF]). For a complete proof of this variational formula, even in the non integrable cases, we refer the reader to [START_REF] Donsker | Asymptotic evaluation of certain markov process expectations for large time. iii[END_REF][START_REF] Catoni | A PAC-Bayesian approach to adaptative classification[END_REF][START_REF] Catoni | PAC-Bayesian supervised classification (the thermodynamics of statistical learning[END_REF].

B.2. Technical lemmas for the proofs of Theorems 2, 6, 7 and 9

Lemma 4. We assume that LowRates(κ) is satisfied for some κ > 0. For any λ > 0 and θ ∈ Θ we have E e λ(R(θ)-rn(θ)) ∨ E e λ(rn(θ)-R(θ)) ≤ exp

λ 2 κ 2 n (1 -k/n) 2 .
Proof of Lemma 4. Let us fix λ > 0 and θ ∈ Θ. Let us define the function h by:

h(x 1 , . . . , x n ) = 1 K(1 + L) n i=k+1 ℓ(f θ (x i-1 , . . . , x i-k ), x i ).
We now check that h satisfies (B.1), remember that ℓ(x, x ′ ) = g(xx ′ ) so

h (x 1 , . . . , x n ) -h (y 1 , . . . y n ) ≤ 1 K(1 + L) n i=k+1 g(f θ (x i-1 , . . . , x i-k ) -x i ) -g(f θ (y i-1 , . . . , y i-k ) -y i ) ≤ 1 1 + L n i=k+1 f θ (x i-1 , . . . , x i-k ) -x i -f θ (y i-1 , . . . , y i-k ) -y i
where we used Assumption LipLoss(K) for the last inequality. So we have

h (x 1 , . . . , x n ) -h (y 1 , . . . y n ) ≤ 1 1 + L n i=k+1 f θ (x i-1 , . . . , x i-k ) -f θ (y i-1 , . . . , y i-k ) + x i -y i ≤ 1 1 + L n i=k+1   k j=1 a j (θ) x i-j -y i-j + x i -y i   ≤ 1 1 + L n i=1   1 + k j=1 a j (θ)   x i -y i ≤ n i=1 x i -y i
where we used Assumption Lip(L). So we can apply Lemma 1 with h(X 1 , . . . ,

X n ) = n-k K(1+L) r n (θ), E(h(X 1 , . . . , X n )) = n-k K(1+L) R(θ), and t = K(1 + L)λ/(n -k): E e λ[R(θ)-rn(θ)] ≤ exp λ 2 K 2 (1 + L) 2 (B + θ ∞,n (1)) 2 2n (1 -k/n) 2 ≤ exp λ 2 K 2 (1 + L) 2 (B + C) 2 2n 1 -k n 2
by Assumption WeakDep(C). This ends the proof of the first inequality. The reverse inequality is obtained by replacing the function h by -h.

We are now ready to state the following key Lemma.

Lemma 5. Let us assume that LowRates(κ) is satisfied satisfied for some κ > 0. Then for any λ > 0 we have

P          ∀ρ ∈ M 1 + (Θ), Rdρ ≤ r n dρ + λκ 2 n(1-k/n) 2 + K(ρ,π)+log(2/ε) λ and r n dρ ≤ Rdρ + λκ 2 n(1-k/n) 2 + K(ρ,π)+log(2/ε) λ          ≥ 1 -ε. (B.3)
Proof of Lemma 5. Let us fix θ > 0 and λ > 0, and apply the first inequality of Lemma 4. We have:

E exp λ R(θ) -r n (θ) - λκ 2 n (1 -k/n) 2 ≤ 1,
and we multiply this result by ε/2 and integrate it with respect to π(dθ). An application of Fubini's Theorem yields

E exp λ(R(θ) -r n (θ)) - λ 2 κ 2 n (1 -k/n) 2 -log (2/ε) π(dθ) ≤ ε 2 .
We apply Lemma 3 and we get:

E exp sup ρ λ (R(θ) -r n (θ))ρ(dθ) - λ 2 κ 2 n (1 -k/n) 2 -log (2/ε) -K(ρ, π) ≤ ε 2 .
As e x ≥ 1 R+ (x), we have:

P sup ρ λ (R(θ) -r n (θ)) ρ(dθ) - λ 2 κ 2 n (1 -k/n) 2 -log (2/ε) -K(ρ, π) ≥ 0 ≤ ε 2 .
Using the same arguments than above but starting with the second inequality of Lemma 4:

E exp λ r n (θ) -R(θ) - λκ 2 n (1 -k/n) 2 ≤ 1.
we obtain:

P sup ρ λ [r n (θ) -R(θ)] ρ(dθ) - λ 2 κ 2 n 1 -k n 2 -log 2 ε -K(ρ, π) ≥ 0 ≤ ε 2 .
A union bound ends the proof.

The following variant of Lemma 5 will also be useful.

Lemma 6. Let us assume that LowRates(κ) is satisfied satisfied for some κ > 0. Then for any λ > 0 we have

P          ∀ρ ∈ M 1 + (Θ), Rdρ ≤ r n dρ + λκ 2 n(1-k/n) 2 + K(ρ,π)+log(2/ε) λ and r n (θ) ≤ R(θ) + λκ 2 n(1-k/n) 2 + log(2/ε) λ          ≥ 1 -ε.
Proof of Lemma 6. Following the proof of Lemma 5 we have:

P sup ρ λ (R(θ) -r n (θ)) ρ(dθ) - λ 2 κ 2 n (1 -k/n) 2 -log (2/ε) -K(ρ, π) ≥ 0 ≤ ε 2 .
Now, we use the second inequality of Lemma 4, with θ = θ:

E exp λ r n (θ) -R(θ) - λκ 2 n (1 -k/n) 2 ≤ 1.
But then, we directly apply Markov's inequality to get:

P r n (θ) ≥ R(θ) + λκ 2 n (1 -k/n) 2 + log (2/ε) λ ≤ ε 2 .
Here again, a union bound ends the proof.

B.3. Proof of Theorems 9 and 7

In this subsection we prove the general result on the Gibbs predictor. Proof of Theorem 9. We apply Lemma 5. So, with probability at least 1ε we are on the event given by (B.3). From now, we work on that event. The first inequality of (B.3), when applied to ρλ (dθ), gives

R(θ)ρ λ (dθ) ≤ r n (θ)ρ λ (dθ) + λκ 2 n (1 -k/n) 2 + 1 λ log (2/ε) + 1 λ K(ρ λ , π).
According to Lemma 3 we have:

r n (θ)ρ λ (dθ) + 1 λ K(ρ λ , π) = inf ρ r n (θ)ρ(dθ) + 1 λ K(ρ, π) so we obtain R(θ)ρ λ (dθ) ≤ inf ρ r n (θ)ρ(dθ) + λκ 2 n (1 -k/n) 2 + K(ρ, π) + log (2/ε) λ .
(B.4) We now estimate from above r(θ) by R(θ). Applying the second inequality of (B.3) and plugging it into Inequality B.4 gives

R(θ)ρ λ (dθ) ≤ inf ρ Rdρ + 2 λ K(ρ, π) + 2λκ 2 n (1 -k/n) 2 + 2 λ log (2/ε) .
We end the proof by the remark that θ → R(θ) is convex and so by Jensen's inequality R(θ)ρ λ (dθ) ≥ R θ ρλ (dθ) = R( θλ ).

Proof of Theorem 7. Let us apply Lemma 5 in each model Θ j , with a fixed λ j > 0 and confidence level ε j > 0. We obtain, for all j,

P            ∀ρ ∈ M 1 + (Θ j ), Rdρ ≤ r n dρ + λj κ 2 j n(1-k/n) 2 + K(ρ,πj)+log(2/εj ) λj and r n dρ ≤ Rdρ + λj κ 2 j n(1-k/n) 2 + K(ρ,πj)+log(2/εj ) λj            ≥ 1 -ε j .
We put ε j = p j ε, a union bound gives leads to:

P              ∀j ∈ {1, ..., M }, ∀ρ ∈ M 1 + (Θ j ), Rdρ ≤ r n dρ + λj κ 2 j n(1-k/n) 2 + K(ρ,πj )+log 2 εp j λj and r n dρ ≤ Rdρ + λj κ 2 j n(1-k/n) 2 + K(ρ,πj )+log 2 εp j λj              ≥ 1 -ε. (B.5)
From now, we only work on that event of probability at least 1ε. Remark that

R( θ) = R( θλ ĵ , ĵ ) ≤ R(θ)ρ λ ĵ , ĵ (dθ) by Jensen's inequality ≤ r n ρλ ĵ , ĵ (dθ) + λ j κ 2 j n (1 -k/n) 2 + K(ρ λ ĵ , ĵ , π j ) + log 2 εpj λ j by (B.5) = inf 1≤j≤M    r n ρλj,j (dθ) + λ j κ 2 j n (1 -k/n) 2 + K(ρ λj ,j , π j ) + log 2 εpj λ j    by definition of ĵ = inf 1≤j≤M inf ρ∈M 1 + (Θj )    r n ρ(dθ) + λ j κ 2 j n (1 -k/n) 2 + K(ρ, π j ) + log 2 εpj λ j    by Lemma 3 ≤ inf 1≤j≤M inf ρ∈M 1 + (Θj )    Rρ(dθ) + 2λ j κ 2 j n (1 -k/n) 2 + 2 K(ρ, π j ) + log 2 εpj λ j    by (B.5) again ≤ inf 1≤j≤M inf δ>0    R(θ j ) + δ + 2λ j κ 2 j n (1 -k/n) 2 + 2 d j log (D j /δ) + log 2 εpj λ j  
 by restricting ρ as in the proof of Cor. 5 page 10

≤ inf 1≤j≤M    R(θ j ) + 2λ j κ 2 j n (1 -k/n) 2 + 2 d j log Dj eλj dj + log 2 εpj λ j    by taking δ = d j λ j = inf 1≤j≤M    R(θ j ) + inf λ>0    2λκ 2 j n (1 -k/n) 2 + 2 d j log Dj eλ dj + log 2 εpj λ       by definition of λ j ≤ inf 1≤j≤M    R(θ j ) + 2 κ j 1 -k/n    d j n log D j e 2 κ j n d j + log 2 εpj nd j       .

B.4. Proof of Theorems 2 and 6

Let us now prove the results about the ERM.

Proof of Theorem 2. We choose π as the uniform probability distribution on Θ and λ > 0. We apply Lemma 6. So we have, with probability at least 1ε,

∀ρ ∈ M 1 + (Θ ′ ), Rdρ ≤ r n dρ + λκ 2 n(1-k/n) 2 + K(ρ,π)+log(2/ε) λ and r n (θ) ≤ R(θ) + λκ 2 n(1-k/n) 2 + log(2/ε) λ .
We restrict the inf in the first inequality to Dirac masses ρ ∈ {δ θ , θ ∈ Θ} and we obtain:

   ∀θ ∈ Θ, R(θ) ≤ r n (θ) + λκ 2 n(1-k/n) 2 + log( 2M ε ) λ and r n (θ) ≤ R(θ) + λκ 2 n(1-k/n) 2 + log(2/ε) λ .
In particular, we apply the first inequality to θERM . We remind that θ minimizes R on Θ and that θERM minimizes r n on Θ, and so we have

R( θERM ) ≤ r n ( θERM ) + λκ 2 n (1 -k/n) 2 + log(M ) + log (2/ε) λ ≤ r n (θ) + λκ 2 n (1 -k/n) 2 + log(M ) + log (2/ε) λ ≤ R(θ) + 2λκ 2 n (1 -k/n) 2 + log(M ) + 2 log (2/ε) λ ≤ R(θ) + 2λκ 2 n (1 -k/n) 2 + 2 log (2M /ε) λ .
The result still holds if we choose λ as a minimizer of

2λκ 2 n (1 -k/n) 2 + 2 log (2M /ε) λ .
Proof of Theorem 6. We put Θ ′ = {θ ∈ R d : θ 1 ≤ D + 1}. We choose π as the uniform probability distribution on Θ ′ . We apply Lemma 6. So we have, with probability at least 1ε,

∀ρ ∈ M 1 + (Θ ′ ), Rdρ ≤ r n dρ + λκ 2 n(1-k/n) 2 + K(ρ,π)+log(2/ε) λ and r n (θ) ≤ R(θ) + λκ 2 n(1-k/n) 2 + log(2/ε) λ . So for any ρ, R( θERM ) = [R( θERM ) -R(θ)]ρ(dθ) + Rdρ ≤ [R( θERM ) -R(θ)]ρ(dθ) + r n dρ + λκ 2 n (1 -k/n) 2 + K(ρ, π) + log (2/ε) λ ≤ [R( θERM ) -R(θ)]ρ(dθ) + [r n (θ) -r n ( θERM )]ρ(dθ) + r n ( θERM ) + λκ 2 n (1 -k/n) 2 + K(ρ, π) + log (2/ε) λ ≤ 2Kψ θ -θERM 1 ρ(dθ) + r n (θ) + λκ 2 n (1 -k/n) 2 + K(ρ, π) + log (2/ε) λ ≤ 2Kψ θ -θERM 1 ρ(dθ) + R(θ) + 2λκ 2 n (1 -k/n) 2 + K(ρ, π) + 2 log (2/ε) λ .
Now we define, for any δ > 0, ρ δ by

dρ δ dπ (θ) = 1{ θ -θERM < δ} t∈Θ ′ 1{ t -θERM < δ}π(dt)
.

So in particular, we have, for any δ > 0, R We just choose λ as the minimizer of the r.h.s., subject to t ≥ 2Kψ/d, to end the proof.

B.5. Some preliminary lemmas for the proof of Theorem 8

Lemma 7. Under the hypothesis of Theorem 8, we have, for any θ ∈ Θ, for any 0 ≤ λ ≤ (nk)/(2kKLBC),

E exp λ 1 - 8kCλ n -k R(θ) -R(θ) -r(θ) + r(θ) ≤ 1, and
E exp λ 1 + 8kCλ n -k R(θ) -R(θ) -r(θ) + r(θ) ≤ 1.
Lemma 7. We apply Lemma 2 to N = nk, Z i = (X i+1 , . . . , X i+k ),

f (Z i ) = 1 n -k R(θ) -R(θ)
ℓ (X i+k , f θ (X i+k-1 , . . . , X i+1 )) + ℓ X i+k , f θ (X i+k-1 , . . . , X i+1 ) 2 , and so S N (f ) = [R(θ) -R(θ)r(θ) + r(θ)],

and the Z i are uniformly mixing with coefficients φ Z r = φ ⌊r/q⌋ . Note that 1 + n-q r=1

φ Z r = 1 + n-q r=1

φ ⌊r/k⌋ ≤ k C by PhiMix(C). For any θ and θ ′ in Θ let us put V (θ, θ ′ ) = E ℓ X k+1 , f θ (X k , ..., X 1 )ℓ X k+1 , f θ ′ (X k , ..., X 1 ) 2 .

We are going to apply Lemma 2. Remark that σ 2 (f ) ≤ V (θ, θ)/(nk) 2 . Also, ℓ (X i+k , f θ (X i+k-1 , . . . , X i+1 ))ℓ X i+k , f θ (X i+k-1 , . . . , X i+1 ) ≤ K f θ (X i+k-1 , . . . , X i+1 )f θ (X i+k-1 , . . . , X i+1 ) ≤ KLB where we used LipLoss(K) for the first inequality and Lip(L) and PhiMix(B, C) for the second inequality. This implies that f ∞ ≤ 2KLB/(nk), so we can apply Lemma 2 for any 0 ≤ λ ≤ (nk)/(2kKLBC)], we have ln E exp λ R(θ) -R(θ)r(θ) + r(θ) ≤ 8kCV (θ, θ)λ 2 nk .

Notice finally that Margin(K) leads to

V (θ, θ) = K R(θ) -R(θ)
This proves the first inequality of Lemma 7. The second inequality is proved exacly in the same way, but replacing f by -f .

We are now ready to state the following key Lemma.

Lemma 8. Under the hypothesis of Theorem 8, we have, for any 0 ≤ λ ≤ (nk)/(2kKLBC), for any 0 < ε < 1,

P            ∀ρ ∈ M 1 + (Θ), 1 -8kCλ n-k
Rdρ -R(θ) ≤ rdρr(θ) + K(ρ,π)+log(2/ε) λ and rdρr(θ) ≤ Rdρ -R(θ) 1 + 8kCλ n-k

+ K(ρ,π)+log(2/ε) λ            ≥ 1 -ε.
Proof of Lemma 8. Let us fix ε, λ and θ ∈ Θ, and apply the first inequality of Lemma 7. We have:

E exp λ 1 - 8kCλ n -k R(θ) -R(θ) -r(θ) + r(θ) ≤ 1,
and we multiply this result by ε/2 and integrate it with respect to π(dθ). Fubini's Theorem gives:

E exp λ 1 - 8kCλ n -k R(θ) -R(θ) -r(θ) + r(θ) + log(ǫ/2) π(dθ) ≤ ε 2 .
We apply Lemma 3 and we get:

E exp sup ρ λ 1 - 8kCλ n -k Rdρ -R(θ) -rdρ + r(θ)
+ log(ǫ/2) -K(ρ, π) ≤ ε 2 .

As e x ≥ 1 R+ (x), we have:

P sup ρ λ 1 - 8kCλ n -k Rdρ -R(θ) -rdρ + r(θ)
+ log(ǫ/2) -K(ρ, π) ≥ 0 ≤ ε 2 .

Let us apply the same arguments starting with the second inequality of Lemma 7. We obtain:

P sup ρ λ 1 + 8kCλ n -k R(θ) -Rdρ -r(θ) + rdρ + log(ǫ/2) -K(ρ, π) ≥ 0 ≤ ε 2 .
A union bound ends the proof.

B.6. Proof of Theorem 8

Proof of Theorem 8. Fix 0 ≤ λ = (nk)/(4kKLBC) ∧ (nk)/(16kC) ≤ (nk)/(2kKLBC). Applying Lemma 8, we assume from now that the event of probability at least 1ε given by this lemma is satisfied. In particular we have ∀ρ ∈ M 1 + (Θ), Rdρ -R(θ) ≤ rdρr(θ) + K(ρ,π)+log(2/ε) λ 1 -8kCλ n-k .

In particular, thanks to Lemma 3, we have:

Rdρ λ -R(θ) ≤ inf ρ∈M 1 + (Θ)
rdρr(θ) + K(ρ,π)+log(2/ε)

λ 1 -8kCλ n-k
. Now, we apply the second inequality of Lemma 8:

Rdρ λ -R(θ) ≤ inf ρ∈M 1 + (Θ) 1 + 8kCλ n-k
Rdρ -R(θ) + 2 K(ρ,π)+log(2/ε)

  Bound(B) and PhiMix(C) ⇒ Bound(B) and WeakDep(CB).

Theorem 5 .

 5 Assume that SlowRates(κ) is satisfied and the existence of d = d(Θ, π) > 0 and D = D(Θ, π) > 0 satisfying the relation

Example 6 .

 6 Consider the linear autoregressive model of AR(k) predictors studied in Theorems 3 and 4. Then Lip(L) is automatically satisfied with L = D +1. The assumptions of Theorem 6 are satisfied with d = k + 1 and ψ = B. Moreover, thanks to Remark 1, the assumptions of Theorem 5 are satisfied with D(Θ, π) = (KB ∨ K 2 B 2 )(R + 1). Then Theorems 3 and 4 are actually direct consequences of Theorems 5 and 6.

  1) δ and the assumptions of Theorem 5 are satisfied for d(Θ, π) = d and D(Θ, π) = (Kψ ∨ K 2 ψ 2 )(D + 1).

Fig 1 .

 1 Fig 1. French GDP forecasting using the quantile loss function with τ = 0.5.

Fig 2 .

 2 Fig 2. French GDP online 50%-confidence intervals (left) and 90%-confidence intervals (right).

  θERM ) ≤ 2Kψδ + R(θ) + 2λκ 2 n (1k/n) 2 + log 1 t∈Θ ′ 1{ t-θERM <δ}π(dt) + 2 log (2/ε) λ .But for any δ ≤ 1,log t∈Θ ′ 1{ t -θERM < δ}π(dt) = d log DWe optimize this result by taking δ = d/(2λKψ), which is smaller than 1 as soon as t ≥ 2Kψ/d, we get:R( θERM ) ≤ R(θ) + 2λκ 2 n (1k/n) 2 + d log 2eKψ(D+1)t d + 2 log (2/ε) λ .

.

  by restricting ρ as in the proof of Theorem 5. First, notice that our choice λ ≤ (nk)/(16kC) leads toRdρ λ -R(θ) ≤ 2 inf j ) + δ -R(θ)Taking δ = d j /λ leads to Rdρ λ -R(θ) replace the last occurences of λ by its value:Rdρ λ -R(θ) j ) -R(θ) + (16kC ∨ 4kKLBC) j ) -R(θ) + 4kC (4 ∨ KLB) d j logDj e(n-k) 16kCdj

  Performances of the ERM and of the INSEE.

	.5	0.2249	0.0812
	INSEE	0.2579	0.0967
			Table 1
		τ	Estimator	Frequency
		0.05	θ ERM,0.05	0.1739
		0.25	θ ERM,0.25	0.4130
		0.5	θ ERM,0.5	0.6304
		0.75	θ ERM,0.75	0.9130
		0.95	θ ERM,0.95	0.9782
			Table

Table 3

 3 Performances of the ERM estimators and ARMA, on the simulations. The first row "ERM abs." is for the ERM estimator with absolute loss, the second row "ERM quad." for the ERM with quadratic loss. The standard deviations are given in parentheses.

	n	Model Innovations	ERM abs.	ERM quad.	QMLE
	100	(8.1)	Gaussian	0.1436 (0.1419)	0.1445 (0.1365)	0.1469 (0.1387)
			Uniform	0.1594 (0.1512)	0.1591(0.1436)	0.1628 (0.1486)
		(8.2)	Gaussian Uniform	0.1770 (0.1733) 0.1520 (0.1572)	0 ¯.1699 (0.1611) 0.1528 (0.1495)	0.1728 (0.1634) 0.1565 (0.1537)
	1000	(8.1)	Gaussian	0.1336 (0.1291)	0.1343 (0.1294)	0.1345 (0.1296)
			Uniform	0.1718 (0.1369)	0.1729 (0.1370)	0.1732 (0.1372)
		(8.2)	Gaussian	0.1612( 0.1375)	0.1610 (0.1367)	0.1613 (0.1369)
			Uniform	0.1696 (0.1418)	0.1687 (0.1404)	0.1691 (0.1407)

Table 4

 4 Performances of the Gibbs, AIC and "full model" predictors on simulations.

	n	Model	Innovations	Gibbs	AIC	Full Model
	100	(8.3)	Uniform	0.165 (0.022)	0.165 (0.023)	0.182 (0.029)
			Gaussian	0.167 (0.023)	0.161 (0.023)	0.173 (0.027)
		(8.4)	Uniform	0.163 (0.020)	0.169 (0.022)	0.178 (0.022)
			Gaussian	0.172 (0.033)	0.179 (0.040)	0.201 (0.049)
		(8.5)	Uniform	0.174 (0.022)	0.179 (0.028)	0.201 (0.040)
			Gaussian	0.179 (0.025)	0.182 (0.025)	0.202 (0.031)
	1000	(8.3)	Uniform	0.163 (0.005)	0.163 (0.005)	0.166 (0.005)
			Gaussian	0.160 (0.005)	0.160 (0.005)	0.162 (0.005)
		(8.4)	Uniform	0.164 (0.004)	0.166 (0.004)	0.167 (0.004)
			Gaussian	0.160 (0.008)	0.161 (0.008)	0.163 (0.008)
		(8.5)	Uniform	0.171 (0.005)	0.172 (0.006)	0.175 (0.006)
			Gaussian	0.173 (0.009)	0.173 (0.009)	0.176 (0.010)

Institut National de la Statistique et des Etudes Economiqueshttp://www.insee.fr/

It has been checked that to replace past flash estimates by the actual GDP growth rate when it becomes available do not improve the quality of the forecasting[START_REF] Minodier | Avantages comparés des séries premières valeurs publiées et des séries des valeurs révisées[END_REF].

We want to mention that a preliminary version of Theorem 9 appeared in proceedings of DS'12 [AL12]. Research partially supported by the "Agence Nationale pour la Recherche", grant ANR-09-BLAN-0128 "PARCIMONIE" and grant ANR-Bandhits. 1 AMS 2000 , 62M45, 62P20.