Rosina Moreno
email: rmoreno@ub.edu

Miquel Clar

Juan-Carlos Carlos Duque

Rosina Moreno Grup

Forecasting Business and Consumer Surveys Indicators. A Time Series Models Competition

Keywords: forecast competition, indicators, business surveys, monitoring forecasts Comparative methods, Evaluating forecast, Forecasting competition, Indicators, Monitoring forecasts, Business Surveys Comparative methods, Evaluating forecast, Forecasting competition, Indicators, Monitoring forecasts, Business Surveys

  L'archive ouverte pluridisciplinaire

Introduction

The amount of information provided by Business and Consumer Surveys concerning agents' perceptions and expectations of their environment means that they are now recognised as a crucial instrument for gathering economic information in today's ever-changing environment. Data obtained from Business and Consumer Surveys are often used in forecasting models and in testing different expectation schemes - Kauppi et al (1996), Batchelor and Dua (1998), Mourougane and Roma (2002) and Nardo (2003)-. The speed with which the results of these surveys are made available and the wide range of variables included make them extremely useful for decisionmaking -see [START_REF] Stuart | Are Business and Consumer Surveys Still of Value?[END_REF] for a deep discussion on the value of Business and Consumer Surveys-. The remarkable growth in business surveys in Europe since the early 1960s, and the need for them to be carried out and presented in a comparable way, led to the implementation of the Joint Harmonised EU Programme by the Commission in 1961.

The present paper tries to compare different time series methods for the short-run forecasting of Business and Consumer Survey Indicators. Certainly, forecast competitions have been considered in the economic literature although focused in quantitative variables such as industrial production (Byers and Peel, 1995;[START_REF] Simpson | Forecasting UK industrial production over the business cycle[END_REF], output growth and employment (Clements and Smith, 2000) and exchange rates (Clements and Smith, 2001;Boreo and Marrocu, 2002), as well as for general macroeconomic time series [START_REF] Stock | A comparison of linear and nonlinear univariate models for forecasting macroeconomic time series[END_REF]. On the other hand, as far as we know, forecast competitions have not been conducted for the case of qualitative variables. However, we consider that this kind of exercise can be useful to 3 analyse which forecasting method presents the best behaviour. The usefulness of this comparison is twofold. First, it will allow having the best qualitative forecast to evaluate whether a series will move up or down in all kind of processes which can be modified according to changing economic conditions (McIntosh and Dorfman, 1992) or to predict business cycle turning points (Diebold and Rudebusch, 1989). And second, it will guarantee that the best forecast could be used as an explanatory variable in quantitative forecast models (Biart and Praet, 1987;Parigi and Schlitzer, 1995).

The objective of this paper is therefore to compare different time series methods for the short-run forecasting of Business and Consumer Survey Indicators. This objective can be summarised in the following two questions: Is it possible to forecast qualitative indicators? And, if so, which is the best procedure for conducting such forecasts? In order to answer these questions, we considered all available data taken from the Business and Consumer Survey Indicators for the Euro area between, in the main, January 1985 and December 2002. The dataset analysed includes 38 indicators (33 of which are monthly and 5 quarterly) and 6 composite indicators. 1 First, we considered raw data in order to test the forecast accuracy of five different sets of models: autoregressions, ARIMA, Self-exciting threshold autoregressions (SETAR), Markov switching regime models and vector autoregressions (traditional VAR and also VAR models considering the joint evolution of different indicators). Some of the conclusions obtained are related to the high volatility presented by An additional aspect considered in this comparison of forecasts is related to the balance forecasts. As survey data are derived from qualitative questions and based on subjective evaluations, the results are usually presented in terms of balances, which show the difference between the positive and negative percentages of answers. Since it is this balance that analysts take into account and the information that is usually forecast, we examine whether it is better to forecast the balance directly or rather to forecast the negative and positive answers first and, then, calculate the balance.

Additionally, we examine the composite indicators, which are calculated from business surveys in which answers are weighted to various questions in the survey.

When looking at these aggregate indicators, we analyse whether it is preferable to forecast them directly or to obtain the forecast by weighting the forecasts for the different components.

Finally, we consider the effects of seasonal adjustment procedures and the removal of outliers on forecast accuracy. In the case of seasonal adjustment procedures, we apply different methods to the indicators so as to obtain seasonally adjusted data (using Tramo/Seats -TS, X12 and Dainties -DA) and trend cycle estimation (using TS, X12 and Wavelets -WAV) in order to evaluate differences. In the case of outliers, we consider how their presence might affect the results by using TS to remove them (additive, transitory changes and level shifts) from the original series, and to what extent the effects of outliers should be removed each time a new observation is available. The outline of the paper is as follows. The next section presents the models that are considered in the forecast competition. The database and the design of the forecast competition experiment is given in section 3. Section 4 offers the main results of the forecast competition not only for raw data but we also consider the effects of seasonality and removing outliers on forecast accuracy. Finally, section 5 concludes.

Models for forecasting Business and Consumer Survey Indicators

In order to assess alternative methods and models for forecasting Business and Consumer Surveys Indicators, we chose to focus on five different sets of model: autoregressions (AR), ARIMA, Self-exciting threshold autoregressions (SETAR), Markov switching regime models (MK) and vector autoregressions (traditional VAR and also VAR models considering the joint evolution of different indicators).

Autoregressions

The widely known autoregressive model (also known as the distributed-lags model) explains the behaviour of the endogenous variable as a linear combination of its own past values:

t p t p t t t Y Y Y Y ε φ φ φ + + + + = - - - ... 2 2 1 1 .
The key question is how to determine the number of lags that should be included in the model. For monthly -quarterly-data we considered different models with a

ARIMA models

Since the study conducted by Box and Jenkins (1970), ARIMA models have been widely used and their forecast performance has also been confirmed. The general expression of an ARIMA model is the following:

( ) ( ) ( ) ( ) t d D s s s s s t L L L L x ε φ θ λ ∆ ∆ Φ Θ = , ( ) ( ) Qs Qs s s s s s s L ... L L L Θ - - Θ - Θ - = Θ 2 2 1
is a seasonal moving average polynomial, ( ) ( )

Ps Ps s s s s s s L ... L L L Φ - - Φ - Φ - = Φ 2 2 1
is a seasonal autoregressive polynomial, ( ) ( )

q q L L L L θ θ θ θ - - - - = ... 1 2 2 1 1
is a regular moving average polynomial, ( ) ( )

p p L L L L φ φ φ φ - - - - = ... 1 2 2 1 1
is a regular autoregressive polynomial, λ is the value of the Box-Cox (1964) transformation, D s ∆ is the seasonal difference operator, d ∆ is the regular difference operator, S is the periodicity of the time series under consideration, and t ε is the innovation assumed to behave as a white noise.

In order to use models of this kind for forecasting, the proper model has to be identified (i.e., giving values to the order of the different polynomials, to the difference operator, etc.). For monthly data, we considered models with up to 12 AR and MA terms (4 in the case of quarterly data) selecting the model with the lowest AIC value. The statistical goodness of the selected model was also checked. In the case of the ARIMA model the relationship between the current value of a variable and its lags is supposed to be linear and constant over time. However, when looking at real data it can be seen that expansions tend to be more prolonged over time than recessions (Hansen, 1997). In fact, in the behaviour of most economic variables there seems to be a cyclical asymmetry that linear models are unable to capture (Clements and Smith, 1999). A Self-Excited Threshold Autoregressive model (SETAR) for the time series X t can be summarised as follows:

t t u X L B + )• ( i f X t-k ≤ X, and t t v X L + )• ( ζ if X t-k > X,
where u t and v t are white noises, B(L) and ) (L ζ are autoregressive polynomials, the value k is known as delay and the value X is known as threshold. This two-regime self-exciting threshold autoregressive process is estimated using monthly and quarterly data for each indicator and the Monte Carlo procedure is used to generate multi-step forecasts. The delay values selected are those that minimise the sum of squared errors among values between 1 and 12 for monthly data and 1 and 4 for quarterly data. The values of the threshold are given by the variation of the variable being analysed.

Markov switching regime models

Threshold autoregressive models are perhaps the simplest generalisations of linear autoregressions. In fact, these models were built on developments of traditional ARMA time series models. As an alternative to these, time series regime-switching F o r P e e r R e v i e w 8 models assume that the distribution of the variable is known conditional on a particular regime or state occurring. When the economy changes from one regime to another, a substantial change occurs in the series. Hamilton (1989) presented the Markov regime-switching model in which the unobserved regime evolves over time as a 1st-order Markov process. The regime completely governs the dynamic behaviour of the series. This implies that once we assume conditions on a particular regime occurring, and assume a particular parameterisation of the model, we can write down the density of the variable of interest. However, as the regime is strictly unobservable, statistical inferences have to be drawn concerning the likelihood of each regime occurring at any point in time. So, we need to obtain the transition probabilities from one regime to the other.

Three approaches have been adopted in estimating these models (Potter, 1999). First, Hamilton (1989) developed a non-linear filter to evaluate the likelihood function of the model and, then, he directly maximised the likelihood function. Second, in a later article, Hamilton (1990) constructed an EM algorithm which proves particularly useful should all the parameters switch. Finally, Albert and Chib (1993) developed a Bayesian approach to estimation.

Here, we employ a Markov-switching threshold autoregressive model (MK-TAR) in which we are able to allow for different regime-dependent intercepts, autoregressive parameters, and variances. The estimation of the models is carried out by maximum likelihood using the Hamilton (1989) filter 2 together with Kim's smoothing filter (1994). Having estimated the probabilities of expansion and recession, we could then construct the following model for the time series X t :

t t u X L B + )• ( i f P[Expansion/X t-k ]≤ P, and t t v X L + )• ( ζ if P[Recession/X t-k ]>P,
where, as in the SETAR models, u t and v t are white noises, B(L) and ) (L ζ are autoregressive polynomials, the value k is the estimated delay and the value P is the estimated threshold 3 . The selected delay values are those that minimise the sum of squared errors for values between 1 and 12 for monthly data and 1 and 4 for quarterly data. The values of the threshold are given by the variation of the probability.

VAR models

In these models, each variable depends on a certain number of lags of the other variables under analysis [START_REF] Sims | Policy analysis with econometric models[END_REF]. The idea is that the positive, neutral and negative answers to each question can be considered jointly. Moreover, as the sum of the percentages of positive (P), neutral (E) and negative (M) answers would, by definition, total one hundred, this restriction could also be introduced in the model improving its forecasting accuracy: In order to use models of this kind for forecasting, the proper model must first be identified (i.e., give values to the number of lags p). For monthly data, we considered models with up to 24 lags (8 in the case of quarterly data), selecting the model with the lowest AIC value. The statistical goodness of the selected model was also checked.

F o r P e e r R e v i e w 10           +                     + +                     =           - - - - - - t t t p t p t p t t t t t t t M E P Z Z Z Z Z Z Z Z Z M E P A A A A A A A A A M E P

Database and design of the forecast competition experiment

The immediate relevance of the results of harmonised business and consumer surveys is one of their main properties, given that they are published shortly after the termination of the month to which they refer. The survey results are presented either in the form of balances for particular questions or as synthetic indicators. They are able to describe the panorama of the current economic situation much quicker than the quantitative indicators and the macroeconomic magnitudes from national accounts, though the latter tend to be more precise in their descriptions. . More details on the dataset can be found in Table 1.

The European

Since the objective of the paper was to assess alternative methods and models for forecasting Business and Consumer Survey Indicators, we initially considered raw data (in all cases, non seasonally adjusted levels of each category of the variables were used) in order to test the forecast accuracy of the five different sets of models presented in the previous section.

In order to evaluate the relative forecasting accuracy of the models, each model was estimated for all the indicators included up to 2000.12 (or 2000.IV for quarterly indicators) and forecasts for 1, 2, 3, 6 and 12 months (or 1, 2, 4 quarters) in the future were computed. The model specifications are based on information up to 2000.12 or 2000.IV and, thereafter, the models were re-estimated in each month or quarter and the forecasts were computed with these estimation results. Given the availability of actual values up to 2002.12 or 2002.III or 2002.IV, we were able to compute the forecast errors for each indicator and method in a recursive way (i.e., for the 1 month forecast horizon, 24 forecast errors were computed for each indicator). In order to summarise this information, the Root Mean Square Error (RMSE) and Mean Before showing the results of this comparative exercise, it should be stressed that the statistical properties of the Business and Consumer Survey Indicators differ substantially from those of the main macroeconomic variables (GDP, CPI, Industrial Production, Industrial Producer Prices, etc.). In Table 2, the variation coefficient is shown. It is worth noting that the variation coefficient values are extremely high for some indicators. This can be interpreted as evidence of the high volatility of the indicators and, with this in mind, the forecast accuracy of the methods considered can be expected to be lower than that for other macroeconomic variables.

A further result of interest is that if we examine the average values of the variation coefficient for categories (i.e., positive answers, negative answers, balance, etc.), the highest value corresponds to the balance. A possible explanation for the higher variance of the balance is purely statistical. The variance of the balance can be decomposed as follows:

) , ( • 2 ) ( ) ( ) ( ) ( m p Cov m Var p Var m p Var b Var - + = - = .
So, if the covariance between positive and negative answers is negative (i.e. they move in opposite directions), the variance of the balance would be higher than the variance of the other two components. In 87% of cases, the sign of this covariance is negative, so the variance of the balance is higher (in most cases) than the other two 3a). To test whether it was significantly higher, we applied a test of equality of variances between the balance and the other two components. In 76% of the cases, the null hypothesis of equality of variances was rejected (Table 3b).

Additionally, we computed some of the most commonly used methods to test the unit root hypothesis: the augmented Dickey and Fuller test (1979), the Phillips and Perron test (1988) and the Kwiatkowski, Phillips, Schmidt and Yongcheol test (1992) (Table 4a). The Perron test (1989) was also adapted so as to allow for different types of structural change: in the level (crash model) or in the slope (breaking trend) (Table 4b). Since the variables under consideration can only take values between 0 and 100, a priori we would expect most of them to be I(0). The most striking result is that in many cases the variables are eventually considered to be I(1) -85% if the structural break is not considered and 52% when it is 4 .

Results: forecast accuracy comparison

Main results for raw data

In Indicator is obtained by weighting the answers to different questions on the survey.

When dealing with these aggregates, there are two possible courses of action:

forecasting them directly or obtaining the forecast by weighting the forecasts for the different components. To the best of our knowledge, no research has been undertaken in order to determine which method provides the best results. Therefore, we chose to forecast composite indicators in two ways: what we call a direct (computation) method and by using the forecasts of the components, an indirect (computation) method.

Among the conclusions that can be extracted from Table 5, we would like to point out the following. First, indirect, as opposed to direct, methods seem to perform best.

Thus, it would appear to be better to forecast these indicators from the forecasts of the components than directly. Second, among the direct methods, the AR model outperformed the rest of the models in almost all cases. By contrast, the ARIMA and TAR models were never found to be the best forecasters. It should also be stressed that unrestricted VAR models usually work better than restricted VAR models and that the errors displayed by the ARIMA, TAR and VAR models for different indicators were generally higher.

Furthermore, taking into account that the variables considered (positive, neutral and negative answers but not the balance) can only take values between 0 and 100, the forecast errors are quite high even in the case of the best model. As expected, the forecasts errors increased for longer horizons in most cases and, in general, variables corresponding to questions with a higher number of possible answers were better forecast than the rest. In most cases, the size of the errors was higher for the balance than for the components (this is related to the higher volatility of these variables), which is a common result for most composite indicators (one notable exception is the Economic Sentiment Indicator).

A further aspect to be considered is related to that of balance forecasts. As survey data are derived from qualitative questions and based on subjective evaluation, the results are usually presented in terms of balances, which show the difference between positive and negative percentages of answers. The balance is the information that analysts take into account and the information that is usually forecast. But, is it better to forecast the balance directly or to forecast negative and positive answers and then calculate the balance? In order to answer this question we replicated the forecasting

The effects of seasonality on data revision and on forecast accuracy

Seasonal adjustment methods are usually applied to these indicators. However, an interesting point that has not been analysed to date is the extent to which the method chosen for seasonal adjustment -TS, X12, DA, WAV, among others -might affect the values of the series under consideration. In order to remedy this situation, we applied all these methods of seasonal adjustment so as to obtain seasonally adjusted data (using TS, X12 and DA) and trend cycle estimations (using TS, X12 and WAV)

for a number of qualitative indicators 5 . From the results shown in Table 7, several conclusions can be drawn:

First, TS and X12 results were very similar but DA results (seasonal adjustment) and WAV results (trend cycle-estimation) differed greatly. Specifically, differences were found to be greater in trend-cycle estimations than in seasonal adjustments (Table 7a and 7b). Second, in order to analyse how the seasonal adjustment method chosen affected the revision of the series, seasonally adjusted data and trend-cycle estimations were computed in a recursive way adding one more observation from 2001.12 to 2002.12.

The results (see Table 7c and7d) showed that:

• There was no relationship between the size of the revision in a given observation and the number of time periods between this observation and the last observation available.

• There were great differences in the size of revision for the different variables in each seasonal adjustment method considered.

• There were no revisions of seasonally adjusted data using the DA method.

• TS revisions of seasonally adjusted data were greater than X12 revisions.

• Trend-cycle estimations using WAV method showed dramatic revisions.

• TS revisions of trend-cycle data were greater than X12 revisions.

Third, the choice of seasonal adjustment method was found to affect forecasts. Thus, we were interested in determining whether it was better to forecast raw data and then apply a seasonal adjustment method or not. As pointed out by Commission staff, the information from Business and Consumer Surveys has little seasonality, and, therefore the results should not vary significantly. To confirm this, we computed the Kruskal-Wallis test (also in Table 7e) for all the qualitative variables in order to verify the importance of seasonality. In almost 87% of cases, the null hypothesis of non-seasonality was not rejected, that is, most series did not present a seasonal component. This, it would seem, despite the large number of studies discussing this

The effects on forecast accuracy of removing outliers using TS

The presence of outliers may well affect the results discussed in the previous sections. Therefore, TS can be used in order to remove the outliers from the original series. TS uses an automatic procedure to detect and eliminate outliers from a series.

Three different types of outliers are considered here: Additive outliers (AO),

Transitory changes (TC) and Level shifts (LS).

In Table 8a the variation coefficient for the qualitative variables from the industrial survey and the balances for the other surveys are shown. From these statistics, it can be seen that for some series, TS did not detect any outliers (i.e., the Economic Sentiment Indicator). However, in most cases, the volatility was substantially lower, although in some the value of the variation coefficient increased (although the standard deviation was lower, the value of the mean -which is clearly affected by the presence of extreme values-was even lower). As before, we now computed the value of the covariance between positive and negative answers and we tested to see if the variance of the balance differed from that of positive or negative answers. As observed in Table 8b, in 95% of the cases (as opposed to 87% when seasonality was not taken into account), the covariance was negative, so that the null hypothesis of equality of variances between the balance and the other two components (positive and negative answers) was rejected in 82% of the cases (compared with 76%) -see Using these data, we repeated the comparison of forecasts as in the previous sections.

The main results are shown in Table 8d. The main conclusions to be drawn from this analysis can be summarised as follows: First, RMSE values were the same or lower in nearly all variables when using data from which the outliers had been removed using TS. However, there were differences between models: while in the AR, ARIMA, TAR and Markov models the values of RMSE were markedly lower than those for raw data, the results for VAR (unrestricted and restricted) were, some times, worse. Finally, the VAR models consisting of different indicators performed much better when the outliers were removed.

Second, when comparing the RMSE for the balance computed from forecasts with positive and negative answers, we observed that the results for the models analysed (AR, ARIMA, TAR and VAR) -once the outliers had been removed -were, in most cases, higher than those for raw data. Similarly, when the composite indicators were computed from forecasts based on their components, here again the models performed worse once the outliers had been removed.

A practical issue that needs to be borne in mind when forecasting these series in real time concerns the effects of incorporating new observations on outlier detection using TS. In other words, is it necessary to remove the effects of outliers each time a new observation is available? To analyse this, we adopted a similar approach to that regarding the effects of seasonal adjustment procedures. In order to evaluate the 9. From these results, we can conclude that the number and type of outliers clearly depend on the available sample. Although there was some 'persistence' in the moment and type of detected outliers, some significant changes in the results might also be recorded using TS once an additional observation is included (sometimes due to changes in the underlying model, but also to the dynamics of the series itself). So, if we take into account the results regarding the forecast accuracy of the various models, and although this has the effect of increasing the computational cost, it would appear to be necessary to apply the TS to remove the effects of outliers each time an additional observation becomes available.

In addition, we should stress that when comparing the various seasonal adjustment methods, the TS and X12 results were very similar, but the DA (seasonal adjustment)

and WAV results (trend cycle-estimation) presented marked differences. In fact, these differences were greater for the trend-cycle estimations than the seasonal adjustments, whereas there were considerable differences in the size of revision for the different variables for each seasonal adjustment method considered.

Additionally, after removing the outliers using TS the forecast accuracy of the various methods was similar to that observed with raw data. The number and type of outliers identified with TS clearly depends on the available sample. Although there is some 'persistence' in the moment and type of detected outliers, significant changes can be reflected in the results. As far as the forecast accuracy of the different methods is concerned, in most cases the univariate autoregressions were not outperformed by the other methods. In fact, only the forecast errors from the VAR and Markov models were, in some cases, lower than those from the AR models. It should also be stressed that unrestricted VAR models usually worked better than restricted VAR models and that the errors displayed by the ARIMA, TAR and VAR models comprising a range of indicators were generally higher. However, the size of the forecast errors was high even in the case of the best model.

Interestingly, both in the case of composite indicators and indicators which belong to a balance category, our results show that it is preferable to forecast them indirectly.

As for the effect of the seasonal adjustment methods that are typically applied to these indicators, we used a range of methods (TS, X12, Dainties and Wavelets) that suggest that the use of raw data has little effect on forecast accuracy. A similar conclusion is obtained when using data from which outliers have been removed

using TS, as here again forecast errors were found to be similar for nearly all variables. 1. More details on the dataset can be found in Table 1.

F

2. The Hamilton filter is an iterative procedure which provides estimates of the probability that a given state is prevailing at each point in time given its previous history. These estimates are dependent upon the parameter values given to the filter. Running the filter through the entire sample provides a log likelihood value for the particular set of estimates used. This filter is then repeated to optimise the log likelihood of obtaining the MLE estimates of the parameters. With the maximum likelihood parameters, the probability of state 0 at each point in time is calculated and these are the probabilities of recession and expansion.

3. An alternative approach would have consisted in imposing the value of P and k instead of estimating them. These models are known as Markov Switching

Autoregressive Models (MS-AR) and, in general, the values of P are 0.7 or 0.8 and the values of k, 0 or 1.

4. It is also interesting to note that all the tests only coincide in 10 series.

5. The seasonality analysis was conducted for variables v1, v2, v3p, v3e, v3m, v3b, v4p, v4e, v4m and v4b. We acknowledge financial support from ECFIN/2002/A3-01. We would like to thank the European Commission for their helpful comments and support. The usual disclaimer applies.

F o r P e e r R e v i e w

28

Introduction

The amount of information provided by Business and Consumer Surveys concerning agents' perceptions and expectations of their environment means that they are now recognised as a crucial instrument for gathering economic information in today's ever-changing environment. Data obtained from Business and Consumer Surveys are often used in forecasting models and in testing different expectation schemes - Kauppi et al (1996), Batchelor and Dua (1998), Mourougane and Roma (2002) and Nardo (2003)-. The speed with which the results of these surveys are made available and the wide range of variables included make them extremely useful for decisionmaking -see [START_REF] Stuart | Are Business and Consumer Surveys Still of Value?[END_REF] for a deep discussion on the value of Business and Consumer Surveys-. The remarkable growth in business surveys in Europe since the early 1960s, and the need for them to be carried out and presented in a comparable way, led to the implementation of the Joint Harmonised EU Programme by the Commission in 1961.

The present paper tries to compare different time series methods for the short-run forecasting of Business and Consumer Survey Indicators. Certainly, forecast competitions have been considered in the economic literature although focused in quantitative variables such as industrial production (Byers and Peel, 1995;[START_REF] Simpson | Forecasting UK industrial production over the business cycle[END_REF], output growth and employment (Clements and Smith, 2000) and exchange rates (Clements and Smith, 2001;Boero and Marrocu, 2002), as well as for general macroeconomic time series [START_REF] Stock | A comparison of linear and nonlinear univariate models for forecasting macroeconomic time series[END_REF]. On the other hand, as far as we know, forecast competitions have not been conducted for the case of qualitative variables. However, we consider that this kind of exercise can be useful to 3 analyse which forecasting method presents the best behaviour. The usefulness of this comparison is twofold. First, it will allow having the best qualitative forecast to evaluate whether a series will move up or down in all kind of processes which can be modified according to changing economic conditions (McIntosh and Dorfman, 1992) or to predict business cycle turning points (Diebold and Rudebusch, 1989). And second, it will guarantee that the best forecast could be used as an explanatory variable in quantitative forecast models (Biart and Praet, 1987;Parigi and Schlitzer, 1995).

The objective of this paper is therefore to compare different time series methods for the short-run forecasting of Business and Consumer Survey Indicators. This objective can be summarised in the following two questions: Is it possible to forecast qualitative indicators? And, if so, which is the best procedure for conducting such An additional aspect considered in this comparison of forecasts is related to the balance forecasts. As survey data are derived from qualitative questions and based on subjective evaluations, the results are usually presented in terms of balances, which show the difference between the positive and negative percentages of answers. Since it is this balance that analysts take into account and the information that is usually forecast, we examine whether it is better to forecast the balance directly or rather to forecast the negative and positive answers first and, then, calculate the balance.

Additionally, we examine the composite indicators, which are calculated from business surveys in which answers are weighted to various questions in the survey.

When looking at these aggregate indicators, we analyse whether it is preferable to forecast them directly or to obtain the forecast by weighting the forecasts for the different components.

Finally, we consider the effects of seasonal adjustment procedures and the removal of outliers on forecast accuracy. In the case of seasonal adjustment procedures, we apply different methods to the indicators so as to obtain seasonally adjusted data (using Tramo/Seats -TS, X12 and Dainties -DA) and trend cycle estimation (using TS, X12 and Wavelets -WAV) in order to evaluate differences. In the case of outliers, we consider how their presence might affect the results by using TS to The outline of the paper is as follows. The next section presents the models that are considered in the forecast competition. The database and the design of the forecast competition experiment is given in section 3. Section 4 offers the main results of the forecast competition not only for raw data but we also consider the effects of seasonality and removing outliers on forecast accuracy. Finally, section 5 concludes.

Models for forecasting Business and Consumer Survey Indicators

In order to assess alternative methods and models for forecasting Business and Consumer Surveys Indicators, we chose to focus on five different sets of model: autoregressions (AR), ARIMA, Self-exciting threshold autoregressions (SETAR), Markov switching regime models (MK) and vector autoregressions (traditional VAR and also VAR models considering the joint evolution of different indicators).

Autoregressions

The widely known autoregressive model (also known as the distributed-lags model) explains the behaviour of the endogenous variable as a linear combination of its own past values:

t p t p t t t Y Y Y Y + + + + = ... 2 2 1 1 .
The key question is how to determine the number of lags that should be included in the model. For monthly -quarterly-data we considered different models with a

ARIMA models

Since the study conducted by Box and Jenkins (1970), ARIMA models have been widely used and their forecast performance has also been confirmed. 3 The general expression of an ARIMA model is the following:

( ) ( ) ( ) ( ) t s s t d D s s s L L x ) L 1 ( ) L 1 ( L L =
, where L is the lag operator, ( ) ( )

Qs Qs s s s s s s L ... L L L = 2 2 1 is a Q-order seasonal moving average polynomial, ( ) ( ) Ps Ps s s s s s s L ... L L L = 2 2 1
is a P-order seasonal autoregressive polynomial, ( ) ( )

q q L L L L = ... 1 2 2 1 1
is a q-order regular moving average polynomial, ( ) ( )

p p L L L L = ... 1 2 2 1 1
is a p-order regular autoregressive polynomial, D is the seasonal difference order, d is the regular difference order, is the value of the Box-Cox (1964) transformation, S is the periodicity of the time series under consideration (S=4 for quarterly data, and S=12 for monthly data), and t is the disturbance assumed to behave as a white noise.

In order to use models of this kind for forecasting, the proper model has to be identified (i.e., giving values to the order of the different polynomials, to the difference operator, etc.). For monthly data, we considered models with up to 12 AR

TAR models

In the case of the ARIMA model the relationship between the current value of a variable and its lags is supposed to be linear and constant over time. However, when looking at real data it can be seen that expansions tend to be more prolonged over time than recessions (Hansen, 1997). In fact, in the behaviour of most economic variables there seems to be a cyclical asymmetry that linear models are unable to capture (Clements and Smith, 1999). A Self-Excited Threshold Autoregressive model (SETAR) for the time series X t can be summarised as follows:

t t u X L B + )• ( if X t-k X, and t t v X L + )• ( if X t-k > X,
where u t and v t are white noises, B(L) and ) (L are autoregressive polynomials, the value k is known as delay and the value X is known as threshold. This two-regime self-exciting threshold autoregressive process is estimated using monthly and quarterly data for each indicator and the Monte Carlo procedure is used to generate multi-step forecasts. The delay values selected are those that minimise the sum of squared errors among values between 1 and 12 for monthly data and 1 and 4 for quarterly data. 4 The values of the threshold are given by the variation of the variable being analysed. Threshold autoregressive models are perhaps the simplest generalisations of linear autoregressions. In fact, these models were built on developments of traditional ARMA time series models. As an alternative to these, time series regime-switching models assume that the distribution of the variable is known conditional on a particular regime or state occurring. When the economy changes from one regime to another, a substantial change occurs in the series. Hamilton (1989) presented the Markov regime-switching model in which the unobserved regime evolves over time as a 1st-order Markov process. The regime completely governs the dynamic behaviour of the series. This implies that once we assume conditions on a particular regime occurring, and assume a particular parameterisation of the model, we can write down the density of the variable of interest. However, as the regime is strictly unobservable, statistical inferences have to be drawn concerning the likelihood of each regime occurring at any point in time. So, we need to obtain the transition probabilities from one regime to the other.

Three approaches have been adopted in estimating these models (Potter, 1999). First, Hamilton (1989) developed a non-linear filter to evaluate the likelihood function of the model and, then, he directly maximised the likelihood function. Second, in a later article, Hamilton (1990) constructed an EM algorithm which proves particularly useful should all the parameters switch. Finally, Albert and Chib (1993) Having estimated the probabilities of expansion and recession, we could then construct the following model for the time series X t :

t t u X L B + )• ( if P[Expansion/X t-k ] P, and t t v X L + )• ( if P[Recession/X t-k ]>P,
where, as in the SETAR models, u t and v t are white noises, B(L) and ) (L are autoregressive polynomials, the value k is the estimated delay and the value P is the estimated threshold 6 . The selected delay values are those that minimise the sum of squared errors for values between 1 and 12 for monthly data and 1 and 4 for quarterly data. The values of the threshold are given by the variation of the probability.

VAR models

In these models, each variable depends on a certain number of lags of the other variables under analysis [START_REF] Sims | Policy analysis with econometric models[END_REF]. The idea is that the positive, neutral and negative answers to each question can be considered jointly. Moreover, as the sum of the percentages of positive (P), neutral (E) and negative (M) answers would, by definition, total one hundred, this restriction could also be introduced in the model improving its forecasting accuracy: In order to use models of this kind for forecasting, the proper model must first be identified (i.e., give values to the number of lags p). For monthly data, we considered models with up to 24 lags (8 in the case of quarterly data), selecting the model with the lowest AIC value. The statistical goodness of the selected model was also checked.

Database and design of the forecast competition experiment

The immediate relevance of the results of harmonised business and consumer surveys is one of their main properties, given that they are published shortly after the . More details on the dataset can be found in Table 1.

Since the objective of the paper was to assess alternative methods and models for forecasting Business and Consumer Survey Indicators, we initially considered raw data (in all cases, non seasonally adjusted levels of each category of the variables were used) in order to test the forecast accuracy of the five different sets of models presented in the previous section.

In Before showing the results of this comparative exercise, it should be stressed that the statistical properties of the Business and Consumer Survey Indicators differ substantially from those of the main macroeconomic variables (GDP, CPI, Industrial Production, Industrial Producer Prices, etc.). In Table 2, the variation coefficient is shown. It is worth noting that the variation coefficient values are extremely high for some indicators. This can be interpreted as evidence of the high volatility of the indicators and, with this in mind, the forecast accuracy of the methods considered can be expected to be lower than that for other macroeconomic variables.

A further result of interest is that if we examine the average values of the variation coefficient for categories (i.e., positive answers, negative answers, balance, etc.), the highest value corresponds to the balance. A possible explanation for the higher variance of the balance is purely statistical. The variance of the balance can be decomposed as follows:

) , ( • 2 ) ( ) ( ) ( ) ( m p Cov m Var p Var m p Var b Var + = = .
So, if the covariance between positive and negative answers is negative (i.e. they move in opposite directions), the variance of the balance would be higher than the sum of the variances of the other two components. In 87% of cases, the sign of this covariance is negative, so the variance of the balance is higher (in most cases) than 3a). To test whether it was significantly higher, we applied a test of equality of variances between the balance and the other two components. In 76% of the cases, the null hypothesis of equality of variances was rejected (Table 3b).

Additionally, we computed some of the most commonly used methods to test the unit root hypothesis: the augmented Dickey and Fuller test (1979), the Phillips and Perron test (1988) and the Kwiatkowski, Phillips, Schmidt and Yongcheol test (1992) (Table 4a). The Perron test (1989) was also adapted so as to allow for different types of structural change: in the level (crash model) or in the slope (breaking trend) (Table 4b). Since the variables under consideration can only take values between 0 and 100, a priori we would expect most of them to be I(0). The most striking result is that in many cases the variables are eventually considered to be I(1) -85% if the structural break is not considered and 52% when it is 7 .

Results: forecast accuracy comparison

Main results for raw data

In Indicator is obtained by weighting the answers to different questions on the survey.

When dealing with these aggregates, there are two possible courses of action:

forecasting them directly or obtaining the forecast by weighting the forecasts for the different components. To the best of our knowledge, no research has been undertaken in order to determine which method provides the best results. Therefore, we chose to forecast composite indicators in two ways: what we call a direct (computation) method and by using the forecasts of the components, an indirect (computation) method.

Among the conclusions that can be extracted from Table 5, we would like to point out the following. First, indirect, as opposed to direct, methods seem to perform best.

Thus, it would appear to be better to forecast these indicators from the forecasts of the components than directly. Second, among the direct methods, the AR model outperformed the rest of the models in almost all cases. By contrast, the ARIMA and TAR models were never found to be the best forecasters. It should also be stressed that unrestricted VAR models usually work better than restricted VAR models and that the errors displayed by the ARIMA, TAR and VAR models for different indicators were generally higher.

Furthermore, taking into account that the variables considered (positive, neutral and negative answers but not the balance) can only take values between 0 and 100, the forecast errors are quite high even in the case of the best model. As expected, the forecasts errors increased for longer horizons in most cases and, in general, variables corresponding to questions with a higher number of possible answers were better forecast than the rest. In most cases, the size of the errors was higher for the balance than for the components (this is related to the higher volatility of these variables), which is a common result for most composite indicators (one notable exception is the Economic Sentiment Indicator).

A further aspect to be considered is related to that of balance forecasts. As survey data are derived from qualitative questions and based on subjective evaluation, the results are usually presented in terms of balances, which show the difference between positive and negative percentages of answers. The balance is the information that analysts take into account and the information that is usually forecast. But, is it better to forecast the balance directly or to forecast negative and positive answers and then calculate the balance? In order to answer this question we replicated the forecasting 6, referred to as 'b') show that it is usually better to forecast the balance from the forecasts of positive and negative answers rather than by doing it directly, and that the AR model outperforms the rest of the models in almost all cases.

The effects of seasonality on data revision and on forecast accuracy

Seasonal adjustment methods are usually applied to these indicators. However, an interesting point that has not been analysed to date is the extent to which the method chosen for seasonal adjustment -TS, X12, DA, WAV, among others -might affect the values of the series under consideration. In order to remedy this situation, we applied all these methods of seasonal adjustment so as to obtain seasonally adjusted data (using TS, X12 and DA) and trend cycle estimations (using TS, X12 and WAV)

for a number of qualitative indicators 8 . From the results shown in Table 7, several conclusions can be drawn:

First, TS and X12 results were very similar but DA results (seasonal adjustment) and WAV results (trend cycle-estimation) differed greatly. Specifically, differences were found to be greater in trend-cycle estimations than in seasonal adjustments (Table 7a and 7b).

Second, in order to analyse how the seasonal adjustment method chosen affected the revision of the series, seasonally adjusted data and trend-cycle estimations were computed in a recursive way adding one more observation from 2001.12 to 2002.12.

The results (see Table 7c and7d) showed that:

• There was no relationship between the size of the revision in a given observation and the number of time periods between this observation and the last observation available.

• There were great differences in the size of revision for the different variables in each seasonal adjustment method considered.

• There were no revisions of seasonally adjusted data using the DA method.

• TS revisions of seasonally adjusted data were greater than X12 revisions.

• Trend-cycle estimations using WAV method showed dramatic revisions.

• TS revisions of trend-cycle data were greater than X12 revisions.

Third, the choice of seasonal adjustment method was found to affect forecasts. Thus, we were interested in determining whether it was better to forecast raw data and then apply a seasonal adjustment method or not. As pointed out by Commission staff, the information from Business and Consumer Surveys has little seasonality, and, therefore the results should not vary significantly. To confirm this, we computed the Kruskal-Wallis test (also in Table 7e) for all the qualitative variables in order to verify the importance of seasonality. In almost 87% of cases, the null hypothesis of non-seasonality was not rejected, that is, most series did not present a seasonal component. This, it would seem, despite the large number of studies discussing this matter, that too much attention has been given to this issue in the context of Business and Consumer Surveys.

The effects on forecast accuracy of removing outliers using TS

The presence of outliers may well affect the results discussed in the previous sections. Therefore, TS can be used in order to remove the outliers from the original series. TS uses an automatic procedure to detect and eliminate outliers from a series [START_REF] Gómez | Programs TRAMO (Time Series Regression with ARIMA Noise, Missing Observations, and Outliers) and SEATS (Signal Extraction in ARIMA Time Series) Instructions for the User[END_REF]. Three different types of outliers are considered here:

Additive outliers (AO), Transitory changes (TC) and Level shifts (LS).

In Table 8a the variation coefficient for the qualitative variables from the industrial survey and the balances for the other surveys are shown. From these statistics, it can be seen that for some series, TS did not detect any outliers (i.e., the Economic Sentiment Indicator). However, in most cases, the volatility was substantially lower, although in some the value of the variation coefficient increased (although the standard deviation was lower, the value of the mean -which is clearly affected by the presence of extreme values-was even lower). As before, we now computed the value of the covariance between positive and negative answers and we tested to see if the variance of the balance differed from that of positive or negative answers. As observed in Table 8b, in 95% of the cases (as opposed to 87% when seasonality was not taken into account), the covariance was negative, so that the null hypothesis of equality of variances between the balance and the other two components (positive and negative answers) was rejected in 82% of the cases (compared with 76%) -see Table 8c-. Finally, the results on the unit root tests did not change at all when the outliers were removed from the variables, so that the main results remained unchanged. Using these data, we repeated the comparison of forecasts as in the previous sections.

The main results are shown in Table 8d. The main conclusions to be drawn from this analysis can be summarised as follows: First, RMSE values were the same or lower in nearly all variables when using data from which the outliers had been removed using TS. However, there were differences between models: while in the AR, ARIMA, TAR and Markov models the values of RMSE were markedly lower than those for raw data, the results for VAR (unrestricted and restricted) were, some times, worse. Finally, the VAR models consisting of different indicators performed much better when the outliers were removed.

Second, when comparing the RMSE for the balance computed from forecasts with positive and negative answers, we observed that the results for the models analysed (AR, ARIMA, TAR and VAR) -once the outliers had been removed -were, in most cases, higher than those for raw data. Similarly, when the composite indicators were computed from forecasts based on their components, here again the models performed worse once the outliers had been removed.

A practical issue that needs to be borne in mind when forecasting these series in real time concerns the effects of incorporating new observations on outlier detection using TS. In other words, is it necessary to remove the effects of outliers each time a new observation is available? To analyse this, we adopted a similar approach to that regarding the effects of seasonal adjustment procedures. In order to evaluate the effects of incorporating new observations, we identified the time periods during which an outlier is found in a recursive way by adding one more observation from 2001.1 to 2002.12. The results of this analysis are shown in Table 9. From these , we can conclude that the number and type of outliers clearly depend on the available sample. Although there was some 'persistence' in the moment and type of detected outliers, some significant changes in the results might also be recorded using TS once an additional observation is included (sometimes due to changes in the underlying model, but also to the dynamics of the series itself). So, if we take into account the results regarding the forecast accuracy of the various models, and although this has the effect of increasing the computational cost, it would appear to be necessary to apply the TS to remove the effects of outliers each time an additional observation becomes available.

In addition, we should stress that when comparing the various seasonal adjustment methods, the TS and X12 results were very similar, but the DA (seasonal adjustment)

and WAV results (trend cycle-estimation) presented marked differences. In fact, these differences were greater for the trend-cycle estimations than the seasonal adjustments, whereas there were considerable differences in the size of revision for the different variables for each seasonal adjustment method considered.

Additionally, after removing the outliers using TS the forecast accuracy of the various methods was similar to that observed with raw data. The number and type of outliers identified with TS clearly depends on the available sample. Although there is some 'persistence' in the moment and type of detected outliers, significant changes can be reflected in the results.

Conclusions

The objective of this paper is to compare time series methods for the short-run forecasting of Business and Consumer Survey Indicators. We analysed a dataset for the Euro area, which included 38 indicators (33 of which were monthly and 5 quarterly) and 6 composite indicators, mostly between January 1985 and December 2002. In order to test the forecast accuracy, we used five different sets of models:

autoregressions, ARIMA, Self-exciting threshold autoregressions, Markov switching regime models and vector autoregressions (traditional VAR and also VAR models considering the joint evolution of different indicators).

As far as the forecast accuracy of the different methods is concerned, in most cases the univariate autoregressions were not outperformed by the other methods. In fact, only the forecast errors from the VAR and Markov models were, in some cases, lower than those from the AR models. It should also be stressed that unrestricted VAR models usually worked better than restricted VAR models and that the errors displayed by the ARIMA, TAR and VAR models comprising a range of indicators were generally higher. However, the size of the forecast errors was high even in the case of the best model.

Interestingly, both in the case of composite indicators and indicators which belong to a balance category, our results show that it is preferable to forecast them indirectly.

As for the effect of the seasonal adjustment methods that are typically applied to these indicators, we used a range of methods (TS, X12, Dainties and Wavelets) that suggest that the use of raw data has little effect on forecast accuracy. A similar 1. More details on the dataset can be found in Table 1. Although the number of observations in any empirical analysis should be ideally as large as possible, the type of analysis performed in this paper is considered to be valid over 50 observations.

2. We provide the AIC since it is one of the criteria most widely used when comparing time-series models. However, the sensitivity of the results has been analysed so as to check whether the results differ significantly if other criteria were used. We have also obtained the Hannan-Quinn criterion as well as the Schwartz criterion (or Bayesian information criterion) for all the models and a wide range of variables. The selected model is the same in all the cases, confirming the robustness of our results.

3. The univariate ARIMA models are usually used in the forecasting literature for comparative purposes when other forecasting methods are analysed (e.g. [START_REF] Debenedictis | A vector autoregressive model of the British Columbia regional economy[END_REF][START_REF] Feng | A SETAR model for Canadian GDP: non-linearities and forecast comparisons[END_REF].

4. We have assumed one year in order to minimise the sum of squared errors since variables in Business and Consumer Surveys only ask for agents' perceptions and expectations of their environment in some next months or as a maximum of one year ahead. Thus, since the short term is the one being analysed in these surveys, this is the one that is taken in this paper when comparing the different time series methods.

5. The Hamilton filter is an iterative procedure which provides estimates of the probability that a given state is prevailing at each point in time given its previous history. These estimates are dependent upon the parameter values given to the for the particular set of estimates used. This filter is then repeated to optimise the log likelihood of obtaining the MLE estimates of the parameters. With the maximum likelihood parameters, the probability of state 0 at each point in time is calculated and these are the probabilities of recession and expansion.

6. An alternative approach would have consisted in imposing the value of P and k instead of estimating them. These models are known as Markov Switching

Autoregressive Models (MS-AR) and, in general, the values of P are 0.7 or 0.8 and the values of k, 0 or 1.

7. It is also interesting to note that all the tests only coincide in 10 series.

8. The seasonality analysis was conducted for variables v1, v2, v3p, v3e, v3m, v3b, v4p, v4e, v4m and v4b. We acknowledge financial support from ECFIN/2002/A3-01. We would like to thank the European Commission for their helpful comments and support. The usual disclaimer applies. -358.2062 -358.1892 -358.1993 v1 -256.5865 -256.5694 -256.5796 -476.0159 -475.9988 -476.009 -225.2675 -225.2497 -225.2603 -396.5351 -396.5181 -396.5282 -419.6729 -419.6559 -419.666 -281.4471 -281.4301 -281.4402 -449.2374 -449.2203 -449.2305 -529.8743 -529.8573 -529.8674 -546.4626 -546.4455 -546.4557 -400.5203 -400.5033 -400.5134 -512 -196.79 -196.7573 -196.777 -182.8637 -182.8311 -182.8508 adjusted v1 -358.2062 -358.1892 -358.1993 -256.5865 -256.5694 -256.5796 -473.4402 -473.4231 -473.4333 -247.1958 -247.1775 -247.1883 -360.8181 -360.8011 -360.8112 -407.2603 -407.2432 -407.2534 -259.5352 -259.5181 -259.5283 -443.5743 -443.5573 -443.5674 -529.8743 -529.8573 -529.8674 Page -4235.1686 -4234.9643 -4235.0858 v2 -1100.2962 -1100.2404 -1100.2736 -1162.5494 -1162.4939 -1162.5269 -1278.3017 -1278.2472 -1278.2796 v12 -1229.053 -1228.9986 -1229.0309 -887.3439 -887.2885 -887.3214 -1028.621 -1028.5652 -1028.5983 -1433.4271 -1433.3756 -1433

  type, so that their statistical properties need to be taken into account when concluding.

  1 lag up to a maximum of 24 -8-(including all the intermediate lags), selecting the model with the lowest Akaike Information Criteria (AIC) value.

  Commission draws up and publishes a wide range of indicators calculated on the basis of the results sent in by more than 40 institutes in 25 countries in the framework of the Joint Harmonised EU Programme of Business and Consumer Surveys. Particular attention is paid to indicators for the Euro area. The EU Programme currently includes surveys for industry, construction, the retail trade, services, investment and consumers.

  , we considered all the information available for the Business and Consumer Survey Indicators in the Euro area. The dataset analysed includes 38 indicators (33 of which are monthly and 5 quarterly) and 6 composite indicators. The starting date of these indicators differs but most of them begin in January 1985 (or in the first quarter of 1985). The latest period to be included in the analysis is December 2002 (or the final quarter of 2002)

  (MAPE) were computed. These values provide useful information for analysing the forecast accuracy of each method, and enabled us to rank the methods according to their values.

  too much attention has been given to this issue in the context of Business and Consumer Surveys.

  new observations, we identified the time periods during which an outlier is found in a recursive way by adding one more observation from 2001.1 to 2002.12. The results of this analysis are shown in Table

  this paper is to compare time series methods for the short-run forecasting of Business and Consumer Survey Indicators. We analysed a dataset for the Euro area, which included 38 indicators (33 of which were monthly and 5 quarterly) and 6 composite indicators, mostly between January 1985 and December 2002. In order to test the forecast accuracy, we used five different sets of models: autoregressions, ARIMA, Self-exciting threshold autoregressions, Markov switching regime models and vector autoregressions (traditional VAR and also VAR models considering the joint evolution of different indicators).

  AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO 05Outlier, TC: Transitory Change, LS: Level shift. 6 No outlier is detected for v3p. 01-88 TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC TC 02-93 TC TC TC 01-97 AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO LS LS LS LS LS LS LS LS LS LS LS LS 03-86 TC TC TC TC TC TC TC TC TC TC TC TC TC 05AO AO AO AO TC AO AO AO AO AO TC TC TC TC TC TC TC TC TC TC TC TC TC 11-88 TC TC TC TC TC TC TC TC TC LS TC

  forecasts?In order to answer these questions, we considered all available data taken from the Business and Consumer Survey Indicators for the Euro area between, in the main, January 1985 and December 2002. The dataset analysed includes 38 indicators (33 of which are monthly and 5 quarterly) and 6 composite indicators. This gives a total of 216 observations for the monthly series and 73 for the quarterly series.1 First, we considered raw data in order to test the forecast accuracy of five different sets of models: autoregressions, ARIMA, Self-exciting threshold autoregressions (SETAR), Markov switching regime models and vector autoregressions (traditional VAR and also VAR models considering the joint evolution of different indicators). Some of the conclusions obtained are related to the high volatility presented by type, so that their statistical properties need to be taken into account when concluding.

  remove them (additive, transitory changes and level shifts) from the original series, and to what extent the effects of outliers should be removed each time a new observation is available.

  1 lag up to a maximum of 24 -8-(including all the intermediate lags), selecting the model with the lowest Akaike Information Criteria (AIC) value. 2

  (4 in the case of quarterly data) selecting the model with the lowest AIC value. The statistical goodness of the selected model was also checked.

  developed a Bayesian approach to estimation.Here, we employ a Markov-switching threshold autoregressive model (MK-TAR) in which we are able to allow for different regime-dependent intercepts, autoregressive variances. The estimation of the models is carried out by maximum likelihood using theHamilton (1989) filter 5 together with Kim's smoothing filter(1994).

  termination of the month to which they refer. The survey results are presented either in the form of balances for particular questions or as synthetic indicators. They are able to describe the panorama of the current economic situation much quicker than the quantitative indicators and the macroeconomic magnitudes from national accounts, though the latter tend to be more precise in their descriptions. The European Commission draws up and publishes a wide range of indicators calculated on the basis of the results sent in by more than 40 institutes in 25 countries in the framework of the Joint Harmonised EU Programme of Business and Consumer Surveys. Particular attention is paid to indicators for the Euro area. The EU Programme currently includes surveys for industry, construction, the retail trade, services, investment and consumers.

  , we considered all the information available for the Business and Consumer Survey Indicators in the Euro area. The dataset analysed includes 38 indicators (33 of which are monthly and 5 quarterly) and 6 composite indicators. The starting date of these indicators differs but most of them begin in January 1985 (or in the first quarter of 1985). The latest period to be included in the analysis is December 2002 (or the final quarter of 2002)

  order to evaluate the relative forecasting accuracy of the models, each model was estimated for all the indicators included up to 2000.12 (or 2000.IV for quarterly indicators) and forecasts for 1, 2, 3, 6 and 12 months (or 1, 2, 4 quarters) in the future were computed. The model specifications are based on information up to 2000.12 or 2000.IV and, thereafter, the models were re-estimated in each month or quarter and the forecasts were computed with these estimation results. Given the availability of actual values up to 2002.12 or 2002.III or 2002.IV, we were able to compute the forecast errors for each indicator and method in a recursive way (i.e., for the 1 month forecast horizon, 24 forecast errors were computed for each indicator). In order to summarise this information, the Root Mean Square Error (RMSE) and Mean (MAPE) were computed. These values provide useful information for analysing the forecast accuracy of each method, and enabled us to rank the methods according to their values.

  the sum of variances of the two components (Table

  , but on this occasion we computed the balance from the forecasts of negative and positive answers using the AR, ARIMA, TAR and VAR models. The results (in the last block of Table

  the filter through the entire sample provides a log likelihood value

  AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO AO 05Outlier, TC: Transitory Change, LS: Level shift.1 No outlier is detected for v3p. of variables taken in order to analyse differences between Information Criterion

  Table 5 we present a summary of the recursive forecasts of the main indicators

	F o r
	P
	e
	e r
	R e v
	i e
	w
	(Economic Sentiment -v1-, Industrial Confidence -v2-, Consumer Confidence -v12-,
	Construction Confidence -v28-, Retail Trade Confidence -v33-and Services
	Confidence -v39-Indicators) for the different models. The table refers to raw data.
	For each indicator, the best model is highlighted.

14

Before analysing the results, it should be pointed out that composite indicators are usually calculated from business surveys. For example, the Economic Sentiment

Table 8c -

 8c . Finally, the results on the unit root tests did not change at all when the

	outliers were removed from the variables, so that the main results remained
	unchanged.
	F o r
	P
	e
	e r
	R e v
	i e
	w
	19

Table 1 . List of Business and Consumer Surveys Indicators for the Euro area

 1

	Description

(Continues next page) Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Submitted Manuscript

F o r P e e r R e v i e w

The letters refer to positive answers (pp and p), neutral answers (e), negative answers (mm and m), non answers (n), balance (b) and composite indicators (i).

Table 2 . Variation coefficient for the Business and Consumer Surveys Indicators

 2

		Average variation coefficient
		Monthly indicators	Quarterly indicators
	pp	27.22	13.90
	p	31.33	15.10
	e	16.03	4.38
	m	33.75	17.02
	mm	33.76	2.55
	n	19.52	36.44
	b	348.47	118.68
	i	67.26

Table 3a . Analysis of the sign of the covariance between positive and negative answers

 3a

	Covariance Negative sign	Positive sign	TOTAL
	Month	28	5	33
	Quarter	5	0	5
	TOTAL	33	5	38
	Covariance Negative sign	Positive sign	TOTAL
	Month	74%	13%	87%
	Quarter	13%	0%	13%
	TOTAL	87%	13%	100%

Table 3b . Tests of equality of variance

 3b

	F o r
		P
		e
		e r
		R e v
	Statistic	H 0 : Equality of variance i e H A : Non-equality of variance Rejection of the Null Non-rejection of the Null w	TOTAL
	Month	27	6	33
	Quarter	2	3	5
	TOTAL	29	9	38
	Statistic	Rejection of the Null	Non-rejection of the Null	TOTAL
	Month	71%	16%	87%
	Quarter	5%	8%	13%
	TOTAL	76%	24%	100%

Table 4a . Results of the tests for unit root hypothesis. Unit root tests without structural break. (I(1) in 85% of the cases)

 4a

	Order Variable ADF PP(k=4) PP(k=8) KPSSa KPSSb
	I(0)	pp	3	1	4	9	6
		p	4	4	13	18	10
		e	1	6	4	16	5
		m	4	3	12	21	9
		mm			1	8	3
		n	1	5	1	1	4
		b	4	9	11	24	10
		v1				1	1
		v2	1		1	1	1
	v12 v28 v33 pp p r Total I(0) I(1) F o	18 9 23	28 11 23	47 8 14	1 1 101 3 9	1 50 6 17
		e m mm P n e 25 23 12 11 b 24 v1 1 v2 e r	20 24 12 7 19 1 1	22 15 11 11 17 1	10 6 4 11 4	21 18 9 8 18
	Total I(1) Total general	v12 v28 v33	1 1 1 131 149 R 1 1 1 121 149 e v 1 1 1 102 149	1 48 149	1 1 99 149
					i e
					w

ADF: augmented

Dickey and Fuller test (1979)

; PP:

Phillips and Perron test (1988)

; KPSS:

Kwiatkowski, Phillips, Schmidt and Yongcheol test (1992)

.

Table 4b . Results of the tests for unit root hypothesis. Perron's (1989) test with structural break. (I(1) in 52 % of the cases)

 4b

			Submitted Manuscript
	Order	Variable Crash model (a) Crash model (b) Breaking trend (a) Breaking trend (b)
	I(0)	pp	4		4	5	5
		p	13		12	13	12
		e	17		17	17	17
		m	14		13	12	13
		mm	5		4	4	4
		n	9		8	9	9
		b	11		11	10	10
		v1
	Total I(0)	v2 v12 v28 v33 F o r 1 74		1 70	1 71	1 71
	I(1)	pp p e m mm n b	8 14 9 P 13 7 e 3 17 e r	8 15 9 14 8 4 17	7 14 9 15 8 3 18	7 15 9 14 8 3 18
	Total I(1) Total general	v1 v2 v12 v28 v33	1 1 1 1 75 149	1 1 1 1 79 R e v 149 i e	1 1 1 1 78 149	1 1 1 1 78 149
					w

Page 32 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

Table 5 . Average RMSE -Recursive forecasts from January 2001 to December 2002. Raw data for main composite indicators

 5

							Economic Sentiment Indicator (v1)
	direct	methods	AR ARIMA TAR MK-TAR	1 month 2 months 3 months 6 months 12 months 0.45* 0.77 1.02 1.51 1.52 1.92 3.21 4.36 7.27 6.25 4.65 7.14 9.33 15.07 20.51 0.61 0.75 1.19 2.20 6.12
	indirect	methods	AR ARIMA TAR VAR * Best model	2.73 0.46 0.71 4.38	2.83 0.46* 0.84 4.55	2.92 0.42* 1.06 4.70	3.08 0.41* 1.80 4.98	1.70 0.46* 3.03 3.49
							Industrial Confidence Indicator (v2)
							1 month 2 months 3 months 6 months 12 months
	direct	methods	AR ARIMA TAR MK-TAR	2.03 10.54 21.88 3.68	3.97 21.70 31.29 4.31	5.43 29.85 39.02 7.33	7.99 40.20 61.76 10.93	4.68 39.19 95.72 87.78
					VAR	3.39	5.63	6.90	12.25	17.82
	indirect	methods	AR ARIMA TAR VAR * Best model	2.04 2.21 4.77 0.08*	3.54 4.14 6.78 2.15*	4.68 5.52 8.45 3.29*	6.22 7.60 14.65 3.28*	3.69 7.97 22.69 2.28*
							Consumer Confidence Indicator (v12)
	direct	methods	AR ARIMA TAR MK-TAR	1 month 2 months 3 months 6 months 12 months 1.68* 2.71* 3.65* 5.49 3.77 9.13 13.57 18.02 30.84 25.56 14.31 18.75 23.70 39.32 47.02 2.68 4.42 6.77 10.38 90.78
	indirect	methods	VAR AR ARIMA TAR VAR * Best model	3.02 1.78 5.42 6.09 9.47	5.24 2.79 4.54 6.54 11.66	7.67 3.69 3.93 7.14 13.42	13.62 5.64 3.00* 9.22 15.31	24.60 3.27* 3.67 11.99 12.92
							Construction Confidence Indicator (v28)
	direct indirect	methods methods	1 month 2 months 3 months 6 months 2.01 2.36 2.38 2.97 15.82 26.90 36.08 50.42 26.32 38.53 50.42 73.45 5.00 8.01 11.38 22.41 1.97 2.09* 2.15* 2.82* 12.62 14.07 15.31 18.56 13.06 14.62 16.96 25.21 0.07* 4.93 9.38 15.92 * Best model AR ARIMA TAR VAR AR ARIMA TAR VAR	12 months 2.59* 44.52 100.06 38.31 2.88 22.76 32.69 11.37

Table 6 . Average RMSE -Recursive forecasts from January 2001 to December 2002. Raw data for qualitative indicators according to the type of answer

 6

	pp
		1 month 2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	1.00* 1.48* 1.91* 2.74*	3.23* 0.61*	0.57*	0.61*
	ARIMA	4.15	5.60	6.71	8.79	9.53	1.79	1.85	2.17
	TAR	10.23 24.04 56.91 836.09193363.17	2.55	3.36	4.35
	MK-TAR	1.53	2.01	3.06	5.21	10.94	na	na	na
	* Best model
	p
		1 month 2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	2.07	2.77 3.40* 4.41*	5.48*	1.87*	2.01*	2.29*
	ARIMA	9.94 12.68 15.07 18.91	19.76	6.30	7.46	7.74
	TAR	17.83 24.60 31.22 48.50	72.76	10.95	11.73	17.79
	MK-TAR 1.92*	3.38	4.27 10.00	42.15	3.69	3.76	5.08
	VAR unr	2.00 2.77*	3.48	5.01	6.25	2.48	2.72	2.91
	VAR rest	2.41	3.41	4.31	6.06	6.91	2.46	2.79	2.90
	* Best model
	e
		1 month 2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	2.22	2.87	3.37	4.28	5.21	3.12	3.52	3.47
	ARIMA	10.42 13.05 15.00 17.95	18.30	9.40	10.13	7.64
	TAR	36.59 39.57 42.46 92.09 167.79	12.38	14.58	19.95
	MK-TAR	2.08 2.46*	4.01	5.78	9.27	na	na	na
	VAR unr 1.88*	2.53 3.03* 3.82*	4.42*	1.50*	1.83*	2.19*
	VAR rest	2.30	3.00	3.54	4.64	5.47	1.68	1.91	2.26
	* Best model
	m
		1 month 2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	1.92*	2.77 3.46* 4.75*	5.41*	2.15*	3.13*	3.43*
	ARIMA	9.67 13.72 16.81 22.32	23.98	7.02	10.47	11.00
	TAR	18.11 39.66 51.40 112.63 781.91	10.53	14.04	15.11
	MK-TAR	2.21 2.59*	3.65	5.15	12.09	3.74	4.34	6.92
	VAR unr	2.03	3.00	3.74	5.21	6.19	2.79	3.58	4.24
	VAR rest	2.23	3.15	3.91	5.46	6.39	2.53	3.22	3.61
	* Best model

* Indirect method.

e) Forecasting raw-data or seasonally adjusted data? The Kruskal-Wallis test for detecting seasonality (summary of the results)

Table 8 . The effects of removing outliers using TS a) Average variation coeffcient

 8

		Monthly indicators
	P	28.92
	E	5.86
	M	32.64
	B	979.29
	b) Analysis

of the sign of the covariance between positive and negative answers

	d) The effects on forecasting accuracy b
				1 month 2 months3 months6 months12 months1 quarter2 quarters4 quarters
		AR			Economic Sentiment Indicator (v1) 2.57* 3.64* 4.78* 6.89* 5.39*	7.17	9.18	9.13*
	direct	1 month 2 months 3 months 6 months 12 months 0.45* 0.77 1.02 1.51 1.52 1.92 3.21 4.36 7.27 6.25 4.51 6.72 8.73 14.25 20.04 0.61 0.75 1.20 2.20 6.15 14.55 21.81 27.82 40.06 MK-TAR AR ARIMA TAR ARIMA direct methods 42.39 26.40 37.28 TAR 26.94 39.04 49.91 88.28 138.05 43.16 53.28 MK-TAR 6.54 7.46 8.83 10.71 12.90 11.16 11.49 methods	37.60 53.19 19.76
	indirect	AR ARIMA TAR VAR indirect AR methods VAR * Best model 3.10 4.09 ARIMA 4.08 TAR 5.73 VAR unr 4.18 methods	2.83 0.47 0.77 4.34 4.97 4.56 5.28 6.87 5.43	2.94 0.44* 0.85 5.77 6.00 4.73 6.33 8.14 12.69 0.41* 3.04 1.04 9.48 8.06 4.89 9.14 6.91 9.75	3.22 0.40* 1.61 7.36 6.69* 12.17 5.19 9.39 8.30 20.76 12.20 11.14 11.56	1.84 0.48* 2.34 8.98* 3.67 13.12 13.44 15.68	9.24 16.61 13.61 14.16
	Covariance Negative sign Month 32 Quarter 4 TOTAL 36 F o r 1 month 2 months3 months6 months12 months 1 quarter2 quarters4 quarters Positive sign TOTAL 1 33 1 5 2 38 1.50* 1.95* 2.50* 3.19* 3.38 3.25* 3.88* 4.68* 8.52 11.42 13.65 18.01 15.99 11.22 13.92 15.08 r o VAR rest ARIMA p AR 4.37 5.45 6.68 9.36 7.27 7.77 10.48 F *	10.47
	Covariance Negative sign Month 84% Quarter 11% TOTAL 95% c) Tests of equality of variance P e e r TAR 15.53 20.80 26.23 45.54 MK-TAR 2.02 2.09 2.56 3.34 VAR unr 1.78 2.49 3.28 4.46 P VAR rest 1.76 2.36 3.06 4.20 e * Best model e r	Positive sign 3% 3% 5% 76.03 4.66 4.65 3.14*	20.94 6.10 5.19 4.06	TOTAL 87% 13% 100% 21.29 5.85 6.50 5.01	34.48 10.38 5.87 4.98
	H 0 : Equality of variance H A : Non-equality of variance Rejection of the Null Non-rejection of the Null 29 4 2 3 31 7 R e v i e 1 month 2 months3 months6 months12 months 1 quarter2 quarters4 quarters Statistic TOTAL Month 33 Quarter 5 TOTAL 38 AR e 1.70* 2.30 2.75* 3.33 3.74* 2.51* 2.95* 2.76* ARIMA 8.53 11.41 13.81 16.64 16.20 7.17 7.96 5.72 TAR 13.31 16.98 19.71 27.24 34.88 10.17 12.37 R e v 18.09 MK-TAR 1.70 2.12* 2.96 3.31* 5.03 na na na VAR unr 1.89 2.54 3.02 3.67 4.23 3.42 3.73 3.08 i e
	Statistic Month VAR rest * Best model	Rejection of the Null 76% 4.76 4.83 4.92	Non-rejection of the Null 11% w 5.00 4.40 2.88 w	TOTAL 87% 3.38 3.40
	Quarter			5%		8%	13%
	TOTAL			82%		18%	100%
	m
		1 month 2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR			1.54	2.32	3.07	4.53	3.92		4.30*	6.15	6.27
	ARIMA		8.68 13.16 16.76 22.93	15.55		14.55	21.98	22.36
	TAR	14.48 21.38 28.02 49.10	83.71		22.46	30.87	35.20
	MK-TAR 0.68* 0.63* 0.81* 0.94*	0.66*		6.73	7.65	12.68
	VAR unr		1.70	2.71	3.61	5.40	6.08		6.80	9.48	8.49
	VAR rest		1.56	2.27	2.95	4.45	3.60		4.34	6.00*	6.03*
	* Best model
										40

Best model Page 41 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Submitted Manuscript F o r P e e r R e v i e w 42Table 9 .

 9 The next tables should be interpreted as follows: the number and type of detected outliers when adding a new observation are shown in columns, while in rows, the moment of time for each outlier can be found.

	v2	jan-01	feb-01	mar-01	apr-01	may-01	jun-01	jul-01	aug-01	set-01	oct-01	nov-01	des-01	jan-02	feb-02	mar-02	apr-02	may-02	jun-02	jul-02	aug-02	set-02	oct-02	nov-02	des-02
	01-																			AO AO
	10-																			AO AO
	07-AO AO AO AO AO AO AO		AO			AO												AO
	06-AO AO AO AO AO AO AO		AO			AO												AO
	10-													LS LS LS LS LS LS				LS LS
	11-																		TC
	12-												AO
	01-																			LS LS
	AO: Additive Outlier, TC: Transitory Change, LS: Level shift.

Page 44 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

		Submitted Manuscript
														TC TC
	03-89						AO			TC AO
	05-90									TC
	01-93 AO		AO
	10-98 AO
	03-99 AO AO AO AO AO AO AO AO AO AO AO AO 01-00 AO AO AO AO AO 02-00 AO AO AO AO AO 07-01 AO AO AO AO AO AO AO: Additive Outlier, TC: Transitory Change, LS: Level shift. F o r F o r				AO			AO AO AO AO AO AO AO
	v8e 03-99 AO AO AO AO AO AO AO AO AO AO AO jan-01 feb-01 mar-01 apr-01 may-01 jun-01 jul-01 aug-01 set-01 oct-01 nov-01 P P 12-01 e e AO: Additive Outlier, TC: Transitory Change, LS: Level shift. e r e r	des-01	jan-02	feb-02	mar-02	apr-02	may-02	jun-02	jul-02	aug-02	set-02	oct-02	nov-02 AO AO des-02 AO AO
	7 No outlier is detected for v8p.	R e v R e v
		i e i e
		w w

  Table 5 we present a summary of the recursive forecasts of the main indicators Before analysing the results, it should be pointed out that composite indicators are usually calculated from business surveys. For example, the Economic Sentiment

	F o r
	P
	e
	e r
	R e v
	i e
	w
	(Economic Sentiment -v1-, Industrial Confidence -v2-, Consumer Confidence -v12-,
	Construction Confidence -v28-, Retail Trade Confidence -v33-and Services
	Confidence -v39-Indicators) for the different models. The table refers to raw data.
	For each indicator, the best model is highlighted.
	14

  Third, among the indirect methods, the AR and VAR models were the ones that provided the lowest RMSE (with the exception of the Economic Sentiment Indicator, where the ARIMA model outperformed the rest of the models). Once again, the TAR model presented the highest values for the RMSE. As for the results of the forecast comparison for the remaining indicators, details figures are shown in Table6, in which the average RMSE is shown for each type of answer and where the best model is highlighted in each case. The main conclusions can be summarised as follows. As far as the forecast accuracy of the different methods is concerned, in most cases the univariate autoregressions are not outperformed by the other methods. In fact, only the forecast errors from the VAR and Markov models were lower than those obtained from AR models in some cases.

	of this analysis
	F o r
	P
	e
	e r
	R e v
	i e
	w
	15

67 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Submitted Manuscript
	conclusion is obtained when using data from which outliers have been removed Simpson, P.W., Osborn, D.R. & Sensier, M. (2001). Forecasting UK industrial Notes:
	using TS, as here again forecast errors were found to be similar for nearly all production over the business cycle, Journal of Forecasting, 20, 405-424.
	variables.
	Sims, C. (1982). Policy analysis with econometric models, Brookings Papers on
	Economic Activity, 1, 107-164.
	Stock, J.H. & Watson, M.W. (1999). A comparison of linear and nonlinear univariate
	F o r F o r F o r F o r F o r F o r models for forecasting macroeconomic time series. In: R.F. Engle & H. White (Eds.), Cointegration, Causality, and Forecasting. A Festschrift in Honour of Clive W.J. Granger (pp. 1-44). Oxford: Oxford University Press. F o r
	P P P P P P P
	e e e e e e Stuart, O.D.J. (1985). Are Business and Consumer Surveys Still of Value?, Journal e
	e r e r e r e r e r e r for Studies in Economics and Econometrics, 0, iss. 23, 29-50. e r
	R e v R e v R e v R e v R e v R e v R e v
	i e i e i e i e i e i e i e
	w w w w w w w
	22 27

Page

Page 74 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Submitted Manuscript
	F o r P e e r R e v i e w
	30

Table 1 . List of Business and Consumer Surveys Indicators for the Euro area

 1

	Description

Table 2 . Variation coefficient for the Business and Consumer Surveys Indicators

 2

		Average variation coefficient
		Monthly indicators	Quarterly indicators
	pp	27.22	13.90
	p	31.33	15.10
	e	16.03	4.38
	m	33.75	17.02
	mm	33.76	2.55
	n	19.52	36.44
	b	348.47	118.68
	i	67.26

Table 3a . Analysis of the sign of the covariance between positive and negative answers

 3a

	Covariance Negative sign	Positive sign	TOTAL
	Month	28	5	33
	Quarter	5	0	5
	TOTAL	33	5	38
	Covariance Negative sign	Positive sign	TOTAL
	Month	74%	13%	87%
	Quarter	13%	0%	13%
	TOTAL	87%	13%	100%

Table 3b . Tests of equality of variance (

 3b

	F o r
		P
		e
		e r
		1 ) w N , 1 N Y X F S e 2 Y 2 X S F = R v ) H 0 : Equality of variance ( 2 Y 2 i e X = H A : Non-equality of variance
	Statistic	Rejection of the Null	Non-rejection of the Null	TOTAL
	Month	27	6	33
	Quarter	2	3	5
	TOTAL	29	9	38
	Statistic	Rejection of the Null	Non-rejection of the Null	TOTAL
	Month	71%	16%	87%
	Quarter	5%	8%	13%
	TOTAL	76%	24%	100%

Table 4a . Results of the tests for unit root hypothesis. Unit root tests without structural break. (I(1) in 85% of the cases)

 4a

	Order	Variable ADF PP(k=4) PP(k=8) KPSSa KPSSb
	I(0)	pp	3	1	4	9	6
		p	4	4	13	18	10
		e	1	6	4	16	5
		m	4	3	12	21	9
		mm			1	8	3
		n	1	5	1	1	4
		b	4	9	11	24	10
		v1				1	1
		v2	1		1	1	1
	v12 v28 v33 pp p r Total I(0) I(1) F o	18 9 23	28 11 23	47 8 14	1 1 101 3 9	1 50 6 17
		e m mm P n e 25 23 12 11 b 24 v1 1 v2 e r	20 24 12 7 19 1 1	22 15 11 11 17 1	10 6 4 11 4	21 18 9 8 18
	v12 v28 v33 ADF: augmented Dickey and Fuller test (1979); PP: Phillips and Perron test (1988); KPSS: 1 1 1 1 1 1 1 1 1 1 1 1 Total I(1) 131 121 102 48 99 Total general 149 149 149 149 149 R e v Kwiatkowski, Phillips, Schmidt and Yongcheol test (1992). i e
					w

Table 4b . Results of the tests for unit root hypothesis. Perron's (1989) test with structural break. (I(1) in 52 % of the cases)

 4b

	Order	Variable Crash model (a) Crash model (b) Breaking trend (a) Breaking trend (b)
	I(0)	pp	4	4	5	5
		p	13	12	13	12
		e	17	17	17	17
		m	14	13	12	13
		mm	5	4	4	4
		n	9	8	9	9
		b	11	11	10	10
		v1
		v2	1	1	1	1
		v12
		v28
		v33
	Total I(0)		74	70	71	71
	I(1)	pp	8	8	7	7
		p	14	15	14	15
		e	9	9	9	9
		m	13	14	15	14
		mm	7	8	8	8
		n	3	4	3	3
		b	17	17	18	18
		v1	1	1	1	1
		v2
		v12	1	1	1	1
		v28	1	1	1	1
		v33	1	1	1	1
	Total I(1)		75	79	78	78
	Total general		149	149	149	149

Page 79 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Submitted Manuscript
	F o r
	P
	e
	e r
	R e v
	i e
	w

Table 5 . Average RMSE -Recursive forecasts from January 2001 to December 2002. Raw data for main composite indicators

 5

							Economic Sentiment Indicator (v1)
							1 month 2 months 3 months 6 months 12 months
	direct	methods	AR ARIMA TAR MK-TAR	0.45* 1.92 4.65 0.61	0.77 3.21 7.14 0.75	1.02 4.36 9.33 1.19	1.51 7.27 15.07 2.20	1.52 6.25 20.51 6.12
	indirect	methods	AR ARIMA TAR VAR	2.73 0.46 0.71 4.38	2.83 0.46* 0.84 4.55	2.92 0.42* 1.06 4.70	3.08 0.41* 1.80 4.98	1.70 0.46* 3.03 3.49
						* Best model
							Industrial Confidence Indicator (v2)
							1 month 2 months 3 months 6 months 12 months
	direct	methods	AR ARIMA TAR MK-TAR	2.03 10.54 21.88 3.68	3.97 21.70 31.29 4.31	5.43 29.85 39.02 7.33	7.99 40.20 61.76 10.93	4.68 39.19 95.72 87.78
					VAR	3.39	5.63	6.90	12.25	17.82
	indirect	methods	AR ARIMA TAR VAR	2.04 2.21 4.77 0.08*	3.54 4.14 6.78 2.15*	4.68 5.52 8.45 3.29*	6.22 7.60 14.65 3.28*	3.69 7.97 22.69 2.28*
					* Best model
							Consumer Confidence Indicator (v12)
							1 month 2 months 3 months 6 months 12 months
	direct	methods	AR ARIMA TAR MK-TAR	1.68* 9.13 14.31 2.68	2.71* 13.57 18.75 4.42	3.65* 18.02 23.70 6.77	5.49 30.84 39.32 10.38	3.77 25.56 47.02 90.78
					VAR	3.02	5.24	7.67	13.62	24.60
	indirect	methods	AR ARIMA TAR VAR	1.78 5.42 6.09 9.47	2.79 4.54 6.54 11.66	3.69 3.93 7.14 13.42	5.64 3.00* 9.22 15.31	3.27* 3.67 11.99 12.92
					* Best model
							Construction Confidence Indicator (v28)
						1 month 2 months 3 months 6 months 12 months
	direct	methods	AR ARIMA TAR VAR	2.01 15.82 26.32 5.00	2.36 26.90 38.53 8.01	2.38 36.08 50.42 11.38	2.97 50.42 73.45 22.41	2.59* 44.52 100.06 38.31
	indirect	methods	AR ARIMA TAR VAR	1.97 12.62 13.06 0.07*	2.09* 14.07 14.62 4.93	2.15* 15.31 16.96 9.38	2.82* 18.56 25.21 15.92	2.88 22.76 32.69 11.37
				* Best model

Table 6 . Average RMSE -Recursive forecasts from January 2001 to December 2002. Raw data for qualitative indicators according to the type of answer

 6

	pp
		1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	1.00*	1.48*	1.91*	2.74*	3.23*	0.61*	0.57*	0.61*
	ARIMA	4.15	5.60	6.71	8.79	9.53	1.79	1.85	2.17
	TAR	10.23	24.04	56.91 836.09193363.17	2.55	3.36	4.35
	MK-TAR	1.53	2.01	3.06	5.21	10.94	na	na	na
	* Best model
	p
		1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	2.07	2.77	3.40*	4.41*	5.48*	1.87*	2.01*	2.29*
	ARIMA	9.94	12.68	15.07	18.91	19.76	6.30	7.46	7.74
	TAR	17.83	24.60	31.22	48.50	72.76	10.95	11.73	17.79
	MK-TAR 1.92*	3.38	4.27	10.00	42.15	3.69	3.76	5.08
	VAR unr	2.00	2.77*	3.48	5.01	6.25	2.48	2.72	2.91
	VAR rest	2.41	3.41	4.31	6.06	6.91	2.46	2.79	2.90
	* Best model
	e
		1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	2.22	2.87	3.37	4.28	5.21	3.12	3.52	3.47
	ARIMA	10.42	13.05	15.00	17.95	18.30	9.40	10.13	7.64
	TAR	36.59	39.57	42.46	92.09 167.79	12.38	14.58	19.95
	MK-TAR	2.08	2.46*	4.01	5.78	9.27	na	na	na
	VAR unr 1.88*	2.53	3.03*	3.82*	4.42*	1.50*	1.83*	2.19*
	VAR rest	2.30	3.00	3.54	4.64	5.47	1.68	1.91	2.26
	* Best model
	m
		1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR	1.92*	2.77	3.46*	4.75*	5.41*	2.15*	3.13*	3.43*
	ARIMA	9.67	13.72	16.81	22.32	23.98	7.02	10.47	11.00
	TAR	18.11	39.66	51.40 112.63 781.91	10.53	14.04	15.11
	MK-TAR	2.21	2.59*	3.65	5.15	12.09	3.74	4.34	6.92
	VAR unr	2.03	3.00	3.74	5.21	6.19	2.79	3.58	4.24
	VAR rest	2.23	3.15	3.91	5.46	6.39	2.53	3.22	3.61
	* Best model

e) Forecasting raw-data or seasonally adjusted data? The Kruskal-Wallis test for detecting seasonality (summary of the results)

	F o r
		P
		e
		H 0 : non-seasonality H A : seasonality e r
	Month Quarter TOTAL	Rejection of the Null 22 4 26 R e v	Non-148 22 170	TOTAL 170 26 196
	Month Quarter TOTAL	Rejection of the Null 11.22% 2.04% i e 13.27% w Non-75.51% 11.22% 86.73% 100.00% TOTAL 86.73% 13.27%
				41

Table 8 . The effects of removing outliers using TS a) Average variation coeffcient

 8

		Monthly indicators
	P	28.92
	E	5.86
	M	32.64
	B	979.29
	b) Analysis

of the sign of the covariance between positive and negative answers

					Page
	Covariance Negative sign	Positive sign	TOTAL
	Month		32	1	33
	Quarter	4	1	5
	TOTAL	36	2	38
	Covariance Negative sign	Positive sign	TOTAL
	Month		84%	3%	87%
	Quarter	11%	3%	13%
	TOTAL	95%	5%	100%
	c) Tests of equality of variance
			H 0 : Equality of variance
		H A : Non-equality of variance
	Statistic	Rejection of the Null	Non-rejection of the Null	TOTAL
	Month	29		4	33
	Quarter	2		3	5
	TOTAL	31		7	38
	Statistic	Rejection of the Null	Non-rejection of the Null	TOTAL
	Month	76%	11%	87%
	Quarter	5%		8%	13%
	TOTAL	82%	18%	100%

86 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

							Submitted Manuscript
	d) The effects on forecasting accuracy b
					1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters
			AR			2.57*	Economic Sentiment Indicator (v1) 3.64* 4.78* 6.89* 5.39*	7.17	9.18	9.13*
	direct	direct methods	1 month 2 months 3 months 6 months 12 months 0.45* 0.77 1.02 1.51 1.52 1.92 3.21 4.36 7.27 6.25 4.51 6.72 8.73 14.25 20.04 14.55 21.81 27.82 40.06 42.39 26.40 37.28 26.94 39.04 49.91 88.28 138.05 43.16 53.28 6.54 7.46 8.83 10.71 12.90 11.16 11.49 MK-TAR AR ARIMA TAR ARIMA TAR MK-TAR methods 0.61 0.75 1.20 2.20 6.15 VAR 3.10 4.34 5.77 9.48 12.17	37.60 53.19 19.76
	indirect	indirect methods	AR ARIMA TAR AR methods VAR ARIMA TAR VAR unr * Best model 4.09 4.08 5.73 4.18 VAR rest 4.37		2.83 0.47 0.77 4.97 4.56 5.28 6.87 5.43 5.45	2.94 0.44* 0.85 6.00 4.73 6.33 8.14 6.91 6.68	3.04 0.41* 1.04 8.06 4.89 9.14 12.69 9.75 9.36	3.22 0.40* 1.61 7.36 6.69* 5.19 9.39 8.30 20.76 12.20 11.14 11.56 7.27 7.77	1.84 0.48* 2.34 8.98* 3.67 13.12 13.44 15.68 10.48	9.24 16.61 13.61 14.16 10.47
	p AR ARIMA *	F o r 1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters 1.50* 1.95* 2.50* 3.19* 3.38 3.25* 3.88* 4.68* 8.52 11.42 13.65 18.01 15.99 11.22 13.92 15.08 r o F
	TAR MK-TAR VAR unr VAR rest * Best model 15.53 2.02 1.78 1.76	P e e r 20.80 26.23 45.54 2.09 2.56 3.34 2.49 3.28 4.46 P 2.36 3.06 4.20 e e r	76.03 4.66 4.65 3.14*	20.94 6.10 5.19 4.06	21.29 5.85 6.50 5.01	34.48 10.38 5.87 4.98
	e AR ARIMA TAR MK-TAR VAR unr	R e v i e 1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters 1.70* 2.30 2.75* 3.33 3.74* 2.51* 2.95* 2.76* 8.53 11.41 13.81 16.64 16.20 7.17 7.96 5.72 13.31 16.98 19.71 27.24 34.88 10.17 12.37 R e v 18.09 1.70 2.12* 2.96 3.31* 5.03 na na na 1.89 2.54 3.02 3.67 4.23 3.42 3.73 3.08 i e
	VAR rest * Best model	4.76	4.83	4.92	5.00	w 4.40 w	2.88	3.38	3.40
	m
				1 month2 months3 months6 months12 months 1 quarter2 quarters4 quarters
	AR				1.54	2.32	3.07	4.53	3.92	4.30*	6.15	6.27
	ARIMA		8.68	13.16	16.76	22.93	15.55	14.55	21.98	22.36
	TAR		14.48	21.38	28.02	49.10	83.71	22.46	30.87	35.20
	MK-TAR 0.68*	0.63*	0.81*	0.94*	0.66*	6.73	7.65	12.68
	VAR unr		1.70	2.71	3.61	5.40	6.08	6.80	9.48	8.49
	VAR rest		1.56	2.27	2.95	4.45	3.60	4.34	6.00*	6.03*
	* Best model

Best model Page 88 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Submitted Manuscript F o r P e e r R e v i e w 44Table 9 .

 9 The next tables should be interpreted as follows: the number and type of detected outliers when adding a new observation are shown in columns, while in rows, the moment of time for each outlier can be found. Additive Outlier, TC: Transitory Change, LS: Level shift.

	v2	jan-01	feb-01	mar-01	apr-01	may-01	jun-01	jul-01	aug-01	set-01	oct-01	nov-01	des-01	jan-02	feb-02	mar-02	apr-02	may-02	jun-02	jul-02	aug-02	set-02	oct-02	nov-02	des-02
	01-																			AO AO
	10-																			AO AO
	07-AO AO AO AO AO AO AO		AO			AO												AO
	06-AO AO AO AO AO AO AO		AO			AO												AO
	10-													LS LS LS LS LS LS				LS LS
	11-																		TC
	12-												AO
	01-																			LS LS
	AO:

restricted Page 93 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Page	DIRECT METHODS INDIRECT METHODS VAR unrestricted on Indicators raw data monthly F AR Worksheet: direct AR VAR Worksheet: direct VAR AR Worksheet: indirect VAR VAR Worksheet: indirect VAR o r P e e r R Composite raw data adjusted All composite indicators v1 v2 v12 v28 v33 v39 v1 raw and adjusted Composite raw data v1 v2 v12 v28 v33 v39 adjusted v1 Composite raw data v1 v2 v12 v28 v33 v39 adjusted v1 e v i e w Indicators raw data monthly quarterly Worksheet: VAR restricted v3 v10 v9 v11 v16 v29 v36 Indicators raw data monthly Worksheet: VAR unrestricted v4 v8 v16 v32 v34 v38 F o r P e e r R e v i e w quarterly v3p v10b v4e v25b v7p v27b v13p v15p v19p v23p v31p v37p v13e v18e v20e v29e v34e v37e v43e VAR Submitted Manuscript Submitted Manuscript	Page 94 of 106

Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

95 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Submitted Manuscript
	F o
	r
	P
	e e r
	R e v i e w

Page 96 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Page 97 of 106 Page 99 of 106					Submitted Manuscript Submitted Manuscript Submitted Manuscript Submitted Manuscript	Page 98 of 106
	COMPOSITE Akaike information criteria COMPOSITE COMPOSITE	AIC AIC		SIC SIC	HQC HQC	AIC
	raw Schwarz criterion or Bayesian information criterion AIC SIC HQC raw	SIC
	v1 Hannan-Quinn criterion raw -3938.2663 -3938.0887 -3938.1971 -417.5935 -417.5755 v1	-417.5862	HQC
	adjusted	v2 -3863.5138 -3863.3362 -3863.4446 -325.6424 -325.6254	-325.6355
		v12		-306.8569		-306.8399	-306.85
		v28		-452.8738		-452.8568	-452.8669
		v33		-412.751		-412.7332	-412.7438
	AIC	v39 Akaike information criteria -135.0856		-135.053	-135.0727
	SIC	Schwarz criterion or Bayesian information criterion
	HQC	adjusted Hannan-Quinn criterion
		v1		-417.5935		-417.5755	-417.5862
	INDICATORS
		raw
		monthly v3p v4e v7p	F o -414.5567 -269.5258 -373.8947 F o F o F o		-414.5397 -269.5088 -373.8777	-414.5498 -269.5189 -373.8878
		v13e v13p		r -222.3528 -299.7041 r r r	-222.3344 -299.6869	-222.3453 -299.6972
		v15p v18e		P -233.9184 -233.9014 -224.9946 -224.9765 P P P	-233.9115 -224.9873
		v19p v20e v23p v29e v31p		-319.0595 -221.4247 -270.2258 -531.3928 -379.8651	e e r -319.0425 -221.4077 -270.2083 -531.3758 -379.8481 e e r e e r e e r	-319.0526 -221.4178 -270.2187 -531.3859 -379.8582
		v34e v37e v37p v43e quarterly v10b v25b v27b		-486.5867 -474.9216 -428.2354 -124.3674 -209.9952 -71.1007 -64.7831		R e v i e w -486.5696 -486.5798 -474.9038 -474.9143 -428.2176 -428.2282 -124.3347 -124.3544 -209.9612 -209.9819 -71.0597 -71.0856 -64.7421 -64.768 R e v i e w R e v i e w R e v i e w

Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

100 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Page 101 of 106	Submitted Manuscript Submitted Manuscript Submitted Manuscript
	AIC	Akaike information criteria
	SIC	Schwarz criterion or Bayesian information criterion
	HQC Economic Sentiment Indicator Hannan-Quinn criterion
	F o F o Industrial Confidence Indicator (v7+v4-v6)/3 F o
		r r r
	P P Consumer Confidence Indicator (v14+v16-v19+v23)/4 P
	e e r e e r Construction Confidence Indicator (v30+v31)/2 e e r
	R e v i e w R e v i e w Retail Trade Confidence Indicator (v34-v35+v37)/3 Services Confidence Indicator (v40+v41+v42)/3 R e v i e w

Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Page 102 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

Page 103 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

		Submitted Manuscript
	COMPOSITE	AIC	SIC	HQC
	raw
	v1
		F o
		r
		P
		e e r
			R e v i e w

Page 104 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

	Page 105 of 106			Submitted Manuscript Submitted Manuscript
	INDICATORS INDICATORS		AIC AIC	SIC SIC	HQC HQC
		raw raw
		monthly monthly
		v3 v4	-726.6391 -1162.5494 -1162.4939 -1162.5269 -726.605 -726.6253
		v9 v8	-479.1296 -1184.0465 -1183.9907 -1184.0238 -479.0927 -479.1146
		v16 v16	-487.9228 -715.7464	-487.8888 -715.6953	-487.909 -715.7257
		v29 v32	-999.5321 -1152.9515 -1152.8957 -1152.9289 -999.4981 -999.5183
		v36 v34	-844.0026 -685.9522	-843.9684 -685.8964	-843.9887 -685.9295
		v38	-755.4727	-755.4163	-755.4498
		quarterly
		v10	-285.8194	-285.7514	-285.7927
		v11	-238.1692	-238.1012	.4062 -238.1425
	AIC	Akaike information criteria
	SIC	v28 Schwarz criterion or Bayesian information criterion -887.0438 -886.988 -887.0212
	AIC SIC HQC HQC	-801.3756 -674.325 -756.8375 F o Akaike information criteria v33 Schwarz criterion or Bayesian information criterion -801.3209 -801.3534 -674.2663 -674.3012 -756.7789 -756.8137 Hannan-Quinn criterion o F Hannan-Quinn criterion
			-639.7203 r r	-639.6617	-639.6965
		v39	-407.1731 -417.0883 P -407.0775 -416.9927 P	-407.1351 -417.0503
		adjusted v1	-437.4502 -4154.8225 -4154.6181 -4154.7397 -437.3531 -437.4117 e e r e e r
	AIC SIC HQC	Akaike information criteria Schwarz criterion or Bayesian information criterion Hannan-Quinn criterion R e v i e w R e v i e w

Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK

Page 106 of 106 Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Submitted Manuscript

Editorial Office, Dept of Economics, Warwick University, Coventry CV4 7AL, UK Submitted Manuscript

* Indirect method.