Benjamin David
email: benjamin.david11@hotmail.fr

Model economic phenomena with CART and Random Forest algorithms

Keywords: decision trees, CART, Random Forest JEL Classication: C4, C18, C38, C55

The aim of this paper is to highlight the advantages of algorithmic methods for economic research with quantitative orientation. We describe four typical problems involved in econometric modeling, namely the choice of explanatory variables, a functional form, a probability distribution and the inclusion of interactions in a model. We detail how those problems can be solved by using CART and Random Forest algorithms in a context of massive increasing data availability. We base our analysis on two examples, the identication of growth drivers and the prediction of growth cycles. More generally, we also discuss the application elds of these methods that come from a machinelearning framework by underlining their potential for economic applications.

1 Introduction: some obstacles to econometric modeling

Over the past few years and decades, economics has placed an increasing emphasis on econometric applications. Thus, almost 70% of the articles published in the three main economics journals in 2011 were empirical works [START_REF] Hamermesh | Six decades of top economics publishing : Who and how ?[END_REF]. This orientation toward quantitative methods is driven by digitalization, enhancements of econometric tools and computer capacities growth, which has deeply reoriented economic research. The more common approach in econometric applications is the consideration of a variable Y explained with J predictors X j linked to Y by a specic functional form determining the type of relationship. A simple example is the use of a linear function such as Y = f (X j ) = α + J j=1 β j X j with α and β j as parameters whose values describe the relationship. The statistical work of an economist is to estimate α and β j on the basis of a limited number of realizations (the observations) of the variables Y et X j which could be considered as random variables 1 2 . This type of approach is consistent with the seminal view of [START_REF] Haavelmo | The probability approach in econometrics[END_REF] in which economic phenomena must be studied in probabilistic framework.

The estimation of the parameters of an econometric model could serve three purposes:

• identication of an eect: does the variable X j aect variable Y ?

• quantication of an eect: what is the importance of a measured eect?

• prediction of the Y values on the basis of the estimated model: for some xed values of X j which might be the probable values of Y ?

This probabilistic approach called stochastic data modelling by Breiman (2001b)is the standard for the most part of econometrics works and manuals. It provides interesting quantitative results by using a large collection of tools built on solid mathematical foundations. Their eectiveness explains why this approach is present in other elds such as biology, medical research, engineering or political sciences.

However, from an economist point of view, the use of this approach could be hindered by several obstacles linked to the complexity and the diversity of economic phenomena. Among these diculties, four seem to be prominent.

(i) Choice of explanatory variables

The rst important question in a modeling attempt is the choice of variables to be included in a model. Which are the best suited variables for explaining the variable of interest? A natural solution is to refer to the economic theory that is in line with the proposition of [START_REF] Frisch | Editor's note[END_REF]. Indeed, this founder of econometrics considers that one of the dimensions of this eld is economic theory, in addition to mathematics and statistics. Estimation, that is, a statistical task, relies primarily on quantiable relationships identied on a theoretical basis. Nevertheless, this interesting strategy 1 The probabilistic nature of the model is made visible by the addition of an error term ( ).

Thus, the relationship takes this following form: Y = f (X j ) = α + J j=1 β j X j + 2 In most cases, the stochastic nature of X j variables is called into question for reasons of simplication. The perspective is to consider Y conditional to xed values of X j . faces the diversity of theoretical proposals. From this perspective, a simple example is the study of growth determinants, because recognized models underline the role of many factors behind this dynamic. The standard models constructed by [START_REF] Ramsey | A mathematical theory of saving[END_REF], [START_REF] Solow | A contribution to the theory of economic growth[END_REF] and [START_REF] Swan | Economic growth and capital accumulation[END_REF] consider two factors, which are capital and labor. Other authors identify several additional factors such as learning by doing [START_REF] Romer | Increasing returns and long-run growth[END_REF], education [START_REF] Lucas | On the mechanics of economic development[END_REF], government spending [START_REF] Barro | Government spending in a simple model of endogenous growth[END_REF], research and development [START_REF] Aghion | A model of growth through creative destruction[END_REF] and energy [START_REF] Kümmel | Thermodynamic laws, economic methods and the productive power of energy[END_REF], to name just a few. Furthermore, Sala-I-Martin (1997b) notes that many other predictors were introduced in empirical papers. He founds some 60 variables (growth drivers) with an associated coecient signicantly dierent from 0 at least once in a regression. It should be noted that since this publication, other variables have been used and the number of predictors can be higher if we consider their lagged values. The choice of variables contains a two-fold objective, which is the identication of the good specication in order to avoid the omitted variables bias and the reach of sucient degrees of freedom to ensure feasibility and precision of estimation. This obstacle could be signicant if data used are macroeconomic with annual frequency and the number of possible predictors is large.

(ii) Choice of a functional form

The choice of a functional form refers to the selection of a function of predictor variables able to approximate the observations of an output variable. This is a very important step in econometric modeling because it determines the kind of relationship to estimate; only parameters values will be estimated. For historical reasons, simplicity and conformity to data, a large number of works address the economic process in a linear framework. Indeed, a method such as Ordinary Least Squares (OLS) has been available to economists for a long time; it is accessible for non-specialists and it might be suitable for a certain number of phenomena with a linear pattern. This is also explained by the fact that it is possible to transform a nonlinear model in a linear model by transforming the variables used. This enables a consideration of dierent functional forms by conserving the linearity in the parameters but not in the variables. From this perspective, we can cite the example of the standard Cobb-Douglas function [START_REF] Cobb | A theory of production[END_REF] or many power laws observed in economy, nance or other elds [START_REF] Mandelbrot | New methods in statistical economics[END_REF][START_REF] Gabaix | Power laws in economics: An introduction[END_REF], which can be transformed with a logarithm function.

However, all economic relationships are not linear or linear after transformation, which complicates the identication of an adapted functional form. Indeed, nonlinearity takes on a wide diversity of forms because the relationships between economic variables could include many specic features such as threshold, structural breaks, deceleration or reinforcing. Nonlinearity seems to be at work in many economic topics such as in the link between wage, education and experience [START_REF] Mincer | Schooling, Experience and Earnings[END_REF], in Ricardian equivalence (OCDE, 2015) or in the link between oil prices and the value of the dollar [START_REF] Coudert | Reassessing the empirical relationship between the oil price and the dollar[END_REF]. In order to avoid potential specication errors, it is possible to make a closer inspection of data by using graphical representations, to refer to previous studies on the same topic or to practice specication tests allowing provision of insights on the form of the relationship studied. In despite the precautions, the risks of misspecications are substantial and can produce some imprecisions in the estimations or even some spurious results if the functional form used is signicantly unsuitable.

(iii) Choice of a probability distribution Another important step in econometric modeling is the selection of a probability distribution P (X, Y ) to model the joint distribution of variables. 3 This choice can be the basis of estimation because some approaches are usable only if a specic probability distribution is dened before estimation. This is the case of all models estimated by maximum likelihood. In addition, the choice of distribution can be necessary to apply statistical tests or to compute condence and prediction intervals. For instance, a student test on slopes from a linear regression is relevant under the normality hypothesis. 4 Therefore, this step is important because it conditions the estimated parameters' values and the level of condence in these results. In some cases, the choice could be not too dicult due to the statistical issue. For a logistic regression, it seems logical to consider a Bernouilli distribution when Y is a binary variable, and a multinomial distribution when the number of categories is higher than two. The form of the problem leads naturally to these choices.

It is also possible to remove the risks associated with the choice of unsuitable distribution by studying the residuals of a model through specic statistical tests, which can establish the adequacy of a particular distribution for a given condence level.

On the other hand, economic literature could guide the modeler since specic probability distributions have been identied in particular contexts. For example, it seems that nancial data have singular characteristics such as heavy tails of returns distribution [START_REF] Cont | Empirical properties of asset returns: stylized facts and statistical issues[END_REF]. It means that the number of extreme events (large deviations around the mean) is higher than if returns were normally distributed. For this reason, estimation of (G)ARCH models [START_REF] Engle | Autoregressive conditional heteroskedasticity with estimates of the variance of u.k. ination[END_REF][START_REF] Bollerslev | Generalized autoregressive conditional heteroskedasticity[END_REF]) by quasi maximum likelihood is possible by using a student distribution or a General Error Distribution (GED), which have heavy tails.

However, the use of specic probability distribution could be viewed as a strong hypothesis because all economic phenomena are not precisely specied and the power of statistical tests could be weak with samples with few observations (see for example [START_REF] Jarque | Ecient test for normality, homoscedasticity and serial independence of residuals[END_REF] test). In addition, the diagnostics could be dependent on the threshold chosen for a type 1 error or dierent according the test used. Thus, despite precautions, there can be uncertainty about the probability distribution or it can be impossible to specify it.

(iv) Inclusion of interactions 5

A particular type of nonlinearity occurs when the impact of a variable on the variable to be modeled depends on the levels or the variations of other explanatory variables. Numerous economic questions suppose this kind of conguration. Will a 3 The choice of a functional form already mentioned is a part of this step because the functional form can be viewed as the conditional expectation function of Y given X.

scal stimulus have the same eect depending on the structure of consumption and propensity to save? Will the eect of a development aid policy have the same eect according to the state of the main macroeconomic variables? [START_REF] Morgan | Problems in the analysis of survey data, and a proposal[END_REF] explain that these interactions are common in social sciences and that certain economic variables are sums of interactions by construction.

For example, interest in industrialization of countries or regions returns to study an interactive process between capital accumulation, labor productivity growth, rms and territory reorganizations, evolution of human capital, etc. Thus, to study the impact of one of these factors without taking into account the other is probably an unproductive approach. It is even possible to wonder if the majority of economic variables interact and if separable eects do not fall within the exception. Indeed, economic systems can be viewed as complex systems [START_REF] Arthur | Complexity and the economy[END_REF] that suppose that the elements in an economic system are heterogeneous, in interaction and linked by nonlinear retroactions. Thus, it appears dicult to describe economy with only mechanical, unidirectional and independent relationships.

From a technical point of view, this situation of interaction makes it hard to simply consider an additive function of predictors to approximate the variable of interest.

It becomes necessary to model more complex links with specic patterns such as threshold and symbiotic relationship. In economic research, interactions are taken into account in a variety of ways. One popular method consists of adding interaction terms in regression models. If the eect of a variable X 1 on Y depends on the level of X 2 , one can create an additional variable X 1 × X 2 whose associated coecient captures the interaction. The marginal eect of X 1 would be a function of the X 2 values.

Nonlinear econometric models such as switching regression models can also integrate this kind of dependence (see [START_REF] Teräsvirta | Modelling nonlinear economic time series[END_REF]). In this way, the output variable is approximated by dierent regimes determined by the value of a transition variable. For specic range of values of the transition variable, Y is described by one regime and for other values by other regimes. Thus, the estimated coecients are a function of the regime considered. These approaches are very useful but have a cost in terms of degrees of freedom and in the capacity to interpret the results.

It is therefore dicult to consider a large number of interactions with these methods.

Big data

Obstacles described in the previous lines are inherent to econometric modeling but they can be reinforced by the evolution of the informational context of economic research. Indeed, econometric works depend on the amount of available data and the ability to process them, which have both been signicantly increased. On data availability, Einav and Levin (2014a) summarize the situation: data is now available faster, has greater coverage and scope, and includes new types of observations and measurements that previously were not available. These authors also identify the end of rectangular data (Einav and Levin, 2014b) that correspond to data where the number of variables is less than the number of observations. This new order of magnitude of the applied statistical problems is technically based on the increase in the number of devices capable of digitizing, storing and transforming information (personal computers, servers, smartphones, etc.). The main providers of these new data are mainly governments and rms that compile huge clients, vendors, house-holds or business le bases (Einav and Levin, 2014b).

It should also be noted that national and international institutions are involved in the creation of very large databases. For instance, the International Monetary Fund (IMF) provides very detailed data on international trade with the DOTS database.

For a country like Canada, we can nd all past information on its imports and exports. Data are available for some trading partners over a period of 65 years at a monthly frequency. This is not really "big data" because variables can be observed a few thousand times; however this is an order of magnitude unprecedented for certain thematics.

This huge amount of these newly available data reinforces the obstacles described above, transforming a problem of lack of data in a problem of plenty of data. Indeed, they give the possibility of using new variables and asking new economic questions for which there are no prior expertise. Uncertainty about the relevant variables, the functional form, the underlying probability distribution and the interactions are therefore amplied.

Blind spots and algorithmic solutions

The four obstacles previously described could therefore pose signicant problems in econometric modeling. In some cases, they are marginal issues or they can be circumvented. However, if they are important or if they are combined, there may be some blind spots for empirical analysis because standard econometric tools cannot intrinsically model the considered phenomena. It is possible that some applications cannot be achieved or that certain results cannot be obtained by construction. In this paper, we argue that one possible access to these blind spots relies on the use of decision trees 6 built with the Classication and Regression Tree (CART) [START_REF] Breiman | Classication and Regression Trees[END_REF] and Random Forest (RF) (Breiman, 2001a) algorithms.

Before presenting these approaches, which come from a machine-learning framework, we can rst remove the problem of their legitimacy in economic research. Indeed, the rst formulation of a tree-based method 7 was done by an economist, James Morgan [START_REF] Morgan | Problems in the analysis of survey data, and a proposal[END_REF], who proposed with his coauthor the Automatic Interaction Detector (AID). Furthermore, we can recall that econometrics, even if it is a part of economics, is clearly linked to other elds such as mathematics or computer science. Historically, it has used many tools from other elds. To give just one example, we refer to the statistical concepts and methods created by Ronald A. Fisher for biology or genetics applications, which have become standard for economists (maximum likelihood, etc.). Inversely, some econometric methods have been successfully applied in other elds such as cointegration in climatology [START_REF] Schmith | Statistical analysis of global surface temperature and sea level using cointegration methods[END_REF] or GARCH modeling in hydrology [START_REF] Wang | Stochasticity, nonlinearity and forecasting of streamow processes[END_REF]. Therefore, it seems appropriate to take an interest in nonstandard approaches if they oer good performance and are able to shed light on economic issues. Among them, decision trees have been highly recognized to the point of being considered as a part of Top 10 algorithms in data mining [START_REF] Wu | Top 10 algorithms in data mining[END_REF], and Random Forest, which is an extension of decision trees, has been viewed as one of the most accurate general-purpose learning techniques available [START_REF] Biau | Analysis of a random forests model[END_REF]. Logically, these methods have been applied in a wide variety of elds, such as particle physics (Collaboration, 2012), genetics [START_REF] Goldstein | An application of random forests to a genome-wide association dataset: Methodological considerations and new ndings[END_REF] or computer vision [START_REF] Shotton | Real-time human pose recognition in parts from a single depth image[END_REF].

Despite this multidisciplinary recognition, these methods have curiously only been minimally presented in manuals and barely used in economic papers. (Einav and Levin, 2014a) note, The common techniques in this sort of data miningclassication and regression trees, lasso and methods to estimate sparsemodels, boosting, model averaging, and cross-validationhave not seen much use in economics. Some research on the econpapers database shows, for instance, that only 14 papers using CART were published in an economic peer-review journal over the period 1984-2016 8 9 while this account is based on a large denition of economics, including nancial purposes. A recent counter-example is the Varian's article [START_REF] Varian | Big data: New tricks for econometrics[END_REF] within which the author tried to promote a set of tools from computer science (including decision trees).

Most of the time, the use of these approaches is justied by the achievement of good performance in prediction (in or out sample). This interesting argument (see section 4) must not hide that they are also able to solve technical problems in econometric modeling. It is on this point that we insist in this paper by showing that CART and Random Forest algorithms could overcome the obstacles described in the introduction.

Before starting the presentation of these methods, it appears necessary to provide some clarications. First, this work is not a criticism of standard econometric tools, but instead, it aims to highlight the advantages of other complementary approaches.

Moreover, this work is not a technical 10 and exhaustive presentation. We only take the major features and we leave aside many adjoining and interesting contributions.

Thus, this analysis is solely focused on the base approaches, namely CART [START_REF] Breiman | Classication and Regression Trees[END_REF] and Random Forest (Breiman, 2001a). We don't consider numerous extensions and enhancements such as the Boosting approach (Freund and Schapire, 1996), the Conditional Inference Forest [START_REF] Hothorn | Unbiased recursive partitioning: A conditional inference framework[END_REF], the oblique trees [START_REF] Heath | Induction of oblique decision trees[END_REF], the time series trees and forests (Sela and Simono, 2011 ;Deng et al, 2013), some concurrent algorithms such as ID3, C4-5 [START_REF] Quinlan | Expert systems in the micro electronic age, chapter Discovering rules by induction from large collections of examples[END_REF][START_REF] Quinlan | C4.5: Programs for machine learning[END_REF]) or adaptations to specic topics such as quantile regression [START_REF] Meinhausen | Quantile regression forests[END_REF], survival analysis [START_REF] H Ishwaran | Random survival forests[END_REF], ranking analysis [START_REF] Clémençon | Ranking forests[END_REF], clustering [START_REF] Yan | Cluster forests[END_REF] or online data [START_REF] Denil | Consistency of online random forests[END_REF].

The rest of this paper is organized as follows. Section 2 presents an example of a regression tree built with CART, while section 3 proposes an example of a Random Forest classier dedicated to a prediction issue. Each of these sections includes a description of how these two methods address the modeling barriers discussed in the rst section. Section 4 proposes a discussion on the application elds in economics of the methods and the last section concludes this work.

8 CART keyword was used in econpapers (http://econpapers.repec.org/) for this research. 9 This count includes decision trees built with CART but not RF's use.

10 For more details on theoretical aspects of these two approaches, see for CART [START_REF] Breiman | Classication and Regression Trees[END_REF], [START_REF] Biau | A random forest tour[END_REF] for RF.

Modeling with CART Identication of growth drivers

In order to underline the benets of CART and Random Forest algorithms, we take the example of the Sala-I-Martin (1997b) work on the identication of growth determinants.

11 As previously stated in the introduction, this topic is an excellent il- lustration of model uncertainty because many variables can be considered as growth drivers. These variables are numerous and dicult to prioritize.

The aim of this analysis is to explain the cross-country dierences in terms of production growth between 119 countries 12 with 62 explanatory variables. Data are in cross-section and the variable of interest is the average growth between 1960 and 1992. 13 To address this issue, Sala-I-Martin (1997b) studies the complete distribu- tion of each βj from many models based on dierent combinations of variables. He ranks the variables according to the sizes of the density functions intervals that do not include zero. This topic was also investigated by [START_REF] Fernández | Model uncertainty in cross-country growth regressions[END_REF], who use Bayesian model averaging (BMA) methodology. This method also relies on the estimation of many models but in a Bayesian framework, the variables' importance is established on the base of the posterior probability of each model.

A solution to avoid the problem of model uncertainty is the construction of a regression tree with CART. The basic idea is to split the predictor's spaces into dierent subspaces and to approximate the response of the output variable by its empirical mean in each subspace. On the question of growth determinants, CART produces the tree plotted in gure 1. The interpretation of this estimated model is as follows.

If a country has an openness index (YRSOPEN) of lower than 0.433, its growth rate depends on the value of its life expectancy (LIFE060). If it is under 43.25 years, the growth rate is 0.318 and otherwise the growth rate is equal to 1.532. If a country has an openness index higher than 0.433, predicted growth depends on the fraction of Buddhists in the population.

14 If it is superior to 8.5% the growth rate is equal to 5.571 and it is 2.610 otherwise.

11 We keep the same labels for the variables.

12 Database contains more countries but there are missing values for the growth variable 13 Data used are described in Table 3 in the appendix.

14 The author has some reservations about the culturalist interpretation that can be made of this result. Note that cultural and religious variables are used here only for comparison with (Sala-I-Martin, 1997b) and [START_REF] Fernández | Model uncertainty in cross-country growth regressions[END_REF]. These variables can also be interpreted in a very dierent way. The BUDDHA variable has a high value only for East Asian countries (Japan, South Korea, Taiwan, etc.), which have developed specic development strategies in line with the "ying goose model" described by Akamatsu (1962). The importance of this variable can be interpreted as the success of this strategy. The ease of interpretation is typical for trees built with CART. It doesn't require any statistical or mathematical skills and it is even familiar to economists who are used to tree representations [START_REF] Varian | Big data: New tricks for econometrics[END_REF]. Initial space is the root node (node 1)

and other spaces can be viewed as nodes (t) (or leaves or regions) and we can dene a part of a tree as a branch (T t ). It is also possible to have an analytical representation 15 of a tree which corresponds to a sum of constants, where ŷ(x) is the predicted (tted) value, I t * a dummy variable taking 1 if the terminal node t * is considered, ĉt * is the predicted response in this node and N t equal to the number of cases in node t:

ŷ(x) = t * I t * ĉt * (1)
I t * = 1, if x ∈ t * 0, otherwise and ĉt = ȳt = 1 N t xn∈t y n

Estimation of regression tree with CART

Building a decision tree is a two-stage procedure in which the rst is the succession of binary partitioning producing a tree structure (see gures 2 and 3). The initial space of predictors is split by maximizing the decrease of errors relative to an explanatory variable and a splitting point (s), which are considered the best splitting variable and point. Formally, we have:

max j,s R(s, t) = R(t) -R(t l ) -R(t r ) (2)
R(s, t) is the decrease of errors at node t, t l and t r indexes child nodes. In a regression case, the error of a node is dened as R(t) = 1 Nt xn∈t (y n -ĉt ) 2 . The two child nodes are also partitioned with the same procedure and so on until there is a very large tree (T max ), which must be pruned. 16 Indeed, the risks include arbitrarily setting the size of the tree, constructing too simple tree or creating one that aords a very accurate (or even perfect) adjustment but at the cost of overtting. A satisfying model must be complex enough to identify the structures in sample but it must have 15 Mathematically, this is a simple function.

16 It is possible to set a hyperparameter controlling the number of observations in the terminal nodes before estimation in order to save computing resources. a scope beyond a specic dataset. To prevent this risk of overtting, CART includes a pruning procedure that penalizes (a posteriori) additional partitions. 17 Precisely, the idea is to minimize the cost-complexity criterion (C α (T )), which is the sum of errors in the terminal nodes and the number of splits carried. This criterion takes the following form: 17 There are other pruning methods not presented in this paper. 18 The procedure to identify the value of α is a cross-validation. For more details, readers may refer to [START_REF] Breiman | Classication and Regression Trees[END_REF] or [START_REF] Hastie | The Elements of Statistical Learning: Prediction, Inference and Data Mining[END_REF].

C α (T ) = R (T ) + α| T | (3) R(T ) = t∈ T N t R(t)

Overcoming modeling problems

On the basis of the previous comments, several remarks can be made. The rst is that the problem of the choice of the predictors (obstacle (i)) is solved by using CART. Indeed, the algorithm considers at each split all explanatory variables as a potential splitting variable. In the example, each partition was realized with the variable that minimizes the sum of quadratic errors. This explains why only three predictors are present in the nal tree, while we initially considered 62 variables.

This automatic selection gives a rst indication about the importance of each variable and the hierarchy between them. However, the need to select just one of them at each split could mask important predictors which are slightly less relevant than the selected variable. To overcome these possible masked eects, [START_REF] Breiman | Classication and Regression Trees[END_REF] dene a variable importance index including all variables, even those that were not selected in a nal tree. Variable importance for X j has the following form:

Importance (X j ) = t∈T R sj t , t (4)
sj t corresponds to the value of the splitting point of surrogate (substitute) split closest to the primary split. For each node t, algorithm searches the same partition with others variables than the splitting variable and R(s j t , t) is the decrease of errors of each j variable.

The ranking presented 19 in Table 1 sheds light on the clear hierarchy between all variables because we observe the exclusion of 46 of them. Of the 16 remaining variables, the importance values are also clearly dierentiated. For instance, one can say that the YRSOPEN is the most important predictor or that life expectancy (LIFEE060) is four times more important than urbanization (URB60). Moreover, our application shows that CART is able to take into account the probable masked eects because the third variable in terms of importance is not in the nal tree, which suggests that it had been masked in the splitting process by the selected variables. 20

19 Values are normalized in order to have a sum of importance equal to 100. For this example, the sum is equal to 99 due to rounding. 20 The details of the execution of the algorithm conrm this point. Note 1: The underlined variables are those selected in the nal tree.

Note 2: The variables having importance equal to 0 are not reported.

In view of the dierences in terms of ranking (see Table 1), one can ask the question of how the results obtained with CART are not similar to those of Sala-I-Martin (1997b) and [START_REF] Fernández | Model uncertainty in cross-country growth regressions[END_REF]. A possibility is that the functional form used in these analyses is not adapted to the relationships considered (obstacle (ii)). Indeed, in these works, authors assume without statistical justication that the identication of the growth determinants can be carried in a linear framework. [START_REF] Fernández | Model uncertainty in cross-country growth regressions[END_REF] simply argue that Following the analyses in [START_REF] Levine | A sensitivity analysis of cross-country growth regressions[END_REF] and Sala-I-Martin (1997b) tradition in the growth regression literature, we will consider linear regression models. Thus, the presence of nonlinear relationships is clearly a blind spot for these approaches because they cannot, by construction, grasp them.

The problem is that a close look at the variables considered as the more important in these previous studies for explaining the dierences in terms of average growth do not conrm without doubt the presence of linear patterns (see Figure 4). For instance, if we consider some of the most important variables according to Fernández et al. 22 The results are not reported.

At least it is possible to recognize that there is uncertainty about the functional form adopted in these works that weakens the identication of key variables for explain average growth. Inversely, CART is not subject to this sort of problem because it is a nonparametric method which allows taking into account a large scope of nonlinearities. This algorithm realizes without a priori automatic detection structures by searching the most suitable functional form for a given dataset. Through multiple combinations, the successive splits can model very complex nonlinear relationships.

In addition, it should be noted that CART can reach the conclusion that there is no link between variables. Indeed, the splits must be more informative than costly to be included in the nal tree because the cost-complexity criterion must be minimized. Therefore, it is possible that for a given analysis, any split can compensate for the complexity of the model. This feature is very interesting because it gives the possibility of avoiding the selection of spurious links between the variables. This distinguishes CART from other nonparametric approaches, such as local regressions (LOESS, [START_REF] Cleveland | Robust locally weighted regression and smoothing scatterplots[END_REF]; LOWESS, [START_REF] Cleveland | Locally-weighted regression: An approach to regression analysis by local tting[END_REF], which necessarily propose an estimated model.

Figure 4 Scatter plots

This very exible approach that combines a nonparametric view and automatic variable selection also has the advantage of simplicity in comparison with other methods such as BMA, which relies on Monte-Carlo Markov Chain (MCMC) to select relevant predictors. 23 Furthermore, estimation with CART doesn't need to specify a partic- ular probability distribution. Indeed, from a theoretical point of view, the variable of interest Y and the predictors X j are considered as random variables whose joint distribution are unknown, while their distribution is precisely dened in standard econometric approaches. Breiman (2001b) states that The one assumption made in the theory is that the data is drawn i.i.d. from an unknown multivariate distribution. This fundamental dierence means that some statistical tools are not dened for the case of tree-based method. Thus, there is no likelihood function or parameters to estimate. The ambition, common to algorithmic methods, is just to build, on the base of a learning sample L(Y, X), a function of explanatory variables able to correctly approximate Y without searching to identify the generating process behind data.

On the other hand, the example of the growth drivers gives the opportunity to stress that CART considers all variables as potential interaction variables because each split is included in a sequence of successive partitioning. At each node, CART evaluates the ability of all variables to be the splitting variables and it tests all possible interactions for building a tree. Finally, the eect of variables will depend, in most of cases, on the values taken by other predictors, except if a terminal node is directly linked with the root node.

In our rst application, the observation that the inuence of YRSOPEN depends on the values of LIFEE060 and BUDDHA is done by eliminating the less relevant other potential interaction variables. This constitutes a clear advantage compared to approach using interaction variables. Indeed, this latter strategy could lead to a problem in terms of degrees of freedom because the quantity of interaction variables hugely increases with the number of predictors 24 (number of interaction variables = J!(2!(J -2)!)) -1 ). Inversely, CART can produce a very precise model containing many interactions without any technical diculty. On this basis, we can argue that this algorithm is able to completely solve the obstacle (iv) described in section 1.

3 From trees to forests

Classication and possible trees instability

Numerous contributions have been made to improve decision trees, the main one being Random Forest elaboration. This algorithm is close to the propositions of [START_REF] Ho | The random subspace method for constructing decision forests[END_REF], [START_REF] Amit | Shape quantization and recognition with randomized trees[END_REF], [START_REF] Dietterich | Ensemble methods in machine learning[END_REF], and was nally dened by Breiman (2001a). The formulation of RF addresses several concerns, namely the enhancement of prediction accuracy and the solution of the problem of tree instability. Indeed, in some cases, trees built with CART (or other algorithms) can be 23 Other variables selection methods are employed in economics such as Bayesian Averaging of Classical Estimates (BACE) [START_REF] Sala-I Martin | Determinants of long-term growth: A bayesian averaging of classical estimates (bace)[END_REF] or GEneral TO Specic (GETS) [START_REF] Hendry | We ran one regression[END_REF]. They also have the drawback of only supposing linear relationships.

24 To consider all possible interactions, it should use 1891 interaction variables.

aected by small modications of learning sample, which weakens their ability to predict and to be interpreted [START_REF] Breiman | Bagging predictors[END_REF].

To present RF and underline its qualities, we mobilize another example inspired by the analysis conducted by [START_REF] Osborn | Predicting growth regimes for European countries[END_REF] on growth cycles prediction. Recall that classical cycles correspond to the alternation of periods of expansion and recession, while growth cycles involve the succession of accelerating and slowing production. We pay attention to this topic for the French economy over the period 1978 to 2014 (for more details on data see Table 4 in the Appendix). This example diers from the previous one because the variable CYCLE takes only two possible values (this is a binary variable). We consider the following coding: If growth is higher than growth observed in the previous month, CYCLE is equal to 1. Inversely, if growth is lower than growth observed in the previous month, CYCLE is equal to 0.

For the identication of cycles, we use the chronology established by the Economic Cycle Research Institute

(ECRI).

As for the rst example, the obstacle (i) is obvious because [START_REF] Osborn | Predicting growth regimes for European countries[END_REF] identies about ten variables able to inuence growth cycle and consider, in addition, some lagged values of explanatory variables. This type of conguration can constitute a blind spot for standard econometric tools due to the number of predictors. For instance, in such cases, it is dicult to use a method such as logistic regression because the ratio of events to the number of observations is insucient. A classication tree 27 built with CART is able to overcome this problem by con- structing a model that predicts growth cycle by considering all predictors. The use of this algorithm shows its important exibility because it can work with continuous or categorical as variables of interest. It should also be noted that explanatory variables can be of these two types. CART structure is the same as in the rst example except that each split is not carried according to the decrease of quadratic errors.

For classication purposes, there are three impurity criteria i(t) that can be used at each node t: error classication rate (1-max k (p tk )), Gini index ( K k=1 p tk (1 -p tk )), cross entropy (-K k=1 p tk ln (p tk )). Each node t is split by maximizing the decrease of impurity:

max j,s i(s, t) = i(t) -p l i(t l ) -p r i(t r ) (5)
p tk corresponds to the share of observations belonging to the class k in the node t while p l and p r are the share of cases falling in child nodes l and r. 27 This is a classication tree because the output variable is binary.

of them belong class 1, the model will predict class 1 with a probability of 70%.

Model built with CART 28 is summarized in Figure 5. As in the rst example, the interpretation of the tree is simple. For instance, if at a given month, EZCLI12 is higher than 99.9 and LIR3 is inferior to 11.71, one can predict that growth would be lower than in the previous month ( ĈYCLE = 0).

ĉ4 * = 0 ĉ5 * = 1 ĉ7 * = 1 ĉ12 * = 0 ĉ26 * = 0 ĉ27 * = 1
Aggregating trees as a solution

As noted before, a possible drawback of this kind of model is its potential instability because a perturbation of the learning sample could aect the tree structure. In these cases, prediction and interpretation are therefore questioned, but it should be noted that all trees are not sensible to dataset modications. The instability problem can be solved and more accurate predictions can be obtained by using Random Forest.

The central idea of this CART improvement is to use a collection of trees from B bootstrap samples created on the base of the original learning sample. Note that each bootstrap sample is created with only a fraction of the cases contained in the learning set and one can speak of Out Of Bag(OBB) observations to designate the data not used to generate each sample. The predicted values of each tree are aggregated to obtain the nal prediction of the forest (bagging, [START_REF] Breiman | Bagging predictors[END_REF]).

A key point is that the trees are not correlated because each of them is estimated by randomly selecting only a portion of predictors. 29 It is possible to demonstrate that this way involves a signicant reduction of the variance of the estimation.

28 For this example, we use the Gini index as impurity criterion.

29 As suggested by the presentation of the algorithm, RF doesn't need a pruning procedure contrary to CART. For classication purpose, the default value of m is xed at √ p while it is equal to p 3 or regression task. In practice, it is possible that another value of in the proximity of √ p or p 3 gives better results. In our application, the default value is √ 17 = 4 but we set m = 3 for this reason. This choice is consistent with the rst suggestion of Breiman who proposed to use the following formula: m = log 2 (p + 1) .

The aggregation is dierent according to the type of tree used. In the case of the regression problem, the predicted values are the average responses of all trees and they correspond to the majority of votes (mode) for classication task. 30 In this case, the total number of votes over the number of trees is interpreted as the probability of the event studied. For instance, for a given observation, if three quarters of trees predict the value 0 and P (0) = 0.75 (see Figure 7 to see this distinction between prediction and probability). RF estimation is summarized in Figure 6. However this loss is compensated by accuracy gains and stability of the model. Indeed, beyond a certain number of trees used, errors committed by a forest reach an asymptotic limit that avoids the risk of overtting. Adding new trees in the forest does not improve the quality of the estimation Breiman (2001a).

30 It should be noted that condence intervals can be computed by using Innitesimal Jackknife [START_REF] Wager | Condence intervals for random forest : The jackknife and the innitesimal jackknife[END_REF]. Blue areas represent dierent cycles. Above 0.5, these periods within growth rates are higher than rates observed in the previous month (CYCLE = 0) and under this threshold there are growth rates lower than rates observed in the previous month (CYCLE = 1).

RF algorithm shares with its predecessor CART many advantages such as its nonparametric nature. Any assumption is needed in order to produce a function of explanatory variables able to t the values of the interest variable. A very large scope of functional forms can be considered without specifying any particular probability distribution that avoids obstacles (ii) and (iii). In the case of regression, a gain in precision is obtained in comparison with CART because the latter proposes vertical partitions and constant tted values in each terminal node. Inversely, RF is based on an aggregation of trees which produces smoothed and individualized predicted values. There is a similar situation in classication because some strong constraints that are common to standard methods are relaxed. Indeed, the forms of link function of probit and logistic models force the predicted values in contrast to RF. For instance, a logistic regression will tend to favor values near to 0 or 1, which could be desirable but could also produce an underestimation or overestimation regarding values taken by explanatory variables. Moreover, it should be stressed that while probit and logistic models have nonlinear link functions, they produce linear separations for classifying statistical individuals. On the contrary, CART and RF aord nonlinear separations which allow them to achieve more accurate classication.

Number, selection and importance of variables

RF as CART has the ability to consider a very large number of predictors, even going so far as to have J > N while having very ecient predictions. For example, [START_REF] Breiman | Manual on setting up, using, and understanding random forests v3.1[END_REF] presents a case with a learning sample containing 81 observations, a number of levels of the output variable of 3 and the number of explanatory variables equal to 4,682. This conguration, which is impossible to study with standard econometrics tools, is modeled with RF, which obtains an error rate of only 1.2%.

Beyond this example, comparisons on datasets with very large dimensions (between 701 and 685,569 predictors) showed that RF is on average more accurate that other algorithms. Its relative performance even improves as the number of dimensions increases [START_REF] Caruana | An empirical evaluation of supervised learning in high dimensions[END_REF].

The consideration of many variables is dierent (relative to CART) when RF is used.

For each tree and at each node, the algorithm selects the most relevant splitting variable and splitting point in order to maximize the decrease of errors (or impurity)

of the child nodes. As said previously, the aggregation of the predicted values by tree provides the predicted value by the forest. With CART, non-selected variables do not inuence the predicted response, while RF uses almost all variables to produce predictions. Indeed, RF considers all trees that are constructed with dierent variables such that the probability that each variable would be selected at least one time in one tree is high. Thus, one can say that an RF model gives prediction on the basis of all explanatory variables. 31

To improve understanding of the predictors' hierarchy, RF produces a ranking by importance but dierently to CART. In fact, there are two important measures.

The rst is the Mean Decrease Accuracy (MDA) (Breiman, 2001a) that is obtained for a j variable by computing the dierence between the error rate on OOB sample (e OBB ) and the error rate based on this OBB sample but with j values permuted with the j values of another OBB sample (e OBBj ).

MDA(X j ) = 1 B B 1 (e OBB -e OBBj ) (6)
The logic behind this calculus is that if variable is not important, the permutation of its values should not aect strongly the quality of the estimation.

32 The second measure is the Mean Decrease Impurity (MDI) [START_REF] Breiman | Manual on setting up, using, and understanding random forests v3.1[END_REF] which, for a given variable X j , is the average value of the decrease in errors (or in impurity) on all nodes of all trees where X j is used (j t * is the index of variable used for splitting at node t.). Impurity is dened as Gini index for classication problems and by sum of squared errors for regression.

31 A variable not used corresponds to a variable not selected at any node of any tree.

32 MDA could be normalized by dividing by the standard deviation of dierences.

MDI(X

j ) classication = 1 B B 1 t∈T b I(j t * = j) N t N i(s, t) (7)
or

MDI(X j ) regression = 1 B B 1 t∈T b I(j t * = j) N t N R(s, t) (8)
In the example of growth determinants, computing variables' importance gives the following ranking: These results mainly inform that the four variables situated in the last positions have a very low importance in contrast to other predictors. We remark that there is a gradual decline of variables' importance until the last four which are far from the others. One of these variables (RM1-3) even has a negative value, demonstrating that its inclusion in the model reduces the accuracy of the adjustment. On the other hand, given that RF components are decision trees, this method can take into account interactions between explanatory variables. These interactions are even considered at a very ne level because the search of relevant interactions is done with multiple samples and with a limited number of predictors at each step. Thus, for each tree in the forest, the risk of a masked interaction is reduced due to the low probability of having two near variables in competition in the same tree. However, it is not possible to interpret these interactions as in simple tree because they are numerous and not specially attached to one tree.

Application elds for economic topics

On the basis of our previous analysis, it is possible to argue that CART and Random

Forest algorithms are able to overcome the four obstacles described in the introduction one by one or if they are present jointly. These methods are therefore very useful in order to explore many economic and nancial topics. In the next section, we discuss the application elds of these approaches by identifying the type of issues most suited for their use.

Generality and exibility

Access to blind spots and complex relationships

The ability of tree-based models gives them a feature of generality. Indeed, they are able to cover a large scope of functional forms (even involving highly nonlinear patterns) while neglecting the denition of a specic probability distribution. They can also take into account a large number of continuous or categorical predictors and interactions. As presented before, these characteristics provide solutions for econometric modeling but also suppose a more profound methodological approach. As outlined in section 1, economy can be viewed as one or several complex system(s)

in the technical sense of the term [START_REF] Arthur | Complexity and the economy[END_REF]. This implies that many agents are in relation and react regarding the behavior of other agents, and that the economic variables change by retroactions and nonlinear relationships. All economy and some parts of it can be considered as a complex system (markets, industries, rms...). [START_REF] Arthur | Complexity and the economy[END_REF] argues that because these objects are dicult to analyze, conventional economic theory simplies the issues in order to make possible an analytical approach. It seems that many works using econometric tools proceed in the same way by assuming particular analytical forms in order to make applicable estimation and models available. It appears that decision trees built with CART, and especially Random Forest, can deviate from these technical constraints and thus better account for the "complexity" of economic phenomena in contrast to more conventional methods. A good example is the ability to include a very large number of interactions with strong nonlinearities that happen to be the counterpart of the many complex relationships between economic variables. RF is also able to take into account complexities in the non-theoretical sense of the term. [START_REF] Goldstein | An application of random forests to a genome-wide association dataset: Methodological considerations and new ndings[END_REF] underline that the methods from a machine-learning framework (including CART and RF) are very ecient for this kind of problem: This means these algorithms may be more suited for identifying variants where the causal mechanism is unknown and complex.

Data mining

As stated in section 1 and shown in the two examples, there is often uncertainty on several structural characteristics of economic issues. This could come from diversity, incompatibility or even non-existence of theoretical frameworks. It also can be justied by the contradictory results of previous studies or a strong diculty to specify the economic question. The increasing data availability (big data) is also susceptible to reinforcing this problem.

A response in this context could be an exploratory analysis (data mining) with CART or RF in order to identify the main characteristics contained in data. On this issue, these algorithms are mostly presented as data mining methods in spite of the fact that they are able to do other kind of task. In order to stress this ability, we can recall that these methods can identify the most relevant predictors for regression and classication problems and to hierarchize them. Identication of the key variables, which is a very important question in economic research, can even be at the core of a statistical analysis with tree-based models instead of prediction [START_REF] Goldstein | An application of random forests to a genome-wide association dataset: Methodological considerations and new ndings[END_REF][START_REF] Verikas | Mining data with random forests: A survey and results of new tests[END_REF].

Tree-based models also give the possibility of grasping interactions and the presence of relatively homogeneous subgroups of statistical individuals. This latter capacity is very interesting for economists because data in cross-section and panel forms are very common in economics. This exploratory capacity is therefore particularly suitable for survey data (on employment, rms, etc.) that uses a large number of individuals and statistical variables. In the rst example on the growth drivers, the regression tree identies four subgroups depending on the values of the splitting variables.

Interpretation of tree-based models

The accuracy of in and out-sample predictions of an econometric model is an important quality. However, as said in section 1, economists are also interested in identifying and quantifying the eects of various explanatory variables on output variables. For this reason, they mostly resort to parametric models able to provide the sign and the magnitude of a given eect (by giving a marginal eect). For instance, in a log-log model, the slopes associated with predictors can be interpreted as elasticities and the coecients obtained after a logistic regression can be viewed as odd ratios. These statistical results give the possibility for formulating economic interpretation and policy recommendations.

A possible criticism of the use of tree-based model in economics is related to their supposed limit in terms of interpretation. Indeed, these algorithms that come from the machine-learning eld can be view as black box (Breiman, 2001b) because they propose pertinent predictions but they may appear dicult to interpret. Their complexity can be linked with the large number of used variables, the large number of interactions (and trees for RF) or the absence of parameters and simple analytical form could reduce the attractiveness for economic works. For example, it may seem dicult to understand a forest based on hundreds of trees and characterized by even more interactions.

Firstly, it should be noted that a systematic use of tree-based models does not seem recommendable. The question of the level of prior information on variables and their relationships is fundamental. In the case where the problem is complex and has an unknown form, it seems very interesting to rely on CART or RF in order to perform an automatic structure identication instead of quantifying an uncertain relationship. Inversely, if the statistical issue is well dened, it seems more ecient to use a parametric method. The adjustment to the data would be more accurate and the results would be interpreted in statistical and economic terms. These two approaches can be complementary because it is possible to use a parametric approach after a preliminary analysis with CART or RF. For example, in the case of the tree estimated in section 2, it is obvious that the relationship between average growth and openness index (YRSOPEN) is positive.

Regarding Random Forest, the situation is dierent because it is not possible to summarize the model in a single tree. It is only possible for a given variable to be studied on the basis of all trees how it determines the variable of interest.

However, a more convenient approach for studying the impact of explanatory variables on output variable consists in relying on the partial dependence function (PDF) dened by [START_REF] Friedman | Greedy function approximation: A gradient boosting machine[END_REF]. This kind of object is particularly useful for tree-based models but it can be used for many other type of model. For this reason, [START_REF] Hastie | The Elements of Statistical Learning: Prediction, Inference and Data Mining[END_REF] stress that partial dependence functions can be used to interpret the results of any `black box' learning method. The general idea is to evaluate the mean predicted response of a model for each given value of one of several variables(s) X S . 33 Formally, by considering a subset X S of matrix X which contains predictors S ⊂ 1, 2, . . . , J and a complement set C. Partial dependence function can be estimated by using:

fS (X S ) = 1 N N 1 f (X S , X Cn ) (9)
An important point is that the slope of fS (X S ) may are not necessarily reect marginal eects because it describes how the predicted response changes relative to X S after taking into account the impact of the other variables. The slope of f (X S ) corresponds to marginal eects only when one variable in X S is considered and that any interaction is present that is rare with tree-based models.

For instance, we plot four partial dependence functions from the applications of this work in Figure 8. The rst plot on the left displays the partial dependence of GROWTH over life expectancy (LIFE060). It shows that for any values of LIFE060, the predicted response is positive and that beyond a given threshold (43.25), the predicted value of growth is higher. This very simple conclusion reects the simple structure of the underlying model, which has a limited size with only three explanatory variables and two interactions selected.

When a dependence function is estimated with a large tree or with a RF model, it is probable that very specic relationships will be obtained. If we take the same example but use an RF model (at the top right of the Figure 8), we observe that a growing and nonlinear relationship indicates that the increase or decrease of life expectancy has an eect concentrated in a specic interval (when LIFE060 ∈ (40, 55)). The two other plots in Figure 6 represent two PDF from the second example. It is important to note that in a classication case, the mean response is computed by using the centered logit. These comments stress that the black box expression often associated with statistical models from the machine-learning framework does not seem adapted for tree-based models. Indeed, these approaches are able to identify relevant predictors, hierarchize them and to describe how the variable of interest evolves with values taken by explanatory variables. to the identication of growth determinants and the second concerns the prediction of growth cycles. More generally, we have described the most suitable tasks for these approaches. It seems that CART and RF are particularly ecient for grasping complex patterns in data with a large number of variables, nonlinear relationships and interactions. They also appear adapted to data mining thanks to their ability to automatically detect structure and to make accurate predictions on the basis of a learning sample. Our analysis also stresses that these methods are able to produce models that can be interpreted, which is crucial from an economist's point of view.

On this basis, it seems possible to argue that these tools are very useful for statistical works in economics in complement with other standard approaches. This claim naturally leads to considering the use in economic research of the numerous extensions of these ground algorithms and of other methods from the machine-learning framework, such as neural networks and Support Vector Machines (SVM).

Figure

  Figure 1 regression tree (example 1)

Figure

  Figure 2 split 1

(

  2001) 21 , the scatter plots do not display linear relationships. The joint distribution of GDPSH60 (GDP per capita in logarithm in 1960) which is view as an obvious variable by Sala-I-Martin (1997b) and the growth is almost circular. The possibility of identifying a linear pattern in this case is only based on few observations, that is conrmed by the value of the correlation coecient (r = 0.24). Graphical inspection of the couples GROWTH/CONFUC and GROWTH/SAFRICA completely excludes a linear specication while the relationship between GROWTH and the variable EQINV seems to be slightly instable (approximately logarithmic). This quick evaluation is conrmed by the application of linearity tests which reject the presence of linear patterns. 22 21 These variables are also important according to the Sala-I-Martin (1997b) ranking.

26

  [START_REF] Osborn | Predicting growth regimes for European countries[END_REF] use a variable selection algorithm (n-search algorithm) for choosing the best model among all combinations of variables but with a maximum number of variables of 9. This constraint allows them to obtain interesting estimations, but at the expense of a loss of information due to the exclusion of predictors.

Figure

  Figure 6 RF Algorithm

Figure 7

 7 Figure 7 Out-of-sample predictions

  PredictionCART and RF are naturally oriented toward predictions and are very ecient in this kind of task. Their procedures of estimation and model validation are always based on the ability to predict by using new values not contained in the learning sample. For instance, building a tree with CART involves a step of cross-validation or the use of a test set for choosing the α value, which penalizes additional splits and identies the nal tree. The quality of a model, the variables' importance and the proximity matrix in RF framework are evaluated with OBB cases.On a practical level, these methods are often used for prediction tasks for which very accurate performances are obtained (see examples inBreiman (2001b) or[START_REF] Caruana | An empirical evaluation of supervised learning in high dimensions[END_REF]). Generally, forests are superior to trees and are positioned very favorably compared to other approaches. To stress this ability, it should be noted that[START_REF] Fernández-Delgado | Do we need hundreds of classiers to solve real world classication problems[END_REF] have evaluated the predictions of 179 classication approaches from 17 algorithm families on 121 datasets and that three of the ve best models are RF models. These very convincing performances are added to other interesting characteristics evocated in this paper and are adapted to many economic issues. Examples are the estimation of default probabilities of banks, states, household or the prediction of key variables from a policy point of view such as ination, unemployment or balance of trade.

  Secondly, forests and trees are not perfectly hermetic black boxes which just produce predicted values without giving information on the data structure. The algorithms evoked in this work are both able to carry a variable selection to build a model and to rank all variables considered. Even if they dier in the methodology, these rankings provide information on the capacity of the variables to reduce the prediction errors. It is quite natural to see the existence of causal eects behind these rankings. For this reason,[START_REF] Archer | Empirical characterization of random forest variable importance measures[END_REF] conclude that the RF methodology is attractive for use in classication problems when the goals of the study are to produce an accurate classier and to provide insight regarding the discriminative ability of individual predictor variables. Furthermore, it is possible to add the sign of the eect to importance measures by inspecting the execution of algorithms. For single trees, the detail of their structure is summarized in the associated graphical representations which clearly present the relationships and interactions identied by CART. Thus, single decision trees are highly interpretable[START_REF] Hastie | The Elements of Statistical Learning: Prediction, Inference and Data Mining[END_REF]).

33

  It is also possible to rene the analysis of partial dependence by decomposing the average response by studying the Individual Conditional Expectation (ICE) as proposed by[START_REF] Goldstein | Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation[END_REF]. It also gives the possibility to use median for aggregate individual responses.

Figure

  Figure 8 Partial Dependance Functions

Table 1

 1 Variables' importance (example 1)

		CART		(Sala-I-Martin, 1997b)	(Fernández et al., 2001)
	Variable	Importance	Rank	Variable	Rank	Variable	Rank
	YRSOPEN	19	1	EQINV	1	GDPSH60	1
	LIFEE060	12	2	YRSOPEN	1	CONFUC	2
	DPOP6090	10	3	CONFUC	1	LIFEE060	3
	CIVLIBB	9	4	RULELAW	1	EQINV	4
	ABSLATIT	9	4	MUSLIM	1	SAFRICA	5
	PRIGHTSB	9	4	PRIGHTSB	6	MUSLIM	6
	BUDDHA	7	7	LAAM	6	RULELAW	7
	CONFUC	5	8	SAFRICA	8	YRSOPEN	8
	URB60	4	9	CIVLIBB	8	ECORG	9
	P60	3	10	REVCOUP	10	PROT	10
	SAFRICA	3	10	MINING	11	MINING	11
	S60	3	10	BMP1	12	NONEQINV	12
	H60	3	10	PRIEXP70	13	LAAM	13
	PI6089	1	14	ECORG	14	P60	14
	FRAC	1	14	WARDUM	15	BUDDHIST	15
	HINDU	1	14	NONEQINV	16	BMP1	16

Table 2

 2

			Variables' importance (example 2)
	Variables	MDA	MDA Rank	MDI	MDI Rank	Average rank
	LIR12	17.893	1	10.320	1	1
	OPTG3	16.234	3	9.924	2	2.5
	OECDCLI3	17.091	2	9.155	4	3
	LIR3	15.851	4	7.411	6	5
	SIR12	15.828	5	8.387	5	5
	EZCLI12	15.497	7	9.193	3	5
	SIR3	15.548	6	7.018	7	6.5
	USCLI3	14.538	8	6.454	9	8.5
	OECDECLI3	14.243	9	6.496	8	8.5
	SBF250-12	13.733	10	5.076	13	11.5
	EZCLI3	13.526	11	5.897	12	11.5
	USCLI12	13.393	12	5.963	11	11.5
	OPTG12	13.327	13	6.108	10	11.5
	RM1-12	7.625	14	2.693	14	14
	EXR	5.193	15	2.377	15	15
	SBF250-3	5.038	16	2.032	16	16
	RM1-3	-0.272	17	0.517	17	17

Table 4

 4 Data used in example 2

	Variable	Denition

This is true with nite samples but under additional hypothesis, β converges asymptotically towards the standard normal distribution.

This problem could be included in the choice of a functional form. However, we devote a specic paragraph to this topic due to its importance.

Decision trees are either regression trees or classication trees. Other algorithms than CART can be used to estimate these models.

For a presentation of the history and the dierent types of regression and classication trees, reader could refers to[START_REF] Loh | Fifty years of classication and regression trees[END_REF].

Powered by TCPDF (www.tcpdf.org)

Appendices