Verónica Escudero
email: escudero@ilo.org.

Günther Schmid

Are active labour market policies effective in activating and integrating low-skilled individuals? An international comparison

Keywords: E24, H53, J08, J65, J68 unemployment, employment, participation rate, ALMPs, implementation, startup incentives unemployment, employment, participation rate, ALMPs, implementation, startup incentives. JEL Codes: E24, H53, J08, J65, J68

Are active labour market policies effective in activating and integrating low-skilled individuals? An international comparison

Verónica Escudero

Introduction

This paper examines the effectiveness of ALMPs in improving labour market outcomes, especially for lowskilled individuals. Much has been written about the effectiveness of activation measures based on evaluations carried out using micro data, yet not enough about their effectiveness at the aggregate level. This is particularly true with regards to the role of ALMPs' implementation characteristics in shaping their overall effects. This paper aims to contribute to this debate through an aggregate impact approach which is better placed to measure both, the direct and indirect effects of ALMPs. It finds that ALMPs matter at the aggregate level. Training, employment incentives, supported employment and direct job creation measures show the most favourable results, both in terms of reducing unemployment, but also in terms of increasing employment and participation.

Spending in start-up incentives is more effective but only in terms of reducing the unemployment rate.

Importantly, results show that ALMPs are more effective for the low skilled than for the overall population. In terms of implementation, the analysis finds that the most favourable aspect is the allocation of resources to programme administration. Interestingly, the size of the coefficients arising from policy and implementation variables, once interactions are included is noticeably higher. This demonstrates that a correct implementation of policies enhances their beneficial effect.

Since the 1990s there has been an increased acceptance in the developed world of the need for activation measures to strengthen the link between social protection, labour market policies and employment. Today, these policies are widely regarded as an important tool in fighting unemployment. As a result, expenditure in ALMPs is sizeable in most advanced economies and continues to increase. Success of ALMPs, however, has not been invariably positive. Although some empirical evidence exists that points to a positive effect on the probability of finding employment (e.g. [START_REF] Layard | Unemployment: Macroeconomic Performance and the Labour Market[END_REF][START_REF] Graversen | How to help unemployed find jobs quickly: Experimental evidence from a mandatory activation program[END_REF][START_REF] Lalive | The Effect of Benefit Sanctions on the Duration of Unemployment[END_REF], the effects tend to be relatively small, making it unclear whether the positive outcomes are enough to compensate for the costs.

Moreover, a central concern that remains is whether activation measures are suitable to address longer-term matters such as skills and employability, especially among the least employable jobless individuals.

Importantly, most of what we know today in terms of the effectiveness of activation measures is based on evaluations carried out using micro data. A well-established microeconomic literature exists today on the effects of these policies and their design and delivery systems at the individual level. The overall effects of ALMPs, however, have been less documented. Macroeconomic evaluations of ALMPs have not been carried out systematically yet, at least not taking into account implementation characteristics. In fact, few comprehensive studies centred specifically on the effects of ALMPs have been carried out since the beginning of the 90s and they point to different results with regards to the effect of ALMPs on unemployment rates. While Layard et al. (1991) find that ALMPs have a negative effect on long-term unemployment, the OECD (1993) argues that results are not robust enough to be conclusive. More recently, [START_REF] Estevão | Do Active Labor Market Policies Increase Employment[END_REF] found positive effects of ALMPs on the employment rate of the business sector in the 1990s but not in the late 1980s. Importantly, studies in which the effectiveness of ALMPs is linked to implementation aspects are even scarcer [START_REF] Schmid | Does active labour market policy matter? An aggregate impact analysis for Germany[END_REF][START_REF] Calmfors | Active Labour Market Policy and Unemployment -A Framework for the Analysis of Crucial Design Features[END_REF][START_REF] De Koning | A quantitative approach to process evaluation: the case of the Vermeend-Moor Act[END_REF]. As such, existing knowledge on the aggregate effects of activation measures and their design remains inconclusive.

This paper aims to contribute to this debate in the literature. The analysis is done by ways of a pooled cross country and time-series analysis based on 31 advanced countries for which detailed annual data on different ALMPs (focusing on those specifically targeted to low-skilled individuals) exists for the period 1985-2010. The study contributes to the empirical evaluation of activation policies beyond what it is already known in four ways: First, the paper includes aspects of the delivery system in the analysis to shed light on the role of implementation characteristics in explaining differences in the performance of ALMPs between countries. Second, it is focused on the labour market outcomes of low-skilled individuals that have been among the least researched marginalized groups. Third, it provides an update of the aggregate assessment approach by extending the time and country coverage of the dataset. 1 Finally, the estimation includes specific econometric techniques (i.e.

instrumental variables) aimed to address the endogeneity problem that has weakened many of the ALMP analyses. As it is argued later in the paper, these methods yielded stable results unveiling reliable estimates of the overall net effect of activation policies in the labour market.

Theoretical justification and transmission mechanisms

From the theoretical point of view, the traditional justification for ALMPs has been to reduce labour market imbalances and counteract rigidities and distortions. This comes from the recognition that governments cannot address sustainably unemployment through demand expansion alone (Bellmann and Jackman, 1996a). ALMPs are therefore needed, first, to facilitate the matching process between the supply and demand for labour so that a given number of job-seekers is associated with fewer vacancies; second, to maintain the level of effective labour supply by keeping the long-term unemployed and other groups of "outsiders" tight to the labour force [START_REF] Layard | Unemployment in Britain[END_REF]Layard et al. 1991); third, to affect the demand for labour therefore increasing the number of available jobs [START_REF] Pissarides | Equilibrium Unemployment Theory[END_REF]; and fourth, to boost the productivity of the labour force, both through the direct effect of activation measures on programme participants, but also through general productivity increases associated with externalities.

The direction and magnitude of these effects would depend on the specific type of measure put in place and its target group (Appendix 1). In general:

•

According to Keynesian theory, measures aimed to overcome structural labour market imbalances (i.e.

employment incentives, job-search assistance, public job creation and certain other direct-job-creation measures such as those offering hiring credits) can increase the level of employment through an income effect and a multiplier effect. However, these measures can lead to displacement and substitution effects, when jobs created for a particular category of workers supplant jobs for other categories (OECD, 1993;[START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF].

•

In the case of placement services and all types of job-search assistance, the benefits to the labour market result from an increased effectiveness of search [START_REF] Schmid | Does active labour market policy matter? An aggregate impact analysis for Germany[END_REF], Bellman and Jackman 1996b, OECD, 1993). Job-search assistance might also increase the number of vacancies because opening posts becomes less costly for firms [START_REF] Pissarides | Equilibrium Unemployment Theory[END_REF][START_REF] Calmfors | Macroeconomic effects of active labour market programmes in a Union Wage-Setting Model[END_REF]OECD, 1993), which in turn would expand labour demand. However, some economists predict a reduction in search efforts -and a raise in 1 From around 20 to 25 years and from around 20 to 31 advanced economies.

wage pressure -since government support may reduce the fear of unemployment (Bellman and Jackman, 1996a;[START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF].

•

Labour-supply-oriented measures (including training, workers' subsidies, supported employment and rehabilitation and job rotation and job sharing measures), are expected to have little, if any, impact on the level of unemployment (Schmid, 1996). Yet, these measures will potentially have a stronger impact on the structure of unemployment by reducing skill bottlenecks and the vulnerability of groups that are more exposed to risks in the labour market, such as the unskilled or long-term unemployed (OECD, 1993). This would have a redistributive and reallocating effect of employment opportunities. Search effectiveness can also be improved thanks to these policies, since participating on training courses can provide a positive signal to potential employers, reducing uncertainty about the employability of job applicants (Bellman and Jackman 1996b; [START_REF] Layard | Unemployment in Britain[END_REF]OECD, 1993). More generally, increases in productivity are thought to have positive externalities that would contribute to general productivity increases (OECD, 1993) and to general technical progress of societies [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF]. However, participants may also reduce their search efforts in the expectation that the course culminates (i.e. lock-in effect) (Bellman and Jackman 1996b). Moreover, labour demand can be reduced if the scale effect resulting of an increase in the marginal productivity of labour (i.e. that shifts labour demand upwards because a fall of the relative unit cost of labour provides an incentive to expand output by using more efficient units of labour) is dominated by the substitution effect (i.e. arising since one unit of product can be produced by less units of labour) [START_REF] Calmfors | Active Labour Market Policy and Unemployment -A Framework for the Analysis of Crucial Design Features[END_REF].

•

Specifically relevant for this paper, ALMPs that focus on marginal groups (long-term unemployed, lowqualified individuals, etc.) can be especially effective. They could raise employability, compel the unemployed to search harder for jobs and lead to lower reservation wages, which can stimulate labour demand and facilitate employment (e.g. Bellman and Jackman 1996a). However, in the absence of specific components aimed to raise employability, ALMPs could result in wage moderation and even in an increase in the number of working poor (e.g. [START_REF] Clasen | Beyond Activation: Reforming European Unemployment Protection Systems in Post-Industrial Labour Markets[END_REF]. In general, the weaker the affiliation of targeted group to the labour market, the less likely it would be for activation measures to affect their employment prospects [START_REF] Layard | Unemployment: Macroeconomic Performance and the Labour Market[END_REF]).

•

In general, in times of crisis, characterized by high levels of unemployment and low unfilled vacancies, a given level of aggregate employment needs to be assumed, which would reduce the effectiveness of ALMPs. Given the lack of new available jobs, ALMPs could lead to substitution and displacement effects.

Under these circumstances, a stricter targeting can be justified economically by its potential to affect specific groups even if the aggregate effects of ALMPs are low. In this context, redistributing employment opportunities in favour of disadvantaged groups (e.g. low-skilled individuals) can be a way of enhancing the effectiveness of ALMPs [START_REF] De Koning | Aggregate impact analysis of active labour market policy: A literature review[END_REF].

In terms of the methodologies available for assessing the impact of ALMPs at the aggregate level, several models exist, which have evolved over time. Analyses based on the flow model approach are, by far, the most commonly exploited method to carry out research at the aggregate level in the field of activation policies. These models study the direct effects of ALMPs on the friction between labour supply and demand and the transitions from unemployment to employment (de [START_REF] De Koning | Aggregate impact analysis of active labour market policy: A literature review[END_REF]. The oldest attempts to measure the impact of ALMPs on the efficiency of the labour market were carried from the mid-60s all through the 70s and 80s. The regional version of flow-model approach has been increasingly applied during the last decade.2 Today, this approach constitutes a well-established methodology to measure the net effect of labour market policies exclusively on the matching process (given windfall, substitution and displacement effects on outflows from unemployment) (Bellmann and Jackman, 1996a).

Meanwhile, some few efforts were developed in the late 80s and beginning of the 90s that presented a more general model to study the impact of ALMPs on a number of critical economic and labour market variables [START_REF] Layard | Unemployment in Britain[END_REF]OECD, 1993). This approach allows capturing the impact of ALMPs on the efficiency of the matching process and on the size of the labour force, while also taking into consideration its indirect effects. The model is based on the assumption that ALMPs have the ability to: (i) enhance the employability of labour supply, facilitating recruitment; and (ii) reduce frictions between labour supply and demand, boosting employment. Indirectly, employability increases also influence wage levels, which again affects the labour market and the economy more broadly. As such, this approach is capable of shedding light on the direct impact of ALMPs, while also taking into account productivity and competition effects (insiders vs.

outsiders), deadweight loss, and substitution, crowding-out and lock-in effects (Bellman and Jackman, 1996a;[START_REF] Calmfors | Active Labour Market Policy and Unemployment -A Framework for the Analysis of Crucial Design Features[END_REF]. No other approach allows capturing empirically the overall net effect of ALMPs on the wide labour market.

Empirically, however, evidence from international comparisons on the impact of ALMPs has been contentious.

Historically, a major drawback encountered by the proponents of this cross-country approach is the existence of a simultaneity bias (reverse causality), stemming from the fact that although the scale of ALMP provisions is meant to affect the size of unemployment, unemployment could also drive spending on ALMPs (e.g. if governments base their expenditure decisions on the magnitude of the problem they wish to address).

During the 90s, the quest for addressing this reverse causality focused on defining a medium-term policy reaction function that could realistically predict policy spending patterns of governments. Two clearly opposed theoretical frameworks arose with regards to this issue. The first one assumed that governments based their decisions on a fix level of ALMP spending per unemployed person, which could vary slightly over time but that could not adjust fully with unemployment (Layard et al., 1991). However, measuring ALMP spending per unemployed person would result in an endogeneity problem. Thus, under this scenario, the suggested solution to address the simultaneity problem was to look at average unemployment rates and average levels of spending on ALMPs per unemployed person over the medium-term. If the assumed policy-reaction function were a realistic representation, the policy stance would indeed be exogenous in the proposed scheme. The second theoretical framework assumed that governments committed a given fraction of GDP to ALMP spending over the mediumterm, which did not adjust with the unemployment rate (OECD, 1993). In this situation, ALMP spending was measured as a share of GDP, which authors sustained, did not have an endogeneity problem.

Importantly, different assumptions about governments' policy stance yielded different results with regards to the effect of ALMPs on unemployment rates. While Layard et al. (1991) found that ALMPs had a reducing effect on long-term unemployment, the OECD (1993) argued that results were not robust enough to be conclusive.

Unfortunately, it is still far from clear what a correct representation of the policy-reaction function is and it might well be the case that whatever the representation it would not be the same across countries. Conscious of this problem and of the importance of addressing endogeneity other efforts arose later, which tackled the issue mainly through fixed-effect estimators and instrumental variables [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF][START_REF] Büttner | Does Active Labour Market Policy Affect Structural Unemployment? An Empirical Investigation for West German Regions, 1986 to 1993[END_REF][START_REF] Schmid | Does active labour market policy matter? An aggregate impact analysis for Germany[END_REF][START_REF] Estevão | Do Active Labor Market Policies Increase Employment[END_REF]. 3 However, given the incipient knowledge about estimators capable of dealing with reverse causality and methods to test the instruments' strength and overidentifying restrictions, it remained far from clear whether the endogeneity problem was actually addressed in these studies.

As such, existing knowledge on the aggregate effects of activation measures remains inconclusive. Deepening the understanding of the broader effects of ALMPs is, however, all the more relevant today. First, many advanced countries are under tight budgets and, given the mixed empirical evidence on the effectiveness of ALMPs, spending on these policies is under careful scrutiny. Second, the last decade has seen an important development of tools and methodologies to treat a wider range of empirical problems. For instance, a variety of econometric estimators, technical fixes and diagnostic tests exists today to treat the endogeneity issue. 4 Finally, the availability of longer time series allows controlling better the sensitivity of policies to the business cycle [START_REF] Schmid | Does active labour market policy matter? An aggregate impact analysis for Germany[END_REF]. As such, the potential for capturing the full effects of ALMPs and deepening the understanding of their effectiveness has increased of late.

Empirical specification

Description of the model

Based on these new developments, the present paper is a renewed effort to capture the overall effects of ALMPs at a macroeconomic level. In order to assess how effective ALMPs are in improving labour market outcomes, especially for low-skilled individuals, this paper uses a panel data model based on a structural equation with the following simple form: , ) where, LM represents the selected labour market indicators, including those relative to the low skilled; ALMP denotes active labour market policy indicators and IMPL, indicators relative to implementation characteristics.

= ( , , ,
The remaining three groups of variables are controls: DC includes determinants of demand conditions, STRUC the structure of the labour market (which influences the speed of adjustment to structural change or demand and supply shocks) and INST a range of institutional arrangements.

The analysis consists of an aggregate impact approach based on a pooled cross-country and time-series database for 31 advanced countries with yearly information during the period 1985-2010. The long time series allows increasing the number of observations and providing greater statistical power.

Selection of variables and hypotheses

Dependant variables

Seven indicators were selected as dependant variables. The first three measure labour market outcomes of the overall population and include the unemployment rate, the employment-to-population ratio and the labour force participation rate (LFPR). The first indicator allows measuring the effects of ALMPs on the long-term level of the unemployment rate, but will not say much about whether its reduction is accompanied by higher employment or higher inactivity. To get the complete picture, the participation rate is taken into account to assess the impact of policies in bringing people back to the labour market and the employment rate to evaluate their effect in bringing people back to employment.

Regarding the specific effects of policies on labour market outcomes of low-skilled individuals, as with the first model, I use the unemployment, employment and labour force participation rates of low-skilled individuals as dependant variables. In addition, I include the share of low-skilled unemployed individuals as a percentage of total unemployment to take into account the effects of policies on the "structuralization" of lack of skills in the labour market [START_REF] Schmid | Does active labour market policy matter? An aggregate impact analysis for Germany[END_REF]. This is based on the assumption that the higher the concentration of unemployment on the low-skilled, the higher the real wage rigidity and the higher the persistence of the unemployment rate -i.e. wages at the lower end are not flexible due to labour institutions in place in charge of protecting low-wage incomes [START_REF] Nickell | Would cutting payroll taxes on the unskilled have a significant impact on unemployment?[END_REF]. This last equation would test whether ALMPs are providing incentives to workers for acquiring new skills or to enterprises for enhancing the demand for lowskilled labour.

Policy intervention

As discussed above, a correct representation of the policy stance of governments -on which base it will be anchored the definition of the policy variable -is the first fundamental step to tackle the endogeneity problem that is so common in policy analyses. I assume in this paper that the medium-term policy reaction function of governments regarding ALMP spending is based on a fixed level of expenditure per unemployed individualwhich could be somewhat adjusted based on a cyclical component (Layard et al., 1991). This would imply a positive correlation between total ALMP spending and unemployment but a negative correlation between ALMP spending per unemployed individual and unemployment, which is in fact the case for the panel of countries analysed in this paper.

Following this policy stance, the policy intervention measure computed for the analysis is defined as real expenditure on ALMPs per unemployed person [START_REF] Heylen | Labour Market Structures, Labour Market Policy and Wage Formation in the OECD[END_REF]Bellmann and Jackman, 1996b). 5 Importantly, different policies can produce different effects depending on their objective, design and population targeted (Appendix 1). To take account of these different effects, total ALMP spending was broken down in three different categories, which are explained in detail in Section 4.

Implementation

Moreover, it can be expected that the design and implementation of ALMPs will shape their overall effects.

Indeed, the benefits of policies can be reduced or cancelled if they are badly targeted and implemented, regardless of the expected positive effects and the magnitude of resources allocated (de Koning, 1993) With this in mind, three different aspects of implementation have been included in the analysis. First, I include the magnitude of resources allocated to the implementation of policies (proxied by public expenditure on programme administration, PES). It is to be expected that policies will be more effective in countries that have higher spending on programme administration per unemployed individual, since that would imply that programmes are better resourced and that their administration is better equipped to deliver employment services efficiently.

In addition, continuity and timing of implementation have been included in the analysis. Continuity matters since large fluctuations in public spending (i.e. exceeding cyclical swings) are likely to compromise the stability needed for implementation, which would be detrimental in terms of effectiveness (Schmid, 1996). The timing is important too and higher effectiveness could be expected from policies that are implemented in a countercyclical manner (i.e. spending on ALMPs that runs in opposite direction of the economic trend and parallel to changes in the unemployment rate).

Demand conditions

Moreover, it is assumed in the literature that the overall and low-skilled unemployment rates are determined by demand conditions, which I illustrate in the paper by the growth rate of GDP. It is expected that an increasing demand should reduce the level of unemployment or at least slow down its growth, although the effect may not be immediate.

Structure of the labour market

The structure of national labour markets is taken into account as well since it can influence structural changes and the speed of adjustment to shocks. Two characteristics of the labour market are especially interesting for this analysis. First, I include the concentration of the population on a particular skill level, measured by the share of the population with tertiary education. It can be assumed that the higher the concentration of the high-skilled, the easier it would be for policies to be effective since highly educated individuals have more probabilities to find a job. Moreover, I include the middle-and high-skilled unemployment rates to control for the effects of large unemployment differences across skill groups.6

Institutional arrangements

Controls for differences in institutional arrangements that can affect wage bargaining and macroeconomic performance were also included in the estimation. Union density (the proportion of workforce unionized) is included to control for insider power in wage bargaining, which may push wages upwards at a cost of lower employment especially for groups whose labour supply is more elastic (e.g. low-skilled workers) [START_REF] Layard | Unemployment: Macroeconomic Performance and the Labour Market[END_REF]. Moreover, the OECD index for the strictness of EPL for the layoff of temporary workers was included.

Temporary layoff regulation can reduce search effectiveness of the unemployed since workers who lose their jobs can be recalled, with detrimental consequences on the level of employment (Bellmann and Jackman, 1996b).

Pure control variables

Finally, EU countries have a relatively integrated labour market due to a relative freedom in the movement of workers. To control for this special feature of the European labour market, a dummy variable was added taking the value of 1 for countries that are members of the European Union.

Empirical strategy

Seven different models were estimated to measure the effectiveness of ALMPs. The first three measure the effects of ALMPs on labour market outcomes of the overall population and the other four the effects on the specific target group of this analysis, the low-skilled.7 For each specification, fixed effects, random effects and pooled ordinary least squares (OLS) models were estimated. The suitability of the random-or fixed-effects specification was examined by the Hausman test. In certain cases (employment rate, LFPR, low-skilled employment rate and low-skilled LFPR) a correlation of the entities' error terms with the regressors was found, which invalidated the use of random effects. Fixed-effect models were used in these four cases.

Moreover, serial correlation is usually expected in macro panels with long time series like the one used in this paper, especially as a result of omitting variables that change gradually over time [START_REF] Lusinyan | Work Absence in Europe[END_REF]. I use the Lagrange-Multiplier test [START_REF] Wooldridge | Econometric Analysis of Cross Section and Panel Data[END_REF][START_REF] Drukker | Testing for Serial Correlation in Linear Panel-Data Models[END_REF] and the Arellano-Bond post-estimation technique (abar) [START_REF] Roodman | How to Do xtabond2: An Introduction to "Difference" and "System" GMM in Stata[END_REF] to test for serial correlation in the idiosyncratic error terms. In all cases the null hypothesis was rejected, concluding that the data suffered from first order autocorrelation. In this context, OLS, random-and fixed-effects models are biased and/or inconsistent, since they underestimate standard errors of the coefficients. To account for this, an additional estimator was used in all specifications: a feasible generalized least squares model (GLS) fitted for panel data. This estimator allows for the assessment in the presence of AR1 autocorrelation within panels, cross-sectional correlation and heteroskedasticity across panels.

Results of the pooled ordinary least squares model (OLS), GLS (either fixed-or random-effects), and FGLS with AR1 correction are detailed in columns 1, 2 and 3, respectively, of the tables presented in Appendix 5.

In addition, as discussed above, given the specification of the models and the shape of the policy variables, it can be expected that the different estimations will suffer from reverse causality. Indeed, it is not only ALMPs that affect unemployment but it may also be the case that changes in unemployment could influence expenditure in ALMPs. In this event, it has been widely demonstrated that coefficients estimated through OLS and GLS might be inconsistent and biased. Some authors have dealt with this problem by either normalizing ALMPs to a fixed fraction of GDP over the medium-term or by using country-specific averages of ALMP expenditures over the period analysed. Neither solution is optimal in my view: the former assumes a policy stance that is not a correct representation of reality (at least in the panel of countries used for this study, as discussed above) and the latter eliminates the time varying property of the variables of analysis, something that seems incorrect giving the long period studied. To address the specification problem caused by endogeneity and take account of the presence of heteroskedasticity, a final estimation was carried out instrumenting (i.e. finding variables correlated with the endogenous variables, but not correlated with the error term) policy variables through a 2-stage least squares (2SLS) estimator. The complete discussion of this estimation and the tests and options used are discussed in section 5.3 and presented in Appendix 6.

Data and descriptive statistics

Construction of the database

The variables used in the analysis draw from different sources of information. The exact definitions and sources can be found in Appendix 2. Labour market variables (employment, unemployment, labour force and working age population) for the overall population for the 31 countries analysed in this paper were collected from the Labour Force Survey dataset of OECD.8

The low-skilled unemployment rate and employment-to-population ratio draw from the Eurostat database for the 23 European countries for which information is available in this dataset. To draw information for the low-skilled unemployment rate of the remaining countries, I used ILO databases9 for Australia, Canada, Israel, Mexico and the United States; and national sources for Japan, Korea and New Zealand. Regarding the low-skilled employment rate, I used OECD Education at Glance indicators (2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011)(2012) to gather information for Australia, Israel and Mexico, and national sources for Canada, Japan, Korea, New Zealand and the United States. The LFPR was calculated on the basis of the unemployment and employment rates. Finally, the share of low-skilled unemployed individuals as a percentage of total unemployment was gathered from the World Bank World Development Indicators database, which contains information for the whole sample of countries, although not for all years -e.g. the latest year for which information is available is 2008.

Skill level is measured by the level of educational attainment as defined by the International Standard Classification of Education (ISCED) (UNESCO, 1997). As such, the low-skilled in this paper include individuals with pre-primary, primary and lower secondary education (levels 0-2 of ISCED); the middle-skilled those with upper secondary and post-secondary non-tertiary education (levels 3-4); and the high-skilled those that have finalized tertiary education (levels 5-6). The definition of the variable in countries where information was gathered from national sources varies slightly. In Japan, low-skilled individuals are those with primary school, junior or senior high school; middle-skilled those with junior college; and high-skilled individuals that have coursed college or university, including graduate school. In Korea, low-skilled are middle-school graduates and below, middle-skilled are high school graduates and high-skilled are college and university graduates. In New Zealand, low-skilled are individuals with no school qualification, middle-skilled those with either school qualification or post school but no school qualification and high-skilled those with post school and school qualification. Finally, the low skilled category in Canada and the United States includes individuals with no schooling and persons who received some schooling but did not obtain a secondary or high-school diploma.

Policy intervention variables derive from OECD Employment Outlook databases. Policy categories used in this paper thus follow the OECD classification and definitions. These categories include:

• Training: Consists of targeted training programmes including institutional, workplace and integrated training and special support for apprenticeships. • Job rotation and job sharing: Includes measures that promote the full substitution of an employee by an individual (unemployed or not) from another target group for a fixed period. Job sharing schemes include all measures that encourage the partial substitution of employees by an individual (unemployed or not) from another target group.

• Employment incentives: Includes recruitment and employment maintenance incentives.

• Supported employment and rehabilitation: Consists of subsidies for the hiring of individuals with a long-term or permanently reduced capacity to work, and for the vocational rehabilitation of people with a reduced working capacity.

• Direct job creation: Includes measures aimed to create additional jobs for the long-term unemployed or individuals otherwise difficult to place. Jobs created are usually of community benefit and are usually located in the public or non-profit sectors. This should not include support for lifetime protected work in a non-productive environment.

• Start-up incentives: Includes measures that encourage entrepreneurship among unemployed and other target groups.

Measures included in the analysis are restricted to targeted policies. This excludes measures that are generally available such as in-work benefits for all employees whose income fall below a threshold, or training and apprenticeship programmes that are generally available to employed adults or youth. Appendix 3 illustrates how country-specific policies fit the ALMP categories described above.

For each category, the variable computed was defined as real expenditure per unemployed person, following the policy stance analysis discussed above. For comparability across countries, expenditures were converted to international dollars using purchasing power parity (PPP) exchange rates. In order to deal with the multicollinearity arising between policy variables, a cluster was computed (policy cluster) bringing together training, employment incentives, supported employment and rehabilitation, and direct job creation policies.

One of the novelties of the analysis presented in this paper is the inclusion of implementation-related variables in the analysis of the effectiveness of ALMPs. Three performance indicators were constructed capturing three different dimensions of implementation: allocation of resources to the implementation of policies, continuity and timing in the implementation of programmes. The first dimension is measured by the overall expenditure on programme administration 10 as a percentage of total expenditure in ALMPs. Second, following Schmid (1996), continuity in the implementation of programmes is measured by the dynamics of ALMP expenditure, i.e. large annual variation in spending (fluctuations that exceed cyclical swings) would be the antithesis of continuity in implementation. This variation was captured by the difference between the fluctuations (measured by the standard deviation) in real GDP growth and the growth rate of ALMP spending. Third, as explained above, the variable timing measures whether policies are implemented in a countercyclical or pro-cyclical manner (Schmid, 1996). To assess this, the regression coefficients between ALMP spending and both output and unemployment were calculated. A dummy variable was then created taking the value of 1 if expenditure on ALMPs ran parallel to changes in the unemployment rate and counter the economic trend (i.e. policies that were implemented counter-cyclically) and 0 otherwise.

Regarding the demand conditions variable, GDP was drawn from the National Accounts section of OECD Economic Outlook dataset. In addition, as explained above, two aspects of the labour market are included to control for the structure of the labour market: (i) the share of the population with tertiary education was gathered from the World Bank World Development Indicators database; (ii) the middle-and high-skilled unemployment rates follow the same sources used for the low-skilled unemployment rates. Finally, the two institutional arrangement variables (i.e. union density and strictness of employment protection for temporary employment) are drawn, respectively, from the ICTWSS 11 and EPL databases of the OECD. EPL is measured by Version 1 of the indicator given that it contains annual information since 1985 (although this version does not incorporate all the data items included in Version 3). 12The analysis is based on data for 31 OECD countries over the 25-year period 1985-2010, which yields a total of 806 observations. Yet, information is not usually available for all countries for every year so most of the regressions are based on a smaller data set. For example, the number of countries is reduced to 27 when using the FGLS estimator with correction for first-order autocorrelation (i.e. the preferred specification) due to the unavailability of information about union density and EPL for temporary workers for Israel, Estonia, Luxembourg and Slovenia.

Evolution of ALMPs

Since the 1990s there has been a growing interest in activation measures. Today, expenditure in ALMPs is sizeable in most advanced economies and continues to increase. Between 2004 and 2009, ALMP expenditure grew continuously at an average annual rate of 5.8 per cent, reaching an accumulated growth of 32.5 per cent and a total spending of US$176.5 billion (PPP) in the five years to 2009. Only in 2010, this expansion was interrupted and ALMP spending fell by close to 0.7 per cent (Figure 1). 10 Which is defined by governments' expenditure in Public Employment Services and includes: (i) public expenditure in placement and related services, (ii) benefit administration, and (iii) other services and activities, including both, the budget of institutions that manage placement and related services and ALMPs but also the budget of institutions that administer unemployment and early retirement benefits. It is important to note that in some countries the share of expenditure in PES corresponding to the latter two subcategories might be significant. For example, Belgium, Italy, New Zealand and the United States have a significant share of (ii) in their total PES. Moreover, Canada, Czech Republic, France, Mexico, Poland, Portugal, Slovak Republic and Spain, have a significant portion of (iii); and in Ireland the mix of both categories represents the biggest share. 11 [START_REF] Visser | Data Base Institutional Characteristics of Trade Unions, Wage Setting, State Intervention and Social Pacts, 1960-2010 (ICTWSS)[END_REF]. In terms of the distribution of expenditure by type of policy, an important share of spending on ALMPs (excluding PES and administration) remained concentrated in training measures, which represented close to 39 per cent of the total in 2010 (Figure 2). Spending in employment incentives, direct job creation and supported employment and rehabilitation measures also represented prominent shares with 22.2, 16.8 and 15.7 per cent, respectively. Meanwhile, the share of expenditure in start-up incentives was much lower -at 6.2 per cent of total expenditure -and that of job rotation and job sharing programmes was negligible. Relative to 2004, the different types of policies have maintained their relative importance, with minor exceptions. For example, a small decrease in the share of spending in supported employment and rehabilitation measures seems to have given way to an increase in spending in direct job creation.

The overall growth in expenditure on ALMPs during the period 2004-2010 was driven by training, in spite of a decrease in its share in 2010. It accounted for one-third of the total increase in ALMP spending (excluding PES and administration) during the period. Employment incentives and direct-job-creation measures are the second and third sources of growth in ALMP expenditure, accounting for around 27 and 24.5 per cent of the total increase, respectively. Conversely, spending in supported employment and rehabilitation fell during the period, thus its contribution to growth only accounted for 7.7 per cent.

In sum, training continues to be the preferred tool of governments to address labour market problems in advanced countries. Meanwhile, employment incentives and direct-job-creation measures have become more prominent. The remaining of this section will test whether targeting spending towards these priority policies would be the most effective way to address labour market challenges and imbalances.

Descriptive statistics

A summary of basic descriptive statistics of the variables used in the analysis are presented in Appendix 4.

Importantly, as shown by the pairwise correlations, implementation variables are closely related to policy intervention ones. This is to be expected since the effect of policy interventions depends on the quality of implementation. To avoid a misspecification arising from the omission of these relationships, a number of interaction terms were tested through individual and joint tests for the non-significance of interactions terms.

Three interactions proved significant indicating they should be added to the model to ensure a correct specification: cluster * PES allocation; cluster * timing; cluster * continuity. 13 Finally, an additional interaction term was added between the variables Cycle and timing to capture the fact that countercyclical policies may be more or less effective depending on the moment of the economic cycle. 13 Omitting relevant interaction terms would constrain the partial derivatives of both Policy Cluster and the three implementation variables to be constant rather than varying, as they would be for the equation including the interaction terms (Baum, 2006, p. 125).

Econometric results

Description of results

Tables 1, 2, 3 and 4 below report the effects of ALMPs expenditures on the unemployment, employment and labour force participation rates (LFPR) of the overall and low-skilled population and on the share of low-skilled unemployed individuals. All models presented in these tables report results estimated by FGLS with AR1 correction, which is the preferred specification. For each parameter, interactions were added one by one to check whether results change with each addition. This allows flagging the size of variations due to the inclusion of interaction terms, which in most cases are correlated with the underlying explanatory variables. 14

Effects on the unemployment rate

Estimates relative to the unemployment rate are presented in Table 1. Results show that the policy cluster (i.e. public expenditure in training, employment incentives, supported employment and rehabilitation and direct job creation measures) has a significant negative effect on the total unemployment rate and the unemployment rate of the low-skilled. The effect of start-up incentives on the unemployment rates of both groups is also negative and significant. Finally, job rotation and job sharing has a negative but non-significant effect on the unemployment rate of these groups, which might be due to the fact that expenditure in these policies is small in relative terms. Interestingly, in general, ALMPs seem to be more effective in reducing the unemployment rate of the low skilled than that of the overall population.

These findings are consistent with empirical evaluations carried out using micro data, which suggest that jobsearch assistance and classroom and on-the-job training are especially effective [START_REF] Card | Active labor market policy evaluations: a meta-analysis[END_REF][START_REF] Kluve | Disentangling Treatment Effects of Active Labor Market Policies: The Role of Labor Force Status Sequences[END_REF]. In fact, job-search assistance appears to be more effective than training in the short term, whilst in the medium term the effect of training tends to increase due to gains to human capital [START_REF] Card | Active labor market policy evaluations: a meta-analysis[END_REF][START_REF] Hotz | Evaluating the differential effects of alternative welfare-to-work training components: a re-analysis of the California GAIN program[END_REF]. Subsidized employment programmes can also be effective, sometimes even exceeding the effects of training [START_REF] Kluve | The effectiveness of European active labor market programs[END_REF], but this is not systematically the case in the literature.

Importantly, implementation seems to matter too. Raising the share of PES in total ALMP expenditure has a reducing effect on the unemployment rate of both population groups, although this effect is significantly different from zero only in the case of the overall unemployment rate. This finding supports the conclusions from a number of micro-econometric studies carried out in OECD countries that show consistently positive outcomes from investing in PES [START_REF] Martin | What works and for whom: a review of OECD countries' experience with active labour market policies[END_REF]. The variable timing, which measures whether policies are implemented in a countercyclical or procyclical manner, matters too, implying that countercyclical policies (timing=1) have an unemployment reducing effect. Finally, the effect of policy continuity is significant only in the case of the low-skilled and has a negative effect as well but only once the cycle-timing interaction is included.

The interaction of timing with the policy cluster is also significant and has a positive effect on the unemployment rate of both population groups. 15 It suggests that when policies are implemented in a countercyclical manner (timing=1), the unemployment reducing effect of the policy cluster is lower. 16 This implies that policies that are implemented procyclically have a stronger unemployment reducing effect during booms but also a stronger unemployment enhancing effect during crises. This confirms the argument in favour of policy continuity that advises investing in ALMPs during booms when resources are available but also during crises when the unemployed need that support the most. This argument is also confirmed by the cycle-timing interaction, which is also significant for the low-skilled. It shows that when policies are implemented countercyclically, the elasticity of the unemployment rate to the cycle is lower. Finally, the interaction between the policy cluster and the share of PES is also significant for the low-skilled group, albeit only at the 10 per cent level. It shows that the unemployment reducing effect of the policy cluster becomes stronger as more ALMP resources are devoted to PES and administration.17

Interestingly, the size of the coefficients arising from policy and implementation variables once interactions are included is noticeably higher. This demonstrates that a correct implementation of policies (e.g. namely right timing and the allocation of resources to PES) enhances the unemployment reducing effect of the policy cluster.

Other control variables -such as of union density, the share of the population with tertiary education, the strictness of employment protection for temporary workers and the unemployment rate of low-skilled individuals -also show significant effects. As explained above in more detail, these variables have been included in the analysis to control for the structure of the national labour markets (which may affect the speed of adjustment to shocks) and for differences in institutional arrangements (that can affect wage bargaining and macroeconomic performance). Their coefficients will not be analysed in this paper since their individual effects are (at least partly) already taken up by the other explanatory variables. 18

Effects on the employment rate

Table 2 presents the results of the employment rate estimations of the two population groups. By and large, these findings show the mirror image of the unemployment rate's results. The policy cluster has a significant positive effect on the employment rate of the overall and low-skilled populations. The effect of start-up incentives is also positive but this time is only significant for the overall population. Job rotation and job sharing has again a negative but non-significant effect on both employment rates. Finally, in line with previous results, ALMPs seem to be more effective in boosting the employment rate of the low skilled.

Likewise, the impact of implementation variables shows the mirror image of unemployment rate estimations. An increased allocation of resources towards PES, for example, has a positive effect on the employment rate of both population groups but this effect is significantly different from zero only in the case of the overall population's employment rate. Moreover, the effect of policy continuity is positive and significant for both population groups suggesting that ALMP spending stability would have a boosting effect on the employment rates of the overall and low-skilled populations.

The interaction term between the policy cluster and the allocation of resources to PES is also significant in the case of the low-skilled and is positively correlated. The analysis of this interaction illustrates that as more ALMP resources are allocated towards PES and administration, the favourable effect of the policy cluster on the lowskilled employment rate becomes stronger. In addition, whether policies are implemented in a countercyclical or procyclical manner also influences the magnitude of the effect of the policy cluster on the employment rate of both population groups (i.e. interaction term between policy cluster and timing). This interaction term has a negative significant effect on the employment rate of both population groups, implying that when policies are implemented in a countercyclical manner (timing=1), the employment enhancing effect of the policy cluster is lower. The interpretation of these effects remains the same as in the case of the unemployment rate. This effect is again confirmed by the cycle-timing interaction, which is significant only for the overall population. It suggests that when policies are implemented countercyclically, the elasticity of the employment rate to the cycle is lower.

Effects on the labour force participation rate (LFPR)

Likewise, ALMP intervention has significant effects in the LFPR of the overall and low-skilled populations.

Table 3 presents these parameter estimates. The policy cluster variable is positively and significantly correlated with the LFPR of both population groups, albeit only at the 10 per cent level for the low-skilled.

Implementation variables also have some level of significance. Policy continuity has a positive and significant effect on both participation rates. Moreover, increasing the allocation of resources towards PES has a positive effect on the LFPR of both population groups but this effect is significantly different from zero only in the case of the overall population.

The interaction term between the policy cluster and the allocation of resources to PES is also significant in the case of the low-skilled participation rate and is positively correlated. As it was the case with the low-skilled employment and unemployment rates, the analysis of this interaction's parameter illustrates that as more ALMP resources are devoted to PES and administration, the favourable effect of the policy cluster on the low-skilled participation rate becomes stronger. Similarly, the cluster-timing interaction is significant only for the low skilled and has a negative effect, in line with the low-skilled employment rate's findings. The analysis and interpretation of this effect is the same as in the case of the employment and unemployment rates. Finally, the effect of ALMPs is enhanced when interactions are included in the analysis. 19Effect on the share of low-skilled unemployed individuals Table 4 presents the parameter estimates for the share of low-skilled unemployed individuals showing that ALMPs have also the potential for reducing the negative "structuralization" of unemployment on weaker groups of the labour market, in this case the low-skilled. Yet, unlike the other models, only the policy cluster and the timing of the intervention show significant results, negative in both cases.

A note of caution is in order regarding the robustness of this last model since results did not always hold across the different estimations, mainly in the case of estimations that did not include country dummies. This might be explained by the fact that the structure of unemployment is particularly heterogeneous across countries. Country dummies were thus included in the OLS and FGLS (AR1) estimations to account for the unexplained country-tocountry variation. Adding country dummies, however, has the risk of saturating the model, as it can be seen by the size of variance in model (1).

Interpretation of results

To provide an interpretation of the magnitude of the effects discussed above, I use the coefficients on the first columns of tables 1 to 4 (which show effects of policy variables before adding the interactions) as basis for some calculations. According to the results, increasing an additional standard deviation in the policy cluster (US$4.8 thousand [PPP] per unemployed) would reduce the overall unemployment rate by around 2 percentage points and the low-skilled unemployment rate by close to 3 percentage points. Importantly, the effect would be more important for the low-skilled since this increase in spending would be accompanied by a decrease of 1.5 percentage points in the share of low-skilled unemployed individuals (in total unemployment). In terms of employment, raising one standard deviation the expenditure on these policies would boost the overall employment rate by around 2.1 percentage points and the employment rate of the low-skilled by 2.7 percentage points. Finally, this would be associated with an increase in the LFPR of the two groups by 1.6 and 1.7 percentage points, respectively. Importantly, in general, ALMPs seem to be more effective in improving labour market outcomes of the low skilled. This seems intuitively correct, first, because most policies are targeted towards this more disadvantaged group; and second, because higher-skilled individuals are expected to be better equipped to find jobs by themselves and so policies targeted to them appear less effective due a potential deadweight loss.

The effect of start-up incentives is also non-negligible, albeit less significant in the case of the low skilled. An increase by one standard deviation (US$140 [PPP] per unemployment) in expenditure allocated to start-up incentives would be accompanied by a 0.43 and 0.62 percentage points decrease in the unemployment rates of the overall and low-skilled populations, respectively. In terms of employment, this increase in spending would raise the employment rate of the overall population by 0.29 percentages points.

Importantly, as shown above, implementation has also significant effects. For example, an increase by one percentage point in the share of PES (in total ALMP expenditure) would be associated with a decrease of 3.3 percentage points in the overall unemployment rate, and an increase by 4.2 and 3.5 percentage points in the overall employment and participation rates, respectively. Moreover, a disruption of policy continuity is associated with a reduction of 0.17 and 0.14 percentage points in the overall and low-skilled employment rates, respectively. It would also affect negatively the overall and low-skilled labour force participation rates by 0.11 and 0.16 percentage points, respectively.

The size of effects arising from policy and implementation variables, once interactions are included, is higher. This means that the interaction of the cluster of policies with the right implementation measures (e.g. namely right timing of policies and the share of PES spending) enhances their unemployment reducing effect. These effects can be attributed to the benefits of individualised service delivery in the case of more generous job search conditions [START_REF] Gaure | Job search incentives and job match quality[END_REF][START_REF] Kluve | The effectiveness of European active labor market programs[END_REF], while continuity in implementation can generate mutual trust between labour market actors and employment service agencies. Moreover, some of the implementation variables become significant only once these interactions are in place.

In more detail, these results suggest that a country with a 10 per cent rate of unemployment (such as France or the United States in 2010) would need to spend around US$25,000 (PPP) in policy cluster-type measures for every fewer unemployed -i.e. reducing 0.4 for every 10 unemployed would cost US$10,000 (PPP) in a country with an unemployment rate of 10 per cent. Following the same logic, this same country would need to spend around US$3,300 (PPP) in start-up incentives for every fewer unemployed. According to this, start-up incentives would be more effective in reducing unemployment than the policy cluster. This finding is in line with results from micro-econometric analyses showing that start-up incentives are associated with a "double dividend" if subsidized firms create more jobs in the future [START_REF] Caliendo | Regional Effect Heterogeneity of Start-Up Subsidies for the Unemployed[END_REF][START_REF] Schmid | Policy Measures to Address Polarisation of the Labour Market: Lessons to Korea from Germany and Europe[END_REF]. Unfortunately, countries usually spend less on start-up incentives, in part because it is commonly believed that these policies benefit more the higher skilled who also need less government assistance. This lower expenditure would mean that attaining efficient levels of spending per unemployed individual would probably be more challenging.

France, for example, would need to raise its expenditure per unemployed individual -relative to its 2010 expenditure -by over 270 and 565 per cent, respectively, in the cluster of policies and start-up incentives if the country is to attain the necessary levels for these policies to be the most effective. Importantly, recognition of the benefits of supporting new entrepreneurs has increased. France and the United Kingdom, for example, have new start-up incentives in place following the 2007 economic crisis (NACRE and Start Up Loans Scheme, respectively), which focus on offering longer-term financial, mentoring and follow-up assistance to entrepreneurs looking to start or rescue a business.

To put these numbers in perspective, one could compare the efficient levels of spending in ALMPs presented above with costs for alternative options governments would have for supporting unemployed individuals. For example, the annual median labour cost per employee in France in 2010 was around US$38,500 (PPP), which would suggest that spending in ALMPs would be both economically and socially more efficient than having the government employ these people directly. Moreover, keeping these unemployed individuals attached to the benefit system has also costs for the government and for society as a whole. Specifically for France, in 2010 the cost for society 20 of having one unemployed under the out-of-work maintenance and support income was around US$12,800 (PPP) per participant. In addition, the government spent over US$7,300 (PPP) per participant in 2010 in income support measures, principally for the unemployed that had exhausted their entitlement to unemployment benefits. 21 Importantly, the longer individuals are unemployed the less likely it is for them to find jobs without assistance. As such, activation measures will be nevertheless needed to facilitate the return to work of these long-term unemployed individuals.

The story is different, however, for a country with a 20 per cent unemployment rate such as Spain. In this country, the necessary expenditure in cluster policies to reduce by one the number of unemployed would be over US$50,000 (PPP), which is above the annual median labour cost per employee at around US$33,400 (PPP) in 2010. In this case, start-up incentives would again offer a "bigger bang for the buck" (around US$6,500 [PPP] for one fewer unemployed). Yet, this level of expenditure would entail an increase of 1,650 per cent of Spain's 2010 expenditure in start-up incentives per unemployed individual, which seems unlikely. 21 This includes the cost of the partial unemployment scheme (transferred to enterprises), the temporary delay allowance (ATA), the 50% of the special employment assistance (AEPE) bared by the Central government; and the 60% of specific solidarity and pension equivalent allowances financed by the State budget (Ibid).

Table 1. Regression results on the unemployment rate adding one interaction at a time

Notes for all tables: Standard errors are in parentheses. Significance levels: *significant at 10 per cent; **significant at 5 per cent; ***significant at 1 per cent. All models have been estimated by FGLS with AR1 correction (preferred specification). The model relative to the share of low-skilled unemployed individuals also includes country dummies.

Unemployment rate

Low-skilled unemployment rate 24

Sensitivity analysis

This section discusses the robustness checks that have been carried out to evaluate the sensitivity of the parameters presented in tables 1, 2, 3 and 4, based on a number of alternative specifications and tests.

The use of different samples (i.e. overall and low-skilled populations) in each labour market equation, as discussed in the previous section, can be considered as the first robustness check. Results hold and seem to be coherent between the two population groups analysed. Moreover, robustness of results was checked by excluding key countries (in terms of the relative size of their ALMP expenditure), namely Denmark, Netherlands and Sweden, and by running the regressions on EU countries only. With some exceptions, changing the sample does not seem to alter the big lines of the estimation results. 22 The window of time was also modified to see whether results held when studying only the last decade. Overall results seem to hold, with two exceptions: first, the policy cluster reveals a loss of significance in affecting the share of low-skilled unemployed. Second, the allocation of resources to PES and the timing of policies no longer affect directly the unemployment rate but through the interaction with the policy cluster.

Reduced estimations were also carried out, although results are not provided in this paper. In particular, I estimated three reduced models for each dependent variable: the first model estimates only the influence of the three policy interventions; the second model tests only implementation variables; and the third model all the variables. The description of results was concentrated on the estimations of the full model only.

Along with these additional specifications carried out, a number of tests were included in the main specifications. First, the augmented Dickey-Fuller test was used to control for the non-stationarity of dependent variables. In all cases (but one) the tests rejected the null hypotheses of non-stationarity at 1 and 5 per cent levels. The exception was the variable share of low-skilled unemployed individuals which seems to be nonstationary. Moreover, the different models were controlled for heteroskedasticity using the robust option available. Robust results did not differ much from non-robust results.

The problems of serial correlation, heteroskedasticity and endogeneity have been taken especially seriously and have been dealt with through a number of tests and additional estimation techniques. First of all, the Lagrange-Multiplier test [START_REF] Wooldridge | Econometric Analysis of Cross Section and Panel Data[END_REF][START_REF] Drukker | Testing for Serial Correlation in Linear Panel-Data Models[END_REF] and the abar post-estimation technique [START_REF] Roodman | How to Do xtabond2: An Introduction to "Difference" and "System" GMM in Stata[END_REF] were used to test for serial correlation in the idiosyncratic error terms. Given that the results of these tests confirmed the presence of first order autocorrelation, all specifications were run with a FGLS estimator, which allows the assessment in the presence of AR1 autocorrelation within panels, cross-sectional correlation and heteroskedasticity across panels.

Moreover, in an effort to address the reverse causality problem, a final estimation was carried out instrumenting (i.e. finding variables correlated with the endogenous variables, but not correlated with error term) two of the three policy variables (policy cluster and job rotation and job sharing) 23 and the implementation-related 22 These results are available upon request and in Table 4 of the Extended Appendix.

variables continuity and timing through a 2-stage least squares (2SLS) estimator. In addition to the other exogenous variables of the model, I used two sets of instruments in the analysis. The first one is a set of macroeconomic and structural variables including fiscal deficit or public debt (depending on the specification), 24 inflation, the share of the population with tertiary education and total expenditure in passive labour market policies. Moreover, the differenced unemployment rate was included in the models explaining the employment rate and the two LFPR and the variable terms of trade in the models explaining the unemployment rate, lowskilled employment rate and LFPR. The second is a set of governability-related indicators. This second set of instruments includes two indicators of the colour of the party; 25 a dummy variable, reform, taking the value of 1 if a reform to ALMPs was put in place in that year in the country and 0 otherwise; 26 and a continuous variable, durable, illustrating the number of years that have passed since a change in governability was implemented in the country [START_REF] Marshall | Political Regime Characteristics and Transition, 1800-2012 -Polity TM IV Project[END_REF]. Reform was excluded from the models explaining the employment rate, and the two LFPR because its addition proved to be redundant. Durable was also excluded from the two LFPR equations for the same reason.

Results are presented in Appendix 6 and broadly confirm the findings discussed in previous sections. Accounting for the endogeneity of policy and implementation variables leads to the same effect of the policy cluster and start-up incentives. In fact, start-up incentives become significant in boosting the low-skilled participation rate and in reducing the share of low-skilled unemployment. A difference in the results however, is that job rotation and job sharing becomes significant in most of the equations (with the exception of the overall employment rate and the low-skilled participation rate) but in an ambivalent manner. An increase in spending in this policy appears to be detrimental for the unemployment and employment rates, but positive in boosting overall participation and in reducing the share of low-skilled unemployment. This seems intuitive given the aim of these policies, i.e. promoting a full or partial replacement of an employee by an individual (unemployed or not) from another target group.

Moreover, the allocation of resources towards PES also gains in significance level in the specifications related to the low-skilled when instruments are added. The same occurs with the variable continuity in implementation, with exception of the low-skilled employment and the overall LFPR. Finally, the variable timing continues to the significant and negatively related with the different employment and participation rates. A change occurs, however, in the sign of this variable's effect on the unemployment rate equations. When instruments are included, the coefficient of timing in these two equations becomes positive, implying that when policies are implemented in a countercyclical manner they would have an unemployment increasing effect. This represents now the mirror image of the employment and participation rates equations. The interpretation of these results would be the same explained above. True, policies that are implemented countercyclically may have stronger 24 To avoid correlation of the instrument with the error term, fiscal deficit was used in the unemployment rate and share of lowskilled unemployed equations and public debt in the remaining equations. 25 Cabinet composition 1, illustrates the percentage of right-wing or left-wing parties (the variable with the lowest correlation with the dependant variable was chosen for each model) in total cabinet posts, weighted by days in office (Armingeon et al., 2013a).

Cabinet composition 2 corresponds to the Schmidt-Index, which takes a value of 1 to 5 depending on whether there is a dominance of right-wing or left-wing parties in the composition of cabinets (Armingeon et al., 2013b). 26 fRDB-IZA Reforms Database (Anelli et al., year not available).

unfavourable effects during booms (when pro-cyclical policies should be implemented) but also stronger favourable effects during crises. This confirms the argument in favour of policy continuity.

All 2SLS models were estimated using the gmm and robust options to compute efficient estimates in presence of heteroskedasticity and serial correlation. The exception was the estimation of the share of low-skilled unemployed where the robust option was not used. The models showed robust results in the test for the validity of instruments in an overidentified context (J-statistic of Hansen to test for overidentifying restrictions) implying that the group of instruments used is suitable and that it satisfies the required orthogonality conditions. To test whether there are sufficient valid instruments to identify the model (i.e. models are not underindentified) and whether models are not relying on weak instruments, the Anderson's canonical correlations approach was used.

The null hypothesis was rejected in all models implying that there are enough adequate instruments to estimate the equations. This was confirmed as well by Shea's partial R2 [START_REF] Baum | An Introduction to Modern Econometrics Using Stata[END_REF].

Finally, an analysis of the variance was carried out in each of the seven models to assess the percentage of the variability of the result that was explained by explanatory variables and how much of that variability was left unexplained. In the case of the overall unemployment rate, the R-squared showed that 84.1 per cent of the variance was explained by independent variables, while it was 90 per cent in the case of the low-skilled unemployment rate. Although high, these are still within the rule of thumb of 90 per cent that is acceptable. The variance analysis for the employment and participation rates showed as well shares of the variance within the limits acceptable. These figures were 74.6 per cent and 49.9 per cent for the employment rates of the overall and low-skilled populations, respectively; and 62.2 per cent and 48 per cent for the LFPR of the overall and lowskilled populations, respectively. In contrast, the segment of the variance explained by the explanatory variables in the estimation of the share of low-skilled unemployed individuals was 92.8 per cent. This shows that the use of country dummies to control for country-specific characteristics might be saturating the model in this last case.

To conclude, the additional analyses carried out broadly confirm the estimation results discussed above.

Particularly, results remain largely robust across the various specifications, including pooled OLS, OLS models with robust standard errors, random-effects and fixed-effects models depending on the results of the Hausman test (not reported here), models with country dummies, FGLS estimator to account for serial correlation, as well as the instrumental variable estimator 2SLS. Importantly, the results concerning the effect of ALMP in shaping the "structuralization" of unemployment on the low-skilled remained more sensitive. Robustness checks show that results not always hold across the different estimations, mainly in the case of estimations that did not include country dummies, which as explained above may respond to the fact that the structure of unemployment is particularly country specific.

Conclusion

This paper examines the effectiveness of ALMPs in improving labour market outcomes, especially for lowskilled individuals. Much has been written about the effectiveness of activation measures based on evaluations carried out using micro data, yet not enough about their effectiveness at the aggregate level. This is particularly true with regards to the role of the implementation characteristics of ALMPs in shaping their overall effects. This paper aims to contribute to this debate through an aggregate impact approach which is better placed to measure both, the direct and indirect effects of ALMPs. This is done by ways of a pooled cross-country and time-series database for 31 advanced countries during the period 1985-2010. Different models were estimated to measure the effects of six different ALMPs and three dimensions of their implementation (i.e. allocation of resources to public administration, continuity and timing in the implementation of programmes) on the unemployment, employment and participation rates of the overall and low-skilled populations and the share of low-skilled unemployed individuals. Controls for demand conditions, the structure of the labour market and differences in institutional arrangements were included as well. For each specification, different estimators were used to control for cross-country heterogeneity, account for serial correlation and address reverse causality.

In sum, I find that ALMPs matter at the aggregate level. Public expenditure in training, employment incentives, supported employment and rehabilitation and direct job creation measures (i.e. the policy cluster) show the most favourable results. In particular, the policy cluster appears to have a significant unemployment reducing effect and a significant employment and labour force participation expanding effect for the overall and low-skilled populations (albeit a low significance level on the LFPR of the low skilled) (Table 5). In addition, results suggest that spending in these policies has the potential of reducing the share of low-skilled unemployed. Spending in start-up incentives is effective as well but only in reducing the unemployment rate of both population groups and in boosting the employment rate of the overall population. Results on the rest of labour market variables studied are non-significant.

In terms of implementation, results show that the most favourable aspect analysed is the allocation of resources to programme administration. Indeed, while the share of PES in total ALMP expenditure has a direct and favourable impact on labour market variables of the overall population; it affects the low skilled through an interaction with the policy cluster. This effect illustrates that as more ALMP resources are devoted to PES, the favourable effect of the policy cluster on the low-skilled unemployment, employment and participation rates becomes stronger. In addition, a disruption of policy continuity would be associated with negative effects for all labour market variables analysed. The effect of policy continuity is significantly negative in the case of the lowskilled unemployment rate and significantly positive in the case of the employment and LFPR of both population groups. It is, however, not significant for the rest of the variables studied.

Finally, the variable timing, which measures whether policies are implemented in a countercyclical manner revealed a significant and negative relationship with all labour market variables analysed but the participation rate of the low-skilled. These results show that countercyclical policies (timing=1) have an unemployment reducing effect but also an employment and participation reducing effect. Importantly, once estimations are run through IV, the effect on the unemployment rate becomes positive showing the mirror image of the employment and participation rates. Moreover, the interaction of timing with the policy cluster is also significant for some of the variables and has a positive effect on the unemployment rate of both population groups and negative effect on the employment rate of both population groups and the participation rate of the low-skilled. This overall picture suggests that policies that are implemented pro-cyclically have stronger favourable effects during booms but also stronger unfavourable effects during crises, confirming the argument in favour of policy continuity.

Policy continuity is also supported by the cycle-timing interaction in the two variables where effects were significant: the low-skilled unemployment rate and the overall population employment rate. Interestingly, the size of the coefficients arising from policy and implementation variables, once interactions are included is noticeably higher. This demonstrates that a correct implementation of policies enhances their beneficial effect. categories [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF][START_REF] De Koning | The impact of active labour market policy on job hirings and unemployment in the Netherlands[END_REF][START_REF] Martin | What works and for whom: a review of OECD countries' experience with active labour market policies[END_REF][START_REF] Dauth | Macroeconometric Evaluation of Active Labour Market Policies in Austria[END_REF].

employment. Total effect on employment would depend on the scale of the substitution effect.

-

  ALMP spending per unemployed individual has also increased in a sustained manner -by an accumulated 25.5 per cent between 2004 and 2008. In 2009 and 2010, however, this upward trend ended abruptly due to the rise in the number of unemployed as a consequence of the crisis.

Figure 1 .

 1 Figure 1. Evolution of total expenditure on ALMPs, 2004-2010

Figure 2 .

 2 Figure 2. Share of the expenditure by type of ALMP in 2004 and 2010 and in the change between 2004-2010 (percentages of total ALMP expenditure*)

Table 2 . Regression results on the employment rate adding one interaction at a time Employment rate Low-skilled employment rate

 2

		No interaction	Only first	Only second	Only third	Complete	No interaction	Only first	Only second	Only third	Complete
		(1)	interact. (2)	interact. (3)	interact. (4)	model (5)	(1)	interact. (2)	interact. (3)	interact. (4)	model (5)
	Cluster (Policy 1)	0.436*** (0.0527)	0.390*** (0.100)	1.396*** (0.314)	0.463*** (0.0529)	1.167*** (0.346)	0.563*** (0.110)	0.190 (0.199)	1.701*** (0.644)	0.685*** (0.118)	1.850*** (0.712)
	Job rotation and job sharing	-0.883	-0.826	-0.773	-1.056	-0.832	-1.264	-0.696	-0.999	-1.926	-0.908
	(Policy 2)	(0.711)	(0.725)	(0.682)	(0.745)	(0.724)	(1.389)	(1.392)	(1.300)	(1.609)	(1.497)
	Start-up incentives (Policy 3)	2.085** (0.947)	2.198** (0.963)	1.559* (0.923)	2.282** (0.975)	1.898** (0.967)	1.146 (1.891)	1.762 (1.889)	0.558 (1.800)	1.929 (2.134)	1.577 (2.030)
	Cluster * PES allocation		0.220 (0.381)			0.297 (0.378)		1.672** (0.754)			1.826** (0.799)
	Cluster * Timing			-0.979*** (0.311)		-0.791** (0.330)			-1.189* (0.638)		-1.638** (0.682)
	PES allocation	4.153*** (1.232)	3.911*** (1.328)	4.658*** (1.215)	4.043*** (1.256)	4.120*** (1.351)	3.103 (2.673)	0.870 (2.822)	3.745 (2.591)	4.182 (2.944)	2.443 (3.012)
	Continuity in implementation	0.172*** (0.0387)	0.174*** (0.0388)	0.159*** (0.0377)	0.213*** (0.0392)	0.191*** (0.0385)	0.135** (0.0629)	0.125** (0.0626)	0.121* (0.0627)	0.236*** (0.0874)	0.183** (0.0828)
	Correct timing of policies	-2.901*** (0.943)	-2.989*** (0.942)	-1.448 (1.022)	-5.943*** (1.061)	-4.247*** (1.240)	-4.550** (2.074)	-4.743** (2.075)	-2.591 (2.328)	-6.666*** (2.074)	-4.052* (2.277)
	Cycle * Timing				0.00109*** (0.000251)	0.000891*** (0.000267)				7.53e-06 (6.25e-06)	5.26e-06 (6.15e-06)
	Cycle	6.30e-06** (2.80e-06)	6.52e-06** (2.81e-06)	5.51e-06** (2.77e-06)	-0.00108*** (0.000251)	-0.00088*** (0.000268)
	Growth rate of real GDP	-0.0265 (0.0377)	-0.0273 (0.0380)	-0.0204 (0.0361)	-0.0355 (0.0397)	-0.0283 (0.0380)	0.0699 (0.0747)	0.0684 (0.0737)	0.0813 (0.0697)	0.0450 (0.0873)	0.0587 (0.0796)
	Unemployment rate of the high	-0.773***	-0.771***	-0.712***	-0.832***	-0.768***	-0.585***	-0.563***	-0.523***	-0.540***	-0.433**
	skilled	(0.0797)	(0.0801)	(0.0797)	(0.0816)	(0.0825)	(0.170)	(0.169)	(0.170)	(0.185)	(0.182)
	Union density	0.0622*** (0.0173)	0.0647*** (0.0176)	0.0642*** (0.0174)	0.0692*** (0.0161)	0.0730*** (0.0168)	0.0872** (0.0408)	0.100** (0.0411)	0.0907** (0.0425)	0.0945** (0.0379)	0.117*** (0.0395)
	EPL for temporary workers	-0.655*** (0.203)	-0.655*** (0.203)	-0.677*** (0.200)	-0.453** (0.206)	-0.516** (0.206)	2.151*** (0.445)	2.149*** (0.442)	1.880*** (0.438)	2.639*** (0.457)	2.381*** (0.445)
	EU	-8.606*** (0.763)	-8.651*** (0.760)	-8.712*** (0.773)	-10.08*** (0.764)	-9.932*** (0.783)	-9.361*** (1.870)	-9.570*** (1.872)	-9.365*** (1.995)	-9.761*** (1.673)	-10.10*** (1.744)
	Constant	76.90*** (1.422)	76.96*** (1.421)	75.15*** (1.495)	80.71*** (1.594)	78.57*** (1.783)	51.43*** (3.163)	51.60*** (3.149)	49.64*** (3.387)	52.11*** (3.216)	49.36*** (3.354)
	Observations	364	364	364	364	364	352	352	352	352	352
	Number of countries	27	27	27	27	27	27	27	27	27	27

Table 3 . Regression results on the labour force participation rate (LFPR) Table 4. Regression results on the share of low-skilled adding one interaction at a time unemployed individuals adding one interaction at a time LFPR Low-skilled LFPR Share of low skilled unemployed individuals

 3

		No interaction	No interaction	Only first	Only second	Complete	No interaction	Only first	Only second	Complete
		(1)	(1)	interact. (2)	interact. (3)	model (4)	(1)	interact. (2)	interact. (3)	model (4)
	Cluster (Policy 1)	0.340*** (0.0535)	0.346*** (0.105)	-0.00243 (0.192)	1.569** (0.666)	1.360* (0.695)	-0.315** (0.157)	-0.723** (0.296)	-1.574* (0.924)	-1.995** (0.955)
	Job rotation and job sharing	-1.481*	-0.571	-0.0946	-0.559	-0.0841	1.967	2.486	1.781	2.287
	(Policy 2)	(0.786)	(1.319)	(1.360)	(1.348)	(1.404)	(2.006)	(2.026)	(1.988)	(2.008)
	Start-up incentives (Policy	0.587	-0.0873	0.729	-0.576	0.223	-2.711	-1.121	-1.409	0.175
	3)	(1.126)	(1.998)	(2.050)	(2.056)	(2.130)	(3.513)	(3.656)	(3.594)	(3.734)
	Cluster * PES allocation			1.681** (0.736)		1.736** (0.755)		1.861 (1.161)		1.816 (1.153)
	Cluster * Timing				-1.202* (0.657)	-1.345** (0.665)			1.261 (0.907)	1.284 (0.903)
	PES allocation	3.540** (1.377)	1.171 (2.570)	-0.773 (2.757)	2.321 (2.657)	0.550 (2.867)	2.574 (4.302)	0.190 (4.574)	1.739 (4.288)	-0.586 (4.553)
	Continuity in	0.112***	0.157***	0.152**	0.160***	0.155**	-0.153	-0.140	-0.150	-0.137
	implementation	(0.0265)	(0.0610)	(0.0607)	(0.0608)	(0.0606)	(0.133)	(0.133)	(0.131)	(0.132)
	Correct timing of policies	-3.748*** (0.794)	-4.362** (2.091)	-4.750** (2.044)	-2.979 (2.219)	-3.246 (2.172)	-12.50** (6.159)	-13.69** (6.184)	-14.98** (6.367)	-16.20** (6.392)
	Growth rate of real GDP	-0.0705* (0.0428)	0.0438 (0.0725)	0.0380 (0.0738)	0.0416 (0.0741)	0.0341 (0.0762)	0.185 (0.119)	0.186 (0.119)	0.188 (0.118)	0.189 (0.118)
	Population with tertiary	0.0494***	-0.0472	-0.0475	-0.0554*	-0.0572*	-0.00653	-0.0117	0.00846	0.00387
	education	(0.0139)	(0.0298)	(0.0298)	(0.0302)	(0.0304)	(0.0414)	(0.0413)	(0.0426)	(0.0425)
	Unemployment rate of the	-0.201**	-0.0954	-0.0517	0.0135	0.0714	-0.651***	-0.609***	-0.795***	-0.756***
	high skilled	(0.0840)	(0.167)	(0.169)	(0.178)	(0.180)	(0.232)	(0.233)	(0.255)	(0.256)
	Union density	0.0999*** (0.0156)	0.127*** (0.0402)	0.140*** (0.0398)	0.133*** (0.0396)	0.148*** (0.0390)	0.154 (0.0971)	0.169* (0.0971)	0.170* (0.0976)	0.185* (0.0976)
	EPL for temporary workers	-0.873*** (0.204)	1.383*** (0.435)	1.465*** (0.435)	1.417*** (0.437)	1.525*** (0.437)	1.630** (0.785)	1.705** (0.784)	1.581** (0.780)	1.652** (0.779)
	EU	-7.094*** (0.691)	-7.905*** (1.874)	-8.230*** (1.830)	-8.390*** (1.840)	-8.806*** (1.784)	-26.76*** (5.949)	-26.91*** (5.917)	-27.90*** (5.991)	-28.07*** (5.960)
	Constant	76.04*** (1.461)	58.87*** (3.436)	58.91*** (3.397)	57.23*** (3.510)	57.05*** (3.470)	58.00*** (6.389)	59.20*** (6.417)	59.78*** (6.469)	60.95*** (6.493)
	Observations	336	326	326	326	326	306	306	306	306
	Number of countries	27	27	27	27	27	27	27	27	27

Table 5 . Synopsis of regression results

 5

		UNR	UNR LSK	EMP RATE	EMP RATE LSK	LFPR	LFPR LSK	Share of LSK UN
	Policy Cluster	(-)***	(-)***	(+)***	(+)***	(+)***	(+)*	(-)**
	Job rotation and job sharing	NS	NS	NS	NS	NS	NS	NS
	Start-up incentives	(-)***	(-)**	(+)**	NS	NS	NS	NS
	Cluster * PES allocation	NS	(-)*	NS	(+)**		(+)**	NS
	Cluster * Timing	(+)***	(+)***	(-)**	(-)**		(-)**	NS
	PES allocation	(-)***	NS	(+)***	NS	(+)**	NS	NS
	Continuity in implementation	NS	(-)**	(+)***	(+)**	(+)***	(+)**	NS
	Correct timing of policies	(-)***	(-)**	(-)***	(-)*	(-)***	NS	(-)**
	Cycle * Timing	NS	(-)***	(+)***	NS
	Cycle			(-)***
	Growth rate of real GDP	(-)**	NS	NS	NS	(-)*	NS	NS
	EU	(+)***	(+)***	(-)***	(-)***	(-)***	(-)***	(-)***
	Constant	(+)***	(+)***	(+)***	(+)***	(+)***	(+)***	(+)***
	Observations	452	336	364	352	336	326	306
	Table	1	1	2	2	3	3	4
	Column	3	6	3	6	3	6	3
	Notes: Significance levels: *significant at 10 per cent; **significant at 5 per cent; ***significant at 1 per cent.
	UNR= unemployment rate; LSK= low-skilled; EMP RATE= employment rate; LFPR= labour force participation rate; UN=
	unemployed; and NS means non-significant.

  Standard errors are in parentheses. Significance levels: *significant at 10 per cent; **significant at 5 per cent; ***significant at 1 per cent. (1) Pooled ordinary least squares model (OLS); (2) GLS (either fixed-or random-effects); and (3) FGLS with AR1 correction. OLS estimates include country dummies.

					Public expenditure in supported employment and rehabilitation
	Job rotation and job sharing -Supported employment and rehabilitation (subsidisation) Public expenditure in supported employment and rehabilitation Public expenditure in direct job creation	Labour supply programmes. Measured as expenditure in thousands of US$ (PPP) per Wage-setting schedule Reduced search unemployed individual. effectiveness Public expenditure in programmes aimed to create additional Short-term OECD. Stat Targeted policies to specific groups (e.g. youth, long-term unemployed, low-skilled) could mitigate the fear of unemployment among the (unintended effect) targeted individuals and thus, reduce the incentives to search for jobs (OECD, 1993). 27 jobs. Measured as expenditure in thousands of US$ (PPP) per OLS GLS FGLS (AR1) OECD. Stat (1) (2) (3)	Reduced employment
	-Job rotation and job sharing -Supported employment and rehabilitation (subsidisation) Public expenditure in start-up incentives -2.469*** Matching process and competition effects for insiders and outsiders unemployed individual. Wage-setting schedule Public expenditure in programmes that promote Increased search effectiveness (intended effect) -2.435*** -2.023*** -2.843*** Cluster (Policy 1) (0.230) (0.227) (0.245) (0.483) entrepreneurship. Job rotation and job -1.377* -1.386* -0.905 -1.795 Measured as expenditure in thousands of US$ (PPP) per sharing (Policy 2) (0.768) (0.764) (0.554) (1.415) unemployed individual. Implementation: Start-up incentives -5.729*** -5.589*** -2.320*** -4.010**	Short-term (effects on the labour market on the medium-term) -2.797*** (0.470) OECD. Stat In general, ALMP that target disadvantaged groups (especially when used in conjunction with benefit conditionality) will put pressure on unemployed people to search harder for jobs, which will be associated with lower wage pressure -2.968*** (0.515) -1.841 -0.231 (1.391) (1.164) and more jobs (Bellman and Jackman, 1996a). -4.268** -3.856**	Lower wages and lower unemployment.
	-Start-up incentives Allocation of resources for the implementation of policies (Policy 3) (0.934) Job creation Public expenditure on programme administration (PES) as a Labour demand Job creation Short-term (0.925) (0.740) (1.877) (1.835) Author's percentage of total ALMP expenditure. calculations based Start-up programmes improve employment and income prospects, especially for disadvantaged (1.725) on OECD. Stat -0.694** -0.650** -0.315 0.00688 -0.175 -1.089* Cluster * PES allocation (0.294) (0.288) (0.278) (0.679) (0.658) (0.601) groups (Caliendo and Künn, 2013). -Start-up incentives Competition for insiders Labour demand Displacement effect (unintended effect) Long-term Increased competition could imply that the businesses set up by the beneficiaries of start-up incentives drive other existing (unsubsidised) businesses out of the market (Román et al., 2013). Continuity of programmes implemented Difference between the standard deviation of real GDP growth and the standard deviation of the growth rate of real expenditure in ALMP during the whole period of analysis. Author's calculations based 2.198*** 2.152*** 1.722*** 2.153*** 2.147*** 2.675*** Cluster * Timing (0.224) (0.222) (0.238) (0.458) (0.446) (0.494) on Schmid (1996) Dummy variable taking the value of 1 when policies are implemented countercyclically (i.e. regression coefficient -6.935*** -6.993*** -4.096*** -3.137 -2.995 -2.633 PES allocation (1.182) (1.123) (1.018) (3.148) (2.989) (2.251) Author's Timing in the implementation of programmes between ALMP spending and output is negative and that of ALMP spending and unemployment is positive); and 0 otherwise. calculations based Continuity in 0.0258* 0.0256* 0.000143 0.257*** 0.138** -0.155** on Schmid (1996) implementation (0.0147) (0.0141) (0.0146) (0.0916) (0.0676) (0.0607)	Reduced unemployment. But also increased welfare for the vulnerable groups targeted. Reduced unemployment and increased employment if new self-employed do not displace unsubsidised
	Correct timing of Demand Conditions: policies				-4.544*** (1.454)	-2.930*** (0.810)	-6.967** (3.331)	-3.625** (1.543)	entrepreneurs.
	Real Gross Domestic Product (GDP) -7.52e-06 Cycle * Timing (5.87e-06) Annual growth rate of real GDP -2.61e-06 -3.10e-06 (3.22e-06) (2.01e-06)	-3.89e-06 (9.94e-06)	OECD. Stat -2.25e-06 -1.79e-05*** (6.67e-06) (4.37e-06)
	Structure of the Labour Market: Share of the population with tertiary education Growth rate of real -0.107** GDP (0.0453)	School enrolment, tertiary (% gross). -0.105** -0.0569** (0.0447) (0.0253)	0.0594 (0.0903)	0.0727 World Bank, WDI -0.0354 database (0.0883) (0.0622)
	Middle-skilled unemployment Population with tertiary 0.0366*** Middle-skilled unemployed individuals as a percentage of the 0.0315*** 0.0240** 0.0154 total middle-skilled in the labour force. education (0.00950) (0.00885) (0.0107) (0.0250) (0.0219) 0.0123 Eurostat, ILO and 0.0592*** (0.0224)
	rate Union density	Middle-skilled individuals are those with upper secondary and post-secondary non-tertiary education (levels 3-4 of ISCED). 0.0962*** 0.0759*** 0.0212 0.0844 (0.0211) (0.0180) (0.0145) (0.0588) (0.0449) National sources 0.0604 0.00771 (0.0259)
	EPL for temporary	-0.518**	High-skilled unemployed individuals as a percentage of the -0.487*** -0.101 -1.439***	-1.479***	-1.079***
	workers High-skilled unemployment rate (0.200) EU	total high-skilled in the labour force. (0.185) (0.161) High-skilled individuals are those that have finalized tertiary (0.483) education (levels 5-6 of ISCED). 3.013** 3.620*** (1.372) (0.709)	Eurostat, ILO and (0.436) (0.313) National sources 6.180** 6.752*** (3.071) (1.117)
	Institutional arrangements:
	Union density Constant	9.089*** (1.364)	Ratio of wage and salary earners that are trade union members, divided by the total number of wage and salary earners. 11.01*** 8.991*** 19.47*** 19.96*** OECD. Stat 11.64*** (1.906) (1.189) (3.522) (4.398) (2.459)
	EPL for temporary workers Observations R-squared Number of countries 28	455 0.841	455 Strictness of regulation on the use of temporary contracts. 452 339 339 Version 1 (1985-2008) of the employment protection legislation indicator. 0.900 OECD. Stat 30 27 30	336 27
	Other controls: Notes:
	Member of the European Union		Dummy variable taking the value of 1 if the country is a member of the European Union and 0 if it is not.

Table A5 .2. Regression results on the employment rate

 A5 Standard errors are in parentheses. Significance levels: *significant at 10 per cent; **significant at 5 per cent; ***significant at 1 per cent. (1) Pooled ordinary least squares model (OLS); (2) GLS (either fixed-or random-effects); and (3) FGLS with AR1 correction.

			Employment rate		Low-skilled employment rate
		OLS	GLS	FGLS (AR1)	OLS	GLS	FGLS (AR1)
		(1)	(2)	(3)	(1)	(2)	(3)
	Cluster (Policy 1)	2.919*** (0.473)	0.917*** (0.270)	1.167*** (0.346)	4.208*** (0.930)	0.941** (0.454)	1.850*** (0.712)
	Job rotation and job	-3.201**	-0.846	-0.832	-9.899***	1.214	-0.908
	sharing (Policy 2)	(1.590)	(0.721)	(0.724)	(3.809)	(1.252)	(1.497)
	Start-up incentives	5.918***	0.914	1.898**	11.07***	-0.777	1.577
	(Policy 3)	(1.516)	(0.848)	(0.967)	(3.595)	(1.481)	(2.030)
	Cluster * PES allocation	-0.381 (0.524)	-0.116 (0.337)	0.297 (0.378)	2.141* (1.245)	1.026* (0.593)	1.826** (0.799)
	Cluster * Timing	-2.154*** (0.442)	-0.711*** (0.254)	-0.791** (0.330)	-3.509*** (0.883)	-0.893** (0.426)	-1.638** (0.682)
	PES allocation	10.35*** (1.826)	2.163 (1.432)	4.120*** (1.351)	15.48*** (4.431)	-3.743 (2.807)	2.443 (3.012)
	Continuity in	0.144***	0.232***	0.191***	0.393***	0.180*	0.183**
	implementation	(0.0379)	(0.0477)	(0.0385)	(0.0916)	(0.0939)	(0.0828)
	Correct timing of	-2.662**		-4.247***	-7.335***		-4.052*
	policies	(1.248)		(1.240)	(2.159)		(2.277)
	Cycle * Timing	0.000595** (0.000240)	-0.000422 (0.000563)	0.000891*** (0.000267)	2.16e-05*** (6.05e-06)	-2.44e-05** (1.04e-05)	5.26e-06 (6.15e-06)
	Cycle	-0.000589** (0.000240)	0.000428 (0.000563)	-0.000884*** (0.000268)
	Growth rate of real	-0.531***	-0.185***	-0.0283	-0.590**	-0.173**	0.0587
	GDP	(0.101)	(0.0454)	(0.0380)	(0.247)	(0.0799)	(0.0796)
	Unemployment rate of	-0.916***	-1.057***	-0.768***	-0.507**	-0.809***	-0.433**
	the high skilled	(0.0997)	(0.0734)	(0.0825)	(0.219)	(0.131)	(0.182)
	Union density	0.0782*** (0.0154)	-0.225*** (0.0242)	0.0730*** (0.0168)	0.140*** (0.0370)	0.142*** (0.0432)	0.117*** (0.0395)
	EPL for temporary	-0.660***	-0.504**	-0.516**	4.067***	-0.125	2.381***
	workers	(0.234)	(0.216)	(0.206)	(0.531)	(0.374)	(0.445)
	EU	-10.12*** (0.575)		-9.932*** (0.783)	-11.59*** (1.249)		-10.10*** (1.744)
	Constant	75.53*** (1.982)	82.02*** (1.331)	78.57*** (1.783)	46.16*** (3.845)	49.58*** (2.409)	49.36*** (3.354)
	Observations	367	367	364	355	355	352
	R-squared	0.746	0.762		0.499	0.331
	Number of countries		30	27		30	27
	Notes:

Table A5 .3. Regression results on the labour force participation rate (LFPR)

 A5 Standard errors are in parentheses. Significance levels: *significant at 10 per cent; **significant at 5 per cent; ***significant at 1 per cent. (1) Pooled ordinary least squares model (OLS); (2) GLS (either fixed-or random-effects); and (3) FGLS with AR1 correction.

			LFPR			Low-skilled LFPR
		OLS	GLS	FGLS (AR1)	OLS	GLS	FGLS (AR1)
		(1)	(2)	(3)	(1)	(2)	(3)
	Cluster (Policy 1)	0.561*** (0.0617)	-0.00223 (0.0451)	0.340*** (0.0535)	5.338*** (0.978)	0.223 (0.479)	1.360* (0.695)
	Job rotation and job	-3.169*	-1.184*	-1.481*	-4.987	0.568	-0.0841
	sharing (Policy 2)	(1.716)	(0.674)	(0.786)	(3.917)	(1.208)	(1.404)
	Start-up incentives	5.774***	1.357	0.587	8.335*	0.513	0.223
	(Policy 3)	(1.752)	(0.889)	(1.126)	(4.272)	(1.677)	(2.130)
	Cluster * PES allocation				2.730** (1.239)	1.552*** (0.580)	1.736** (0.755)
	Cluster * Timing				-4.957*** (0.940)	-0.551 (0.446)	-1.345** (0.665)
	PES allocation	8.395*** (1.714)	-2.154 (1.381)	3.540** (1.377)	22.23*** (4.412)	-5.856** (2.753)	0.550 (2.867)
	Continuity in	0.0852***	0.214***	0.112***	0.136**	0.190**	0.155**
	implementation	(0.0229)	(0.0439)	(0.0265)	(0.0580)	(0.0899)	(0.0606)
	Correct timing of	-4.681***		-3.748***	-1.006		-3.246
	policies	(0.673)		(0.794)	(2.100)		(2.172)
	Growth rate of real	-0.560***	-0.247***	-0.0705*	-0.403	-0.204***	0.0341
	GDP	(0.105)	(0.0430)	(0.0428)	(0.246)	(0.0781)	(0.0762)
	Population with tertiary	0.0816***	0.0369***	0.0494***	-0.0901**	0.00879	-0.0572*
	education	(0.0146)	(0.0112)	(0.0139)	(0.0372)	(0.0219)	(0.0304)
	Unemployment rate of	-0.230**	-0.479***	-0.201**	0.291	-0.274**	0.0714
	the high skilled	(0.0963)	(0.0616)	(0.0840)	(0.223)	(0.131)	(0.180)
	Union density	0.0848*** (0.0142)	-0.102*** (0.0278)	0.0999*** (0.0156)	0.200*** (0.0376)	0.176*** (0.0516)	0.148*** (0.0390)
	EPL for temporary	-0.856***	-0.629***	-0.873***	3.666***	-0.562	1.525***
	workers	(0.228)	(0.234)	(0.204)	(0.533)	(0.419)	(0.437)
	EU	-7.047*** (0.559)		-7.094*** (0.691)	-11.71*** (1.316)		-8.806*** (1.784)
	Constant	73.80*** (1.492)	81.50*** (1.724)	76.04*** (1.461)	44.05*** (3.589)	54.10*** (3.175)	57.05*** (3.470)
	Observations	339	339	336	329	329	326
	R-squared	0.622	0.507		0.481	0.141
	Number of countries		30	27		30	27
	Notes:

Table A5 .4. Regression results on the share of low-skilled unemployed individuals Share of low skilled unemployed individuals

 A5 Standard errors are in parentheses. Significance levels: *significant at 10 per cent; **significant at 5 per cent; ***significant at 1 per cent. (1) Pooled ordinary least squares model (OLS); (2) GLS (either fixed-or random-effects); and (3) FGLS with AR1 correction. Model (3) includes country dummies.

		OLS	GLS	FGLS (AR1)
		(1)	(2)	(3)
	Cluster (Policy 1)	-3.044*** (1.008)	-2.431** (0.988)	-1.995** (0.955)
	Job rotation and job	5.036**	4.922**	2.287
	sharing (Policy 2)	(2.483)	(2.474)	(2.008)
	Start-up incentives	-1.131	-1.099	0.175
	(Policy 3)	(4.222)	(4.157)	(3.734)
	Cluster * PES allocation	3.258** (1.367)	3.505*** (1.335)	1.816 (1.153)
	Cluster * Timing	1.892** (0.939)	1.275 (0.920)	1.284 (0.903)
	PES allocation	2.203 (5.921)	1.831 (5.734)	-0.586 (4.553)
	Continuity in	-0.140	0.0215	-0.137
	implementation	(0.196)	(0.137)	(0.132)
	Correct timing of		-6.246	-16.20**
	policies		(7.092)	(6.392)
	Growth rate of real	0.388**	0.318*	0.189
	GDP	(0.169)	(0.167)	(0.118)
	Population with tertiary	-0.00589	-0.0475	0.00387
	education	(0.0450)	(0.0411)	(0.0425)
	Unemployment rate of	-0.977***	-0.831***	-0.756***
	the high skilled	(0.273)	(0.264)	(0.256)
	Union density	0.252** (0.106)	0.186** (0.0865)	0.185* (0.0976)
	EPL for temporary	3.059***	3.342***	1.652**
	workers	(0.927)	(0.858)	(0.779)
	EU		0.235 (6.358)	-28.07*** (5.960)
	Constant	27.66*** (6.714)	37.26*** (9.422)	60.95*** (6.493)
	Observations	309	309	306
	R-squared	0.928
	Number of countries		30	27
	Notes:

See, for example, Anxo et al. (2001) for a regional aggregate impact analysis of ALMPs in France and Sweden, carried out by means of an augmented matching function.[START_REF] Schmid | Does active labour market policy matter? An aggregate impact analysis for Germany[END_REF] carried out a similar analysis for Germany, with special focus on the long-term unemployed, and de[START_REF] De Koning | The impact of active labour market policy on job hirings and unemployment in the Netherlands[END_REF] for the Netherlands. More recently,[START_REF] Hujer | Estimating the macroeconomic effects of active labour market policies using spatial econometric methods[END_REF] used this approach for Western Germany and[START_REF] Dauth | Macroeconometric Evaluation of Active Labour Market Policies in Austria[END_REF] for Austria.

[START_REF] Bassanini | Employment Patterns in OECD Countries: Reassessing the Role of Policies and Institutions[END_REF] and[START_REF] Scarpetta | Assessing the Role of Labour Market Policies and Institutional Settings on Unemployment: A Cross-country Study[END_REF] also analyse the effects of ALMPs on unemployment from a cross-country perspective but not as their main objective. Their central aim is to unveil the effects of institutional settings on unemployment and therefore they include ALMPs as a single variable or as an interaction.

In fact, there are a number of studies that have successfully used these techniques in international empirical analysis of labour market policies other than ALMPs.[START_REF] Cahuc | Is Short-Time Work a Good Method to Keep Unemployment Down?[END_REF], for example, used IV and GMM methods in their cross-country analysis of short time work arrangements during the 2008 recession to treat the endogeneity bias that plagued their results. Both IV and GMM methods yielded stable results, successfully correcting the endogeneity bias. In a similar study (using somewhat different estimation strategies),[START_REF] Hijzen | The Role of Short-Time Work Schemes during the Global Financial Crisis and Early Recovery: A Cross-Country Analysis[END_REF] address the problem of endogeneity through the use of an instrumental variable for short-time work based on the number of years for which a scheme has been in existence.

The ratio of government expenditure in ALMPs to GDP has also been used in the past as discussed above(OECD, 1993). In this paper, this definition does not seem to represent the correct policy stance of governments since in our sample of countries the ratio of ALMPs to GDP does not remain constant over time. It declines even when considering periods of economic growth only.

However, results were not robust when both rates were used in the analysis, thus the middle-skilled unemployment rate was dropped to avoid the presence of multicollinearity.

The augmented Dickey-Fuller test was used to control for the non-stationarity of dependent variables. In all cases (but one) the tests rejected the null hypotheses of non-stationarity at 1 and 5 per cent levels. The exception was the variable share of low-skilled unemployed individuals which did not pass the test and can be therefore assumed to be non-stationary.

All variables obtained from OECD were gathered from OECD.Stat, which is an online repository of data and metadata for OECD countries and selected non-member economies.

KILM (Key Indicators of the Labour Market) and ILO.Stat.

This is, item 16 (authorisation and reporting requirements for TWAs) and 17 (equal treatment for TWA workers) (www.oecd.org/employment/protection).

Note that the analysis is based on 27 countries. Israel was dropped from all regressions due to total unavailability of information about union density. Estonia and Luxembourg were dropped because not enough temporal information is available on EPL for temporary workers to carry out the analysis. The same occurred with Slovenia but with two variables, union density and EPL for temporary workers. This is relevant for all equations presented in tables 1, 2, 3 and 4.

The graphic interpretation of the different interaction effects of all estimations is available upon request and in Table2of the Extended Appendix.

Given the equation of the interaction effect: unr = a + b1 cluster + b2 (cluster*timing) + b3 timing + e, the effect of the interaction term when timing=1 is given by unr = (b1 + b2)*cluster + b3 and the effect of the interaction term when timing=0 is given by unr = b1*cluster.

It is important to note that although the policy cluster*PES allocation and cycle*timing interactions are non-significant for the overall unemployment rate, they have been included in the specification given the results of the joint test of interaction coefficients in favour of their inclusion (i.e. interaction terms are jointly significant). In this circumstance, a misspecification would occur when considering a reduced version of the model without these interactions[START_REF] Baum | An Introduction to Modern Econometrics Using Stata[END_REF].

In particular, these control variables show significant results when used as explanatory variables of the two main policy variables policy cluster and start-up incentives. These results are available upon request and in Table3of the Extended Appendix.

It is important to note, that in the case of the LFPR of the overall population individual and joint tests for the non-significance of interactions terms could not be rejected so all interactions were dropped from the equation.

This mainly includes the cost of the unemployment insurance system, but also the 50 per cent of the special employment assistance programme (AEPE) bared by the unemployment insurance system; and the 40 per cent specific solidarity allowance and pension equivalent allowance financed by the solidarity contributions of State employees (Author's calculations based on Eurostat, 2013b).

The variable start-up incentives was not included among the endogenous variables because the results of the C test of exogeneity did not favour its inclusion. Indeed, when the orthogonality conditions of the model including this variable as exogenous were tested, the C test indicated robust results.

The above, however, would imply that individuals are myopic in terms of the actual risks they face of unemployment. If this were true, myopic individuals would compare themselves with the rest of the population rather than with their vulnerable group. This will give them a sense of an increased competition, which will reduce their wage bargain (Bellman and Jackman 1996a). Moreover, many authors believe that state dependence underlying reduced search effectiveness is not a sufficient explanation and that the assumption of myopia is overrated[START_REF] Calmfors | Macroeconomic effects of active labour market programmes in a Union Wage-Setting Model[END_REF] Huger et al. 2009).

Israel has been dropped from all regressions due to total unavailability of information about union density. Estonia and Luxembourg were dropped from GLS (AR1) regressions because not enough information on EPL for temporary workers was available to carry out the analysis. The same occurred with Slovenia but with two variables, union density and EPL for temporary workers. This is relevant for all equations presented in this appendix.

Short-term

Participants may reduce their search efforts in the knowledge that the employment services will find work for them (Bellman and Jackman 1996b).

Reduced employment -Direct job creation Job creation

Wage-setting schedule (workers subsidies). Labour demand (hiring credits).

Job creation (intended effect)

Short-term (effects on the labour market on the medium-term)

Wage subsidies reduce workers' wage expectations increasing labour supply at any given market wage and reducing unemployment [START_REF] Ohlsson | Job Creation Programmes as Stabilisation Policy Measures[END_REF]. Hiring credits, on the other hand, reduce the effective wage paid by employers shifting labour demand upwards [START_REF] Neumark | Policies to Encourage Job Creation: Hiring Credits vs. Worker Subsidies[END_REF].

Increased employment and effective wages.

No inflation of real wages in the long term.

-Direct job creation -Employment incentives and other subsidised employment policies.

Deadweight loss Labour demand

Deadweight loss (unintended effect)

Short-to medium-term Reduced efficiency of programmes since hiring from the target group would have occurred even in the absence of the programme [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF][START_REF] Martin | What works and for whom: a review of OECD countries' experience with active labour market policies[END_REF][START_REF] Cebrián | La Estabilidad Laboral y los Programas de Fomento de la Contratación Indefinida[END_REF].

Long-term

Training would adjust the qualifications of jobseekers to the structure of demand (OECD, 1993) and reduce "structuralization" [START_REF] Schmid | Does active labour market policy matter? An aggregate impact analysis for Germany[END_REF]. Specifically for the low-skilled, it would increase cross-sector mobility by qualifying them for work in sectors where the demand for labour is growing (Bellman and Jackman 1996a).

Lower unemployment at least among targeted groups.

-Training -To certain degree supported employment and rehabilitation Productivity Labour demand Increased productivity (intended effect)

Long-term

Increases in productivity can have externalities that contribute to general productivity increases (OECD, 1993) and to general technical progress of societies [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF].

Reduced unemployment (subsidisation) and directjob-creation measures with on-the-job training components.

-Training -To certain degree supported employment and rehabilitation (subsidisation) and direct job creation measures with on-the-job training components.

Productivity

Labour demand Substitution effect (unintended effect)

Long-term Importantly, the labour demand can be reduced if this scale effect of labour productivity outweighs the substitution effect arising because a given output can be produced by fewer, more efficient workers (usually when the labour demand is elastic) [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF].

Increase in employment and wages if scale effect offsets substitution effect.

-Training -To some extent, subsidised employment policies.

Labour supply Wage-setting schedule

Increased search effectiveness (intended effect)

Short-term

Maintain the unemployed active and available during recessions, which would reduce the number of vacancies and lower the wage pressure [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF]OECD, 1993).

Positive effect on the effective supply of labour.

-PES -To some extent, employment incentives

Competition for insiders

Wage-setting schedule Increased competition (intended effect)

Short-term Downward pressure on wages due to increased competition for vacancies (Bellman and Jackman 1996b;OECD, 1993;Layard et al. 1991).

Lower wages and lower unemployment.

-

PES -Training

Matching process

Wage-setting schedule

Improved search effectiveness (intended effect)

Short-to medium-term

Placement services can improve the effectiveness of search [START_REF] Blasco | Employment policies, hiring practices and firm performance[END_REF]Bellman and Jackman 1996b). Some believe, however, that this positive effect depends on quality of implementation (de Koning, 1993).

Reduced unemployment and reduced vacancies.

-PES Matching process

Wage-setting schedule

Reduced search effectiveness (unintended effect)

Short-term

Assistance with job search might raise wage pressure by reducing the fear of unemployment [START_REF] Calmfors | Does active labour market policy increase employment? -Theoretical considerations and some empirical evidence from Sweden[END_REF].

Reduced employment

-PES -To some extent, direct job creation Matching process Labour demand Improved search effectiveness (intended effect)

Medium-to long-term

Assistance with job search might also increase the number of vacancies because opening posts becomes less costly for firms [START_REF] Pissarides | Equilibrium Unemployment Theory[END_REF][START_REF] Calmfors | Macroeconomic effects of active labour market programmes in a Union Wage-Setting Model[END_REF]OECD, 1993).

Increased labour demand and reduced unemployment.

-

Employment incentives:

Recruitment incentives: Promotion of regional mobility and entry into employment (travel allowance); Promotion of regional mobility and entry into employment (childcare allowance); Integration subsidy (EB); Allowance for enterprises without employees.

Employment maintenance incentives: Promotion of investment and restructuring.

Supported employment and rehabilitation:

Supported employment: Support for employment of the disabled through the BSBs; Integration enterprises (BSB).

Rehabilitation: None reported.

Direct job creation:

Socio-economic enterprises (SÖB) and non-profit employment projects (GBP); Childcare institutions.

Start-up incentives:

Business start-up programme (UGP+GB). Notes: Standard errors are in parentheses. Significance levels: *significant at 10 per cent; **significant at 5 per cent; ***significant at 1 per cent.

All models have been estimated by 2SLS, using gmm and robust options to compute efficient estimates in presence of heteroskedasticity and autocorrelation. The exception is (7) where the robust option was not used.