Tommaso Agasisti

Antonio Dal Bianco

Paolo Landoni

Alessandro Sala

Mario Salerno

Evaluating the efficiency of research in academic departments: an empirical analysis in an Italian Region

Keywords: University research, efficiency, research policies, Data Envelopment Analysis, Malmquist index University research, efficiency, research policies, Data Envelopment Analysis, Malmquist index University research, efficiency, research policies, Data Envelopment Analysis, Malmquist index JEL Codes. C14, H41, H52

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

In recent years, Higher Education Institutions (HEIs) have been increasingly studied.

In nowadays ''knowledge economy'' their importance for economic development, social equity, mobility, social cohesion and integration is widely acknowledged (Brennan & Teichler, 2008). Furthermore, given the difficult situation of public finances, considerations about resources allocation have been raised in many countries, calling for more evaluations and accountability. For these reasons the so called 'New Public Management' movement, started at the beginning of the 1980s in the public sector in general, has been also increasingly affecting the universities.

New Public Management (Hood, 1991;1995) represents the request to public organisations to become more managerial, in particular by drawing on practices and models from the private sector. This process has been driven both by universities themselves and by governments through reforming efforts. The results, especially in many countries within Europe, share some common characteristics: i) the introduction of incentives schemes based on financial models for allocating public funds, ii) new or strengthened assessment procedures and evaluation agencies, and iii) more emphasis on universities' autonomy (OECD, 2003).

In this work we explored how the recent changes in the university system affected the efficiency of university research. In particular we were interested in understanding whether an overall increase in efficiency can be recognised and/or whether differences are present between universities given the previously noted enlarged autonomy. As a matter of facts, normally the majority of the reforming efforts have been directed towards education activities, whilst an increase in efficiency of university research has been assigned to incentive systems and evaluation processes. These are expected to have had an overall positive effect on the system, but at the same time they have left the universities in charge of finding and managing the ways to obtain better results. Given the autonomy of the universities and the decentralisation of responsibilities significant differences in terms of efficiency changes are thus expected between and within universities.

Furthermore, the idea that performance differentials in terms of research are key for competition among departments is well-established in the literature (Curran, 2000).

We decided to focus on universities research activities and not on education or knowledge/technology transfer activities. Universities' education activities constitute the first and original mission of the universities, but at the same time, they are normally carefully regulated by laws and already widely studied in terms of efficiency and quality (e.g. Madden et al., 1997; section 2). Indicators such as number of degrees awarded, rate of success, years of study, etc. have been widely used both to evaluate the universities and to introduce new policies. The opposite holds for universities' knowledge/technology transfer activities. First of all, even if some argue for this to be a third mission for academia and call for "entrepreneurial universities" (Clark, 1998;Etzkowitz, 2003), others refuse this perspective and signal the risks of focusing on and expanding the role of these activities. Moreover, the efficiency analysis of these activities is hampered by the difficulties in finding the right indicators. As a matter of facts, there are many different knowledge/technology transfer activities (e.g., spin-off, consultancies, research projects with firms, patents, etc.) and their evaluation can be based only partially on economic values given the not for profit nature of universities (Landoni and Verganti, 2006). Finally, we decided to focus on the research activities because we share the vision that they are the core activities of universities, driving and informing both education activities and knowledge/technology transfer activities. This relationship especially holds for transfer activities because the value of the results transferred heavily rely on the underlying research they are based on.

In this work with the term "research" we adopt the standard definition provided by the OECD: "Research and development is a term covering three activities: basic research, applied research, and experimental development" (OECD, 2002) but we consider only research activities that are normally carried out within universities.

Furthermore in this work, as it is done in the evaluation systems previously cited, we focus our efficiency analysis on university departments and not on the university as a whole. Universities are composed of different research departments and usually they significantly differ in terms of fields of activities and thus publication outputs, coauthorship behaviors and thus efficiency. We believe that an analysis at the level of the university can put forward mixed results, while an analysis at the level of single departments is more precise and coherent with the objectives of an efficiency analysis. For this reason, in order to obtain a sample of comparable departments we focused our attention only on natural science departments, not considering the arts and humanities ones. The choice of the department level for the analysis is coherent with the focus on research activities: normally education activities are managed by organizational entities inside the university that are transversal to more departments and many transfer activities are centralized in a single organizational unit (i.e. Liason office or Knowledge / Technology transfer office).

Our analyses consider departments located in the Lombardy region (Italy). We have chosen a regional focus in order to have similar contextual variables in terms of policies, economic environment, etc. for the universities included in the analysis.

Deleted: conjoint laboratories, Deleted: research activities Deleted: The link to education activities is more complex (Elsen et al., 2009). A strong relation is generally understood to be present in a modern university (Clark 1997), but some authors underline that it could be weaker than expected and needed [START_REF] Robertson | The research/teaching relation: A view from the 'edge[END_REF]. For instance literature indicates that students' understandings of research, even at postgraduate level, may be poor (e.g. [START_REF] Deleted ; Zamorski | Research-led teaching and learning in higher education: A case[END_REF]. And others (e.g. Bonaccorsi et al., 2006) point at a potential trade-off between education and research performances. ¶ The importance of university research activities is also recognized at a political and economical level (e.g. in the 2000 Lisbon Declaration) and by the presence of specific evaluation system in many countries (e.g., in the U.S., U.K. and Australia) that are continuously improved and adapted to changing times (Hicks, 2009). Moreover, the growing phenomena of externally-funded academic research raised concerns for a renewed attention to the results of research activities (Hagermann, 2009). Furthermore, Lombardy is one of the more advanced European regions both in terms of (i) economic development and (ii) research activities and universities: more than 9.4 millions of habitants, GDP per capita >34,000 €; 12 universities are located in the Region -out of about 70 in all the country, and more than 230,000 students.

Finally, the regional level is increasingly recognized as a relevant level of analysis in terms of research and innovation activities. Many authors (e.g. Cooke et. al., 1997;Porter, 1998;Lechner e Dowling, 1999) underline that it is at a local level that the majority of the connections between the different innovation actors take place. The localization of knowledge and the role of proximity as a driver of economic development (e.g. clusters) have also highlighted the importance of regional and local government in fostering and strengthening the actors and their relationships.

To summarize, the focus of the paper is on efficiency of research activities in scientific departments in the Lombardy Region (Italy). Our research questions are two:

• did changes in the overall efficiency of departments' research occurred between 2004 and 2007?

• which are the main factors affecting efficiency?

The paper is organized as follows: the next section present the more recent literature on the topic, then the methodology and the data used are introduced, finally results are presented and conclusions are drawn.

Background

The international literature about universities' efficiency -through non-parametric techniques -has been developed both at institution-level (Athanassopoulos & Shale, 1997;McMillan & Datta, 1998;Ng & Li, 2000;Avkiran, 2001;Abbott & Doucouliagos, 2003;Flagg, et al, 2004;Warning, 2004) and department-level. The latter is more relevant for the context of the present study.

The contributions by Johnes & Johnes (1993;1995) highly influenced our methodological approach. In a first paper Johnes & Johnes (1993) measured the research performance of UK economics departments, by using staff numbers as inputs and publication counts and grants as outputs; the same strategy was adopted again for deriving efficiency scores of economics department in a subsequent paper (Johnes & Johnes, 1995). The authors ran several DEA models to test the robustness of results and claim for a possible use of results for policy purposes, These two contributions are the most important as they focus exclusively on the research dimension of the universities' work. Madden et al. (1997) studied the efficiency of economics departments in 29 Australian universities. Inputs were teaching and research staff, outputs were graduates and publications (for teaching and research respectively). As in other studies listed below, here the focus was both on teaching and research performance, even if the analysis were conducted at subunit level (that is at departmental level instead of institutional level). The specific objective of the Madden et al. (1997) study was to verify whether policies promoted by the governments impacted on the productivity; they confirmed that there was actually an increase in productive efficiency. Thursby (2000) analyzed the efficiency of the Economics Research Departments in the US. He defined efficiency as follows: "(…) a department is deemed technically efficient if, when compared to departments with similar level of inputs, it could produce greater research outputs without increasing its inputs usage, or equivalently, it is one which, compared to departments with similar levels of outputs, could produce the current levels of outputs with fewer inputs" (p. 400). He used publication, citation data and number of Ph.D.s as outputs, and faculty size, full professors, graduate students, federal grants, library acquisitions and a dummy for private ownership as inputs. He found that the average level of efficiency in US is incredibly high (67% of economics departments appear as efficient). Seen from a European perspective, this result is not so unexpected. Moreno & Tadepalli (2002) focused on 42 academic units distributed across 7 colleges in the US. They used staff, financial resources and proxies for structures (e.g., building space) as inputs, and number of students, FTE enrolments and grant awards as outputs. In discussing their results, they evidenced that efficiency scores do not illustrate information about effectiveness -that is, they argue that a qualitative dimension is necessary for this purpose. Lastly, as their background is strongly managerial, they also suggest a use of efficiency analysis as a planning tool, not only for evaluating universities' activities. Koksal & Nalcaci (2006) derived efficiency scores for 14 university departments at the Middle East Technical University (METU) in Turkey, by using staff salaries, a composite indicator of previous research quality and number of students as inputs, and number of publications, graduates, and teaching indicators as outputs. Thus, in their contribution, the authors focus also on the teaching activity of the departments. university's administrators through a preliminary survey. The paper suggests that the models adopted for computing efficiency can serve to set new targets for inefficient departments. Gimenez & Martinez (2006) focused on cost efficiency of the departments at the Autonomous University of Barcelona. They computed cost-efficiency scores and separated cost excess into three categories: technical efficiency, fixed factors, scale of size. The units of analysis were 42 departments, and again the focus is on both teaching and research activities. The inputs chosen were: expenditure on temporary hired teaching and research staff, operational expenditures, expenditures on tenure staff. To measure the outputs, they employed the following variables: results of a government evaluation process of research, teaching load (measured through credits granted) and students' opinions. The authors suggest that their analyses can be used to assess the differences between (i) the observed cost of the units and (ii) the cost they would achieve assuming long-term cost minimization. Their result tells that size does not matter in determining efficiency, while higher proportion of non-tenured staff is associated with higher efficiency levels.

In another exercise presented by Kocher et al. (2006) the analysis is focused on the productivity of research in economics across 21 OECD countries. The focus of the paper is on research and the outputs selected are top publications (and impact factors), while inputs are R&D expenditures, number of universities and total population. This study is quite different, however, with respect to the relevant contributions for the present paper, as the unit of analysis is the country (neither the university nor the department). The results suggest that research productivity can be raised through an increase in the scale of operations.

An interesting issue is to analyze the related literature about the Italian universities' research. To the best of the authors' knowledge, there are only two specific contributes which specifically analyze the efficiency of universities in producing research outputs (Pesenti & Ukovich, 1996;Rizzi, 1999), and just one attempting to focus on teaching production (Agasisti & Dal Bianco, 2009).

The two contributions at departmental level (Pesenti & Ukovich, 1996;Rizzi, 1999) use a similar set of variables and focus both on teaching and research outputs. The main scope for these contributions is to provide information and suggestion for allocating resources within a university.

Contributes about the efficiency of Italian higher education has been conducted at institution-level (thus, not considering separately the structures devoted to teaching and research). The reason for this choice can be summarized by the idea that the strategy and the mission of universities is defined at macro(institution) level, then the balance between teaching and research is decided by the decision centres at institution level (Bonaccorsi & Daraio, 2007, p.22 ).

Among these contributions at an institutional level Agasisti & Dal Bianco (2006) studied the efficiency of 58 Italian public universities, by using number of staff, financial resources and students as inputs, and graduates and research grants as outputs. They suggest the use of efficiency analysis for policy purposes, especially to evaluate the differences in terms of performance between efficient and inefficient units. Later, Agasisti & Salerno (2007) analysed the cost efficiency of another sample of Italian universities, by adding some qualitative measures for inputs and outputs.

They conclude that a redistribution of inputs across universities could lead to an overall improvement of efficiency in the Italian HE sector. Bonaccorsi et al. (2006) used a robust non-parametric technique (developed by Daraio & Simar, 2005) to take in account external conditions affecting the productivity of universities. The inputs and outputs adopted are quite similar to previous studies (that is, staff numbers, expenses and spaces as inputs, and publications and graduates as outputs), but they also include some "external" factors: university size, number of departments within the university, percentage of private funding, number of curricula activated. Their conclusions are there economies of scale and scope are not relevant factors in explaining (in)efficiency, but that a strong role is instead played by the external factors -clearly, such result affects the policy considerations.

To sum up, the literature seems to suggest that, to analyse research efficiency of universities -that is, the efficiency in the production of research outputs:

• inputs can be classified under three main groups: human resources, financial resources and structures (e.g. facilities, buildings, etc.);

• outputs can be grouped into publications and grants.

Clearly, the best indicators to measure these classes of inputs/outputs are questionable, but the general emerging consensus is on a productive process that can be described with such indicators. The present paper moves from these suggestions, and it focuses on the efficiency of research activities in scientific departments in the Lombardy Region (Italy).

Methodology and data

Methodology

The methodology is articulated accordingly to the two steps adopted to answer the research questions of the paper: on one hand the efficiency analysis (including The first choice is about the methodology to compute efficiency, because there are different methods to target the analysis of efficiency among different units.. Data Envelopment Analysis (DEA) is a frontier, linear programming technique, introduced by the seminal work of Charnes, et al. (1978). It deals with the efficiency problem of complex organizations that produce many outputs using several inputs at a time.

In this paper we have only briefly discussed some characteristics of DEA, referring the interested reader to Johnes (2004); Cooper et al., (2006), andZhu (2003).

In a DEA model, the notion of efficiency is that of "technical efficiency", defined as the relative ability of each Decision Making Unit (DMU) in producing outputs, where the term "relative" means that each organization is compared with any other homogeneous unit. DEA could be represented by a linear programming technique where each DMU tries to maximize the efficiency ratio (output over inputs) choosing the best set of weights. There are two ways to deal with a DEA formulation: (i) input oriented (that is, the ability to minimize inputs when outputs are given), and (ii) output oriented (maximization of outputs given a certain bundle of inputs). In this paper, we adopt this latter approach, as the typical problem for a research unit is to maximize outputs (e.g., publications, grants, etc.) given its inputs (e.g. research staff, laboratories, etc. DEA mathematical formulation can deal with both constant returns to scale (CRS) and variable returns (VRS). The seminal work of Charnes et al. (1978) introduced a constant return to scale (CRS) model where DMU dimension has no importance in defining efficiency performance (see equation 2). DMUs face the same efficiency frontier, independently of their relative size. Obviously, one can cast doubts on the comparability of small and large units. Larger units exploit common inputs to produce different outputs, whereas smaller ones benefit from substantial advantages in organizing activities. The BCC model, developed by Banker et al. (1984) introduces in DEA modelling the dimension factor. Smaller units are associated with increasing economies of scale, and bigger units with decreasing economies of scale. In the middle, there is a "grey area" of constant returns to scale. So, it is also possible to compute the scale efficiency. This is defined as the ratio CRS/VRS efficiencies, and interpreted as the ability of each institution to benefit (in terms of productivity) from its size -in other words, in a VRS formulation, each DMU is compared only with those units having a comparable size.

We also measure changes in efficiency across time, and our relevant period is between 2004 and 2007. For measuring the determinants of the changes in efficiency scores, we use Malmquist index (for more information about this index see Formatted: Tabs: Not at 153.75 pt Deleted: There are two main categories: parametric and nonparametric methods Deleted: , that is the method chosen in the present study, belongs to the latter group, as it does not require a priori functional specification to mix input and output such as a production function. DEA Deleted: The choice of a set of weights that combine several outputs and several inputs is the core of DEA analysis.

Deleted: .). Following Johnes (2006) Johnes, 2004 andCoelli et. al., 2006). Let us consider a DMU, named P that has a change in production position from P t (in period t) to P t+1 (in period t+1). The change in the production frontier could have occurred in a non-neutral way -that is, it could be determined both by efficiency changes and shifts in the efficiency frontier. Thus, for measuring the change in the efficiency score, we have to break down the score in these two different components. Both of the components have a specific mean: the resulting score for the DMU could have positive or negative effects due to the own efficiency change or due to the shift (movement) of the frontier. Thus, the resulting efficiency index is composed by two parts. The first component of the equation is the ratio of technical efficiency in time period t+1 on the technical efficiency in the period t. Since it measures the change in technical efficiency, it is equal to 1 if no changes occurred, greater than 1 if the change was positive, and less than 1 if it was negative.

The second component measures the shift in the production frontier: if it is equal to 1 it has no effect on the efficiency score, if it is greater than 1 it has a positive effect and if it is less than 1 it has a negative effect. In other words, a shift in a production frontier that makes worse the relative position of the DMU (the other DMUs are performing relative better than before) provides a second component less than 1.

The index (Malmquist index) resulting from the combination of these two components will be equal to 1 if there is no net effect of changes in technical efficiency and frontier changes; it will be greater than 1 if the net effect is positive, and less than 1 if it is negative. In our analysis, we name "efficiency change" the first component and "frontier shift" the second one.

Moreover, we would detect the eventual stability of efficiency rankings across the period (again, the relevant period is between 2004 and 2007). In this case, we adopted one of the statistics suggested by Brockett et al. (1998). More specifically, we are interested in knowing if the efficiency of departments changed influencing the rankings or not over time. "Ranking the set of nxk efficiency ratings and observing the sum of ranks associated with each DMU, one can learn about the relative position of the DMUs vis-avìs each other across the whole period" (Brockett et al., 1998, p. 174). For this purpose, the basis of the analysis is a Kruskal-Wallis test (Kruskal & Wallis, 1952) where "(…) the rejection of the null hypothesis leads to the conclusion that, in general, the different DMUs maintain their relative efficiency positions over time" (p. 175).

The second task in our methodology is to study some potential determinants of efficiency. To pursue this objective, we identified some external factors (e.g. "location" effects, "discipline" effects, etc.) and then we compared efficiency scores Deleted: (called P) Deleted: From an analytical point of view, the index is defined as (Johnes, 2004, pp. 621-624) (3) Deleted: ¶ ¶ where OPi is the radial distance between the origin O and the point Pi, and OPi,j measures the distance from the origin to the point Pi,j,.

Deleted: . ¶ Assuming the existence of n populations simultaneously under investigation, and that all n populations have the same distribution of ratings, then the statistic is: ¶ ¶ (Kruskal & Wallis, 1952). This test allows the researcher to detect if are there statistical differences across the groups analyzed (e.g., when comparing the efficiency scores of units located in a metropolitan area with those located elsewhere, the test indicates if this location effect generates a statistical difference between the average scores of the two groups).

H = 12 n ⋅ k ⋅ (n ⋅ k + 1) ⋅ R 1 2 k + R k    (

Data

In this paper, we used a unique dataset that consists of input and output variables about research and technology transfer activities in university departments located within the Lombardy Region. This dataset is based on information coming from QuESTIO system, a mapping of Research and Technological Transfer organisations developed in Lombardy Region. The QuESTIO system, launched in 2004 by the Regional Government, was intended to be a transparency and incentive tool for the Research and Technological Transfer market by reducing any information unevenness often binding cooperation among producers and users of knowledge.

The system pursues such scopes by collecting, updating and publishing financial, organizational and structural characteristics of research organisations and Knowledge Business Intelligence Services in Lombardy.

Since it aims at presenting the most detailed picture of Regional community active in Research and Technological Transfer, the dataset has some specific criteria that are pivotal for our analyses. First, the system is focused on every structure where such activities are carried out, even where that structure is a part of a wider organisation.

Second, these structures should have significant management autonomy so that their accountancy shall be determinable. The first criterion enables the regional administration to collect information on every single department, being acknowledged that departments belonging to the same university can differ significantly in terms of performances. As a consequence the university departments, as well as sections and institute of National Research Council or R&D unit belonging to a Company, provide data making solely reference to their specific resources and outputs. The second criterion assures that each structure included in the dataset is responsible (to some extent) for its strategy and performances thus justifying application of DEA technique.

University departments are main actors in Lombardy Innovation System; as a matter of fact they are 147 out of 364 structures mapped, which balance the presence of Knowledge Business Intelligence Services, which count for 151. The map is Formatted: Tabs: Not at 153.75 pt Deleted: The analyses will be presented in the next section. In order to better point out the capillarity of these structures within the territory, the geographical analysis shows a concentration in the Province of Milan, and a little less in Bergamo, Brescia, Pavia and Varese, which is explained by the presence of universities in those cities, all of them accredited with their single departments.

Deleted: ¶ Mathematically, the test is computed as: ¶ ¶ H = 12 N(N +1) R i 2 n i i=1 k ∑       -3(N (
We built our dataset by considering those university departments that provided information for the whole period 2004-2007 (units for which there are not missing data), by selecting the following variables: As discussed in the previous section the choice of input and output variables is coherent with the most recent literature. The final sample contains 75 departments, for 4 years -overall, there are 300 observations. Descriptive statistics are illustrated in Table 1. <Table 1> around here

Results

In this section, we illustrate the results obtained through the DEA analysis on the university departments in the Lombardy Region.

The results are presented in two main sections. The first is devoted to a description of the efficiency scores with reference to the year 2007 (the last year available, where the results are more stable as previously discussed). In this section one subparagraph analyses the change in efficiency between the first year of our dataset This section explores the results of the efficiency analyses both looking at a single year (2007) and at differences between years.

DEA efficiency scores, year 2007

Table 2 reports the descriptive statistics about the efficiency scores derived with reference to the year 2007 (the annex reports the specific scores for each department). As explained in the methodological section, we calculated CRS efficiency and VRS efficiency, as well as the "scale" efficiency computed as CRS/VRS scores (it is important to recall here that the model is output-oriented).

<Table 2> around here

The research efficiency of university departments in the Lombardy Region looks quite high (0.75 in terms of CRS efficiency, 0.78 in terms of VRS efficiency); actually, the high average efficiency is coherent with previous results derived across the world in the higher educations sector (Salerno, 2003). What is worth pointing out is a strong heterogeneity across departments -indeed, the standard deviation is >0.2. A look at the lowest scores suggests that there are very poor performers which shows scores <0.5. Data about scale efficiency suggests that all the departments have almost reached their optimal scale of operations (given their present scale), as the scale efficiency score is very high (>0.93) and standard deviation is very low (about 0.1) in this respect.

The picture that emerges from this first analysis of efficiency scores is a sector where there is a number of very efficient units (both assuming constant or variable returns to scale) and a number of departments which experienced low levels of technical efficiency; thus, as the anecdotic evidence suggests, there is quite a strong differentiation among departments in terms of research efficiency.

As the focus of our paper is the efficiency of academic departments in the Lombardy Region, we also checked the robustness of our results by considering revenues from entities located in the Lombardy Region as a separate output (REV_LOMB). The correlation among results obtained by the baseline model and this modified model is very high (>0.97). the VRS efficiency scores for each department comparing the two models (also in this case the reference year is 2007) is available, on request, from the authors. 1.2. Efficiency changes between 2004 and2007 -a Malmquist analysis Following the methodology described above, we derived the Malmquist index, which allows us to disentangle the efficiency changes due to "pure" efficiency improvement (or worsening) and technological improvement (or worsening). The descriptive statistics about the results are reported in Table 3, while the detailed results are available on request from the authors.

<Table 3> around here

The picture that emerges highlights an interesting phenomenon. It looks like an improvement in productivity did not occur (because the Malmquist index is around 1) but, looking at the decomposition of the index, the reality appears more complex: an efficiency changes happened, it was very positive (the index is >1.6) and involved the major part of the departments (56 out of 70); on the other side, there was a relevant detriment of the technology environment (the "frontier shift" indicator is <0.7). It looks like the academic units improved their technical efficiency (that is, the ability to transform inputs into outputs) but, at the same time, the frontier worsened. In other words, what happened is that the "technological change" (in the Malmquist language) was negative.

What is "technology" in a regional research system, when we define research performance as the ability to attract financial resources and to publish in academic journals? Technology could be defined as the bundle of policies that should help in improving research efficiency, e.g. Information Technologies, research grants, support for publishing research results or to attract financial resources from international grants or companies, etc. Thus, the results contained in Table 3 suggest that, while the majority of the universities improved their efficiency, the system as a whole did not experienced an improvement in terms of technology available. A discussion about the potential responsibilities for such phenomenon will be reported in the last section, which deals with policy implications of our work.

Productivity changes between 2004 and 2007 -an analysis of "rankings stability"

Lastly, we analyzed the stability of rankings across our analysis period (between 2004 and 2007) by means of the H index proposed by Brockett et al. (1998). The index is equal to 177.96, and it must be compared with H 2 that is equal to 102.99.

The rejection of the null hypothesis leads to the conclusion that some of the academic departments exhibit consistently better economic performance than others (as measured by the input-output factors that were included in our model) (see Brockett et al., 1998).

Reading this results together with the Malmquist index, it looks like the academic departments improved their performance, but some units continue to perform better than others. At the same time, the results show that some units are not efficientthat is, the frontier actually worsened.

DEA efficiency scores, year 2007 -the external factors affecting efficiency

In this second section, given the evidence on the efficiency results, it is interesting to explore which are the determinants of this efficiency or, in other words, what could actually affect a department's performance.

To answer this question, we tested a number of different factors, by using the Kruskal-Wallis test as methodological approach (see the data section). All the results are listed in Table 4. <Table 4> around here 4.2.1 First external factor: are departments located in Milan metropolitan area (regional capital) more efficient than other departments?

We analysed this factor on the assumption that in a metropolitan area the competition among research centres enforces departments' managerial efficiency. In Milan, there is an important concentration of public and private research institutions and the competition for resources is not limited to departments. The high concentration of research facilities may justify a greater effort by departments in fostering a more balanced use in input resources or may impulse an effort in control for managerial quality. Elsewhere, Agasisti (2009) demonstrated that the effect of competition on Italian universities' performance is positive -but, there, output was defined through teaching dimension's measures.

We ran a Kruskal-Wallis test to verify the statistical difference between the efficiency scores of departments located in Milan versus the other departments. We do not find evidence of difference in efficiency performance between the two groups (indeed, the observed value of the H statistic is nearly 0, and the p-value >0.9). This evidence suggests that there is no "concentration" effect or "location" effect on efficiency performance of university departments or, at least, in the relative rankings. At the same time, this result may suggest i) the existence of no competitive pressure in a high concentration area as Milan, or that ii) we have to look at overall concentration of research centres, and not only to university departments (e.g., there could be a "university" effect which could dominate a "location" effect, or that iii) there is competition but at the same time more opportunities to find resources (more demand for research, and consequently the amount of resources is enough to guarantee a sufficient level of grants for different departments without the need to compete one against the others).

Second external factor: can different scientific sectors affect university department efficiency?

As we measured "output" by using i) publication counts and ii) research grants (of different kinds), it is important to recall that there are some differences in the average number and quality of publication across different disciplines (e.g. medicine versus engineering or biology), as well as differences in terms of attractiveness of research funds (e.g., in engineering fields it is simpler to attract grants for applied research than in physics).

The idea that disciplines greatly vary in terms of costs and productivity is not new, and the suggestion to take this factor into account is important to obtain not-biased results (Sarrico & Dyson, 2004). Recently, Sarrico et al. (2009) demonstrated a strong role of "subject mix" (defined as the disciplinary composition of institutions) in explaining the efficiency differentials among Portuguese higher education institutions.

The authors acknowledged that "(…) research production also varies significantly from subject to subject, with medical, some natural and technical sciences being more prolific in the number of articles than most of the social sciences and humanities" (p.290). Also Ramsden (1994), in his empirical analysis of Australian research departments, adjusted the coefficients on the basis of discipline subjects.

We find a small, but statistically significant difference among university departments classified by scientific sectors: the observed value of the H statistic is >8.4, near the critical value (9.4), and the p-value is 0.07 (then the effect has statistical significance at the 10% level). However, given the limited power of this effect, it is not easy to determine which sectors outperform the others straightforwardly.

Third external factor: is there a "university effect"?

An alternative explanation of efficiency can refer to the existence of a "university effect", that is if a university is more efficient than others, then its departments could be more productive than those belonging to other universities. This assumption finds its justification in the setting of strategies at university levels, not at subunits levels. In this respect, Bonaccorsi & Daraio (2007, pp. 22-23) The fundamental reason for assuming the university level as the unit of analysis is that at the university level the problem of attribution of inputs (in particular, human resources, funding and physical capital) to specific units of outputs, can be kept under control". Even if these authors recognized that a lower unit of analysis could provide useful information on the actual research process, they suggest that a "university effect" (due to different strategies and characteristics at macro-level) can be detected.

Our tests suggest that such effect is not explaining efficiency differentials for our sample (observed value of the H statistic is 9.4, well below the critical value of 14.06, and the p-value is 0.22).

Fourth external factor: does the number of tenured academic staff influence efficiency?

The proportion of tenured research staff theoretically can affect the efficiency of a department: there is anecdotic evidence that tenured and non-tenured staff face quite different stimulus and incentives. Nevertheless, the direction of this influence is not clear a priori: tenured staff is certainly more expert in attracting research grants and in publishing academic papers, also thanks to reputational advantages; conversely, non-tenured staff can have an incentive to be more productive, for instance as a mean to obtain a tenured position. The promotion system can affect productivity if researchers are satisfied with rules promoting this dimension (a discussion of motivations behind researchers' activity is in Ramsden, 1994, pp.214-215).

A complementary idea behind this question is that the different human capital of researchers (in our paper, indirectly measured via the tenured status) can affect the research productivity of the research groups -and so, of the department. In this respect, an interesting example is the paper by Rodgers & Neri (2007), who specifically attempted to correlate the human capital of researchers with the research productivity of Australian departments of Economics. They have found that human capital differential account for about 50% of the observed differences in research productivity. If human capital is (even indirectly) captured by the tenured status (e.g., researchers with more experience should have reached their tenured position), then we should expect a correlation between the proportion of tenured researchers and the department's results. Ramsden (1994) showed that productivity of professors and readers was much higher than that of lecturers. Our analysis did not find evidence of a statistical influence of this factor on research efficiency (the value of the H statistic is 0.16 and the related p-value is >0.6).

Fifth external factor: can departments' "longevity" affect research efficiency?

Finally, we looked at the possible effects due to the longevity of departments. As we measured inputs in terms of numbers of academic staff and laboratories/facilities, it is reasonable that older departments accumulated higher stocks of inputs that could influence their outputs.

A key assumption here is that older departments can favour more cohesion among its members, then facilitating the climate of "collaboration" and "common motivation" that some authors pointed out as a determinant of research productivity (Ramsden, 1994).

Another interesting point of view about this potential phenomenon is that it is related to the "critical mass" dimension. Among the economic aspects of research, Hare & Wyatt (1992) underlined that the ability in producing research relies upon the human capital accumulated by the individual researchers; thus, "combining this process with the notion that there are economies of scale in research may dispose university departments to feel they need to have at least some critical mass of researchers working in the same area in order to function properly as research units" (pp. 55-56).

Consequently, it could be reasonable to assume that older departments are more likely to have reached such "optimal" scale dimension than the more recently established ones.

The statistical analysis did not provide statistically robust results on this effect, indeed the value of the H statistic is 1.172 -while the critical value is around 3.8, so that the p-value is 0.27).

Discussion and policy conclusions

As highlighted discussing the literature on the topic, the analyses presented in this work are innovative in terms of data and methodology. Indeed, in this paper we have investigated the efficiency of university departments on science and technology and medicine in an Italian Region (Lombardy). The aim of the paper was (i) to analyze the changes in productivity in recent years (from 2004 and 2007), and (ii) to detect factors that are potentially affecting efficiency.

Discussing our results about the efficiency change between 2004 and 2007, some questions must be posed: how can a Malmquist index equal to 1 exactly be interpreted? How can we judge a frontier shift that is less than 1? Each academic the vast majority of the units actually improved their productivity in the period ("efficiency change" >1) but the "frontier shift" was <1: it means that the performance differential is much less dispersed in 2007, but at the same time it happens because the frontier is not improved -the best performers did not improve, or even worsened their efficiency. The summary statistics confirms this idea (look at the really lower mean in 2004, with a much higher dispersion -standard deviation -around it). The result is a "research system" in the region that performs better on average, but without having excellence.

The importance of evaluation and accountability has been growing in these recent years and criteria used are widely acknowledged: publications and the ability to gather financial resources (from abroad or from firms). Since the application of these evaluations for funding allocation is more and more concrete, we believe that the weakest departments raised their attention towards those performances. Thanks to this new attention the weakest departments have increased their efficiency and become much closer to the best performers that, on the contrary, had little margin for further improvements. This result requires an important policy discussion. Indeed, the objective of the Lombardy Region was to foster the performance of excellent research units. This purpose animated some policy interventions such as the creation of "clusters", the policies for improving contacts between companies' demand and research units through vouchers (the idea behind is that the best research units would be able to collect more vouchers), etc. Apparently, the results obtained are in the opposite direction.

This fact raises two reflections. The first is that maybe the universities (and, consequently, the research units) look not at the regional incentives, but at the national ones (as the Ministry is still the main financer of the universities' activities) and at international ones (the research reputation strongly depends upon the ability to publish in international journals and to attract international grants). If this is the case, a question arises about the opportunity for a regional government to be involved in this sector.

A second (possibly alternative) reflection is that the policies adopted for boosting research productivity were not adequate in terms of policy design. To properly design a policy in this field the necessary first step is to understand what affect the academic productivity. In this context, Faria (2002) suggests that policies for fostering the Our second analysis regarded how five external factors ("external" because they are not under the control of the departments) affect efficiency (section 4.2.). Our empirical analysis suggests that none of these factors (location in metropolitan area, scientific sector, university effect, proportion of tenured academic staff, departments "longevity") is able to adequately explain efficiency differentials. We consider this result as positive, especially when thinking to the use of efficiency scores for policy purposes (e.g., resources allocation). Indeed, a usual critic to the efficiency analyses is that they provide results which are affected by factors which are not under the control of the units themselves: when this is the case, a procedure of resource allocation following efficiency results would be inequitable. Among others, Marinova & Newman (2008) At the same time, we argue that there are some elements, which actually could influence efficiency and which are related to some "soft" characteristics of the academic units, typically (i) the incentive systems and (ii) the governance/management. For instance, Ramsden (1994) pointed out that the perception of the academic staff about the importance of publications for their career was consistently related to the production of academic articles and books. Rodgers & Neri (2007) suggested that the "departmental environment" is important and they define it through several measures centred on the "human capital" of researchers who work in the department. Unfortunately, QuESTIO dataset does not collect such data: an extension of this dataset along this dimension could help to improve significantly the understanding of this pattern.

However, if the incentive system really matters, another conclusion can be drawn from our study: as we measured a potential "university effect", and it is not

F o r R e v i e w O n l y 21
statistically relevant, then we can conclude that no incentive system works at university level. Different reasons can be proposed to explain this result. In particular, it can be both the cases that (i) the incentive system is designed at an higher level (in Italy, at the central government level), or (ii) that it is designed at a lower (department) level -and given the department-level of our data, we cannot exploit such dimension, which requires individual-level data. The evidence about the process for recruiting academic staff and its career in Italy suggests that, however, no government-level incentive exists -indeed, the career of professors is very poorly related to their scientific production. Levin (1991) underlined the importance of incentives to raise the productivity of universities' activities. More specifically, he acknowledged that higher education shows productivity declines due to two main reasons: (i) it is a "stagnant industry", as it is based on labour intensity, and (ii) clear rewards and property rights are absent.

Thus, the suggestion provided by the author is to develop a reward system within the university based on clear goals, financial incentives and evaluation procedures.

Whether such schemes have been actually implemented in departments analysed in this paper is not clear; probably detailed case studies can help to shed more light on this topic through future research.

The governance/management of the academic departments has not been analysed in our study, as the only information available in this respect (the presence of a quality system or a management control system) is a very indirect measure of the governance process and the data recovery on this topic was quite imprecise. Another way to improve the QuESTIO dataset should be to better investigate this dimension, by including some qualitative descriptions about the way Department managers organise and coordinate the activities and resources, formulate strategies and foresight studies and manage networking activities (see for instance Sala et al., forthcoming;Salerno et al., 2008). Without knowing the characteristics of the "management style" of each department, it is impossible to derive policy implications from such dimension. At the same time, a possible way to better understand the potential incidence of such phenomenon is to conduct case studies about the successful academic departments (e.g. those with higher efficiency scores or those which highly improved the scores in the last years).

Finally, a potential candidate to analyse the efficiency differentials is the interaction between research and teaching activities conducted by the departments. Typically, the literature did not paid enough attention to the interaction between teaching and research at department level, mainly because both activities were considered as inputs and outputs (see section 2). In this paper, the focus was exclusively on the research activities, but it raises also questions about the effects that the (unmeasured) teaching activities play on the research result. This issue is very relevant both in theoretical terms (are there economies of scope, or tradeoffs, between the two activities?) and in empirical ones (which is the optimal allocation of energies between teaching and research?). This interesting issue was at the centre of several contributes since the seminal paper by Cohn et al. (1989), that for the first time addressed and modelled the multi-outputs nature of academic activities.

Recently, the theme has been explored by Bonaccorsi & Daraio (2007)

Introduction and motivation

In recent years, Higher Education Institutions (HEIs) have been increasingly studied.

In nowadays ''knowledge economy'' their importance for economic development, social equity, mobility, social cohesion and integration is widely acknowledged (Brennan & Teichler, 2008). Furthermore, given the difficult situation of public finances, considerations about resources allocation have been raised in many countries, calling for more evaluations and accountability. For these reasons the so called 'New Public Management' movement, started at the beginning of the 1980s in the public sector in general, has been also increasingly affecting the universities.

New Public Management (Hood, 1991;1995) represents the request to public organisations to become more managerial, in particular by drawing on practices and models from the private sector. This process has been driven both by universities themselves and by governments through reforming efforts. The results, especially in many countries within Europe, share some common characteristics: i) the introduction of incentives schemes based on financial models for allocating public funds, ii) new or strengthened assessment procedures and evaluation agencies, and iii) more emphasis on universities' autonomy (OECD, 2003).

In this work we explored how the recent changes in the university system affected the efficiency of university research. In particular we were interested in understanding whether an overall increase in efficiency can be recognised and/or whether differences are present between universities given the previously noted enlarged autonomy. As a matter of facts, normally the majority of the reforming efforts have been directed towards education activities, whilst an increase in efficiency of university research has been assigned to incentive systems and evaluation processes. These are expected to have had an overall positive effect on the system, but at the same time they have left the universities in charge of finding and managing the ways to obtain better results. Given the autonomy of the universities and the decentralisation of responsibilities significant differences in terms of efficiency changes are thus expected between and within universities.

Furthermore, the idea that performance differentials in terms of research are key for competition among departments is well-established in the literature (Curran, 2000).

We to evaluate the universities and to introduce new policies. The opposite holds for universities' knowledge/technology transfer activities. First of all, even if some argue for this to be a third mission for academia and call for "entrepreneurial universities" (Clark, 1998;Etzkowitz, 2003), others refuse this perspective and signal the risks of focusing on and expanding the role of these activities. Moreover, the efficiency analysis of these activities is hampered by the difficulties in finding the right indicators. As a matter of facts, there are many different knowledge/technology transfer activities (e.g., spin-off, consultancies, research projects with firms, patents, etc.) and their evaluation can be based only partially on economic values given the not for profit nature of universities (Landoni and Verganti, 2006) Furthermore in this work, as it is done in the evaluation systems previously cited, we focus our efficiency analysis on university departments and not on the university as a whole. Universities are composed of different research departments and usually they significantly differ in terms of fields of activities and thus publication outputs, coauthorship behaviors and thus efficiency. We believe that an analysis at the level of the university can put forward mixed results, while an analysis at the level of single departments is more precise and coherent with the objectives of an efficiency analysis. For this reason, in order to obtain a sample of comparable departments we focused our attention only on natural science departments, not considering the arts and humanities ones. The choice of the department level for the analysis is coherent with the focus on research activities: normally education activities are managed by organizational entities inside the university that are transversal to more departments and many transfer activities are centralized in a single organizational unit (i.e. Liason office or Knowledge / Technology transfer office).

Our analyses consider departments located in the Lombardy region (Italy). We have chosen a regional focus in order to have similar contextual variables in terms of policies, economic environment, etc. for the universities included in the analysis. Furthermore, Lombardy is one of the more advanced European regions both in terms of (i) economic development and (ii) research activities and universities: more than 9.4 millions of habitants, GDP per capita >34,000 €; 12 universities are located in the Region -out of about 70 in all the country, and more than 230,000 students.

Finally, the regional level is increasingly recognized as a relevant level of analysis in terms of research and innovation activities. Many authors (e.g. Cooke et. al., 1997;Porter, 1998;Lechner e Dowling, 1999) underline that it is at a local level that the majority of the connections between the different innovation actors take place. The localization of knowledge and the role of proximity as a driver of economic development (e.g. clusters) have also highlighted the importance of regional and local government in fostering and strengthening the actors and their relationships.

To summarize, the focus of the paper is on efficiency of research activities in scientific departments in the Lombardy Region (Italy). Our research questions are two:

• did changes in the overall efficiency of departments' research occurred between 2004 and 2007?

• which are the main factors affecting efficiency?

The paper is organized as follows: the next section present the more recent literature on the topic, then the methodology and the data used are introduced, finally results are presented and conclusions are drawn.

Background

The international literature about universities' efficiency -through non-parametric techniques -has been developed both at institution-level (Athanassopoulos & Shale, 1997;McMillan & Datta, 1998;Ng & Li, 2000;Avkiran, 2001;Abbott & Doucouliagos, 2003;Flagg, et al, 2004;Warning, 2004) and department-level. The latter is more relevant for the context of the present study.

The contributions by Johnes & Johnes (1993;1995) highly influenced our methodological approach. In a first paper Johnes & Johnes (1993) measured the research performance of UK economics departments, by using staff numbers as inputs and publication counts and grants as outputs; the same strategy was adopted again for deriving efficiency scores of economics department in a subsequent paper (Johnes & Johnes, 1995). The authors ran several DEA models to test the robustness of results and claim for a possible use of results for policy purposes, especially for establishing an optimal allocation of resources across departments.

These two contributions are the most important as they focus exclusively on the research dimension of the universities' work. Madden et al. (1997) perspective, this result is not so unexpected. Moreno & Tadepalli (2002) focused on 42 academic units distributed across 7 colleges in the US. They used staff, financial resources and proxies for structures (e.g., building space) as inputs, and number of students, FTE enrolments and grant awards as outputs. In discussing their results, they evidenced that efficiency scores do not illustrate information about effectiveness -that is, they argue that a qualitative dimension is necessary for this purpose. Lastly, as their background is strongly managerial, they also suggest a use of efficiency analysis as a planning tool, not only for evaluating universities' activities. Koksal & Nalcaci (2006) university's administrators through a preliminary survey. The paper suggests that the models adopted for computing efficiency can serve to set new targets for inefficient departments. Gimenez & Martinez (2006) focused on cost efficiency of the departments at the Autonomous University of Barcelona. They computed cost-efficiency scores and separated cost excess into three categories: technical efficiency, fixed factors, scale of size. The units of analysis were 42 departments, and again the focus is on both teaching and research activities. The inputs chosen were: expenditure on temporary hired teaching and research staff, operational expenditures, expenditures on tenure staff. To measure the outputs, they employed the following variables: results of a government evaluation process of research, teaching load (measured through credits granted) and students' opinions. The authors suggest that their analyses can be used to assess the differences between (i) the observed cost of the units and (ii) the cost they would achieve assuming long-term cost minimization. Their result tells that size does not matter in determining efficiency, while higher proportion of non-tenured staff is associated with higher efficiency levels.

In another exercise presented by Kocher et al. (2006) the analysis is focused on the productivity of research in economics across 21 OECD countries. The focus of the paper is on research and the outputs selected are top publications (and impact factors), while inputs are R&D expenditures, number of universities and total population. This study is quite different, however, with respect to the relevant contributions for the present paper, as the unit of analysis is the country (neither the university nor the department). The results suggest that research productivity can be raised through an increase in the scale of operations.

An interesting issue is to analyze the related literature about the Italian universities' research. To the best of the authors' knowledge, there are only two specific contributes which specifically analyze the efficiency of universities in producing research outputs (Pesenti & Ukovich, 1996;Rizzi, 1999), and just one attempting to focus on teaching production (Agasisti & Dal Bianco, 2009).

The two contributions at departmental level (Pesenti & Ukovich, 1996;Rizzi, 1999) use a similar set of variables and focus both on teaching and research outputs. The main scope for these contributions is to provide information and suggestion for allocating resources within a university.

Contributes about the efficiency of Italian higher education has been conducted at institution-level (thus, not considering separately the structures devoted to teaching and research). The reason for this choice can be summarized by the idea that the strategy and the mission of universities is defined at macro(institution) level, then the balance between teaching and research is decided by the decision centres at institution level (Bonaccorsi & Daraio, 2007, p.22 ).

Among these contributions at an institutional level Agasisti & Dal Bianco (2006) studied the efficiency of 58 Italian public universities, by using number of staff, financial resources and students as inputs, and graduates and research grants as outputs. They suggest the use of efficiency analysis for policy purposes, especially to evaluate the differences in terms of performance between efficient and inefficient units. Later, Agasisti & Salerno (2007) analysed the cost efficiency of another sample of Italian universities, by adding some qualitative measures for inputs and outputs.

They conclude that a redistribution of inputs across universities could lead to an overall improvement of efficiency in the Italian HE sector. Bonaccorsi et al. (2006) used a robust non-parametric technique (developed by Daraio & Simar, 2005) to take in account external conditions affecting the productivity of universities. The inputs and outputs adopted are quite similar to previous studies (that is, staff numbers, expenses and spaces as inputs, and publications and graduates as outputs), but they also include some "external" factors: university size, number of departments within the university, percentage of private funding, number of curricula activated. Their conclusions are there economies of scale and scope are not relevant factors in explaining (in)efficiency, but that a strong role is instead played by the external factors -clearly, such result affects the policy considerations.

To sum up, the literature seems to suggest that, to analyse research efficiency of universities -that is, the efficiency in the production of research outputs:

• inputs can be classified under three main groups: human resources, financial resources and structures (e.g. facilities, buildings, etc.);

• outputs can be grouped into publications and grants.

Clearly, the best indicators to measure these classes of inputs/outputs are questionable, but the general emerging consensus is on a productive process that can be described with such indicators. The present paper moves from these suggestions, and it focuses on the efficiency of research activities in scientific departments in the Lombardy Region (Italy).

Methodology and data

Methodology

The methodology is articulated accordingly to the two steps adopted to answer the research questions of the paper: on one hand the efficiency analysis (including Charnes, et al. (1978). It deals with the efficiency problem of complex organizations that produce many outputs using several inputs at a time.

F
In this paper we have only briefly discussed some characteristics of DEA, referring the interested reader to Johnes (2004); Cooper et al., (2006), andZhu (2003).

In a DEA model, the notion of efficiency is that of "technical efficiency", defined as the relative ability of each (1984) introduces in DEA modelling the dimension factor.

Smaller units are associated with increasing economies of scale, and bigger units with decreasing economies of scale. In the middle, there is a "grey area" of constant returns to scale. So, it is also possible to compute the scale efficiency. This is defined as the ratio CRS/VRS efficiencies, and interpreted as the ability of each institution to benefit (in terms of productivity) from its size -in other words, in a VRS formulation, each DMU is compared only with those units having a comparable size.

We also measure changes in efficiency across time, and our relevant period is between 2004 and 2007. For measuring the determinants of the changes in efficiency scores, we use Malmquist index (for more information about this index see

Deleted: ¶ ¶ Deleted: ¶ F o r R e v i e w O n l y 9
Johnes , 2004 andCoelli et. al., 2006). Let us consider a DMU, named P that has a change in production position from P t (in period t) to P t+1 (in period t+1 The index (Malmquist index) resulting from the combination of these two components will be equal to 1 if there is no net effect of changes in technical efficiency and frontier changes; it will be greater than 1 if the net effect is positive, and less than 1 if it is negative. In our analysis, we name "efficiency change" the first component and "frontier shift" the second one.

Moreover, we would detect the eventual stability of efficiency rankings across the period (again, the relevant period is between 2004 and 2007). In this case, we adopted one of the statistics suggested by Brockett et al. (1998). More specifically, we are interested in knowing if the efficiency of departments changed influencing the rankings or not over time. "Ranking the set of nxk efficiency ratings and observing the sum of ranks associated with each DMU, one can learn about the relative position of the DMUs vis-avìs each other across the whole period" (Brockett et al., 1998, p. 174). For this purpose, the basis of the analysis is a Kruskal-Wallis test (Kruskal & Wallis, 1952) where the rejection of the null hypothesis leads to the conclusion that, in general, the different DMUs maintain their relative efficiency positions over time" (p. 175).

The second task in our methodology is to study some potential determinants of efficiency. To pursue this objective, we identified some external factors (e.g. "location" effects, "discipline" effects, etc.) and then we compared efficiency scores Wallis test (Kruskal & Wallis, 1952). This test allows the researcher to detect if are there statistical differences across the groups analyzed (e.g., when comparing the efficiency scores of units located in a metropolitan area with those located elsewhere, the test indicates if this location effect generates a statistical difference between the average scores of the two groups).

Data

In this paper, we used a unique dataset that consists of input and output variables about research and technology transfer activities in university departments located within the Lombardy Region. This dataset is based on information coming from QuESTIO system, a mapping of Research and Technological Transfer organisations developed in Lombardy Region. The QuESTIO system, launched in 2004 by the Regional Government, was intended to be a transparency and incentive tool for the Research and Technological Transfer market by reducing any information unevenness often binding cooperation among producers and users of knowledge.

The system pursues such scopes by collecting, updating and publishing financial, organizational and structural characteristics of research organisations and Knowledge Business Intelligence Services in Lombardy.

Since it aims at presenting the most detailed picture of Regional community active in Research and Technological Transfer, the dataset has some specific criteria that are pivotal for our analyses. First, the system is focused on every structure where such activities are carried out, even where that structure is a part of a wider organisation.

Second, these structures should have significant management autonomy so that their accountancy shall be determinable. The first criterion enables the regional administration to collect information on every single department, being acknowledged that departments belonging to the same university can differ significantly in terms of performances. As a consequence the university departments, as well as sections and institute of National Research Council or R&D unit belonging to a Company, provide data making solely reference to their specific resources and outputs. The second criterion assures that each structure included in the dataset is responsible (to some extent) for its strategy and performances thus justifying application of DEA technique. In order to better point out the capillarity of these structures within the territory, the geographical analysis shows a concentration in the Province of Milan, and a little less in Bergamo, Brescia, Pavia and Varese, which is explained by the presence of universities in those cities, all of them accredited with their single departments.

University departments are main actors in Lombardy Innovation

We built our dataset by considering those university departments that provided information for the whole period 2004-2007 (units for which there are not missing data), by selecting the following variables: As discussed in the previous section the choice of input and output variables is coherent with the most recent literature. The final sample contains 75 departments, for 4 years -overall, there are 300 observations. Descriptive statistics are illustrated in Table 1.

<Table 1> around here

Results

In this section, we illustrate the results obtained through the DEA analysis on the university departments in the Lombardy Region.

The results are presented in two main sections. The first is devoted to a description of the efficiency scores with reference to the year 2007 (the last year available, where the results are more stable as previously discussed). In this section one subparagraph analyses the change in efficiency between the first year of our dataset This section explores the results of the efficiency analyses both looking at a single year ( 2007) and at differences between years.

DEA efficiency scores, year 2007

Table 2 reports the descriptive statistics about the efficiency scores derived with reference to the year 2007 (the annex reports the specific scores for each department). As explained in the methodological section, we calculated CRS efficiency and VRS efficiency, as well as the "scale" efficiency computed as CRS/VRS scores (it is important to recall here that the model is output-oriented).

<Table 2> around here

The research efficiency of university departments in the Lombardy Region looks quite high (0.75 in terms of CRS efficiency, 0.78 in terms of VRS efficiency); actually, the high average efficiency is coherent with previous results derived across the world in the higher educations sector (Salerno, 2003). What is worth pointing out is a strong heterogeneity across departments -indeed, the standard deviation is >0.2. A look at the lowest scores suggests that there are very poor performers which shows scores <0.5.

Data about scale efficiency suggests that all the departments have almost reached their optimal scale of operations (given their present scale), as the scale efficiency score is very high (>0.93) and standard deviation is very low (about 0.1) in this respect.

The picture that emerges from this first analysis of efficiency scores is a sector where there is a number of very efficient units (both assuming constant or variable returns to scale) and a number of departments which experienced low levels of technical efficiency; thus, as the anecdotic evidence suggests, there is quite a strong differentiation among departments in terms of research efficiency.

As the focus of our paper is the efficiency of academic departments in the Lombardy Region, we also checked the robustness of our results by considering revenues from entities located in the Lombardy Region as a separate output (REV_LOMB). The picture that emerges highlights an interesting phenomenon. It looks like an improvement in productivity did not occur (because the Malmquist index is around 1)

but, looking at the decomposition of the index, the reality appears more complex: an efficiency changes happened, it was very positive (the index is >1.6) and involved the major part of the departments (56 out of 70); on the other side, there was a relevant detriment of the technology environment (the "frontier shift" indicator is <0.7). It looks like the academic units improved their technical efficiency (that is, the ability to transform inputs into outputs) but, at the same time, the frontier worsened. In other words, what happened is that the "technological change" (in the Malmquist language) was negative.

What is "technology" in a regional research system, when we define research performance as the ability to attract financial resources and to publish in academic journals? Technology could be defined as the bundle of policies that should help in improving research efficiency, e.g. Information Technologies, research grants, support for publishing research results or to attract financial resources from international grants or companies, etc. Thus, the results contained in Table 3 suggest that, while the majority of the universities improved their efficiency, the system as a whole did not experienced an improvement in terms of technology available. A discussion about the potential responsibilities for such phenomenon will be reported in the last section, which deals with policy implications of our work.

Productivity changes between 2004 and 2007 -an analysis of "rankings stability"

Lastly, we analyzed the stability of rankings across our analysis period (between 2004 and 2007) by means of the H index proposed by Brockett et al. (1998). The index is equal to 177.96, and it must be compared with H 2 that is equal to 102.99.

The rejection of the null hypothesis leads to the conclusion that some of the academic departments exhibit consistently better economic performance than others

F o r R e v i e w O n l y
15 (as measured by the input-output factors that were included in our model) (see Brockett et al., 1998).

Reading this results together with the Malmquist index, it looks like the academic departments improved their performance, but some units continue to perform better than others. At the same time, the results show that some units are not efficientthat is, the frontier actually worsened.

DEA efficiency scores, year 2007 -the external factors affecting efficiency

In this second section, given the evidence on the efficiency results, it is interesting to explore which are the determinants of this efficiency or, in other words, what could actually affect a department's performance.

To answer this question, we tested a number of different factors, by using the Kruskal-Wallis test as methodological approach (see the data section). All the results are listed in Table 4.

<Table 4> around here 4.2.1 First external factor: are departments located in Milan metropolitan area (regional capital) more efficient than other departments?

We analysed this factor on the assumption that in a metropolitan area the competition among research centres enforces departments' managerial efficiency. In Milan, there is an important concentration of public and private research institutions and the competition for resources is not limited to departments. The high concentration of research facilities may justify a greater effort by departments in fostering a more balanced use in input resources or may impulse an effort in control for managerial quality. Elsewhere, Agasisti (2009) demonstrated that the effect of competition on Italian universities' performance is positive -but, there, output was defined through teaching dimension's measures.

We ran a Kruskal-Wallis test to verify the statistical difference between the efficiency scores of departments located in Milan versus the other departments. We do not find evidence of difference in efficiency performance between the two groups (indeed, the observed value of the H statistic is nearly 0, and the p-value >0.9). This evidence suggests that there is no "concentration" effect or "location" effect on efficiency performance of university departments or, at least, in the relative rankings. At the same time, this result may suggest i) the existence of no competitive pressure in a high concentration area as Milan, or that ii) we have to look at overall concentration

F o r R e v i e w O n l y 16
of research centres, and not only to university departments (e.g., there could be a "university" effect which could dominate a "location" effect, or that iii) there is competition but at the same time more opportunities to find resources (more demand for research, and consequently the amount of resources is enough to guarantee a sufficient level of grants for different departments without the need to compete one against the others).

Second external factor: can different scientific sectors affect university department efficiency?

As we measured "output" by using i) publication counts and ii) research grants (of different kinds), it is important to recall that there are some differences in the average number and quality of publication across different disciplines (e.g. medicine versus engineering or biology), as well as differences in terms of attractiveness of research funds (e.g., in engineering fields it is simpler to attract grants for applied research than in physics).

The idea that disciplines greatly vary in terms of costs and productivity is not new, and the suggestion to take this factor into account is important to obtain not-biased results (Sarrico & Dyson, 2004). Recently, Sarrico et al. (2009) demonstrated a strong role of "subject mix" (defined as the disciplinary composition of institutions) in explaining the efficiency differentials among Portuguese higher education institutions.

The authors acknowledged that "(…) research production also varies significantly from subject to subject, with medical, some natural and technical sciences being more prolific in the number of articles than most of the social sciences and humanities" (p.290). Also Ramsden (1994), in his empirical analysis of Australian research departments, adjusted the coefficients on the basis of discipline subjects.

We find a small, but statistically significant difference among university departments classified by scientific sectors: the observed value of the H statistic is >8.4, near the critical value (9.4), and the p-value is 0.07 (then the effect has statistical significance at the 10% level). However, given the limited power of this effect, it is not easy to determine which sectors outperform the others straightforwardly.

Third external factor: is there a "university effect"?

An alternative explanation of efficiency can refer to the existence of a "university effect", that is if a university is more efficient than others, then its departments could be more productive than those belonging to other universities. This assumption finds its justification in the setting of strategies at university levels, not at subunits levels. In this respect, Bonaccorsi & Daraio (2007, pp. 22-23) The fundamental reason for assuming the university level as the unit of analysis is that at the university level the problem of attribution of inputs (in particular, human resources, funding and physical capital) to specific units of outputs, can be kept under control". Even if these authors recognized that a lower unit of analysis could provide useful information on the actual research process, they suggest that a "university effect" (due to different strategies and characteristics at macro-level) can be detected.

Our tests suggest that such effect is not explaining efficiency differentials for our sample (observed value of the H statistic is 9.4, well below the critical value of 14.06, and the p-value is 0.22).

Fourth external factor: does the number of tenured academic staff influence efficiency?

The proportion of tenured research staff theoretically can affect the efficiency of a department: there is anecdotic evidence that tenured and non-tenured staff face quite different stimulus and incentives. Nevertheless, the direction of this influence is not clear a priori: tenured staff is certainly more expert in attracting research grants and in publishing academic papers, also thanks to reputational advantages; conversely, non-tenured staff can have an incentive to be more productive, for instance as a mean to obtain a tenured position. The promotion system can affect productivity if researchers are satisfied with rules promoting this dimension (a discussion of motivations behind researchers' activity is in Ramsden, 1994, pp.214-215).

A complementary idea behind this question is that the different human capital of researchers (in our paper, indirectly measured via the tenured status) can affect the research productivity of the research groups -and so, of the department. In this respect, an interesting example is the paper by Rodgers & Neri (2007), who specifically attempted to correlate the human capital of researchers with the research productivity of Australian departments of Economics. They have found that human capital differential account for about 50% of the observed differences in research productivity. If human capital is (even indirectly) captured by the tenured status (e.g., researchers with more experience should have reached their tenured position), then we should expect a correlation between the proportion of tenured researchers and the department's results. Ramsden (1994) showed that productivity of professors and readers was much higher than that of lecturers. Our analysis did not find evidence of a statistical influence of this factor on research efficiency (the value of the H statistic is 0.16 and the related p-value is >0.6).

Fifth external factor: can departments' "longevity" affect research efficiency?

Finally, we looked at the possible effects due to the longevity of departments. As we measured inputs in terms of numbers of academic staff and laboratories/facilities, it is reasonable that older departments accumulated higher stocks of inputs that could influence their outputs.

A key assumption here is that older departments can favour more cohesion among its members, then facilitating the climate of "collaboration" and "common motivation" that some authors pointed out as a determinant of research productivity (Ramsden, 1994).

Another interesting point of view about this potential phenomenon is that it is related to the "critical mass" dimension. Among the economic aspects of research, Hare & Wyatt (1992) underlined that the ability in producing research relies upon the human capital accumulated by the individual researchers; thus, "combining this process with the notion that there are economies of scale in research may dispose university departments to feel they need to have at least some critical mass of researchers working in the same area in order to function properly as research units" (pp. 55-56).

Consequently, it could be reasonable to assume that older departments are more likely to have reached such "optimal" scale dimension than the more recently established ones.

The statistical analysis did not provide statistically robust results on this effect, indeed the value of the H statistic is 1.172 -while the critical value is around 3.8, so that the p-value is 0.27).

Discussion and policy conclusions

As highlighted discussing the literature on the topic, the analyses presented in this work are innovative in terms of data and methodology. Indeed, in this paper we have investigated the efficiency of university departments on science and technology and medicine in an Italian Region (Lombardy). The aim of the paper was (i) to analyze the changes in productivity in recent years (from 2004 and 2007), and (ii) to detect factors that are potentially affecting efficiency.

Discussing our results about the efficiency change between 2004 and 2007, some questions must be posed: how can a Malmquist index equal to 1 exactly be interpreted? How can we judge a frontier shift that is less than 1? Each academic the vast majority of the units actually improved their productivity in the period ("efficiency change" >1) but the "frontier shift" was <1: it means that the performance differential is much less dispersed in 2007, but at the same time it happens because the frontier is not improved -the best performers did not improve, or even worsened their efficiency. The summary statistics confirms this idea (look at the really lower mean in 2004, with a much higher dispersion -standard deviation -around it). The result is a "research system" in the region that performs better on average, but without having excellence.

The importance of evaluation and accountability has been growing in these recent years and criteria used are widely acknowledged: publications and the ability to gather financial resources (from abroad or from firms). Since the application of these evaluations for funding allocation is more and more concrete, we believe that the weakest departments raised their attention towards those performances. Thanks to this new attention the weakest departments have increased their efficiency and become much closer to the best performers that, on the contrary, had little margin for further improvements.

This result requires an important policy discussion. Indeed, the objective of the Lombardy Region was to foster the performance of excellent research units. This purpose animated some policy interventions such as the creation of "clusters", the policies for improving contacts between companies' demand and research units through vouchers (the idea behind is that the best research units would be able to collect more vouchers), etc. Apparently, the results obtained are in the opposite direction.

This fact raises two reflections. The first is that maybe the universities (and, consequently, the research units) look not at the regional incentives, but at the national ones (as the Ministry is still the main financer of the universities' activities)

and at international ones (the research reputation strongly depends upon the ability to publish in international journals and to attract international grants). If this is the case, a question arises about the opportunity for a regional government to be involved in this sector.

A second (possibly alternative) reflection is that the policies adopted for boosting research productivity were not adequate in terms of policy design. To properly design a policy in this field the necessary first step is to understand what affect the academic productivity. In this context, Faria (2002) suggests that policies for fostering the Our second analysis regarded how five external factors ("external" because they are not under the control of the departments) affect efficiency (section 4.2.). Our empirical analysis suggests that none of these factors (location in metropolitan area, scientific sector, university effect, proportion of tenured academic staff, departments "longevity") is able to adequately explain efficiency differentials. We consider this result as positive, especially when thinking to the use of efficiency scores for policy purposes (e.g., resources allocation). Indeed, a usual critic to the efficiency analyses is that they provide results which are affected by factors which are not under the control of the units themselves: when this is the case, a procedure of resource allocation following efficiency results would be inequitable. At the same time, we argue that there are some elements, which actually could influence efficiency and which are related to some "soft" characteristics of the academic units, typically (i) the incentive systems and (ii) the governance/management. For instance, Ramsden (1994) pointed out that the perception of the academic staff about the importance of publications for their career was consistently related to the production of academic articles and books. Rodgers & Neri (2007) suggested that the "departmental environment" is important and they define it through several measures centred on the "human capital" of researchers who work in the department. Unfortunately, QuESTIO dataset does not collect such data: an extension of this dataset along this dimension could help to improve significantly the understanding of this pattern.

However, if the incentive system really matters, another conclusion can be drawn from our study: as we measured a potential "university effect", and it is not

F o r R e v i e w O n l y 21
statistically relevant, then we can conclude that no incentive system works at university level. Different reasons can be proposed to explain this result. In particular, it can be both the cases that (i) the incentive system is designed at an higher level (in Italy, at the central government level), or (ii) that it is designed at a lower (department) level -and given the department-level of our data, we cannot exploit such dimension, which requires individual-level data. The evidence about the process for recruiting academic staff and its career in Italy suggests that, however, no government-level incentive exists -indeed, the career of professors is very poorly related to their scientific production. Levin (1991) underlined the importance of incentives to raise the productivity of universities' activities. More specifically, he acknowledged that higher education shows productivity declines due to two main reasons: (i) it is a "stagnant industry", as it is based on labour intensity, and (ii) clear rewards and property rights are absent.

Thus, the suggestion provided by the author is to develop a reward system within the university based on clear goals, financial incentives and evaluation procedures.

Whether such schemes have been actually implemented in departments analysed in this paper is not clear; probably detailed case studies can help to shed more light on this topic through future research.

The governance/management of the academic departments has not been analysed in our study, as the only information available in this respect (the presence of a quality system or a management control system) is a very indirect measure of the governance process and the data recovery on this topic was quite imprecise. Another way to improve the QuESTIO dataset should be to better investigate this dimension, by including some qualitative descriptions about the way Department managers organise and coordinate the activities and resources, formulate strategies and foresight studies and manage networking activities (see for instance Sala et al., forthcoming;Salerno et al., 2008). Without knowing the characteristics of the "management style" of each department, it is impossible to derive policy implications from such dimension. At the same time, a possible way to better understand the potential incidence of such phenomenon is to conduct case studies about the successful academic departments (e.g. those with higher efficiency scores or those which highly improved the scores in the last years).

Finally, a potential candidate to analyse the efficiency differentials is the interaction between research and teaching activities conducted by the departments. Typically, the literature did not paid enough attention to the interaction between teaching and research at department level, mainly because both activities were considered as inputs and outputs (see section 2). In this paper, the focus was exclusively on the

F o r R e v i e w O n l y 22
research activities, but it raises also questions about the effects that the (unmeasured) teaching activities play on the research result. This issue is very relevant both in theoretical terms (are there economies of scope, or tradeoffs, between the two activities?) and in empirical ones (which is the optimal allocation of energies between teaching and research?). This interesting issue was at the centre of several contributes since the seminal paper by Cohn et al. (1989), that for the first time addressed and modelled the multi-outputs nature of academic activities.

Recently, the theme has been explored by Bonaccorsi & Daraio (2007)

Abstract.

This paper investigates the efficiency of university departments on science, technology and medicine in an Italian Region (Lombardy). The aim of the paper is twofold: (i) to analyse the changes in productivity in recent years (from 2004 and 2007); and (ii) to detect factors that are potentially affecting efficiency. The research benefited from a new and unique dataset (called QuESTIO) developed by the Lombardy Regional Government. Using facilities and academic staff as inputs and research grants and publications as outputs, the research activity of academic departments was modelled. The methodological approach for computing efficiency scores is Data Envelopment Analysis (DEA). Malmquist indexes have been used to measure changes in productivity, while Kruskal-Wallis tests were employed to study the potential determinants of efficiency. The main results were, first, that in the period considered, the academic departments improved their efficiency but, at the same time, the efficiency frontier worsened. Second, external and measurable factors (such as, scientific sector, proportion of tenured staff, location) have a limited impact in explaining efficiency differentials. Policy implications of the results are discussed.

Keywords.

University research, efficiency, research policies, Data Envelopment Analysis, Malmquist index, higher education policy.

JEL

Introduction and motivation

In recent years, higher education institutions have been increasingly studied. In the ''knowledge economy'' their importance for economic development, social equity, mobility, social cohesion and integration is widely acknowledged (Brennan & Teichler, 2008). Furthermore, given the difficult situation of public finances, considerations about resources allocation have been raised in many countries, calling for more evaluations and accountability. For these reasons the so-called 'New Public Management' movement, started at the beginning of the 1980s in the public sector in general, has also increasingly been affecting the universities. New Public Management (Hood, 1991;1995) represents the request to public organisations to become more managerial, in particular by drawing on practices and models from the private sector. This process has been driven both by universities themselves and by governments through reforming efforts. The results, especially in many countries within Europe, share some common characteristics: first, the introduction of incentives schemes based on financial models for allocating public funds; second, new or strengthened assessment procedures and evaluation agencies; third, more emphasis on universities' autonomy (OECD, 2003).

This work explores how the recent changes in the university system affected the efficiency of university research. In particular, the research sought to understand whether an overall increase in efficiency can be recognised or whether differences are present between universities given the previously-noted increased autonomy.

Normally, the majority of the reforming efforts have been directed towards education activities, whilst an increase in efficiency of university research has been assigned to incentive systems and evaluation processes. These are expected to have had an overall positive effect on the system but, at the same time, they have left the universities in charge of finding and managing the ways to obtain better results. Given the autonomy of the universities and the decentralisation of responsibilities, significant efficiency changes are thus expected between and within universities. Furthermore, the idea that differentials in research performance are key for competition among departments is well established in the literature (Curran, 2000). widely studied (Madden et al., 1997). Indicators such as number of degrees awarded, rate of success, years of study, have been widely used both to evaluate the universities and to introduce new policies. The opposite holds for universities' knowledge/technology transfer activities. Even if some argue for this to be a third mission for academia and call for "entrepreneurial universities" (Clark, 1998;Etzkowitz, 2003), others refuse this perspective and signal the risks of focusing on and expanding the role of these activities. Moreover, the efficiency analysis of these activities is hampered by the difficulties in finding the right indicators. There are many different knowledge/technology transfer activities (for example, spin-off, consultancies, research projects with firms, patents) and their evaluation can be based only partially on economic values given the not-for-profit nature of (most)

universities (Landoni and Verganti, 2006). Finally, the focus of the study is on the research activities as they are the core activities of universities; driving and informing The research considers departments located in the Lombardy region (Italy). A regional focus has been adopted to ensure similar contextual variables, such as policies and economic environment, for the universities included in the analysis.

Furthermore, Lombardy is one of the more advanced European regions both for (i) economic development and (ii) research activities and universities: more than 9.4 million habitants, GDP per capita >€34,000; 12 universities are located in the Region (out of about 70 in all the country) with more than 230,000 students. Finally, the regional level is increasingly recognised as a relevant level when analysing research and innovation activities. Many authors (Cooke et al., 1997;Porter, 1998;Lechner & Dowling, 1999) underline that it is at a local level that the majority of the connections between the different innovation actors take place. The localisation of knowledge and the role of proximity as a driver of economic development (for example, clusters)

have also highlighted the importance of regional and local government in fostering and strengthening the actors and their relationships.

To summarise, the focus of the paper is on efficiency of research activities in scientific departments in the Lombardy Region (Italy). The research questions are:

• did changes in the overall efficiency of departments' research occur between 2004 and 2007?

• Which are the main factors affecting efficiency?

The next section presents recent literature on the topic, then the methodology and the data used are introduced, finally results are presented and conclusions are drawn.

Background

The international literature about universities' efficiency (through non-parametric techniques) has been developed both at institution-level (Athanassopoulos & Shale, 1997;McMillan & Datta, 1998;Ng & Li, 2000;Avkiran, 2001;Abbott & Doucouliagos, 2003;Flagg, et al, 2004;Warning, 2004) and department-level. The latter is more relevant for the context of the present study.

The contributions by Johnes & Johnes (1993;1995) had a major influence on the methodological approach. Johnes & Johnes (1993) measured the research performance of UK economics departments by using staff numbers as inputs and

F o r R e v i e w O n l y
publication counts and grants as outputs; the same strategy was adopted again for deriving efficiency scores of economics department in a subsequent paper (Johnes & Johnes, 1995). The authors ran several Data Envelopment Analysis (DEA) models to test the robustness of results and claim for a possible use of results for policy purposes, especially for establishing an optimal allocation of resources across departments. These two contributions are the most important as they focus exclusively on the research dimension of the universities' work. Madden et al. (1997) that a qualitative dimension is necessary for this purpose. Lastly, as their background is strongly managerial, they also suggested a use of efficiency analysis as a planning tool, not only for evaluating universities' activities. Koksal & Nalcaci (2006) derived efficiency scores for 14 university departments at the Middle East Technical University (METU) in Turkey, by using staff salaries, a composite indicator of previous research quality and number of students as inputs and number of publications, graduates and teaching indicators as outputs. Thus, in their contribution, the authors focused also on the teaching activity of the departments.

They decided to use weights for teaching and research loads as suggested by university's administrators through a preliminary survey. The paper suggests that the models adopted for computing efficiency can serve to set new targets for inefficient departments. Gimenez & Martinez (2006) tenure staff. To measure the outputs, they employed the following variables: results of a government evaluation process of research, teaching load (measured through credits granted) and students' opinions. The authors suggested that their analyses can be used to assess the differences between (i) the observed cost of the units and (ii) the cost they would achieve assuming long-term cost minimisation. Their result tells that size does not matter in determining efficiency, while higher proportion of non-tenured staff is associated with higher efficiency levels.

In another exercise presented by Kocher et al. (2006) the analysis focused on the productivity of research in economics across 21 OECD countries. The focus of the paper is on research and the outputs selected are top publications (and impact factors), while inputs are expenditure on research and development, number of universities and total population. This study is quite different, however, with respect to the relevant contributions for the present paper, as the unit of analysis is the country (neither the university nor the department). The results suggest that research productivity can be raised through an increase in the scale of operations. An interesting issue is to analyse the related literature about the Italian universities' research. To the best of the authors' knowledge, there are only two specific contributes that specifically analyse the efficiency of universities in producing research outputs (Pesenti & Ukovich, 1996;Rizzi, 1999) and just one attempting to focus on teaching production (Agasisti & Dal Bianco, 2009).

The two contributions at departmental level (Pesenti & Ukovich, 1996;Rizzi, 1999) use a similar set of variables and focus both on teaching and research outputs. The main scope for these contributions is to provide information and suggestions for allocating resources within a university.

Contributions about the efficiency of Italian higher education has been conducted at institution-level (thus, not considering separately the structures devoted to teaching and research). The reason for this choice can be summarised by the idea that the strategy and the mission of universities is defined at macro (institution) level, then the balance between teaching and research is decided by the decision centres at institution level (Bonaccorsi & Daraio, 2007, p. 22). This is because the strategy and the mission of universities and the balance between teaching and research are defined at institutional level (Bonaccorsi & Daraio, 2007, p. 22).

Among these contributions at an institutional level Agasisti & Dal Bianco (2006) studied the efficiency of 58 Italian public universities, by using, as inputs, number of staff, financial resources and students and, as outputs, graduates and research grants.

They suggested the use of efficiency analysis for policy purposes, especially to evaluate the differences in performance between efficient and inefficient units. Later, Agasisti & Salerno (2007) analysed the cost efficiency of another sample of Italian universities, by adding some qualitative measures for inputs and outputs. They concluded that a redistribution of inputs across universities could lead to an overall improvement of efficiency in the Italian higher education sector. Bonaccorsi et al. (2006) used a robust non-parametric technique (developed by Daraio & Simar, 2005) to take in account external conditions affecting the productivity of universities. The inputs and outputs adopted are quite similar to previous studies (that is, staff numbers, expenses and spaces as inputs and publications and graduates as outputs) but they also included some "external" factors:

university size, number of departments within the university, percentage of private To sum up, the literature seems to suggest that, to analyse research efficiency of universities, that is, the efficiency in the production of research outputs:

• inputs can be classified under three main groups: human resources, financial resources and structures (e.g., facilities, buildings);

• outputs can be grouped into publications and grants.

Methodology and data

Methodology

The methodology is articulated accordingly to the two steps adopted to answer the research questions of the paper: on one hand the efficiency analysis (including efficiency scores of the departments, changes across time and time trends), on the other hand the effort to identify the determinants of efficiency.

The first choice is about the methodology to compute efficiency, because there are different methods to target the analysis of efficiency among different units Data Envelopment Analysis (DEA) is a frontier, linear programming technique, introduced by the seminal work of Charnes et al. (1978). It deals with the efficiency problem of complex organisations that produce many outputs using several inputs at a time. Data Envelopment Analysis (DEA) (Charnes et al., 1978) was the methodology used to analyse efficiency. It deals with the efficiency problem of complex organisations that produce many outputs using several inputs at a time.

This paper only briefly discusses some characteristics of DEA, referring the interested reader to Johnes (2004), Cooper et al., (2006) and Zhu (2003).

F o r R e v i e w O n l y
In a DEA model, the notion of efficiency is that of "technical efficiency", defined as the relative ability of each decision making unit (DMU) in producing outputs, where the term "relative" means that each organisation is compared with any other homogeneous unit. DEA could be represented by a linear programming technique where each DMU tries to maximise the efficiency ratio (output over inputs) choosing the best set of weights. There are two ways to deal with a DEA formulation: (i) input oriented (that is, the ability to minimise inputs when outputs are given) and (ii) output oriented (maximisation of outputs given a certain bundle of inputs). The latter approach is used in this paper, as the typical problem for a research unit is to maximise outputs (publications, grants) given its inputs (research staff, laboratories). [WHAT DOES THIS MEAN?]

Smaller units are associated with increasing economies of scale and bigger units with decreasing economies of scale. In the middle, there is a "grey area" of constant returns to scale. So, it is also possible to compute the scale efficiency. This is defined as the ratio CRS/VRS efficiencies and interpreted as the ability of each institution to benefit in terms of PLEASE REPHRASE TO AVOID "in terms of" WHICH FAILS We also measure changes in efficiency across time and our relevant period is between 2004 and 2007. For measuring the determinants of the changes in efficiency scores, we use Malmquist index (for more information about this index see Johnes (2004) and equal to 1 it has no effect on the efficiency score, if it is greater than 1 it has a positive effect and if it is less than 1 it has a negative effect. In other words, a shift in a production frontier that makes worse the relative position of the DMU (the other DMUs are performing relative better than before) provides a second component less than 1. The index (Malmquist index) resulting from the combination of these two components will be equal to 1 if there is no net effect of changes in technical efficiency and frontier changes; it will be greater than 1 if the net effect is positive and less than 1 if it is negative. In our analysis, we name "efficiency change" the first component and "frontier shift" the second one. I THINK A CONCRETE EXAMPLE IS NECESSARY HERE TO GUIDE THE READER.

Moreover, we would detect the eventual stability of efficiency rankings across the period (again, the relevant period is between 2004 and 2007). In this case, we adopted one of the statistics suggested by Brockett et al. (1998). More specifically, we are interested in knowing if the efficiency of departments changed influencing the rankings or not over time. "Ranking the set of nxk [WHAT IS THIS?] efficiency ratings and observing the sum of ranks associated with each DMU, one can learn about the relative position of the DMUs vis-à-vis each other across the whole period" (Brockett et al., 1998, p. 174). For this purpose, the basis of the analysis is a Kruskal-Wallis test (Kruskal & Wallis, 1952, p. 175) where "the rejection of the null hypothesis leads to the conclusion that, in general, the different DMUs maintain their The second task in our methodology is to study some potential determinants of efficiency. To pursue this objective, we identified some external factors (for example, location effects, discipline effects) and then we compared efficiency scores across different groups of units. The comparison was made by means of the Kruskal-Wallis test (Kruskal & Wallis, 1952). This test allows the researcher to detect if are there statistical differences across the groups analysed (for example, when comparing the efficiency scores of units located in a metropolitan area with those located elsewhere, the test indicates if this location effect generates a statistical difference between the average scores of the two groups).

Data

The research used a unique dataset that consists of input and output variables about Since it aims at presenting the most detailed picture of regional community active in research and technological transfer, the dataset has some specific criteria that are pivotal for the analyses. First, the system is focused on every structure where such activities are carried out, even where that structure is a part of a wider organisation.

Second, these structures should have significant management autonomy so that their accountancy shall be determinable. The first criterion enables the regional administration to collect information on every single department, being acknowledged that the performance of departments belonging to the same university can differ significantly. As a consequence the university departments, as well as sections and

DEA efficiency scores, year 2007

This section explores the results of the efficiency analyses both looking at a single year (2007) and at differences between years. REPETITIVE (Salerno, 2003). BUT WHAT IS THIS TELLING US? What is worth pointing out is a strong heterogeneity across departments: indeed, the standard deviation is >0.2. A look at the lowest scores suggests that there are very poor performers which shows scores [??] <0.5.

DEA efficiency scores, year 2007

Data about scale efficiency suggests that all the departments have almost reached their optimal scale of operations (given their present scale), as the scale efficiency score is very high (>0.93) and standard deviation is very low (about 0.1) in this respect.

The picture that emerges from this first analysis of efficiency scores is a sector where there is a number of very efficient units (both assuming constant or variable returns to scale) and a number of departments that experienced low levels of technical efficiency; thus, as the anecdotic anecdotal evidence suggests, there is quite a strong differentiation among departments in terms of PLEASE REPHRASE TO AVOID "in terms of" research efficiency.

As the focus of our paper is the efficiency of academic departments in the Lombardy region, we also checked the robustness of our results by considering revenues from entities located in the Lombardy region as a separate output (REV_LOMB). The

Efficiency changes between 2004 and 2007 -a Malmquist analysis

Following the methodology described above, we derived the Malmquist index, which allows us to disentangle the efficiency changes due to "pure" efficiency improvement (or worsening) and technological improvement (or worsening). The descriptive statistics about the results are reported in Table 3, while the detailed results are available on request from the authors.

<Table 3> about here

The picture that emerges highlights an interesting phenomenon. It looks like an improvement in productivity did not occur (because the Malmquist index is around 1)

but, looking at the decomposition of the index, the reality appears more complex: an efficiency changes happened, it was very positive (the index is >1.6) and involved the What is "technology" in a regional research system, when we define research performance as the ability to attract financial resources and to publish in academic journals? Technology could be defined as the bundle of policies that should help in improving research efficiency, for example, information technologies, research grants, support for publishing research results or to attract financial resources from international grants or companies. Thus, the results (Table 3) suggest that, while the majority of the universities improved their efficiency, the system as a whole did not experience an improvement in terms of PLEASE REPHRASE TO AVOID "in terms of" technology available. A discussion about the potential responsibilities for such phenomenon will be reported in the last section, which deals with policy implications of the study. The stability of rankings across the analysis period (between 2004 and 2007) were analysed by means of the H index proposed by Brockett et al. (1998). The index is equal to 177.96 and it must be compared with H 2 that is equal to 102.99. The rejection of the null hypothesis leads to the conclusion that some of the academic departments exhibit consistently better economic performance than others (as measured by the input-output factors that were included in our model) (see Brockett et al., 1998).

Reading this results together with the Malmquist index, it looks like the academic departments improved their performance but some units continue to perform better than others. At the same time, the results show that some units are not efficient, that is, the frontier actually worsened.

DEA efficiency scores, year 2007: the external factors affecting efficiency

This section, given the evidence on the efficiency results, it is interesting to explore which are the determinants of this efficiency or, in other words, what could actually affect a department's performance.

To answer this question, we tested a number of different factors, by using the Kruskal-Wallis test as methodological approach (see the data section). All the results are listed in Table 4.

This section explores what affects a department's performance. Several factors were tested using the Kruskal-Wallis test, which showed…. WHAT? (Table 4) <Table 4> about here 4.2.1 First external factor: are departments located in Milan metropolitan area (regional capital) more efficient than other departments?

We analysed this factor on the assumption that in a metropolitan area the competition among research centres enforces departments' managerial efficiency. In Milan, there is an important concentration of public and private research institutions and the competition for resources is not limited to departments. The high concentration of research facilities may justify a greater effort by departments in fostering a more balanced use in input resources or may impulse an effort in control for managerial quality. Elsewhere, Agasisti (2009) demonstrated that the effect of competition on Italian universities' performance is positive -but, there, output was defined through teaching dimension's measures. We ran a Kruskal-Wallis test to verify the statistical difference between the efficiency scores of departments located in Milan versus the other departments. We do not find evidence of difference in efficiency performance between the two groups (indeed, the observed value of the H statistic is nearly 0 and the p-value >0.9). This evidence suggests that there is no "concentration" effect or "location" effect on efficiency performance of university departments or, at least, in the relative rankings. At the same time, this result may suggest i) the existence of no competitive pressure in a high concentration area as Milan, or that ii) we have to look at overall concentration of research centres and not only to university departments (e.g., there could be a "university" effect which could dominate a "location" effect, or that iii) there is competition but at the same time more opportunities to find resources (more demand for research and consequently the amount of resources is enough to guarantee a sufficient level of grants for different departments without the need to compete one against the others).

Second external factor: can different scientific sectors affect university department efficiency?

As we measured "output" by using i) publication counts and ii) research grants (of different kinds), it is important to recall that there are some differences in the average number and quality of publication across different disciplines (e.g. medicine versus engineering or biology), as well as differences in terms of PLEASE REPHRASE TO AVOID "in terms of" attractiveness of research funds (e.g., in engineering fields it is simpler to attract grants for applied research than in physics).

The idea that disciplines greatly vary in terms of PLEASE REPHRASE TO AVOID "in terms of" costs and productivity is not new and the suggestion to take this factor into account is important to obtain not-biased results (Sarrico & Dyson, 2004).

Recently, Sarrico et al. (2009) demonstrated a strong role of "subject mix" (defined as the disciplinary composition of institutions) in explaining the efficiency differentials among Portuguese higher education institutions. The authors acknowledged that "(…) research production also varies significantly from subject to subject, with medical, some natural and technical sciences being more prolific in the number of articles than most of the social sciences and humanities" (p.290). Also Ramsden (1994), in his empirical analysis of Australian research departments, adjusted the coefficients on the basis of discipline subjects. We find a small but statistically significant difference among university departments classified by scientific sectors: the observed value of the H statistic is >8.4, near the critical value (9.4) and the p-value is 0.07 (then the effect has statistical significance at the 10% level WHY CHOOSE 10%? WHY NOT SAY NOT SIGNICANT (AT 5%, WHICH TENDS TO BE THE NORM?)). However, given the limited power of this effect, it is not easy to determine which sectors outperform the others straightforwardly.

4.2.3. Third external factor: is there a "university effect"?

An alternative explanation of efficiency can refer to the existence of a "university effect", that is if a university is more efficient than others, then its departments could be more productive than those belonging to other universities. This assumption finds its justification in the setting of strategies at university levels, not at subunits levels. In this respect, Bonaccorsi & Daraio (2007, pp. 22-23) pointed out that:

the notion of strategy is much more relevant and interesting if assumed at the overall university level. On the one hand, most strategic decisions are made at university level.… The fundamental reason for assuming the university level as the unit of analysis is that at the university level the problem of attribution of inputs (in particular, human resources, funding and physical capital) to specific units of outputs, can be kept under control.

Even if these authors recognised that a lower unit of analysis could provide useful information on the actual research process, they suggest that a "university effect" (due to different strategies and characteristics at macro-level) can be detected.

Our tests suggest that such effect is not explaining efficiency differentials for our sample (observed value of the H statistic is 9.4, well below the critical value of 14.06 and the p-value is 0.22).

Fourth external factor: does the number of tenured academic staff influence efficiency?

The proportion of tenured research staff theoretically can affect the efficiency of a department: there is anecdotic evidence that tenured and non-tenured staff face quite different stimulus and incentives. Nevertheless, the direction of this influence is not clear a priori: tenured staff is certainly more expert in attracting research grants and in publishing academic papers, also thanks to reputational advantages; conversely, non- we should expect a correlation between the proportion of tenured researchers and the department's results. Ramsden (1994) showed that productivity of professors and readers was much higher than that of lecturers.

Our analysis did not find evidence of a statistical influence of this factor on research efficiency (the value of the H statistic is 0.16 and the related p-value is >0.6).

Fifth external factor: can departments' "longevity" affect research efficiency?

Finally, we looked at the possible effects due to the longevity of departments. As we measured inputs in terms of PLEASE REPHRASE TO AVOID "in terms of" numbers of academic staff and laboratories/facilities, it is reasonable that older departments accumulated higher stocks of inputs that could influence their outputs.

A key assumption here is that older departments can favour more cohesion among its members, facilitating the climate of collaboration and common motivation which enhances research productivity.

Another interesting point of view about this potential phenomenon is that it is related to the "critical mass" dimension. Among the economic aspects of research, Hare & Wyatt (1992) Consequently, it could be reasonable to assume that older departments are more likely to have reached such "optimal" scale dimension than the more recently established ones.

The statistical analysis did not provide statistically robust results on this effect, indeed the value of the H statistic is 1.172, while the critical value is around 3.8, so that the p-value is 0.27). Discussing our results about the efficiency change between 2004 and 2007, some questions must be posed: how can a Malmquist index equal to 1 exactly be interpreted? How can we judge a frontier shift that is less than 1? Each academic unit, on average, is in the 2007 in the same position relative to the frontier than in the 2004 (indeed, also the H index calculated above suggests small changes in the rankings in the period considered). However, what happened is something different: the vast majority of the units actually improved their productivity in the period ("efficiency change" >1) but the "frontier shift" was <1: it means that the performance differential is much less dispersed in 2007 but at the same time it happens because the frontier is not improved -the best performers did not improve, or even worsened their efficiency. The summary statistics confirms this idea (look at the really lower mean in 2004, with a much higher dispersion -standard deviation -around it). The result is a "research system" in the region that performs better on average but without having excellence. The importance of evaluation and accountability has been growing in these recent years and criteria used are widely acknowledged: publications and the ability to gather financial resources (from abroad or from firms). Since the application of these evaluations for funding allocation is more and more concrete, we believe that the weakest departments raised their attention towards those performances. Thanks to this new attention the weakest departments have increased their efficiency and become much closer to the best performers that, on the contrary, had little margin for further improvements.

Discussion and policy conclusions

This result requires an important policy discussion. Indeed, the objective of the Lombardy Region was to foster the performance of excellent research units. This purpose animated some policy interventions such as the creation of "clusters", the policies for improving contacts between companies' demand and research units through vouchers (the idea behind is that the best research units would be able to collect more vouchers), etc. Apparently, the results obtained are in the opposite direction.

This fact raises two reflections. The first is that maybe the universities (and, consequently, the research units) look not at the regional incentives but at the national ones (as the Ministry is still the main financer of the universities' activities) and at international ones (the research reputation strongly depends upon the ability to publish in international journals and to attract international grants). If this is the case, a question arises about the opportunity for a regional government to be involved in this sector.

A second (possibly alternative) reflection is that the policies adopted for boosting research productivity were not adequate in terms of PLEASE REPHRASE TO AVOID "in terms of" policy design. To properly design a policy in this field the Our second analysis regarded how five external factors ("external" because they are not under the control of the departments) affect efficiency (section 4.2.). Our empirical analysis suggests that none of these factors (location in metropolitan area, scientific sector, university effect, proportion of tenured academic staff, departments "longevity") is able to adequately explain efficiency differentials. We consider this result as positive, especially when thinking to the use of efficiency scores for policy purposes (e.g., resources allocation). Indeed, a usual critic to the efficiency analyses is that they provide results which are affected by factors which are not under the control of the units themselves: when this is the case, a procedure of resource allocation following efficiency results would be inequitable. Among others, Marinova & Newman (2008) At the same time, we argue that there are some elements, which actually could influence efficiency and which are related to some "soft" characteristics of the academic units, typically (i) the incentive systems and (ii) the governance/management. For instance, Ramsden (1994) pointed out that the perception of the academic staff about the importance of publications for their career was consistently related to the production of academic articles and books. Rodgers & Neri (2007) suggested that the "departmental environment" is important and they define it through several measures centred on the "human capital" of researchers who work in the department. Unfortunately, QuESTIO dataset does not collect such data:

an extension of this dataset along this dimension could help to improve significantly the understanding of this pattern.

However, if the incentive system really matters, another conclusion can be drawn from our study: as we measured a potential "university effect", and it is not statistically relevant, then we can conclude that no incentive system works at university level. Different reasons can be proposed to explain this result. In particular, it can be both the cases that (i) the incentive system is designed at an higher level (in

F o r R e v i e w O n l y
Italy, at the central government level), or (ii) that it is designed at a lower (department) level -and given the department-level of our data, we cannot exploit such dimension, which requires individual-level data. The evidence about the process for recruiting academic staff and its career in Italy suggests that, however, no government-level incentive exists -indeed, the career of professors is very poorly related to their scientific production. Levin (1991) underlined the importance of incentives to raise the productivity of universities' activities. More specifically, he acknowledged that higher education shows productivity declines due to two main reasons: (i) it is a "stagnant industry", as it is based on labour intensity; (ii) clear rewards and property rights are absent. Thus, the suggestion provided by the author is to develop a reward system within the university based on clear goals, financial incentives and evaluation procedures.

Whether such schemes have been actually implemented in departments analysed in this paper is not clear; probably detailed case studies can help to shed more light on this topic through future research.

The governance/management of the academic departments has not been analysed in our study, as the only information available in this respect (the presence of a quality system or a management control system) is a very indirect measure of the governance process and the data recovery on this topic was quite imprecise. Another way to improve the QuESTIO dataset should be to better investigate this dimension, by including some qualitative descriptions about the way Department managers organise and coordinate the activities and resources, formulate strategies and foresight studies and manage networking activities (see for instance Sala et al., forthcoming;Salerno et al., 2008). Without knowing the characteristics of the "management style" of each department, it is impossible to derive policy implications from such dimension. At the same time, a possible way to better understand the potential incidence of such phenomenon is to conduct case studies about the successful academic departments (for example, those with higher efficiency scores or those which highly improved the scores in the last years).

Finally, a potential candidate to analyse the efficiency differentials is the interaction between research and teaching activities conducted by the departments. Typically, the literature did not paid enough attention to the interaction between teaching and research at department level, mainly because both activities were considered as inputs and outputs (see section 2). In this paper, the focus was exclusively on the research activities but it raises also questions about the effects that the (unmeasured) teaching activities play on the research result. This issue is very relevant both in theoretical terms (are there economies of scope, or tradeoffs, between the two activities?) and in empirical ones (which is the optimal allocation of energies between teaching and research?). This interesting issue was at the centre of several contributes since the seminal paper by Cohn et al. (1989), that for the first time addressed and modelled the multi-outputs nature of academic activities. Recently, the theme has been explored by Bonaccorsi & Daraio (2007) for a sample of European universities. In the present paper, we cannot explore this topic because QuESTIO dataset does not record data about the teaching dimension of the departments' activities (as before, it could represent a substantial improvement of the dataset).

Finally, the issue of research quality still remains open. Indeed, our analysis is focused only on a quantitative dimension of research, while quality is important to adequately assess the performance. Typically the lack of data on this respect justifies the scarce attention to this problem. However, the QuESTIO dataset will include in the next years some qualitative indicators, like H and citation indexes. We will devote attention to this topic in future research, also exploring trade-offs between research quantity and quality.

The analyses and the data have allowed an interesting discussion in terms of PLEASE REPHRASE TO AVOID "in terms of" policy implication but we hope that further

Abstract.

This paper investigates the efficiency of university departments on science, technology and medicine in an Italian Region (Lombardy). The aim of the paper is twofold: (i) to analyse the changes in productivity in recent years (from 2004 to and 2007); and (ii) to detect factors that are potentially affecting efficiency. The research benefited from a new and unique dataset (called QuESTIO) developed by the Lombardy Regional Government. Using facilities and academic staff as inputs and research grants and publications as outputs, the research activity of academic departments was modelled. The methodological approach for computing efficiency scores is Data Envelopment Analysis (DEA). Malmquist indexes have been used to measure changes in productivity, while Kruskal-Wallis tests were employed to study the potential determinants of efficiency. The main results were that in the period considered, the academic departments improved their efficiency but, at the same time, the efficiency frontier worsened. Second, Moreover, external and measurable factors (such as, scientific sector, proportion of tenured staff, location) have a limited impact in explaining efficiency differentials. Policy implications of the results are discussed.

Keywords.

University research, efficiency, research policies, Data Envelopment Analysis, Malmquist index, higher education policy.

JEL

Introduction and motivation

In recent years, higher education institutions have been increasingly studied. In the ''knowledge economy'' their importance for economic development, social equity, mobility, social cohesion and integration is widely acknowledged (Brennan & Teichler, 2008). Furthermore, given the difficult situation of public finances, considerations about resources allocation have been raised in many countries, calling for more evaluations and accountability. For these reasons the so-called 'New Public Management' movement, started at the beginning of the 1980s in the public sector in general, has also increasingly been affecting the universities. New Public Management (Hood, 1991;1995) represents the request to public organisations to become more managerial, in particular by drawing on practices and models from the private sector. This process has been driven both by universities themselves and by governments through reforming efforts. The results, especially in many countries within Europe, share some common characteristics: first, the introduction of incentives schemes based on financial models for allocating public funds; second, new or strengthened assessment procedures and evaluation agencies; third, more emphasis on universities' autonomy (OECD, 2003).

This work explores how the recent changes in the university system affected the efficiency of university research. In particular, the research sought to understand whether an overall increase in efficiency can be recognised or whether differences are present between universities given the previously-noted increased autonomy.

Normally, the majority of the reforming efforts have been directed towards education activities, whilst an increase in efficiency of university research has been assigned to incentive systems and evaluation processes. These are expected to have had an overall positive effect on the system but, at the same time, they have left the universities in charge of finding and managing the ways to obtain better results. Given the autonomy of the universities and the decentralisation of responsibilities, significant efficiency changes are thus expected between and within universities. Furthermore, the idea that differentials in research performance are key for competition among departments is well established in the literature (Curran, 2000).

The research focused on universities research activities and not on education or knowledge/technology transfer activities. Universities' education activities constitute the first and original mission of the universities but, at the same time, they are normally carefully regulated by laws and their efficiency and quality are already widely studied (Madden et al., 1997). Indicators such as number of degrees awarded, rate of success, years of study, have been widely used both to evaluate the universities and to introduce new policies. The opposite holds for universities' knowledge/technology transfer activities. Even if some argue for this to be a third mission for academia and call for "entrepreneurial universities" (Clark, 1998;Etzkowitz, 2003), others refuse this perspective and signal the risks of focusing on and expanding the role of these activities. Moreover, the efficiency analysis of these activities is hampered by the difficulties in finding the right indicators. There are many different knowledge/technology transfer activities (for example, spin-off, consultancies, research projects with firms, patents) and their evaluation can be based only partially on economic values given the not-for-profit nature of (most)

universities (Landoni and Verganti, 2006). Finally, the focus of the study is on the research activities as they are the core activities of universities; driving and informing both education activities and knowledge/technology transfer activities. This The research considers departments located in the Lombardy region (Italy). A regional focus has been adopted to ensure similar contextual variables, such as policies and economic environment, for the universities included in the analysis.

Furthermore, Lombardy is one of the more advanced European regions both for (i) economic development and (ii) research activities and universities: more than 9.4 million habitants, GDP per capita >€34,000; 12 universities are located in the Region (out of about 70 in all the country) with more than 230,000 students. Finally, the regional level is increasingly recognised as a relevant level when analysing research and innovation activities. Many authors (Cooke et al., 1997;Porter, 1998;Lechner & Dowling, 1999) underline that it is at a local level that the majority of the connections between the different innovation actors take place. The localisation of knowledge and the role of proximity as a driver of economic development (for example, clusters)

have also highlighted the importance of regional and local government in fostering and strengthening the actors and their relationships.

To summarise, the focus of the paper is on efficiency of research activities in scientific departments in the Lombardy Region (Italy). The research questions are:

• did changes in the overall efficiency of departments' research occur between 2004 and 2007?

• Which are the main factors affecting efficiency?

The next section presents recent literature on the topic, then the methodology and the data used are introduced, finally results are presented and discussed. and conclusions are drawn.

Background

The international literature about universities' efficiency (through non-parametric techniques) has been developed both at institution-level (Athanassopoulos & Shale, 1997;McMillan & Datta, 1998;Ng & Li, 2000;Avkiran, 2001;Abbott & Doucouliagos, 2003;Flagg, et al, 2004;Warning, 2004) and department-level. The latter is more relevant for the context of the present study.

The contributions by Johnes & Johnes (1993;1995) performance of UK economics departments by using staff numbers as inputs and publication counts and grants as outputs; the same strategy was adopted again for deriving efficiency scores of economics department in a subsequent paper (Johnes & Johnes, 1995). The authors ran several Data Envelopment Analysis (DEA) models to test the robustness of results and claim for a possible use of results for policy purposes, especially for establishing an optimal allocation of resources across departments. These two contributions are the most important as they focus exclusively on the research dimension of the universities' work. Madden et al. (1997) enrolments and grant awards as outputs. In discussing their results, they showed that efficiency scores do not illustrate information about effectiveness: that is, they argued that a qualitative dimension is necessary for this purpose. Lastly, as their background is strongly managerial, they also suggested a use of efficiency analysis as a planning tool, not only for evaluating universities' activities. Koksal & Nalcaci (2006) derived efficiency scores for 14 university departments at the Middle East Technical University (METU) in Turkey, by using staff salaries, a composite indicator of previous research quality and number of students as inputs and number of publications, graduates and teaching indicators as outputs. Thus, in their contribution, the authors focused also on the teaching activity of the departments.

They decided to use weights for teaching and research loads as suggested by university's administrators through a preliminary survey. The paper suggests that the models adopted for computing efficiency can serve to set new targets for inefficient departments. Gimenez & Martinez (2006) tenure staff. To measure the outputs, they employed the following variables: results of a government evaluation process of research, teaching load (measured through credits granted) and students' opinions. The authors suggested that their analyses can be used to assess the differences between (i) the observed cost of the units and (ii) the cost they would achieve assuming long-term cost minimisation. Their result tells that size does not matter in determining efficiency, while higher proportion of non-tenured staff is associated with higher efficiency levels.

In another exercise presented by Kocher et al. (2006) respect to the relevant contributions for the present paper, as the unit of analysis is the country (neither the university nor the department). The results suggest that research productivity can be raised through an increase in the scale of operations.

An interesting issue is to analyse the related literature about the Italian universities' research. To the best of the authors' knowledge, there were are only two specific contributes that specifically analyse the efficiency of universities in producing research outputs (Pesenti & Ukovich, 1996;Rizzi, 1999) and just one attempting to focus on teaching production (Agasisti & Dal Bianco, 2009).

The two contributions at departmental level (Pesenti & Ukovich, 1996;Rizzi, 1999) use used a similar set of variables and focus both on teaching and research outputs.

The main scope for these contributions is was to provide information and suggestions for allocating resources within a university.

Contributions about the efficiency of Italian higher education have been conducted at institution-level (thus, not considering separately the structures devoted to teaching and research). The reason for this choice can be summarised by the idea that the strategy and the mission of universities is defined at macro (institution) level, then the balance between teaching and research is decided by the decision centres at institution level (Bonaccorsi & Daraio, 2007, p. 22). This is because the strategy and the mission of universities and the balance between teaching and research are defined at institutional level (Bonaccorsi & Daraio, 2007, p. 22).

Among these contributions at an institutional level Agasisti & Dal Bianco (2006) studied the efficiency of 58 Italian public universities, by using, as inputs, number of staff, financial resources and students and, as outputs, graduates and research grants.

They suggested the use of efficiency analysis for policy purposes, especially to evaluate the differences in performance between efficient and inefficient units. Later, Agasisti & Salerno (2007) analysed the cost efficiency of another sample of Italian universities, by adding some qualitative measures for inputs and outputs. They concluded that a redistribution of inputs across universities could lead to an overall improvement of efficiency in the Italian higher education sector. Bonaccorsi et al. (2006) used a robust non-parametric technique (developed by Daraio & Simar, 2005) to take in account external conditions affecting the productivity of universities. The inputs and outputs adopted are quite similar to previous studies (that is, staff numbers, expenses and spaces as inputs and publications and graduates as outputs) but they also included some "external" factors: To sum up, the literature seems to suggest that, to analyse research efficiency of universities, that is, the efficiency in the production of research outputs:

F o r R e v i
• inputs can be classified under three main groups: human resources, financial resources and structures (e.g., facilities, buildings);

• outputs can be grouped into publications and grants.

The best indicators to measure these classes of inputs/outputs are questionable but the general emerging consensus is on a productive process that can be described with

Methodology and data

Methodology

The methodology is articulated accordingly to the two steps adopted to answer the research questions of the paper: on one hand the efficiency analysis (including efficiency scores of the departments, changes across time and time trends), on the other hand the effort to identify the determinants of efficiency.

The first choice is about the methodology to compute efficiency, because there are different methods to target the analysis of efficiency among different units Data Envelopment Analysis (DEA) is a frontier, linear programming technique, introduced by the seminal work of Charnes et al. (1978). It deals with the efficiency problem of complex organisations that produce many outputs using several inputs at a time. Data Envelopment Analysis (DEA) (Charnes et al., 1978) was the methodology used to analyse efficiency. It deals with the efficiency problem of complex organisations that produce many outputs using several inputs at a time. This paper only briefly discusses some characteristics of DEA, referring the interested reader to Johnes (2004), Cooper et al., (2006) and Zhu (2003).

In a DEA model, the notion of efficiency is that of "technical efficiency", defined as the relative ability of each decision making unit (DMU) in producing outputs, where the term "relative" means that each organisation is compared with any other homogeneous unit. DEA could be represented by a linear programming technique where each DMU tries to maximise the efficiency ratio (output over inputs) choosing the best set of weights. There are two ways to deal with a DEA formulation: (i) input oriented (that is, the ability to minimise inputs when outputs are given) and (ii) output oriented (maximisation of outputs given a certain bundle of inputs). The latter approach is used in this paper, as the typical problem for a research unit is to maximise outputs (publications, grants) given its inputs (research staff, laboratories).

DEA mathematical formulation can deal with both constant returns to scale (CRS)

and variable returns to scale (VRS). The seminal work of Charnes et al. (1978) introduced Smaller units are associated with increasing economies of scale and bigger units with decreasing economies of scale. In the middle, there is a "grey area" of constant returns to scale. So, it is also possible to compute the scale efficiency. This is defined as the ratio CRS/VRS efficiencies and interpreted as the ability of each institution to benefit in terms of PLEASE REPHRASE TO AVOID "in terms of" WHICH FAILS TO EXPLAIN THE RELATIONSHIP productivity from its size: operate at its optimal scale dimension; in other words, in a VRS formulation, each DMU is compared only with those units having a comparable size. We It is also possible to measure changes in efficiency across time and our (the relevant period here is between 2004 and 2007. For measuring the determinants of the changes in efficiency scores, we use Malmquist index has been used (for more information about this index see Johnes (2004) and Coelli et al., (2006)). Let us Considering a DMU, named P that has a change in production position from P t (in period t) to P t+1 (in period t+1), the change in the production frontier could have occurred in a non-neutral way; that is, it could be determined both by efficiency effect on the efficiency score, if it is greater than 1 it has a positive effect and if it is less than 1 it has a negative effect. In other words, a shift in a production frontier that makes worse the relative position of the DMU (the other DMUs are performing relative better than before) provides a second component less than 1. The index (Malmquist index) resulting from the combination of these two components will be equal to 1 if there is no net effect of changes in technical efficiency and frontier changes; it will be greater than 1 if the net effect is positive and less than 1 if it is negative. In our the analysis, we name "efficiency change" is the first component and "frontier shift" the second one. I THINK A CONCRETE EXAMPLE IS NECESSARY HERE TO GUIDE THE READER.

Moreover, we would detect the eventual stability of efficiency rankings across the period (again, the relevant period is between 2004 and 2007). In this case, we adopted one of the statistics suggested by Brockett et al. (1998) ratings and observing the sum of ranks associated with each DMU, one can learn about the relative position of the DMUs vis-à-vis each other across the whole period" (Brockett et al., 1998, p. 174). For this purpose, the basis of the analysis is a Kruskal-Wallis test (Kruskal & Wallis, 1952, p. 175) where "the rejection of the null hypothesis leads to the conclusion that, in general, the different DMUs maintain their relative efficiency positions over time". AGAIN A CONCRETE EXAMPLE WOULD HELP HERE.

The second task in our the methodology is to study some potential determinants of efficiency. To pursue this objective, we identified some external factors (for example, location effects, discipline effects) were identified and then we compared efficiency scores were compared across different groups of units. The comparison was made by means of the Kruskal-Wallis test (Kruskal & Wallis, 1952). This test allows the researcher to detect if are there statistical differences across the groups analysed (for example, when comparing the efficiency scores of units located in a metropolitan area with those located elsewhere, the test indicates if this location effect generates a statistical difference between the average scores of the two groups).

Data

The research used a unique dataset that consists of input and output variables about Since it aims at presenting the most detailed picture of regional community active in research and technological transfer, the dataset has some specific criteria that are pivotal for the analyses. First, the system is focused on every structure where such activities are carried out, even where that structure is a part of a wider organisation. Second, these structures should have significant management autonomy so that their accountancy shall be determinable. The first criterion enables the regional administration to collect information on every single department, being acknowledged that the performance of departments belonging to the same university can differ significantly. As a consequence the university departments, as well as sections and institutes of the National Research Council or a research and development unit belonging to a company, provide data that refers solely to their specific resources and outputs. The second criterion assures that each structure included in the dataset is responsible (to some extent) for its strategy and performances thus justifying compatible? with previous results derived across the world in the higher educations sector (Salerno, 2003). BUT WHAT IS THIS TELLING US? What is worth pointing out is a strong heterogeneity across departments: indeed, the standard deviation is >0.2. A look at the lowest scores suggests that there are very poor performers which

shows scores [??] <0.5.

Data about scale efficiency suggests that all the departments have almost reached their optimal scale of operations (given their present scale), as the scale efficiency score is very high (>0.93) and standard deviation is very low (about 0.1) in this respect.

The picture that emerges from this first analysis of efficiency scores is a sector where there is a number of very efficient units (both assuming constant or variable returns to scale) and a number of departments that experienced low levels of technical efficiency; thus, as the anecdotic anecdotal evidence suggests, there is quite a strong

Efficiency changes between 2004 and 2007 -a Malmquist analysis

Following the methodology described above, we derived the the Malmquist index has been calculated, which allows us to disentangle the efficiency changes due to "pure" efficiency improvement (or worsening) and technological improvement (or worsening). The descriptive statistics about the results are reported in Table 3. while the detailed results are available on request from the authors.

<Table 3> about here

The picture that emerges highlights an interesting phenomenon. It looks like an improvement in productivity did not occur (because the Malmquist index is around 1)

but, looking at the decomposition of the index, the reality appears more complex: an efficiency changes happened, it was very positive (the index is >1.6) and involved the majority of the departments (56 out of 70); conversely, there was a worsening of the technology frontier (the "frontier shift" indicator is <0.7). What is "technology" in a regional research system, when we define research performance is defined as the ability to attract financial resources and to publish in academic journals? Technology could be defined as the bundle of policies that should help in improving research efficiency, for example, information technologies, research grants, support for publishing research results or to attract financial resources from international grants or companies. This fact raises two reflections. The first is that maybe the universities (and, consequently, the research units) look not at the regional incentives but at the national ones (as the Ministry is still the main financer of the universities' activities) and at international ones (the research reputation strongly depends upon the ability to publish in international journals and to attract international grants). If this is the case, a question arises about the opportunity for a regional government to be involved in this sector. A second (possibly alternative) reflection is that the policies adopted for boosting research productivity were not adequately designed in terms of PLEASE REPHRASE TO AVOID "in terms of" policy design. To properly design a policy in this field the necessary first step is to understand what affect the academic productivity. In this context, Faria (2002) suggests that policies for fostering the overall productivity of the research sector are:

(i) competition in the labour market among researchers (e.g., through career or salary incentives) and (ii) increasing research funds. Did the policies of Lombardy Region respect these elements? Are there other elements that affect research productivity?

These are interesting starting point for further research in this field.

Thus, the results (Table 3) suggest that, while the majority of the universities improved their efficiency, the system as a whole did not experience an improvement of in terms of PLEASE REPHRASE TO AVOID "in terms of" technology available.

A discussion about the potential responsibilities for such phenomenon will be reported in the last section, which deals with policy implications of the study. As a result, however, each academic unit, on average, is in the 2007 in the same position relative to the frontier than in the 2004. The vast majority of the units actually improved their productivity in the period ("efficiency change" >1) but the "frontier shift" was <1: it means that the performance differential is much less dispersed in 2007 but at the same time it happens because the frontier is not improved -the best performers did not improve, or even worsened their efficiency. The result is a "research system" in the region that performs better on average but without having excellence.

Productivity changes between 2004 and 2007: an analysis of "rankings stability"

The stability of rankings across the analysis period (between 2004 and 2007) were analysed by means of the H index proposed by Brockett et al. (1998). The index is equal to 177.96 and it must be compared with H 2 that is equal to 102.99. The rejection of the null hypothesis leads to the conclusion that some of the academic departments exhibit consistently better economic performance than others (as measured by the input-output factors that were included in our model) (see Brockett et al., 1998).

Reading this results together with the Malmquist index, it looks like the academic departments improved their performance but some units continue to perform better than others. At the same time, the results show that some units are not efficient, that is, the frontier actually worsened.

DEA efficiency scores, year 2007: the external factors affecting efficiency

This section, given the evidence on the efficiency results, it is interesting to explore which are the determinants of this efficiency or, in other words, what could actually affect a department's performance.

To answer this question, we tested a number of different factors, by using the Kruskal-Wallis test as methodological approach (see the data section). All the results are listed in Table 4.

This section explores what affects a department's performance. Several factors were tested using the Kruskal-Wallis test, which showed…. WHAT? (Table 4) <Table 4> about here 4.2.1 First external factor: are departments located in Milan metropolitan area (regional capital) more efficient than other departments?

We analysed this factor on the The assumption was that in a metropolitan area the competition among research centres enforces departments' managerial efficiency. In for managerial quality. Elsewhere, Agasisti (2009) demonstrated that the effect of competition on Italian universities' performance is positive -but, there, output was defined through teaching dimension's measures.

We ran a Kruskal-Wallis test to verify the statistical difference between the efficiency scores of departments located in Milan versus the other departments. We do not find

There is no evidence of difference in efficiency performance between the two groups (indeed, the observed value of the H statistic is nearly 0 and the p-value >0.9). This evidence suggests that there is no "concentration" effect or "location" effect on efficiency performance of university departments or, at least, in the relative rankings.

At the same time, this result may suggest i) the existence of no competitive pressure in a high concentration area as Milan., or that ii) we have to look at overall concentration of research centres and not only to university departments (e.g., there could be a "university" effect which could dominate a "location" effect, or that iii)

there is competition but at the same time more opportunities to find resources (more demand for research and consequently the amount of resources is enough to guarantee a sufficient level of grants for different departments without the need to compete one against the others).

Second external factor: can different scientific sectors affect university department efficiency?

As we measured "output" is measured by using i) publication counts and ii) research grants (of different kinds), it is important to recall that there are some differences in the average number and quality of publication across different disciplines (e.g. medicine versus engineering or biology), as well as differences in terms of PLEASE REPHRASE TO AVOID "in terms of" in the attractiveness of research funds (e.g., in engineering fields it is simpler to attract grants for applied research than in physics).

The idea that disciplines greatly vary in terms of PLEASE REPHRASE TO AVOID "in terms of" in costs and productivity is not new and the suggestion to take this factor into account is important to obtain not-biased results (Sarrico & Dyson, 2004).

Recently, Sarrico et al. (2009) demonstrated a strong role of "subject mix" (defined as the disciplinary composition of institutions) in explaining the efficiency differentials among Portuguese higher education institutions. The authors acknowledged that "(…) research production also varies significantly from subject to subject, with medical, some natural and technical sciences being more prolific in the number of articles than

F o r R e v i e w O n l y 20
most of the social sciences and humanities" (p.290). Also Ramsden (1994), in his empirical analysis of Australian research departments, adjusted the coefficients on the basis of discipline subjects.

We It was not found a small but statistically significant difference among university departments classified by scientific sectors: the observed value of the H statistic is >8.4, near the critical value (9.4) and the p-value is 0.07 (then the effect has statistical significance at the 10% level WHY CHOOSE 10%? WHY NOT SAY NOT SIGNICANT (AT 5%, WHICH TENDS TO BE THE NORM?)). However, given the limited power of this effect, it is not easy to determine which sectors outperform the others straightforwardly.

4.2.3. Third external factor: is there a "university effect"?

An alternative explanation of efficiency can refer to the existence of a "university effect", that is if a university is more efficient than others, then its departments could be more productive than those belonging to other universities. This assumption finds its justification in the setting of strategies at university levels, not at subunits levels. In this respect, Bonaccorsi & Daraio (2007, pp. 22-23) pointed out that:

"the notion of strategy is much more relevant and interesting if assumed at the overall university level. On the one hand, most strategic decisions are made at university level.… The fundamental reason for assuming the university level as the unit of analysis is that at the university level the problem of attribution of inputs (in particular, human resources, funding and physical capital) to specific units of outputs, can be kept under control".

Even if these authors recognised that a lower unit of analysis could provide useful information on the actual research process, they suggest that a "university effect" (due to different strategies and characteristics at macro-level) can be detected.

Our The tests suggest that such effect is not explaining efficiency differentials for the our sample (observed value of the H statistic is 9.4, well below the critical value of 14.06 and the p-value is 0.22).

Fourth external factor: does the number of tenured academic staff influence efficiency?

The proportion of tenured research staff theoretically can affect the efficiency of a department: there is anecdotic evidence that tenured and non-tenured staff face quite

F o r R e v i e w O n l y
different stimulus and incentives. Nevertheless, the direction of this influence is not clear a priori: tenured staff is certainly more expert in attracting research grants and in publishing academic papers, also thanks to reputational advantages; conversely, nontenured staff can have an incentive to be more productive, for instance as a mean to obtain a tenured position. The promotion system can affect productivity if researchers are satisfied with rules promoting this dimension.

A complementary idea behind this question is that the different human capital of researchers (in our the paper, indirectly measured via the tenured status) can affect the research productivity of the research groups and, so, of the department. In this respect, an interesting example is the paper by Rodgers & Neri (2007), who specifically attempted to correlate the human capital of researchers with the research productivity of Australian departments of Economics. They have found that human capital differential account for about 50% of the observed differences in research productivity. If human capital is (even indirectly) captured by the tenured status (e.g., researchers with more experience should have reached their tenured position), then we it should be expected a correlation between the proportion of tenured researchers and the department's results. Ramsden (1994) showed that productivity of professors and readers was much higher than that of lecturers.

Our The analysis did not find evidence of a statistical influence of this factor on research efficiency (the value of the H statistic is 0.16 and the related p-value is >0.6).

Fifth external factor: can departments' "longevity" affect research efficiency?

Finally, we it was looked at the possible effects due to the longevity of departments.

As we measured inputs were measured as in terms of PLEASE REPHRASE TO AVOID "in terms of" numbers of academic staff and laboratories/facilities, it is reasonable that older departments accumulated higher stocks of inputs that could influence their outputs.

A key assumption here is that older departments can favour more cohesion among its members, facilitating the climate of collaboration and common motivation which enhances research productivity.

Another interesting point of view about this potential phenomenon is that it is related to the "critical mass" dimension. Among the economic aspects of research, Hare & Wyatt (1992) underlined that the ability in producing research relies upon the human capital accumulated by the individual researchers; thus:

F o r R e v i e w O n l y
"combining this process with the notion that there are economies of scale in research may dispose university departments to feel they need to have at least some critical mass of researchers working in the same area in order to function properly as research units". (Hare & Wyatt, 1992, pp. 55-56) Consequently, it could be reasonable to assume that older departments are more likely to have reached such "optimal" scale dimension than the more recently established ones.

The statistical analysis did not provide statistically robust results on this effect, indeed the value of the H statistic is 1.172, while the critical value is around 3.8, so that the p-value is 0.27).

To sum up, the empirical analysis suggests that none of these factors (location in metropolitan area, scientific sector, university effect, proportion of tenured academic staff, departments "longevity") is able to adequately explain efficiency differentials.

We consider This result should be considered as positive, especially when thinking to the use of efficiency scores for policy purposes (e.g., resources allocation). Indeed, a usual critic to the efficiency analyses is that they provide results which are affected by factors which are not under the control of the units themselves: when this is the case, a procedure of resource allocation following efficiency results would be inequitable.

At the same time, we argue it could be possible that there are some elements, which actually could influence efficiency and which are related to some "soft" characteristics of the academic units, typically (i) the incentive systems and (ii) the governance/management. For instance, Ramsden (1994) pointed out that the perception of the academic staff about the importance of publications for their career was consistently related to the production of academic articles and books. Rodgers & Neri (2007) suggested that the "departmental environment" is important and they define it through several measures centred on the "human capital" of researchers who work in the department. Unfortunately, QuESTIO dataset does not collect such data:

an extension of this dataset along this dimension could help to improve significantly the understanding of this pattern.

The governance/management of the academic departments has not been analysed in the our study, as the only information available in this respect (the presence of a quality system or a management control system) is a very indirect measure of the governance process and the data recovery on this topic was quite imprecise. Another way to improve the QuESTIO dataset should be to better investigate this dimension, by including some qualitative descriptions about the way Department managers organise and coordinate the activities and resources, formulate strategies and foresight studies and manage networking activities (see for instance Sala et al., forthcoming;Salerno et al., 2008). Without knowing the characteristics of the "management style"

of each department, it is impossible to derive policy implications from such dimension. At the same time, a possible way to better understand the potential incidence of such phenomenon is to conduct case studies about the successful academic departments (for example, those with higher efficiency scores or those which highly improved the scores in the last years).

Finally, a potential candidate to analyse the efficiency differentials is the interaction between research and teaching activities conducted by the departments. Typically, the literature did not paid enough attention to the interaction between teaching and research at department level, mainly because both activities were considered as inputs and outputs (see section 2). In this paper, the focus was exclusively on the research activities but it raises also questions about the effects that the (unmeasured) teaching activities play on the research result. This issue is very relevant both in theoretical terms (are there economies of scope, or tradeoffs, between the two activities?) and in empirical ones (which is the optimal allocation of energies between teaching and research?). This interesting issue was at the centre of several contributes since the seminal paper by Cohn et al. (1989), that for the first time addressed and modelled the multi-outputs nature of academic activities. Recently, the theme has been explored by Bonaccorsi & Daraio (2007) for a sample of European universities. In the present paper, we cannot explore this topic has not been explored because QuESTIO dataset does not record data about the teaching dimension of the departments' activities (as before, it could represent a substantial improvement of the dataset).

Finally, the issue of research quality still remains open. Indeed, our the analysis is focused only on a quantitative dimension of research, while quality is important to adequately assess the performance. Typically the lack of data on this respect justifies the scarce attention to this problem. However, the QuESTIO dataset will include in the next years some qualitative indicators, like H and citation indexes. We It will be devoted attention to this topic in future research, also exploring trade-offs between research quantity and quality. As highlighted discussing the literature on the topic, the analyses presented in this work are innovative for the in terms of PLEASE REPHRASE TO AVOID "in terms of" data and the methodology. Indeed, in this paper we have investigated the efficiency of university departments on science and technology and medicine in an Italian region (Lombardy). The aim of the paper was (i) to analyse the changes in productivity in recent years (from 2004 and 2007) and (ii) to detect factors that are potentially affecting efficiency.

Discussing our results about the efficiency change between 2004 and 2007, some questions must be posed: how can a Malmquist index equal to 1 exactly be interpreted? How can we judge a frontier shift that is less than 1? Each academic unit, on average, is in the 2007 in the same position relative to the frontier than in the 2004 (indeed, also the H index calculated above suggests small changes in the rankings in the period considered). However, what happened is something different: the vast majority of the units actually improved their productivity in the period ("efficiency change" >1) but the "frontier shift" was <1: it means that the performance differential is much less dispersed in 2007 but at the same time it happens because the frontier is not improved -the best performers did not improve, or even worsened their efficiency. The summary statistics confirms this idea (look at the really lower mean in 2004, with a much higher dispersion -standard deviation -around it). The result is a "research system" in the region that performs better on average but without having excellence.

The importance of evaluation and accountability has been growing in these recent years and criteria used are widely acknowledged: publications and the ability to gather financial resources (from abroad or from firms). Since the application of these evaluations for funding allocation is more and more concrete, we believe that the weakest departments raised their attention towards those performances. Thanks to this new attention the weakest departments have increased their efficiency and become This result requires an important policy discussion. Indeed, the objective of the Lombardy Region was to foster the performance of excellent research units. This purpose animated some policy interventions such as the creation of "clusters", the policies for improving contacts between companies' demand and research units through vouchers (the idea behind is that the best research units would be able to collect more vouchers), etc. Apparently, the results obtained are in the opposite direction.

This fact raises two reflections. The first is that maybe the universities (and, consequently, the research units) look not at the regional incentives but at the national ones (as the Ministry is still the main financer of the universities' activities) and at international ones (the research reputation strongly depends upon the ability to publish in international journals and to attract international grants). If this is the case, a question arises about the opportunity for a regional government to be involved in this sector.

A second (possibly alternative) reflection is that the policies adopted for boosting research productivity were not adequate in terms of PLEASE REPHRASE TO AVOID "in terms of" policy design. To properly design a policy in this field the necessary first step is to understand what affect the academic productivity. In this context, Faria (2002) suggests that policies for fostering the overall productivity of the research sector are: (i) competition in the labour market among researchers (e.g., through career or salary incentives) and (ii) increasing research funds. Did the policies of Lombardy Region respect these elements? Are there other elements that affect research productivity? These are interesting starting point for further research in this field.

Our second analysis regarded how five external factors ("external" because they are not under the control of the departments) affect efficiency (section 4.2.). Our empirical analysis suggests that none of these factors (location in metropolitan area, scientific sector, university effect, proportion of tenured academic staff, departments "longevity") is able to adequately explain efficiency differentials. We consider this result as positive, especially when thinking to the use of efficiency scores for policy purposes (e.g., resources allocation). Indeed, a usual critic to the efficiency analyses is At the same time, we argue that there are some elements, which actually could influence efficiency and which are related to some "soft" characteristics of the academic units, typically (i) the incentive systems and (ii) the governance/management. For instance, Ramsden (1994) pointed out that the perception of the academic staff about the importance of publications for their career was consistently related to the production of academic articles and books. Rodgers & Neri (2007) suggested that the "departmental environment" is important and they define it through several measures centred on the "human capital" of researchers who work in the department. Unfortunately, QuESTIO dataset does not collect such data:

an extension of this dataset along this dimension could help to improve significantly the understanding of this pattern.

However, if the incentive system really matters, another conclusion can be drawn from our study: as we measured a potential "university effect", and it is not statistically relevant, then we can conclude that no incentive system works at university level. Different reasons can be proposed to explain this result. In particular, it can be both the cases that (i) the incentive system is designed at an higher level (in Italy, at the central government level), or (ii) that it is designed at a lower (department) level -and given the department-level of our data, we cannot exploit such dimension, which requires individual-level data. The evidence about the process for recruiting academic staff and its career in Italy suggests that, however, no government-level incentive exists -indeed, the career of professors is very poorly related to their scientific production. Levin (1991) underlined the importance of incentives to raise the productivity of universities' activities. More specifically, he acknowledged that higher education shows productivity declines due to two main reasons: (i) it is a "stagnant industry", as it is based on labour intensity; (ii) clear rewards and property rights are absent. Thus, the suggestion provided by the author is to develop a reward system within the university based on clear goals, financial incentives and evaluation procedures.

Whether such schemes have been actually implemented in departments analysed in this paper is not clear; probably detailed case studies can help to shed more light on this topic through future research.

The governance/management of the academic departments has not been analysed in our study, as the only information available in this respect (the presence of a quality system or a management control system) is a very indirect measure of the governance process and the data recovery on this topic was quite imprecise. Another way to improve the QuESTIO dataset should be to better investigate this dimension, by including some qualitative descriptions about the way Department managers organise and coordinate the activities and resources, formulate strategies and foresight studies and manage networking activities (see for instance Sala et al., forthcoming;Salerno et al., 2008). Without knowing the characteristics of the "management style" of each department, it is impossible to derive policy implications from such dimension. At the same time, a possible way to better understand the potential incidence of such phenomenon is to conduct case studies about the successful academic departments (for example, those with higher efficiency scores or those which highly improved the scores in the last years).

Finally, a potential candidate to analyse the efficiency differentials is the interaction between research and teaching activities conducted by the departments. Typically, the literature did not paid enough attention to the interaction between teaching and research at department level, mainly because both activities were considered as inputs and outputs (see section 2). In this paper, the focus was exclusively on the research activities but it raises also questions about the effects that the (unmeasured) teaching activities play on the research result. This issue is very relevant both in theoretical terms (are there economies of scope, or tradeoffs, between the two activities?) and in empirical ones (which is the optimal allocation of energies between teaching and research?). This interesting issue was at the centre of several contributes since the seminal paper by Cohn et al. (1989), that for the first time addressed and modelled the

F o r R e v i e w O n l y
multi-outputs nature of academic activities. Recently, the theme has been explored by Bonaccorsi & Daraio (2007) for a sample of European universities. In the present paper, we cannot explore this topic because QuESTIO dataset does not record data about the teaching dimension of the departments' activities (as before, it could represent a substantial improvement of the dataset).

Finally, the issue of research quality still remains open. Indeed, our analysis is focused only on a quantitative dimension of research, while quality is important to adequately assess the performance. Typically the lack of data on this respect justifies the scarce attention to this problem. However, the QuESTIO dataset will include in the next years some qualitative indicators, like H and citation indexes. We will devote attention to this topic in future research, also exploring trade-offs between research quantity and quality.

The analyses and the data have allowed an interesting discussion in terms of PLEASE REPHRASE TO AVOID "in terms of" policy implication but we hope that further This paper investigates the efficiency of university departments on science, technology and medicine in an Italian Region (Lombardy). The aim of the paper is twofold: (i) to analyse the changes in productivity in recent years (from 2004 to 2007); and (ii) to detect factors that are potentially affecting efficiency. The research benefited from a new and unique dataset (called QuESTIO) developed by the Lombardy Regional Government. Using facilities and academic staff as inputs and research grants and publications as outputs, the research activity of academic departments was modelled. The methodological approach for computing efficiency scores is Data Envelopment Analysis. Malmquist indexes have been used to measure changes in productivity, while Kruskal-Wallis tests were employed to study the potential determinants of efficiency. The main results were that in the period considered, the academic departments improved their efficiency but, at the same time, the efficiency frontier worsened. Moreover, external and measurable factors (such as, scientific sector, proportion of tenured staff, location) have a limited impact in explaining efficiency differentials. Policy implications of the results are discussed.

Keywords.

University research, efficiency, research policies, Data Envelopment Analysis, Malmquist index, higher education policy.

Introduction and motivation

In recent years, higher education institutions have been studied increasingly. In the ''knowledge economy'' their importance for economic development, social equity, mobility, social cohesion and integration is widely acknowledged (Brennan & Teichler, 2008). Furthermore, given the difficult situation of public finances, considerations about resources allocation have been raised in many countries, calling for more evaluations and accountability. For these reasons the so-called 'New Public Management' movement, started at the beginning of the 1980s in the public sector in general, has also increasingly been affecting the universities. New Public Management (Hood, 1991(Hood, , 1995) ) represents the request to public organisations to become more managerial, in particular by drawing on practices and models from the private sector. This process has been driven both by universities themselves and by governments through reforming efforts. The results, especially in many countries within Europe, share some common characteristics: first, the introduction of incentives schemes based on financial models for allocating public funds; second, new or strengthened assessment procedures and evaluation agencies; third, more emphasis on universities' autonomy (OECD, 2003).

This work explores how the recent changes in the university system affected the efficiency of university research. In particular, the research sought to understand whether an overall increase in efficiency can be recognised or whether differences are present between universities given the previously-noted increased autonomy.

Normally, the majority of the reforming efforts have been directed towards education activities, whilst an increase in efficiency of university research has been assigned to incentive systems and evaluation processes. These are expected to have had an overall positive effect on the system but, at the same time, they have left the universities in charge of finding and managing the ways to obtain better results. Given the autonomy of the universities and the decentralisation of responsibilities, significant efficiency changes are thus expected between and within universities. Furthermore, the idea that differentials in research performance are key for competition among departments is well established in the literature (Curran, 2000). widely studied (Madden et al., 1997). Indicators such as number of degrees awarded, rate of success, years of study, have been widely used both to evaluate the universities and to introduce new policies. The opposite holds for universities' knowledge/technology transfer activities. Even if some argue for this to be a third mission for academia and call for "entrepreneurial universities" (Clark, 1998;Etzkowitz, 2003), others refuse this perspective and signal the risks of focusing on and expanding the role of these activities. Moreover, the efficiency analysis of these activities is hampered by the difficulties in finding the right indicators. There are many different knowledge/technology transfer activities (for example, spin-off, consultancies, research projects with firms, patents) and their evaluation can be based only partially on economic values given the not-for-profit nature of (most)

universities (Landoni & Verganti, 2006). Finally, the focus of the study is on the research activities as they are the core activities of universities; driving and informing The research considers departments located in the Lombardy region (Italy). A regional focus has been adopted to ensure similar contextual variables, such as policies and economic environment, for the universities included in the analysis.

Furthermore, Lombardy is one of the more advanced European regions both for (i) economic development and (ii) research activities and universities: more than 9.4 million habitants, GDP per capita >€34,000; 12 universities are located in the Region (out of about 70 in all the country) with more than 230,000 students. Finally, the regional level is increasingly recognised as a relevant level when analysing research and innovation activities. Many authors (Cooke et al., 1997;Porter, 1998;Lechner & Dowling, 1999) underline that it is at a local level that the majority of the connections between the different innovation actors take place. The localisation of knowledge and the role of proximity as a driver of economic development (for example, clusters)

have also highlighted the importance of regional and local government in fostering and strengthening the actors and their relationships.

To summarise, the focus of the paper is on efficiency of research activities in scientific departments in the Lombardy Region (Italy). The research questions are:

• did changes in the overall efficiency of departments' research occur between 2004 and 2007?

• Which are the main factors affecting efficiency?

The next section presents recent literature on the topic, then the methodology and the data used are introduced, finally results are presented and discussed.

Background

The international literature about universities' efficiency (through non-parametric techniques) has been developed both at institution-level (Athanassopoulos & Shale, 1997;McMillan & Datta, 1998;Ng & Li, 2000;Avkiran, 2001;Abbott & Doucouliagos, 2003;Flagg, et al., 2004;Warning, 2004) and department-level. The latter is more relevant for the context of the present study.

The contributions by Johnes & Johnes (1993, 1995) had a major influence on the methodological approach. Johnes & Johnes (1993) publication counts and grants as outputs; the same strategy was adopted again for deriving efficiency scores of economics department in a subsequent paper (Johnes & Johnes, 1995). The authors ran several Data Envelopment Analysis (DEA) models to test the robustness of results and claim for a possible use of results for policy purposes, especially for establishing an optimal allocation of resources across departments. These two contributions are the most important as they focus exclusively on the research dimension of the universities' work. Madden et al. (1997) studied the efficiency of economics departments in 29

Australian universities. Inputs were teaching and research staff, outputs were graduates and publications (for teaching and research respectively). As in other studies listed below, here the focus was both on teaching and research performance, even if the analysis were conducted at sub-unit level (that is at departmental level instead of institutional level). The specific objective of the Madden et al. (1997) study was to verify whether policies promoted by governments impacted on productivity; they confirmed that there was actually an increase in productive efficiency. Thursby (2000, p. 400) analysed the efficiency of the Economics Research Departments in the United States. He defined efficiency as follows:

a department is deemed technically efficient if, when compared to departments with similar level of inputs, it could produce greater research outputs without increasing its inputs usage, or equivalently, it is one which, compared to departments with similar levels of outputs, could produce the current levels of outputs with fewer inputs.

Thursby used publication, citation data and number of Ph.D.s as outputs and the number of researchers and professors, graduate students, federal grants, library acquisitions and a dummy for private ownership as inputs. He found that the average level of efficiency in the United States is high (67% of economics departments appear as efficient). Moreno and Tadepalli (2002) focused on 42 academic units distributed across seven colleges in the United States. They used staff, financial resources and proxies for structures (for example, building space) as inputs and number of students, full-time equivalent enrolments and grant awards as outputs. In discussing their results, they showed that efficiency scores do not illustrate information about effectiveness: that is, they argued that a qualitative dimension is necessary for this purpose. Lastly, as their background is strongly managerial, they also suggested a use of efficiency analysis as a planning tool, not only for evaluating universities' activities. Koksal and Nalcaci (2006) derived efficiency scores for 14 university departments at the Middle East Technical University in Turkey, by using staff salaries, a composite indicator of previous research quality and number of students as inputs and number of publications, graduates and teaching indicators as outputs. Thus, in their contribution, the authors focused also on the teaching activity of the departments. They decided to use weights for teaching and research loads as suggested by university's administrators through a preliminary survey. The paper suggests that the models adopted for computing efficiency can serve to set new targets for inefficient departments. Gimenez and Martinez (2006) focused on cost efficiency of the departments at the Autonomous University of Barcelona. They computed cost-efficiency scores and separated cost excess into three categories: technical efficiency, fixed factors, and "scale effects" (due to the number of academics in the department). The units of analysis were 42 departments and again the focus is on both teaching and research activities. The inputs chosen were: expenditure on temporary hired teaching and research staff, operational expenditures, expenditures on tenure staff. To measure the outputs, they employed the following variables: results of a government evaluation process of research, teaching load (measured through credits granted) and students' opinions. The authors suggested that their analyses can be used to assess the differences between (i) the observed cost of the units and (ii) the cost they would achieve assuming long-term cost minimisation. Their result tells that size does not matter in determining efficiency, while higher proportion of non-tenured staff is associated with higher efficiency levels.

In another exercise presented by Kocher et al. (2006) the analysis focused on the productivity of research in economics across 21 OECD countries. The focus of the paper was on research and the outputs selected are top publications (and impact factors), while inputs are expenditure on research and development, number of universities and total population. This study was quite different, however, with respect to the relevant contributions for the present paper, as the unit of analysis is the country (neither the university nor the department). The results suggest that research productivity can be raised through an increase in the scale of operations. • inputs can be classified under three main groups: human resources, financial resources and structures (for example, facilities, buildings);

• outputs can be grouped into publications and grants.

Methodology and data

Methodology

The methodology is articulated accordingly to the two steps adopted to answer the research questions of the paper: on one hand the efficiency analysis (including efficiency scores of the departments, changes across time and time trends), on the other hand the effort to identify the determinants of efficiency.

Data Envelopment Analysis (DEA) (Charnes et al., 1978) was the methodology used to analyse efficiency. It deals with the efficiency problem of complex organisations that produce many outputs using several inputs at a time. This paper only briefly discusses some characteristics of DEA, referring the interested reader to Johnes (2004), Cooper et al., (2006) and Zhu (2003).

In a DEA model, the notion of efficiency is that of "technical efficiency", defined as the relative ability of each decision making unit (DMU) in producing outputs, where the term "relative" means that each organisation is compared with any other homogeneous unit. DEA could be represented by a linear programming technique where each DMU tries to maximise the efficiency ratio (output over inputs) choosing the best set of weights. There are two ways to deal with a DEA formulation: (i) input oriented (that is, the ability to minimise inputs when outputs are given) and (ii) output oriented (maximisation of outputs given a certain bundle of inputs). The latter approach is used in this paper, as the typical problem for a research unit is to maximise outputs (publications, grants) given its inputs (research staff, laboratories).

DEA mathematical formulation can deal with both constant returns to scale (CRS)

and variable returns to scale (VRS). The seminal work of Charnes et al. (1978) introduced a constant-return-to-scale (CRS) model where DMU dimension has no importance in defining efficiency performance (see equation 2). DMUs face the same efficiency frontier, independently of their relative size. One can cast doubts on the comparability of small and large units. Larger units exploit common inputs to produce different outputs, whereas smaller ones benefit from substantial advantages in organising activities. A further model, developed by Banker et al. (1984) introduces

F o r R e v i e w O n l y
in DEA modelling the dimension factor, by comparing units according to their dimension; that is, by considering that the unit's dimension can affect efficiency (VRS efficiency).

Smaller units are associated with increasing economies of scale and bigger units with decreasing economies of scale. In the middle, there is a "grey area" of constant returns to scale. So, it is also possible to compute the scale efficiency. This is defined as the ratio CRS/VRS efficiencies and interpreted as the ability of each institution to operate at its optimal scale dimension; in other words, in a VRS formulation, each DMU is compared only with those units having a comparable size.

It is also possible to measure changes in efficiency across time (the relevant period here is between 2004 and 2007). For measuring the determinants of the changes in efficiency scores, Malmquist index has been used (for more information about this index see Johnes (2004) andCoelli et al., (2006)). Considering a DMU, named P that has a change in production position from P t (in period t) to P t+1 (in period t+1), the change in the production frontier could have occurred in a non-neutral way; that is, it could be determined both by efficiency changes and shifts in the efficiency frontier. it has a positive effect and if it is less than 1 it has a negative effect. In other words, a shift in a production frontier that makes worse the relative position of the DMU (the other DMUs are performing relatively better than before) provides a second component less than 1. The index (Malmquist index) resulting from the combination of these two components will be equal to 1 if there is no net effect of changes in technical efficiency and frontier changes; it will be greater than 1 if the net effect is positive and less than 1 if it is negative. The second task in the methodology is to study some potential determinants of efficiency. To pursue this objective, some external factors (for example, location effects, discipline effects) were identified and then efficiency scores were compared across different groups of units. The comparison was made by means of the Kruskal-Wallis test (Kruskal & Wallis, 1952). This test allows the researcher to detect if there are statistical differences across the groups analysed (for example, when comparing the efficiency scores of units located in a metropolitan area with those located elsewhere, the test indicates if this location effect generates a statistical difference between the average scores of the two groups).

Data

The research used a unique dataset that consists of input and output variables about research and technology transfer activities in university departments located within the Lombardy Region. This dataset is based on information coming from QuESTIO system, a mapping of research and technological transfer organisations developed in the Lombardy region. The QuESTIO system, launched in 2004 by the regional government, was intended to be a transparency and incentive tool for the research and technological transfer market by reducing any information unevenness often binding cooperation among producers and users of knowledge. The system collects, updated and publishes financial, organisational and structural characteristics of research organisations and knowledge business intelligence services in Lombardy.

Since it aims at presenting the most detailed picture of regional community active in research and technological transfer, the dataset has some specific criteria that are pivotal for the analyses. First, the system is focused on every structure where such activities are carried out, even where that structure is a part of a wider organisation.

Second, these structures should have significant management autonomy so that their accountancy shall be determinable. The first criterion enables the regional administration to collect information on every single department, being acknowledged that the performance of departments belonging to the same university can differ significantly. As a consequence the university departments, as well as sections and institutes of the National Research Council or a research and development unit belonging to a company, provide data that refers solely to their specific resources and outputs. The second criterion assures that each structure included in the dataset is O4 Yearly number of publications: yearly publications in academic international journals. This is the traditional and widespread variable used to measure the output of research activities.

O5 Doctorates in cooperation with external bodies: sheds light on the capability of departments in attracting funds for doctoral students and to establish cooperation with institutions and companies. Since this variable could be interpreted such as an indicator of technology transfer activities, it was investigated if it could alter the efficiency scores and a DEA model has been estimated without considering it.

As discussed in the previous section, the choice of input and output variables is compatible with the most recent literature. The final sample contains 75 departments, for 4 years; overall, there are 300 observations, revealing considerable variation (Table 1).

<Table 1> about here

Results and discussion

This section illustrates and discusses the results obtained through the DEA analysis of the university departments in the Lombardy Region.

The results are presented in two main sections. The first is devoted to a description of the efficiency scores with reference to the year 2007 (the last year available, where the results are more stable, as previously discussed). It analyses the change in efficiency between the first year of the dataset ( 2004) and the last year (2007) as well as the presence of time trends in the efficiency scores. The second section examines statistical differences in efficiency scores due to external factors, such as localisation and discipline subject. Data about scale efficiency suggests that all the departments have almost reached their optimal scale of operations (given their present scale), as the scale efficiency score is very high (>0.93) and standard deviation is very low (about 0.1)

DEA efficiency scores, year 2007

The picture that emerges from this first analysis of efficiency scores is a sector where there is a number of very efficient units (both assuming constant or variable returns to scale) and a number of departments that experienced low levels of technical efficiency; thus, as the anecdotal evidence suggests, there is quite a strong differentiation among departments.

As the focus of the paper is the efficiency of academic departments in the Lombardy region, the results' robustness was checked by considering revenues from entities located in the Lombardy region as a separate output (REV_LOMB). The correlation among results obtained by the baseline model and this modified model is very high (>0.97), suggesting that both models report a similar picture.

Efficiency changes between 2004 and 2007: a Malmquist analysis

Following the methodology described above, the Malmquist index has been calculated to disentangle the efficiency changes due to "pure" efficiency improvement (or worsening) and technological improvement (or worsening). The descriptive statistics about the results are reported in Table 3.

<Table 3> about here

The picture that emerges highlights an interesting phenomenon. It looks like an improvement in productivity did not occur (because the Malmquist index is around 1)

but, looking at the decomposition of the index, the reality appears more complex: an efficiency change happened, it was very positive (the index is >1.6) and involved the majority of the departments (56 out of 70); conversely, there was a worsening of the technology frontier (the "frontier shift" indicator is <0.7). What is "technology" in a regional research system, when research performance is defined as the ability to attract financial resources and to publish in academic journals? Technology could be defined as the bundle of policies that should help in improving research efficiency, for example, information technologies, research grants, support for publishing research results or to attract financial resources from international grants or companies. This fact raises two reflections. The first is that maybe the universities (and, consequently, the research units) look not at the regional incentives but at the national ones (as the Ministry is still the main financer of the universities' activities) and at international ones (the research reputation strongly depends upon the ability to publish in international journals and to attract international grants). If this is the case, a question arises about the opportunity for a regional government to be involved in this sector. A second (possibly alternative) reflection is that the policies adopted for boosting research productivity were not adequately designed. To properly design a policy in this field the necessary first step is to understand what affect the academic productivity. In this context, Faria (2002) suggests that policies for fostering the overall productivity of the research sector are:

(i) competition in the labour market among researchers (for example, through career or salary incentives) and (ii) increasing research funds. Did the policies of Lombardy Region respect these elements? Are there other elements that affect research productivity? These are interesting starting point for further research in this field.

As a result, however, each academic unit in 2007, on average, is in the same position relative to the frontier as in 2004. The vast majority of the units actually improved their productivity in the period ("efficiency change" >1) but the "frontier shift" was <1: it means that the performance differential is much less dispersed in 2007 but at the same time it happens because the frontier is not improved: the best performers did not improve, or even worsened their efficiency. The result is a "research system" in the region that performs better on average but without having excellence.

DEA efficiency scores, year 2007: the external factors affecting efficiency

This section explores what affects a department's performance. Several factors were tested using the Kruskal-Wallis test ( The assumption was that in a metropolitan area the competition among research centres enforces departments' managerial efficiency. In Milan, there is an important concentration of public and private research institutions and the competition for resources is not limited to departments. The high concentration of research facilities may justify a greater effort by departments in fostering a more balanced use in input resources or may impulse an effort in control for managerial quality. Elsewhere, Agasisti (2009) demonstrated that the effect of competition on Italian universities' performance is positive; but, there, output was defined through teaching dimension's measures.

There is no evidence of difference in efficiency performance between the two groups (indeed, the observed value of the H statistic is nearly 0 and the p-value >0.9). This evidence suggests that there is no "concentration" effect or "location" effect on efficiency performance of university departments or, at least, in the relative rankings.

At the same time, this result may suggest i) the existence of no competitive pressure in a high concentration area as Milan.

Second external factor: can different scientific sectors affect university department efficiency?

As "output" is measured by using (i) publication counts and (ii) research grants (of different kinds), it is important to recall that there are some differences in the average number and quality of publication across different disciplines (for example, medicine versus engineering or biology), as well as in the attractiveness of research funds (for example, in engineering fields it is simpler to attract grants for applied research than in physics).

The idea that disciplines greatly vary in costs and productivity is not new and the suggestion to take this factor into account is important to obtain not-biased results (Sarrico & Dyson, 2004) subject to subject, with medical, some natural and technical sciences being more prolific in the number of articles than most of the social sciences and humanities" (Sarrico et al., 2009, p. 290).

No significant difference among university departments classified by scientific sectors was found: the observed value of the H statistic is >8.4.

4.2.3. Third external factor: is there a "university effect"?

An alternative explanation of efficiency can refer to the existence of a "university effect", that is if a university is more efficient than others, then its departments could be more productive than those belonging to other universities. This assumption finds its justification in the setting of strategies at university levels, not at subunits levels. In this respect, Bonaccorsi & Daraio (2007, pp. 22-23) pointed out that:

the notion of strategy is much more relevant and interesting if assumed at the overall university level. On the one hand, most strategic decisions are made at university level.… The fundamental reason for assuming the university level as the unit of analysis is that at the university level the problem of attribution of inputs (in particular, human resources, funding and physical capital) to specific units of outputs, can be kept under control.

Even if these authors recognised that a lower unit of analysis could provide useful information on the actual research process, they suggest that a "university effect" (due to different strategies and characteristics at macro-level) can be detected.

The tests suggest that such effect is not explaining efficiency differentials for the sample (observed value of the H statistic is 9.4, well below the critical value of 14.06 and the p-value is 0.22).

Fourth external factor: does the number of tenured academic staff influence efficiency?

The proportion of tenured research staff theoretically can affect the efficiency of a department: there is anecdotal evidence that tenured and non-tenured staff face quite different stimulus and incentives. Nevertheless, the direction of this influence is not clear a priori: tenured staff are certainly more expert in attracting research grants and in publishing academic papers, also thanks to reputational advantages; conversely, non-tenured staff can have an incentive to be more productive, for instance as a mean A complementary idea behind this question is that the different human capital of researchers (in the paper, indirectly measured via the tenured status) can affect the research productivity of the research groups and, so, of the department. In this respect, an interesting example is the paper by Rodgers & Neri (2007), who specifically attempted to correlate the human capital of researchers with the research productivity of Australian departments of economics. They found that human capital differential accounts for about 50% of the observed differences in research productivity. If human capital is (even indirectly) captured by the tenured status (for eample, researchers with more experience should have reached their tenured position), then a correlation between the proportion of tenured researchers and the department's results should be expected.

The analysis did not find evidence of a statistical influence of this factor on research efficiency (the value of the H statistic is 0.16 and the related p-value is >0.6).

Fifth external factor: can departments' "longevity" affect research efficiency?

Finally, the possible effects due to the longevity of departments was examined. As inputs were measured as numbers of academic staff and laboratories/facilities, it is reasonable that older departments accumulated higher stocks of inputs that could influence their outputs.

A key assumption here is that older departments can favour more cohesion among its members, facilitating the climate of collaboration and common motivation, which enhances research productivity.

Another interesting point of view about this potential phenomenon is that it is related to the "critical mass" dimension. Among the economic aspects of research, Hare & Wyatt (1992) underlined that the ability in producing research relies upon the human capital accumulated by the individual researchers; thus: combining this process with the notion that there are economies of scale in research may dispose university departments to feel they need to have at least some critical mass of researchers working in the same area in order to function properly as research units. (Hare & Wyatt, 1992, pp. 55-56)

F o r R e v i e w O n l y
Consequently, it could be reasonable to assume that older departments are more likely to have reached such "optimal" scale dimension than the more recently established ones.

The statistical analysis did not provide statistically robust results on this effect, indeed the value of the H statistic is 1.172, while the critical value is around 3.8, so that the p-value is 0.27).

To sum up, the empirical analysis suggests that none of these factors (location in metropolitan area, scientific sector, university effect, proportion of tenured academic staff, departments "longevity") is able to adequately explain efficiency differentials.

This result should be considered as positive, especially when thinking to the use of efficiency scores for policy purposes (for example, resources allocation). Indeed, a usual criticism of the efficiency analyses is that they provide results which are affected by factors that are not under the control of the units themselves: when this is the case, a procedure of resource allocation following efficiency results would be inequitable. At the same time, it could be possible that there are some elements that could influence efficiency and that are related to some "soft" characteristics of the academic units, typically (i) the incentive systems and (ii) the governance/management. For instance, the perception of the academic staff about the importance of publications for their career. Rodgers & Neri (2007) suggested that the "departmental environment" is important and they define it through several measures centred on the "human capital" of researchers who work in the department.

Unfortunately, the QuESTIO dataset does not collect such data: an extension of this dataset along this dimension could help to improve significantly the understanding of this pattern.

The governance/management of the academic departments has not been analysed in the study, as the only information available in this respect (the presence of a quality system or a management control system) is a very indirect measure of the governance process and the data recovery on this topic was quite imprecise. Another way to improve the QuESTIO dataset should be to better investigate this dimension, by including some qualitative descriptions about the way department managers organise and coordinate the activities and resources, formulate strategies and foresight studies and manage networking activities (Sala et al., forthcoming;Salerno et al., 2008).

Without knowing the characteristics of the "management style" of each department, it is impossible to derive policy implications from such dimension. At the same time, a possible way to better understand the potential incidence of such phenomenon is to conduct case studies about the successful academic departments (for example, those with higher efficiency scores or those that substantially improved scores in recent years).

Finally, a potential candidate to analyse the efficiency differentials is the interaction between research and teaching activities conducted by the departments. Typically, the literature did not pay enough attention to the interaction between teaching and research at department level, mainly because both activities were considered as inputs and outputs (see section 2). In this paper, the focus was exclusively on the research activities but it raises also questions about the effects that the (unmeasured) teaching activities play on the research result. This issue is very relevant both in theoretical terms (are there economies of scope, or tradeoffs, between the two activities?) and in empirical ones (which is the optimal allocation of energies between teaching and research?). This interesting issue was at the centre of several contributions since the seminal paper by Cohn et al. (1989) that, for the first time, addressed and modelled the multi-outputs nature of academic activities. Recently, the theme has been explored by Bonaccorsi & Daraio (2007) for a sample of European universities. In the present paper, this topic has not been explored because QuESTIO dataset does not record data about the teaching dimension of the departments' activities (as before, it could represent a substantial improvement of the dataset).

Finally, the issue of research quality still remains open. Indeed, the analysis is focused only on a quantitative dimension of research, while quality is important to adequately assess the performance. Typically the lack of data on this respect justifies the scarce attention to this problem. However, the QuESTIO dataset will, in future, include some qualitative indicators, like H and citation indexes.

Introduction and motivation

In recent years, higher education institutions have been studied increasingly. In the ''knowledge economy'' their importance for economic development, social equity, mobility, social cohesion and integration is widely acknowledged (Brennan & Teichler, 2008). Furthermore, given the difficult situation of public finances, considerations about resources allocation have been raised in many countries, calling for more evaluations and accountability. For these reasons the so-called 'New Public Management' movement, started at the beginning of the 1980s in the public sector in general, has also increasingly been affecting the universities. New Public Management (Hood, 1991(Hood, , 1995) ) represents the request to public organisations to become more managerial, in particular by drawing on practices and models from the private sector. This process has been driven both by universities themselves and by governments through reforming efforts. The results, especially in many countries within Europe, share some common characteristics: first, the introduction of incentives schemes based on financial models for allocating public funds; second, new or strengthened assessment procedures and evaluation agencies; third, more emphasis on universities' autonomy (OECD, 2003).

This work explores how the recent changes in the university system affected the efficiency of university research. In particular, the research sought to understand whether an overall increase in efficiency can be recognised or whether differences are present between universities given the previously-noted increased autonomy.

Normally, the majority of the reforming efforts have been directed towards education activities, whilst an increase in efficiency of university research has been assigned to incentive systems and evaluation processes. These are expected to have had an overall positive effect on the system but, at the same time, they have left the universities in charge of finding and managing the ways to obtain better results. Given the autonomy of the universities and the decentralisation of responsibilities, significant efficiency changes are thus expected between and within universities. Furthermore, the idea that differentials in research performance are key for competition among departments is well established in the literature (Curran, 2000). widely studied (Madden et al., 1997). Indicators such as number of degrees awarded, rate of success, years of study, have been widely used both to evaluate the universities and to introduce new policies. The opposite holds for universities' knowledge/technology transfer activities. Even if some argue for this to be a third mission for academia and call for "entrepreneurial universities" (Clark, 1998;Etzkowitz, 2003), others refuse this perspective and signal the risks of focusing on and expanding the role of these activities. Moreover, the efficiency analysis of these activities is hampered by the difficulties in finding the right indicators. There are many different knowledge/technology transfer activities (for example, spin-off, consultancies, research projects with firms, patents) and their evaluation can be based only partially on economic values given the not-for-profit nature of (most)

universities (Landoni & Verganti, 2006). Finally, the focus of the study is on the research activities as they are the core activities of universities; driving and informing The research considers departments located in the Lombardy region (Italy). A regional focus has been adopted to ensure similar contextual variables, such as policies and economic environment, for the universities included in the analysis.

Furthermore, Lombardy is one of the more advanced European regions both for (i) economic development and (ii) research activities and universities: more than 9.4 million habitants, GDP per capita >€34,000; 12 universities are located in the Region (out of about 70 in all the country) with more than 230,000 students. Finally, the regional level is increasingly recognised as a relevant level when analysing research and innovation activities. Many authors (Cooke et al., 1997;Porter, 1998;Lechner & Dowling, 1999) underline that it is at a local level that the majority of the connections between the different innovation actors take place. The localisation of knowledge and the role of proximity as a driver of economic development (for example, clusters)

have also highlighted the importance of regional and local government in fostering and strengthening the actors and their relationships.

To summarise, the focus of the paper is on efficiency of research activities in scientific departments in the Lombardy Region (Italy). The research questions are:

• did changes in the overall efficiency of departments' research occur between 2004 and 2007?

• Which are the main factors affecting efficiency?

The next section presents recent literature on the topic, then the methodology and the data used are introduced, finally results are presented and discussed.

Background

The international literature about universities' efficiency (through non-parametric techniques) has been developed both at institution-level (Athanassopoulos & Shale, 1997;McMillan & Datta, 1998;Ng & Li, 2000;Avkiran, 2001;Abbott & Doucouliagos, 2003;Flagg, et al., 2004;Warning, 2004) and department-level. The latter is more relevant for the context of the present study.

The contributions by Johnes & Johnes (1993, 1995) had a major influence on the methodological approach. Johnes & Johnes (1993) publication counts and grants as outputs; the same strategy was adopted again for deriving efficiency scores of economics department in a subsequent paper (Johnes & Johnes, 1995). The authors ran several Data Envelopment Analysis (DEA) models to test the robustness of results and claim for a possible use of results for policy purposes, especially for establishing an optimal allocation of resources across departments. These two contributions are the most important as they focus exclusively on the research dimension of the universities' work. Madden et al. (1997) background is strongly managerial, they also suggested a use of efficiency analysis as a planning tool, not only for evaluating universities' activities. Koksal and Nalcaci (2006) derived efficiency scores for 14 university departments at the Middle East Technical University in Turkey, by using staff salaries, a composite indicator of previous research quality and number of students as inputs and number of publications, graduates and teaching indicators as outputs. Thus, in their contribution, the authors focused also on the teaching activity of the departments. They decided to use weights for teaching and research loads as suggested by university's administrators through a preliminary survey. The paper suggests that the models adopted for computing efficiency can serve to set new targets for inefficient departments. Gimenez and Martinez (2006) focused on cost efficiency of the departments at the Autonomous University of Barcelona. They computed cost-efficiency scores and separated cost excess into three categories: technical efficiency, fixed factors, and "scale effects" (due to the number of academics in the department). The units of analysis were 42 departments and again the focus is on both teaching and research activities. The inputs chosen were: expenditure on temporary hired teaching and research staff, operational expenditures, expenditures on tenure staff. To measure the outputs, they employed the following variables: results of a government evaluation process of research, teaching load (measured through credits granted) and students' opinions. The authors suggested that their analyses can be used to assess the differences between (i) the observed cost of the units and (ii) the cost they would achieve assuming long-term cost minimisation. Their result tells that size does not matter in determining efficiency, while higher proportion of non-tenured staff is associated with higher efficiency levels.

In another exercise presented by Kocher et al. (2006) the analysis focused on the productivity of research in economics across 21 OECD countries. The focus of the paper was on research and the outputs selected are top publications (and impact factors), while inputs are expenditure on research and development, number of universities and total population. This study was quite different, however, with respect to the relevant contributions for the present paper, as the unit of analysis is the country (neither the university nor the department). The results suggest that research productivity can be raised through an increase in the scale of operations. • inputs can be classified under three main groups: human resources, financial resources and structures (for example, facilities, buildings);

• outputs can be grouped into publications and grants.

Methodology and data

Methodology

The methodology is articulated accordingly to the two steps adopted to answer the research questions of the paper: on one hand the efficiency analysis (including efficiency scores of the departments, changes across time and time trends), on the other hand the effort to identify the determinants of efficiency.

Data Envelopment Analysis (DEA) (Charnes et al., 1978) was the methodology used to analyse efficiency. It deals with the efficiency problem of complex organisations that produce many outputs using several inputs at a time. This paper only briefly discusses some characteristics of DEA, referring the interested reader to Johnes (2004), Cooper et al., (2006) and Zhu (2003).

In a DEA model, the notion of efficiency is that of "technical efficiency", defined as the relative ability of each decision making unit (DMU) in producing outputs, where the term "relative" means that each organisation is compared with any other homogeneous unit. DEA could be represented by a linear programming technique where each DMU tries to maximise the efficiency ratio (output over inputs) choosing the best set of weights. There are two ways to deal with a DEA formulation: (i) input oriented (that is, the ability to minimise inputs when outputs are given) and (ii) output oriented (maximisation of outputs given a certain bundle of inputs). The latter approach is used in this paper, as the typical problem for a research unit is to maximise outputs (publications, grants) given its inputs (research staff, laboratories).

DEA mathematical formulation can deal with both constant returns to scale (CRS)

and variable returns to scale (VRS). The seminal work of Charnes et al. (1978) introduced a constant-return-to-scale (CRS) model where DMU dimension has no importance in defining efficiency performance (see equation 2). DMUs face the same efficiency frontier, independently of their relative size. One can cast doubts on the comparability of small and large units. Larger units exploit common inputs to produce different outputs, whereas smaller ones benefit from substantial advantages in organising activities. A further model, developed by Banker et al. (1984) introduces in DEA modelling the dimension factor, by comparing units according to their dimension; that is, by considering that the unit's dimension can affect efficiency (VRS efficiency).

Smaller units are associated with increasing economies of scale and bigger units with decreasing economies of scale. In the middle, there is a "grey area" of constant returns to scale. So, it is also possible to compute the scale efficiency. This is defined as the ratio CRS/VRS efficiencies and interpreted as the ability of each institution to operate at its optimal scale dimension; in other words, in a VRS formulation, each DMU is compared only with those units having a comparable size.

It is also possible to measure changes in efficiency across time (the relevant period here is between 2004 and 2007). For measuring the determinants of the changes in efficiency scores, Malmquist index has been used (for more information about this index see Johnes (2004) andCoelli et al., (2006)). Considering a DMU, named P that has a change in production position from P t (in period t) to P t+1 (in period t+1), the change in the production frontier could have occurred in a non-neutral way; that is, it could be determined both by efficiency changes and shifts in the efficiency frontier. equal to 1 if no changes occurred, greater than 1 if the change was positive and less than 1 if it was negative. The second component measures the shift in the production frontier: if it is equal to 1 it has no effect on the efficiency score, if it is greater than 1 it has a positive effect and if it is less than 1 it has a negative effect. In other words, a shift in a production frontier that makes worse the relative position of the DMU (the other DMUs are performing relatively better than before) provides a second component less than 1. The index (Malmquist index) resulting from the combination of these two components will be equal to 1 if there is no net effect of changes in technical efficiency and frontier changes; it will be greater than 1 if the net effect is positive and less than 1 if it is negative. The second task in the methodology is to study some potential determinants of efficiency. To pursue this objective, some external factors (for example, location effects, discipline effects) were identified and then efficiency scores were compared across different groups of units. The comparison was made by means of the Kruskal-Wallis test (Kruskal & Wallis, 1952). This test allows the researcher to detect if there are statistical differences across the groups analysed (for example, when comparing the efficiency scores of units located in a metropolitan area with those located elsewhere, the test indicates if this location effect generates a statistical difference between the average scores of the two groups).

Data

The research used a unique dataset that consists of input and output variables about research and technology transfer activities in university departments located within the Lombardy Region. This dataset is based on information coming from QuESTIO system, a mapping of research and technological transfer organisations developed in the Lombardy region. The QuESTIO system, launched in 2004 by the regional government, was intended to be a transparency and incentive tool for the research and technological transfer market by reducing any information unevenness often binding cooperation among producers and users of knowledge. The system collects, updated and publishes financial, organisational and structural characteristics of research organisations and knowledge business intelligence services in Lombardy.

Since it aims at presenting the most detailed picture of regional community active in research and technological transfer, the dataset has some specific criteria that are pivotal for the analyses. First, the system is focused on every structure where such activities are carried out, even where that structure is a part of a wider organisation.

Second, these structures should have significant management autonomy so that their accountancy shall be determinable. The first criterion enables the regional administration to collect information on every single department, being acknowledged that the performance of departments belonging to the same university can differ significantly. As a consequence the university departments, as well as sections and institutes of the National Research Council or a research and development unit belonging to a company, provide data that refers solely to their specific resources and outputs. The second criterion assures that each structure included in the dataset is O4 Yearly number of publications: yearly publications in academic international journals. This is the traditional and widespread variable used to measure the output of research activities.

O5 Doctorates in cooperation with external bodies: sheds light on the capability of departments in attracting funds for doctoral students and to establish cooperation with institutions and companies. Since this variable could be interpreted such as an indicator of technology transfer activities, it was investigated if it could alter the efficiency scores and a DEA model has been estimated without considering it.

As discussed in the previous section, the choice of input and output variables is compatible with the most recent literature. The final sample contains 75 departments, for 4 years; overall, there are 300 observations, revealing considerable variation (Table 1).

<Table 1> about here

Results and discussion

This section illustrates and discusses the results obtained through the DEA analysis of the university departments in the Lombardy Region.

The results are presented in two main sections. The first is devoted to a description of the efficiency scores with reference to the year 2007 (the last year available, where the results are more stable, as previously discussed). It analyses the change in efficiency between the first year of the dataset ( 2004) and the last year (2007) as well as the presence of time trends in the efficiency scores. The second section examines statistical differences in efficiency scores due to external factors, such as localisation and discipline subject.

DEA efficiency scores, year 2007

Table 2 reports the descriptive statistics about the efficiency scores derived with reference to the year 2007 (the annex reports the specific scores for each department). The picture that emerges from this first analysis of efficiency scores is a sector where there is a number of very efficient units (both assuming constant or variable returns to scale) and a number of departments that experienced low levels of technical efficiency; thus, as the anecdotal evidence suggests, there is quite a strong differentiation among departments.

As the focus of the paper is the efficiency of academic departments in the Lombardy region, the results' robustness was checked by considering revenues from entities located in the Lombardy region as a separate output. The correlation among results

obtained by the baseline model and this modified model is very high (>0.97), suggesting that both models report a similar picture.

Efficiency changes between 2004 and 2007: a Malmquist analysis

Following the methodology described above, the Malmquist index has been calculated to disentangle the efficiency changes due to "pure" efficiency improvement (or worsening) and technological improvement (or worsening). The descriptive statistics about the results are reported in Table 3.

<Table 3> about here

The picture that emerges highlights an interesting phenomenon. It looks like an improvement in productivity did not occur (because the Malmquist index is around 1) but, looking at the decomposition of the index, the reality appears more complex: an efficiency change happened, it was very positive (the index is >1.6) and involved the majority of the departments (56 out of 70); conversely, there was a worsening of the technology frontier (the "frontier shift" indicator is <0.7). The assumption was that in a metropolitan area the competition among research centres enforces departments' managerial efficiency. In Milan, there is an important concentration of public and private research institutions and the competition for resources is not limited to departments. The high concentration of research facilities may justify a greater effort by departments in fostering a more balanced use in input resources or may impulse an effort in control for managerial quality. Elsewhere, Agasisti (2009) demonstrated that the effect of competition on Italian universities' performance is positive; but, there, output was defined through teaching dimension's measures.

There is no evidence of difference in efficiency performance between the two groups (indeed, the observed value of the H statistic is nearly 0 and the p-value >0.9). This evidence suggests that there is no "concentration" effect or "location" effect on efficiency performance of university departments or, at least, in the relative rankings.

At the same time, this result may suggest i) the existence of no competitive pressure in a high concentration area as Milan.

Second external factor: can different scientific sectors affect university department efficiency?

As "output" is measured by using (i) publication counts and (ii) research grants (of different kinds), it is important to recall that there are some differences in the average number and quality of publication across different disciplines (for example, medicine versus engineering or biology), as well as in the attractiveness of research funds (for example, in engineering fields it is simpler to attract grants for applied research than in physics).

The idea that disciplines greatly vary in costs and productivity is not new and the suggestion to take this factor into account is important to obtain not-biased results (Sarrico & Dyson, 2004). Recently, Sarrico et al. (2009) prolific in the number of articles than most of the social sciences and humanities" (Sarrico et al., 2009, p. 290).

No significant difference among university departments classified by scientific sectors was found: the observed value of the H statistic is >8.4.

4.2.3. Third external factor: is there a "university effect"?

An alternative explanation of efficiency can refer to the existence of a "university effect", that is if a university is more efficient than others, then its departments could be more productive than those belonging to other universities. This assumption finds its justification in the setting of strategies at university levels, not at subunits levels. In this respect, Bonaccorsi & Daraio (2007, pp. 22-23) pointed out that:

the notion of strategy is much more relevant and interesting if assumed at the overall university level. On the one hand, most strategic decisions are made at university level.… The fundamental reason for assuming the university level as the unit of analysis is that at the university level the problem of attribution of inputs (in particular, human resources, funding and physical capital) to specific units of outputs, can be kept under control.

Even if these authors recognised that a lower unit of analysis could provide useful information on the actual research process, they suggest that a "university effect" (due to different strategies and characteristics at macro-level) can be detected.

The tests suggest that such effect is not explaining efficiency differentials for the sample (observed value of the H statistic is 9.4, well below the critical value of 14.06 and the p-value is 0.22).

Fourth external factor: does the number of tenured academic staff influence efficiency?

The proportion of tenured research staff theoretically can affect the efficiency of a department: there is anecdotal evidence that tenured and non-tenured staff face quite different stimulus and incentives. Nevertheless, the direction of this influence is not clear a priori: tenured staff are certainly more expert in attracting research grants and in publishing academic papers, also thanks to reputational advantages; conversely, non-tenured staff can have an incentive to be more productive, for instance as a mean A complementary idea behind this question is that the different human capital of researchers (in the paper, indirectly measured via the tenured status) can affect the research productivity of the research groups and, so, of the department. In this respect, an interesting example is the paper by Rodgers & Neri (2007), who specifically attempted to correlate the human capital of researchers with the research productivity of Australian departments of economics. They found that human capital differential accounts for about 50% of the observed differences in research productivity. If human capital is (even indirectly) captured by the tenured status (for eample, researchers with more experience should have reached their tenured position), then a correlation between the proportion of tenured researchers and the department's results should be expected.

The analysis did not find evidence of a statistical influence of this factor on research efficiency (the value of the H statistic is 0.16 and the related p-value is >0.6).

Fifth external factor: can departments' "longevity" affect research efficiency?

Finally, the possible effects due to the longevity of departments was examined. As inputs were measured as numbers of academic staff and laboratories/facilities, it is reasonable that older departments accumulated higher stocks of inputs that could influence their outputs.

A key assumption here is that older departments can favour more cohesion among its members, facilitating the climate of collaboration and common motivation, which enhances research productivity.

Another interesting point of view about this potential phenomenon is that it is related to the "critical mass" dimension. Among the economic aspects of research, Hare & Wyatt (1992) underlined that the ability in producing research relies upon the human capital accumulated by the individual researchers; thus: combining this process with the notion that there are economies of scale in research may dispose university departments to feel they need to have at least some critical mass of researchers working in the same area in order to function properly as research units. (Hare & Wyatt, 1992, pp. 55-56) Consequently, it could be reasonable to assume that older departments are more likely to have reached such "optimal" scale dimension than the more recently established ones.

The statistical analysis did not provide statistically robust results on this effect, indeed the value of the H statistic is 1.172, while the critical value is around 3.8, so that the p-value is 0.27).

To sum up, the empirical analysis suggests that none of these factors (location in metropolitan area, scientific sector, university effect, proportion of tenured academic staff, departments "longevity") is able to adequately explain efficiency differentials.

This result should be considered as positive, especially when thinking to the use of efficiency scores for policy purposes (for example, resources allocation). Indeed, a usual criticism of the efficiency analyses is that they provide results which are affected by factors that are not under the control of the units themselves: when this is the case, a procedure of resource allocation following efficiency results would be inequitable. At the same time, it could be possible that there are some elements that could influence efficiency and that are related to some "soft" characteristics of the academic units, typically (i) the incentive systems and (ii) the governance/management. For instance, the perception of the academic staff about the importance of publications for their career. Rodgers & Neri (2007) suggested that the "departmental environment" is important and they define it through several measures centred on the "human capital" of researchers who work in the department.

Unfortunately, the QuESTIO dataset does not collect such data: an extension of this dataset along this dimension could help to improve significantly the understanding of this pattern.

The governance/management of the academic departments has not been analysed in the study, as the only information available in this respect (the presence of a quality system or a management control system) is a very indirect measure of the governance process and the data recovery on this topic was quite imprecise. Another way to improve the QuESTIO dataset should be to better investigate this dimension, by including some qualitative descriptions about the way department managers organise and coordinate the activities and resources, formulate strategies and foresight studies and manage networking activities (Sala et al., forthcoming;Salerno et al., 2008).

Without knowing the characteristics of the "management style" of each department, it is impossible to derive policy implications from such dimension. At the same time, a possible way to better understand the potential incidence of such phenomenon is to conduct case studies about the successful academic departments (for example, those with higher efficiency scores or those that substantially improved scores in recent years).

A potential candidate to analyse the efficiency differentials is the interaction between research and teaching activities conducted by the departments. Typically, the literature did not pay enough attention to the interaction between teaching and research at department level, mainly because both activities were considered as inputs and outputs (see section 2). In this paper, the focus was exclusively on the research activities but it raises also questions about the effects that the (unmeasured) teaching activities play on the research result. This issue is very relevant both in theoretical terms (are there economies of scope, or tradeoffs, between the two activities?) and in empirical ones (which is the optimal allocation of energies between teaching and research?). This interesting issue was at the centre of several contributions since the seminal paper by Cohn et al. (1989) that, for the first time, addressed and modelled the multi-outputs nature of academic activities. Recently, the theme has been explored by Bonaccorsi & Daraio (2007) for a sample of European universities. In the present paper, this topic has not been explored because QuESTIO dataset does not record data about the teaching dimension of the departments' activities (as before, it could represent a substantial improvement of the dataset).

Finally, the issue of research quality still remains open. Indeed, the analysis is focused only on a quantitative dimension of research, while quality is important to adequately assess the performance. Typically the lack of data on this respect justifies the scarce attention to this problem. However, the QuESTIO dataset will, in future, include some qualitative indicators, like H and citation indexes. Attention will be devoted to this topic in future research, also exploring trade-offs between research quantity and quality.

F

  an optimal allocation of resources across departments.

  They decided to use weights for teaching and research loads as suggested by

  the departments, changes across time and time trends), on the other hand the effort to identify the determinants of efficiency.

  are s outputs and m inputs. yrk is the output r produced by the kth unit. xik is the amount of input i used by the same kth unit. Lastly, sr is the output slack, and sj the input slack. ¶ Deleted: (although it has a different meaning to that usually applied in economics), simply adding the following constraint

  4) ¶ ¶ where n k scores have been ordering (ascending order), and Rj denotes the sum of ranks corresponding to DMUj. of units. The comparison was made by means of the Kruskal-Wallis test

  5) ¶ ¶ where N is the size of the full sample, k is the number of subsamples, ni is the size of the sub-sample i, and Ri is the sum of the rank indices of the sub-sample i. Hospital and medical research centres, 34 Public Research Organizations and 19 company R&D units.

  the areas (measured in square meters) devoted to laboratories belonging to or accessible by the department. This data provides a dimensional data related to the most distinctive and characterizing areas equipped for scientific research. I.2. High-Qualified Human Resources It refers to persons who perform research and technology transfer activities, regardless of their qualifications. They are mainly full professors, associated professors, permanent researchers, post-Doc scholarships and PhD students. It includes employees dedicated to activities in support of research, like: certification, tests and measurements, patenting and commercialisation, financial assistance, partners search, technologies monitoring, events organisation, editing of technicalscientific "spreading publications". I.3. Other personnel This variable considers administrative personnel that should ensure the efficient working of human resources devoted to research and technology transfer activities. OUTPUTS O.1. Revenues from financed activitiesAccomplished activities financed with regional or national calls for bids. This variable denotes the ability of the department to compete with other research organizations in getting public-funded research, thus measuring the amount of research that meet requirements of national and regional R&I strategies.O.2. Revenues from financed activities from abroadAccomplished activities financed shows the capacity of drawing on international funding, like the European projects, resulting by working with third countries reflects department down stream internationalization and it is a sign of the research activities that are acknowledged at international level.O.3. Revenue from ordersThis variable measures how far research activities carried out in a department are committed directly by companies or other institutions.O.4. Yearly number of publicationsThis variable reports yearly articles signed by department human resources on the international scientific press. This is the traditional and widespread variable used to measure the output of research activities.O.5. Doctorates in cooperation with external bodiesThis variable sheds light on the capability of departments in attracting investments for education finalized to research and to establish cooperation with institutions and companies. Since this variable could be interpreted such as an indicator of technology transfer activities, we investigated if it could alter the efficiency scores and we ran a DEA model without considering it. The results are the same for almost all the DMUs so we decided to keep it in the model.

(

  2004) and the last year (2007) and another one explore the presence of time trends in the efficiency scores. The second section refers to the study of statistical differences among efficiency scores due to "external factors", such as localization, discipline subject, etc.

  average, is in the 2007 in the same position relative to the frontier than in the 2004 (indeed, also the H index calculated above suggests small changes in the rankings in the period considered). However, what happened is something different:

  the research sector are: (i) competition in the labour market among researchers (e.g., through career or salary incentives) and (ii) increasing research funds. Did the policies of Lombardy Region respect these elements? Are there other elements that affect research productivity? These are interesting starting point for further research in this field.

  , describing the new funding model for Australian universities' research, pointed out several counterproductive mechanisms that can be created by linking academic performance and resources without taking into account factors affecting efficiency in a proper way. The authors concluded their paper stating that among the "two necessary pre-conditions for Australia to have a healthy, strong and world-class university research sector" one is that "the funding model used should allow for diversity and flexibility to properly reflect the complexity of academic world" (p.285).

  of units. The comparison was made by means of theKruskal-

  System; as a matter of fact they are 147 out of 364 structures mapped, which balance the presence of Knowledge Business Intelligence Services, which count for 151. The map is and medical research centres, 34 Public Research Organizations and 19 company R&D units.

  the areas (measured in square meters) devoted to laboratories belonging to or accessible by the department. This data provides a dimensional data related to the most distinctive and characterizing areas equipped for scientific research. I.2. High-Qualified Human Resources It refers to persons who perform research and technology transfer activities, regardless of their qualifications. They are mainly full professors, associated professors, permanent researchers, post-Doc scholarships and PhD students. It includes employees dedicated to activities in support of research, like: certification, tests and measurements, patenting and commercialisation, financial assistance, partners search, technologies monitoring, events organisation, editing of technicalscientific "spreading publications". I.3. Other personnel This variable considers administrative personnel that should ensure the efficient working of human resources devoted to research and technology transfer activities. OUTPUTS O.1. Revenues from financed activitiesAccomplished activities financed with regional or national calls for bids. This variable denotes the ability of the department to compete with other research organizations in getting public-funded research, thus measuring the amount of research that meet requirements of national and regional R&I strategies.O.2. Revenues from financed activities from abroadAccomplished activities financed shows the capacity of drawing on international funding, like the European projects, resulting by working with third countries reflects department down stream internationalization and it is a sign of the research activities that are acknowledged at international level.O.3. Revenue from ordersThis variable measures how far research activities carried out in a department are committed directly by companies or other institutions.O.4. Yearly number of publicationsThis variable reports yearly articles signed by department human resources on the international scientific press. This is the traditional and widespread variable used to measure the output of research activities.O.5. Doctorates in cooperation with external bodiesThis variable sheds light on the capability of departments in attracting investments for education finalized to research and to establish cooperation with institutions and companies. Since this variable could be interpreted such as an indicator of technology transfer activities, we investigated if it could alter the efficiency scores and we ran a DEA model without considering it. The results are the same for almost all the DMUs so we decided to keep it in the model.

( 2004 )

 2004 and the last year(2007) and another one explore the presence of time trends in the efficiency scores. The second section refers to the study of statistical differences among efficiency scores due to "external factors", such as localization, discipline subject, etc.

  average, is in the 2007 in the same position relative to the frontier than in the 2004 (indeed, also the H index calculated above suggests small changes in the rankings in the period considered). However, what happened is something different:

  the research sector are: (i) competition in the labour market among researchers (e.g., through career or salary incentives) and (ii) increasing research funds. Did the policies of Lombardy Region respect these elements? Are there other elements that affect research productivity? These are interesting starting point for further research in this field.

  Among others, Marinova & Newman (2008), describing the new funding model for Australian universities' research, pointed out several counterproductive mechanisms that can be created by linking academic performance and resources without taking into account factors affecting efficiency in a proper way. The authors concluded their paper stating that among the "two necessary pre-conditions for Australia to have a healthy, strong and world-class university research sector" one is that "the funding model used should allow for diversity and flexibility to properly reflect the complexity of academic world" (p.285).

  The research focused on universities research activities and not on education or knowledge/technology transfer activities. Universities' education activities constitute the first and original mission of the universities but, at the same time, they are normally carefully regulated by laws and their efficiency and quality are already

  both education activities and knowledge/technology transfer activities. This relationship especially holds for transfer activities because the value of the results transferred relies heavily on the underlying research on which they are based. This paper adopts the standard definition of 'research' provided by the Organisation for Economic Co-operation and Development (OECD): "Research and development is a term covering three activities: basic research, applied research, and experimental development" (OECD, 2002). However, only research activities that are normally carried out within universities are considered.Furthermore, the focus of the efficiency analysis is on university departments and not on the university as a whole. Universities are composed of different research departments and their publication outputs, co-authorship behaviours and thus efficiency usually differ significantly. An analysis at the level of the university can put forward mixed results, while an analysis at the level of single departments is more precise and coherent with the objectives of an efficiency analysis. For this reason, to obtain a sample of comparable departments the research focused on natural science departments. The choice of the department level for the analysis is coherent with the focus on research activities: normally education activities are managed by organisational entities inside the university that are transversal to more departments and many transfer activities are centralised in a single organisational unit (such as a liaison office or a knowledge/technology transfer office).

  focused on cost efficiency of the departments at the Autonomous University of Barcelona. They computed cost-efficiency scores and separated cost excess into three categories: technical efficiency, fixed factors, scale of size [WHAT IS THIS? DO YOU MEAN NUMBER OF ACADEMICS IN THE DEPARTMENT?]. The units of analysis were 42 departments and again the focus is on both teaching and research activities. The inputs chosen were: expenditure on temporary hired teaching and research staff, operational expenditures, expenditures on

  DEA mathematical formulation can deal with both constant returns to scale (CRS) and variable returns to scale (VRS). The seminal work ofCharnes et al. (1978) introduced a constant-return-to-scale (CRS) model where DMU dimension has no importance in defining efficiency performance (see equation 2). DMUs face the same efficiency frontier, independently of their relative size. One can cast doubts on the comparability of small and large units. Larger units exploit common inputs to produce different outputs, whereas smaller ones benefit from substantial advantages in organising activities. The BCC [WHAT DOES BCC STAND FOR?] model, developed byBanker et al. (1984) introduces in DEA modelling the dimension factor.

  Coelli et al., (2006)). Let us consider a DMU, named P that has a change in production position from P t (in period t) to P t+1 (in period t+1). The change in the production frontier could have occurred in a non-neutral way; that is, it could be determined both by efficiency changes and shifts in the efficiency frontier. Thus, for measuring the change in the efficiency score, we have to break down the score in these two different components. Both of the components have a specific mean: the resulting score for the DMU could have positive or negative effects due to the own efficiency change or due to the shift (movement) of the frontier. Thus the resulting efficiency index is composed of two parts. The first component of the equation is the ratio of technical efficiency in time period t+1 on the technical efficiency in the period t. Since it measures the change in technical efficiency, it is equal to 1 if no changes occurred, greater than 1 if the change was positive and less than 1 if it was negative. The second component measures the shift in the production frontier: if it is

  research and technology transfer activities in university departments located within the Lombardy Region. This dataset is based on information coming from QuESTIO system, a mapping of research and technological transfer organisations developed in the Lombardy region. The QuESTIO system, launched in 2004 by the regional government, was intended to be a transparency and incentive tool for the research and technological transfer market by reducing any information unevenness often binding cooperation among producers and users of knowledge. The system pursues such scopes [??] by collecting, updating and publishing financial, organisational and structural characteristics of research organisations and knowledge business intelligence services in Lombardy.

  National Research Council or a research and development unit belonging to a company, provide data that refers solely to their specific resources and outputs. The second criterion assures that each structure included in the dataset is responsible (to some extent) for its strategy and performances thus justifying application of the DEA technique. University departments are main actors in Lombardy innovation system; they are 147 out of 364 structures mapped, which balance the presence of Knowledge Business Intelligence Services, which count for 151. The map is completed with 13 Hospital and medical research centres, 34 Public Research Organisations and 19 company R&D units. The Lombardy innovation system maps 364 structures of which 147 are university departments, 151 are knowledge business intelligence services, 13 are hospital and medical research centres, 34 are public research organisations and 19 are company research and development units. In order to better point out the capillarity of these structures within the territory, the geographical analysis shows a concentration in the Province of Milan and a little less in Bergamo, Brescia, Pavia and Varese, which is explained by the presence of universities in those cities, all of them accredited with their single departments. [??] We built our dataset by considering those university departments that provided information for the whole period 2004-2007 (units for which there are not missing data), by selecting the following variables: Inputs I1 Laboratories: the areas (measured in square meters) devoted to laboratories belonging to or accessible by the department. This data provides a dimensional data related to the most distinctive and characterising areas equipped for scientific research. [Not clear what this means] I2 Highly-qualified human resources: persons who perform research and technology transfer activities, regardless of their qualifications. They are mainly full professors, associated professors, permanent researchers, post-Doc post-doctoral scholarships and PhD students.

  The results are presented in two main sections. The first is devoted to a description of the efficiency scores with reference to the year 2007 (the last year available, where the results are more stable, as previously discussed). In this section one sub-paragraph It analyses the change in efficiency between the first year of the dataset (2004) and the last year (2007) and another one explore as well as the presence of time trends in the efficiency scores. The second section refers to the study of examines statistical differences in efficiency scores due to external factors, such as localisation and discipline subject.

  correlation among results obtained by the baseline model and this modified model is very high (>0.97). WHICH MEANS WHAT? The VRS efficiency scores for each department comparing the two models (also in this case the reference year is 2007) is available, on request, from the authors. NEED TO SAY WHAT THE ESSENCE OF AND DROP THIS REFERENCE IN THE TEXT TO SEPARATE DATA BEING AVAILABLE.

  majority of the departments (56 out of 70); conversely, there was a relevant detriment of the technology environment [MEANING NOT CLEAR] (the "frontier shift" indicator is <0.7). It looks like the academic units improved their technical efficiency (that is, the ability to transform inputs into outputs) but, at the same time, the frontier worsened. In other words, what happened is that the "technological change" (in the Malmquist language) was negative. WHICH MEANS WHAT?

  Productivity changes between 2004 and 2007: an analysis of "rankings stability"

  have an incentive to be more productive, for instance as a mean to obtain a tenured position. The promotion system can affect productivity if researchers are satisfied with rules promoting this dimension.A complementary idea behind this question is that the different human capital of researchers (in our paper, indirectly measured via the tenured status) can affect the research productivity of the research groups and, so, of the department. In this respect, an interesting example is the paper byRodgers & Neri (2007), who specifically attempted to correlate the human capital of researchers with the research productivity of Australian departments of Economics. They have found that human capital differential account for about 50% of the observed differences in research productivity. If human capital is (even indirectly) captured by the tenured status (e.g., researchers with more experience should have reached their tenured position), then

  underlined that the ability in producing research relies upon the human capital accumulated by the individual researchers; thus: combining this process with the notion that there are economies of scale in research may dispose university departments to feel they need to have at least some critical mass of researchers working in the same area in order to function properly as research units.(Wyatt, 1992, pp. 55-56)

  IT WOULD PROBABLY BE BETTER TO INCLUDE MUCH OF THIS DISCUSSION IN THE PRESENTATION OF RESULTS TO (A) AVOID REPETITION AND (B) TO PROVIDE A CLEARER STORY AS TO WHAT HE RESULTS MEAN.As highlighted discussing the literature on the topic, the analyses presented in this work are innovative in terms of PLEASE REPHRASE TO AVOID "in terms of" data and methodology. Indeed, in this paper we have investigated the efficiency of university departments on science and technology and medicine in an Italian region (Lombardy). The aim of the paper was (i) to analyse the changes in productivity in recent years(from 2004 and 2007) and (ii) to detect factors that are potentially affecting efficiency.

  necessary first step is to understand what affect the academic productivity. In this context,Faria (2002) suggests that policies for fostering the overall productivity of the research sector are: (i) competition in the labour market among researchers (e.g., through career or salary incentives) and (ii) increasing research funds. Did the policies of Lombardy Region respect these elements? Are there other elements that affect research productivity? These are interesting starting point for further research in this field.

  , describing the new funding model for Australian universities' research, pointed out several counterproductive mechanisms that can be created by linking academic performance and resources without taking into account factors affecting efficiency in a proper way. The authors concluded their paper stating that among the "two necessary pre-conditions for Australia to have a healthy, strong and world-class university research sector" one is that "the funding model used should allow for diversity and flexibility to properly reflect the complexity of academic world" (p.285).

  relationship especially holds for transfer activities because the value of the results transferred relies heavily on the underlying research on which they are based. This paper adopts the standard definition of 'research' provided by the Organisation for Economic Co-operation and Development (OECD): "Research and development is a term covering three activities: basic research, applied research, and experimental development" (OECD, 2002). However, only research activities that are normally carried out within universities are considered.Furthermore, the focus of the efficiency analysis is on university departments and not on the university as a whole. Universities are composed of different research departments and their publication outputs, co-authorship behaviours and thus efficiency usually differ significantly. An analysis at the level of the university can put forward mixed results, while an analysis at the level of single departments is more precise and coherent with the objectives of an efficiency analysis. For this reason, to obtain a sample of comparable departments the research focused on natural science departments. The choice of the department level for the analysis is coherent with the focus on research activities: normally education activities are managed by organisational entities inside the university that are transversal to more departments and many transfer activities are centralised in a single organisational unit (such as a liaison office or a knowledge/technology transfer office).

  focused on cost efficiency of the departments at the Autonomous University of Barcelona. They computed cost-efficiency scores and separated cost excess into three categories: technical efficiency, fixed factors, and "scale effects" (due to the number of academics in the department). scale of size [WHAT IS THIS? DO YOU MEAN NUMBER OF ACADEMICS IN THE DEPARTMENT?]. The units of analysis were 42 departments and again the focus is on both teaching and research activities. The inputs chosen were: expenditure on temporary hired teaching and research staff, operational expenditures, expenditures on

  such indicators WHAT DOES THIS MEAN?. The present paper moves from these suggestions and it focuses on the efficiency of research activities in scientific departments in the Lombardy Region (Italy). [what struck me was inconsistency in the research about economies of scale!]

  a constant-return-to-scale (CRS) model where DMU dimension has no importance in defining efficiency performance (see equation 2). DMUs face the same efficiency frontier, independently of their relative size. One can cast doubts on the comparability of small and large units. Larger units exploit common inputs to produce different outputs, whereas smaller ones benefit from substantial advantages in organising activities. The BCC [WHAT DOES BCC STAND FOR?] A further model, developed by Banker et al. (1984) introduces in DEA modelling the dimension factor, by comparing units according to their dimension -that is, by considering that the unit's dimension can affect efficiency (VRS efficiency). [WHAT DOES THIS MEAN?]

  changes and shifts in the efficiency frontier. Thus, for measuring the change in the efficiency score, we have it is possible to break down the score in these two different components. Both of the components have a specific mean: the resulting score for the DMU could have positive or negative effects due to the own efficiency change or due to the shift (movement) of the frontier. Thus the resulting efficiency index is composed of two parts. The first component of the equation is the ratio of technical efficiency in time period t+1 on the technical efficiency in the period t. Since it measures the change in technical efficiency, it is equal to 1 if no changes occurred, greater than 1 if the change was positive and less than 1 if it was negative. The second component measures the shift in the production frontier: if it is equal to 1 it has no

  research and technology transfer activities in university departments located within the Lombardy Region. This dataset is based on information coming from QuESTIO system, a mapping of research and technological transfer organisations developed in the Lombardy region. The QuESTIO system, launched in 2004 by the regional government, was intended to be a transparency and incentive tool for the research and technological transfer market by reducing any information unevenness often binding cooperation among producers and users of knowledge. The system pursues such scopes [??] by collects, updated and publishes financial, organisational and structural characteristics of research organisations and knowledge business intelligence services in Lombardy.

  NEEDS THE RESULTS REPORTED TO BE DISCUSSED AND EXPLAINED BUILDING A PICTURE RATHER THAN LEAVING THE DISCUSSION UNTIL A FINAL SECTION AND THESE RESULTS STANDING IN A KIND OF VACUUM. SEE JOURNAL STYLE. THIS WILL ALSO HELP REDUCE REPETITION AND REDUCE THE WORD COUNT This section illustrates and discusses the results obtained through the DEA analysis of the university departments in the Lombardy Region.The results are presented in two main sections. The first is devoted to a description of the efficiency scores with reference to the year 2007 (the last year available, where the results are more stable, as previously discussed). In this section one sub-paragraph It analyses the change in efficiency between the first year of the dataset (2004) and the last year (2007) and another one explore as well as the presence of time trends in the efficiency scores. The second section refers to the study of examines statistical differences in efficiency scores due to external factors, such as localisation and discipline subject.DEA efficiency scores, year 2007This section explores the results of the efficiency analyses both looking at a single year (2007) and at differences between years. REPETITIVE DEA efficiency scores, year 2007

  in terms of PLEASE REPHRASE TO AVOID "in terms of" research efficiency. As the focus of our the paper is the efficiency of academic departments in the Lombardy region, we also checked the robustness of our the results' robustness was checked by considering revenues from entities located in the Lombardy region as a separate output (REV_LOMB). The correlation among results obtained by the baseline model and this modified model is very high (>0.97), suggesting that both models report a similar picture. WHICH MEANS WHAT? The VRS efficiency scores for each department comparing the two models (also in this case the reference year is 2007) is available, on request, from the authors. NEED TO SAY WHAT THE ESSENCE OF THIS EVIDENCE IS AND DROP THIS REFERENCE IN THE TEXT TO SEPARATE DATA BEING AVAILABLE.

  relevant detriment of the technology environment [MEANING NOT CLEAR] It looks like the academic units improved their technical efficiency (that is, the ability to transform inputs into outputs) but, at the same time, the frontier worsened. In other words, what happened is that the "technological change" (in the Malmquist language) was negative. WHICH MEANS WHAT?

  Milan, there is an important concentration of public and private research institutions and the competition for resources is not limited to departments. The high concentration of research facilities may justify a greater effort by departments in fostering a more balanced use in input resources or may impulse an effort in control

  BE BETTER TO INCLUDE MUCH OF THIS DISCUSSION IN THE PRESENTATION OF RESULTS TO (A) AVOID REPETITION AND (B) TO PROVIDE A CLEARER STORY AS TO WHAT HE RESULTS MEAN.

  the best performers that, on the contrary, had little margin for further improvements.

  results which are affected by factors which are not under the control of the units themselves: when this is the case, a procedure of resource allocation following efficiency results would be inequitable. Among others,Marinova & Newman (2008), describing the new funding model for Australian universities' research, pointed out several counterproductive mechanisms that can be created by linking academic performance and resources without taking into account factors affecting efficiency in a proper way. The authors concluded their paper stating that among the "two necessary pre-conditions for Australia to have a healthy, strong and world-class university research sector" one is that "the funding model used should allow for diversity and flexibility to properly reflect the complexity of academic world" (p.285).

  The research focused on universities research activities and not on education or knowledge/technology transfer activities. Universities' education activities constitute the first and original mission of the universities but, at the same time, they are normally carefully regulated by laws and their efficiency and quality are already

  both education activities and knowledge/technology transfer activities. This relationship especially holds for transfer activities because the value of the results transferred relies heavily on the underlying research on which they are based. This paper adopts the standard definition of 'research' provided by the Organisation for Economic Co-operation and Development (OECD): "Research and development is a term covering three activities: basic research, applied research, and experimental development" (OECD, 2002). However, only research activities that are normally carried out within universities are considered.Furthermore, the focus of the efficiency analysis is on university departments and not on the university as a whole. Universities are composed of different research departments and their publication outputs, co-authorship behaviours and thus efficiency usually differ significantly. An analysis at the level of the university can put forward mixed results, while an analysis at the level of single departments is more precise and coherent with the objectives of an efficiency analysis. For this reason, to obtain a sample of comparable departments the research focused on natural science departments. The choice of the department level for the analysis is coherent with the focus on research activities: normally education activities are managed by organisational entities inside the university that are transversal to more departments and many transfer activities are centralised in a single organisational unit (such as a liaison office or a knowledge/technology transfer office).

  Thus, for measuring the change in the efficiency score, it is possible to break down the score in these two different components. Both of the components have a specific mean: the resulting score for the DMU could have positive or negative effects due to the own efficiency change or due to the shift (movement) of the frontier. Thus the resulting efficiency index is composed of two parts. The first component of the equation is the ratio of technical efficiency in time period t+1 on the technical efficiency in the period t. Since it measures the change in technical efficiency, it is equal to 1 if no changes occurred, greater than 1 if the change was positive and less than 1 if it was negative. The second component measures the shift in the production frontier: if it is equal to 1 it has no effect on the efficiency score, if it is greater than 1

  some extent) for its strategy and performances thus justifying application of the DEA technique. The Lombardy innovation system maps 364 structures of which 147 are university departments, 151 are knowledge business intelligence services, 13 are hospital and medical research centres, 34 are public research organisations and 19 are company research and development units. The geographical analysis shows a concentration in the Province of Milan and a little less in Bergamo, Brescia, Pavia and Varese, which is explained by the presence of universities in those cities. The dataset was built by considering those university departments that provided information for the whole period 2004-2007 (units for which there are not missing data), by selecting the following variables: Inputs I1 Laboratories: the areas (measured in square meters) devoted to laboratories belonging to or accessible by the department. I2 Highly-qualified human resources: persons who perform research and technology transfer activities, regardless of their qualifications. They are mainly full professors, associated professors, permanent researchers, post-doctoral scholarships and PhD students. It includes employees dedicated to activities in support of research, such as, certification, tests and measurements, patenting and commercialisation, financial assistance, partners search, technologies monitoring, events organisation, editing of technical-scientific "spreading publications". I3 Other personnel: administrative personnel that should ensure the efficient working of human resources devoted to research and technology transfer activities.Outputs O1 Revenues from financed activities: activities financed with regional or national calls for bids. This variable denotes the ability of the department to compete with other research organisations in getting public-funded research, thus measuring the amount of research that meet requirements of national and regional research and innovation strategies.O2 Revenues from financed activities from abroad: internationally-funded activities that show the capacity to draw on international funding, such as European projects, orders: measures how far research activities carried out in a department are funded by companies or other institutions.

  tenured position. The promotion system can affect productivity if researchers are satisfied with rules promoting this dimension.

  The research focused on universities research activities and not on education or knowledge/technology transfer activities. Universities' education activities constitute the first and original mission of the universities but, at the same time, they are normally carefully regulated by laws and their efficiency and quality are already

  both education activities and knowledge/technology transfer activities. This relationship especially holds for transfer activities because the value of the results transferred relies heavily on the underlying research on which they are based. This paper adopts the standard definition of 'research' provided by the Organisation for Economic Co-operation and Development (OECD): "Research and development is a term covering three activities: basic research, applied research, and experimental development" (OECD, 2002). However, only research activities that are normally carried out within universities are considered.Furthermore, the focus of the efficiency analysis is on university departments and not on the university as a whole. Universities are composed of different research departments and their publication outputs, co-authorship behaviours and thus efficiency usually differ significantly. An analysis at the level of the university can put forward mixed results, while an analysis at the level of single departments is more precise and coherent with the objectives of an efficiency analysis. For this reason, to obtain a sample of comparable departments the research focused on natural science departments. The choice of the department level for the analysis is coherent with the focus on research activities: normally education activities are managed by organisational entities inside the university that are transversal to more departments and many transfer activities are centralised in a single organisational unit (such as a liaison office or a knowledge/technology transfer office).

  Thus, for measuring the change in the efficiency score, it is possible to break down the score in these two different components. Both of the components have a specific mean: the resulting score for the DMU could have positive or negative effects due to the own efficiency change or due to the shift (movement) of the frontier. Thus the resulting efficiency index is composed of two parts. The first component of the equation is the ratio of technical efficiency in time period t+1 on the technical efficiency in the period t. Since it measures the change in technical efficiency, it is

  some extent) for its strategy and performances thus justifying application of the DEA technique. The Lombardy innovation system maps 364 structures of which 147 are university departments, 151 are knowledge business intelligence services, 13 are hospital and medical research centres, 34 are public research organisations and 19 are company research and development units. The geographical analysis shows a concentration in the Province of Milan and a little less in Bergamo, Brescia, Pavia and Varese, which is explained by the presence of universities in those cities. The dataset was built by considering those university departments that provided information for the whole period 2004-2007 (units for which there are not missing data), by selecting the following variables: Inputs I1 Laboratories: the areas (measured in square meters) devoted to laboratories belonging to or accessible by the department. I2 Highly-qualified human resources: persons who perform research and technology transfer activities, regardless of their qualifications. They are mainly full professors, associated professors, permanent researchers, post-doctoral scholarships and PhD students. It includes employees dedicated to activities in support of research, such as, certification, tests and measurements, patenting and commercialisation, financial assistance, partners search, technologies monitoring, events organisation, editing of technical-scientific "spreading publications". I3 Other personnel: administrative personnel that should ensure the efficient working of human resources devoted to research and technology transfer activities.Outputs O1 Revenues from financed activities: activities financed with regional or national calls for bids. This variable denotes the ability of the department to compete with other research organisations in getting public-funded research, thus measuring the amount of research that meet requirements of national and regional research and innovation strategies.O2 Revenues from financed activities from abroad: internationally-funded activities that show the capacity to draw on international funding, such as European projects, orders: measures how far research activities carried out in a department are funded by companies or other institutions.

  of university departments in the Lombardy region is high (CRS efficiency = 0.75, VRS efficiency = 0.78). What is worth pointing out is a strong heterogeneity across departments: indeed, the standard deviation is >0.2. Data about scale efficiency suggests that all the departments have almost reached their optimal scale of operations (given their present scale), as the scale efficiency score is very high (>0.93) and standard deviation is very low (about 0.1)

  external factor: are departments located in Milan metropolitan area (regional capital) more efficient than other departments?

  demonstrated a strong role of "subject mix" (defined as the disciplinary composition of institutions) in explaining the efficiency differentials among Portuguese higher education institutions. The authors acknowledged that "(…) research production also varies significantly from subject to subject, with medical, some natural and technical sciences being more

  tenured position. The promotion system can affect productivity if researchers are satisfied with rules promoting this dimension.

  pointed out that "(…) the notion much more relevant and interesting if assumed at the overall university level. On the one hand, most strategic decisions are made at university level.(…)

	of strategy is
	F o r
	R
	e v
	i e
	w
	O n l	Deleted: , but further tests suggest that these effects are really small, and it
	y
	17

Table 4 .

 4 Factors affecting efficiency(Kruskal-Wallis analysis)

	Effects of Metropolitan area (Milan)
	H (observed value)	0.007
	H (critic value)	3.481
	p-value	0.934
	Effects of subject mix
	H (observed value)	8.451
	H (critic value)	9.488
	p-value	0.076
	University effects
	H (observed value)	9.426
	H (critic value)	14.067
	p-value	0.224
	Effects of tenured staff proportion
	H (observed value)	0.163
	H (critic value)	3.841
	p-value	0.687
	Effects of departments' age
	H (observed value)	1.172
	H (critic value)	3.841
	p-value	0.279

  studied the efficiency of economics departments in 29

	Australian universities. Inputs were teaching and research staff, outputs were
	graduates and publications (for teaching and research respectively). As in other
	studies listed below, here the focus was both on teaching and research performance,
	even if the analysis were conducted at subunit level (that is at departmental level
	instead of institutional level). The specific objective of the Madden et al. (1997) study
	was to verify whether policies promoted by the governments impacted on the
	productivity; they confirmed that there was actually an increase in productive
	efficiency.
	Thursby (2000) analyzed the efficiency of the Economics Research Departments in
	the US. He defined efficiency as follows: "(…) a department is deemed technically
	efficient if, when compared to departments with similar level of inputs, it could
	produce greater research outputs without increasing its inputs usage, or equivalently,
	it is one which, compared to departments with similar levels of outputs, could
	produce the current levels of outputs with fewer inputs" (p. 400). He used publication,
	citation data and number of Ph.D.s as outputs, and faculty size, full professors,
	graduate students, federal grants, library acquisitions and a dummy for private
	ownership as inputs. He found that the average level of efficiency in US is incredibly
	high (67% of economics departments appear as efficient). Seen from a European

  Following the methodology described above, we derived the Malmquist index, which allows us to disentangle the efficiency changes due to "pure" efficiency improvement (or worsening) and technological improvement (or worsening). The descriptive statistics about the results are reported in Table3, while the detailed results are available on request from the authors.

	<Table 3> around here
	F o r
	R
	e v
	i e
	w
	O n l
	y

The correlation among results obtained by the baseline model and this modified model is very high (>0.97). the VRS efficiency scores for each department comparing the two models (also in this case the reference year is 2007) is available, on request, from the authors. 14 4.

1.2. Efficiency changes between 2004 and 2007 -a Malmquist analysis

Table 1 .

 1 Inputs/outputs descriptive statistics

	for a sample

adequately assess the performance. Typically the lack of data on this respect justifies the scarce attention to this problem. However, the QuESTIO dataset will include in the next years some qualitative indicators, like H and citation indexes. We will devote attention to this topic in future research, also exploring trade-offs between research quantity and quality.

The analyses and the data have allowed an interesting discussion in terms of policy implication but we hope that further research will follow both including new data (e.g. referring to teaching activities and research quality) and considering other geographical regions. Daraio, C., Simar L., (2005), Introducing environmental variables in nonparametric frontier models: a probabilistic approach, Journal of Productivity Analysis, 24(1), 93-121. Etzkowitz, H., (2003), Research groups as 'quasi-firms': the invention of the entrepreneurial university, Research Policy, 32(1), 109-121. Faria, J.R., Scientific, business and political networks in academia, Research in Economics, 56(1), 187-198. Flagg, A.T., Allen, D.O., Field, K., Thurlow, T.W., (2004), "Measuring the Efficiency of British Universities: a Multi-Period Data Envelopment Analysis", Education Economics, 12(3), 231-249. Gimenez, V.M., Martinez, J.L., (2006), Cost efficiency in the university: a departmental evaluation model, Economics of Education Review, 25, 543-553.

Hare, P., Wyatt, G., (1992)

, Economics of academic research and its implications for higher education, Oxford Review of Economic Policy, 8(2), 48-66.

Table 2 .

 2 Efficiency scores (DEA on 2007 data) -descriptive statistics

	CRS	VRS	Scale
	Efficiency	Efficiency	Efficiency

Table 3 .

 3 Malmquist index 2004Malmquist index -2007 -descriptive statistics -descriptive statistics

	Department	Malqmuist index	Efficiency change	Frontier shift
	Mean	1.060	1.666	0.652
	St. dev.	0.492	0.882	0.141
	Min	0.072	0.529	0.104
	Max	2.534	4.600	1.031
	# improving units	35	56	2
	Source: authors' elaborations on QuESTIO data, Lombardy Region and IRER.

Table 4 .

 4 Factors affecting efficiency(Kruskal-Wallis analysis)

	Effects of Metropolitan area (Milan)
	H (observed value)	0.007
	H (critic value)	3.481
	p-value	0.934
	Effects of subject mix
	H (observed value)	8.451
	H (critic value)	9.488
	p-value	0.076
	University effects
	H (observed value)	9.426
	H (critic value)	14.067
	p-value	0.224
	Effects of tenured staff proportion
	H (observed value)	0.163
	H (critic value)	3.841
	p-value	0.687
	Effects of departments' age
	H (observed value)	1.172
	H (critic value)	3.841
	p-value	0.279

1 Evaluating

the efficiency of research in academic departments: an empirical analysis in an Italian Region Please address the comments in red font below. Leave the comment in situ. All changes in blue font, please. Need to depersonalise throughout. I've done a lot of this already but it needs completing. The tense keeps changing and for reference to published work I have put into past tense (journal style) as much as I have noticed but needs checking. Many style things have been changed below (e.g. United Kingdom English grammar and spelling) but authors need to check re journal style. Please rephrase sentences to avoid "in terms of" throughout, I have done a lot of this but the remaining ones need to be done. Finally, and most importantly, the article is too long with repetition and needs to be trimmed to no more than 7000 words in total.

  studied the efficiency of economics departments in 29 Australian universities. Inputs were teaching and research staff, outputs were graduates and publications (for teaching and research respectively). As in other studies listed below, here the focus was both on teaching and research performance,

	F o r
	R
	e v
	i e
	w
	O n l
	y
	high (67%
	of economics departments appear as efficient). Seen from a European perspective, this
	result is not so unexpected [WHY?].
	Moreno & Tadepalli (2002) focused on 42 academic units distributed across 7
	colleges in the United States. They used staff, financial resources and proxies for

even if the analysis were conducted at sub-unit level (that is at departmental level instead of institutional level). The specific objective of the

Madden et al. (1997)

study was to verify whether policies promoted by governments impacted on productivity; they confirmed that there was actually an increase in productive efficiency.

Thursby (2000, p. 400)

analysed the efficiency of the Economics Research Departments in the United States. He defined efficiency as follows: a department is deemed technically efficient if, when compared to departments with similar level of inputs, it could produce greater research outputs without increasing its inputs usage, or equivalently, it is one which, compared to departments with similar levels of outputs, could produce the current levels of outputs with fewer inputs.

Thursby used publication, citation data and number of Ph.D.s as outputs and faculty size [WHAT DO YOU MEAN BY FACULTY SIZE?], full professors, graduate students, federal grants, library acquisitions and a dummy for private ownership as inputs. He found that the average level of efficiency in the United States is structures (for example, building space) as inputs and number of students, FTE enrolments and grant awards as outputs. In discussing their results, they showed that efficiency scores do not illustrate information about effectiveness: that is, they argued

  Doctorates in cooperation with external bodies: sheds light on the capability of departments in attracting investments for education finalised to research [not clear what this means] and to establish cooperation with institutions and companies. Since this variable could be interpreted such as an indicator of technology transfer activities, we investigated if it could alter the efficiency scores and we ran a DEA model without considering it. The results are the same for almost all the DMUs so we decided to keep it in the model. [NOT SURE WHETHER YOU ARE SAYING IT WOULDN'TAs discussed in the previous section the choice of input and output variables is coherent compatible? with the most recent literature. The final sample contains 75 departments, for 4 years; overall, there are 300 observations, revealing considerable variation (Table1).

	financial assistance, partners search, technologies monitoring, events organisation,
	editing of technical-scientific "spreading publications".
	I3 Other personnel: administrative personnel that should ensure the efficient working
	of human resources devoted to research and technology transfer activities.
	Outputs <Table 1> about here
	O1 Revenues from financed activities: activities financed with regional or national
	calls for bids. This variable denotes the ability of the department to compete with Results THIS RESULTS SECTION NEEDS THE RESULTS REPORTED TO BE
	other research organisations in getting public-funded research, thus measuring the DISCUSSED AND EXPLAINED BUILDING A PICTURE RATHER THAN
	amount of research that meet requirements of national and regional R&I [in full] LEAVING THE DISCUSSION UNTIL A FINAL SECTION AND THESE
	strategies.	F o r F o r
		R R
		e v e v
		i e i e
		w w
		O n l O n l
		y y
	MATTER IF IT WASN'T IN THE MODEL?]

It includes employees dedicated to activities in support of research, such as, certification, tests and measurements, patenting and commercialisation, O2 Revenues from financed activities from abroad: accomplished activities financed [do you mean internationally funded activities?] that show the capacity to draw on international funding, such as European projects, resulting by working with third countries organisations [not clear what this is referring to?]. It reflects department down stream internationalisation [not clear what this means] and it is a sign of the research activities that are acknowledged at international level. O3 Revenue from orders: measures how far research activities carried out in a department are committed directly by companies [not clear what this means] or other institutions. O4 Yearly number of publications: yearly articles signed by department human resources on the international scientific press [not clear what this means, do you mean publication in international journals?]. This is the traditional and widespread variable used to measure the output of research activities. O5 RESULTS STANDING IN A KIND OF VACUUM. SEE JOURNAL STYLE. THIS WILL ALSO HELP REDUCE REPETITION AND REDUCE THE WORD COUNT This section illustrates the results obtained through the DEA analysis of the university departments in the Lombardy Region.

  Table2reports the descriptive statistics about the efficiency scores derived with reference to the year 2007 (the annex reports the specific scores for each department). methodological section, we calculated CRS efficiency and VRS efficiency, as well as the "scale" efficiency computed as CRS/VRS scores (it is important to recall here that the model is output-oriented). AVOID TOO MUCH REPETITION AS ARTICLE

	TOO LONG.
	<Table 2> about here
	The research efficiency of university departments in the Lombardy region looks quite
	[?? IS IT OR ISN'T IT HIGH, THIS IS TOO VAGUE] high (0.75 in terms of
	PLEASE REPHRASE TO AVOID "in terms of" CRS efficiency, 0.78 in terms of
	F o r
	R
	e v
	i e
	w
	O n l
	y

NOT JOURNAL STYLE. INSTEAD SAY WHAT THE DATA SHOWS AND REFER TO 'EVIDENCE' (VIZ TABLE) IN PARENTHESES. As explained in the

PLEASE REPHRASE TO AVOID "in terms of" VRS efficiency); actually, the high average efficiency is coherent compatible? with previous results derived across the world in the higher educations sector

Table 1 .

 1 Inputs/outputs descriptive statistics

	F o r
	R
	e v
	i e
	w
	O n l
	y

research will follow both including new data (e.g. referring to teaching activities and research quality) and considering other geographical regions.

Table 3 .

 3 Malmquist index 2004Malmquist index -2007 -descriptive statistics -descriptive statistics

	Department

Table 4 .

 4 Factors affecting efficiency(Kruskal-Wallis analysis)

	Effects of Metropolitan area (Milan)
	H (observed value)

1 Evaluating

the efficiency of research in academic departments: an empirical analysis in an Italian Region Please address the comments in red font below. Leave the comment in situ. All changes in blue font, please. Need to depersonalise throughout. I've done a lot of this already but it needs completing. The tense keeps changing and for reference to published work I have put into past tense (journal style) as much as I have noticed but needs checking. Many style things have been changed below (e.g. United Kingdom English grammar and spelling) but authors need to check re journal style. Please rephrase sentences to avoid "in terms of" throughout, I have done a lot of this but the remaining ones need to be done. Finally, and most importantly, the article is too long with repetition and needs to be trimmed to no more than 7000 words in total.

  studied the efficiency of economics departments in 29 Australian universities. Inputs were teaching and research staff, outputs were

	F o r
	R
	e v
	i e
	w
	O n l
	Thursby used publication, citation data and number of Ph.D.s as outputs and the y
	number of researchers and professors faculty size [WHAT DO YOU MEAN BY
	FACULTY SIZE?], full professors, graduate students, federal grants, library
	acquisitions and a dummy for private ownership as inputs. He found that the average
	level of efficiency in the United States is high (67% of economics departments appear
	as efficient). Seen from a European perspective, this result is not so unexpected
	[WHY?].

graduates and publications (for teaching and research respectively). As in other studies listed below, here the focus was both on teaching and research performance, even if the analysis were conducted at sub-unit level (that is at departmental level instead of institutional level). The specific objective of the

Madden et al. (1997)

study was to verify whether policies promoted by governments impacted on productivity; they confirmed that there was actually an increase in productive efficiency.

Thursby (2000, p. 400)

analysed the efficiency of the Economics Research Departments in the United States. He defined efficiency as follows: a department is deemed technically efficient if, when compared to departments with similar level of inputs, it could produce greater research outputs without increasing its inputs usage, or equivalently, it is one which, compared to departments with similar levels of outputs, could produce the current levels of outputs with fewer inputs.

Moreno & Tadepalli (2002)

focused on 42 academic units distributed across 7 colleges in the United States. They used staff, financial resources and proxies for structures (for example, building space) as inputs and number of students, FTE

  university size, number of departments within the university, percentage of private funding, number of curricula activated. They concluded that economies of scale and scope are not relevant factors in explaining (in)efficiency. However, a strong role is played by the external factors: clearly, such result affects the policy considerations.

	[NOT SURE IT IS CLEAR. MIGHT NEED TO ELABORATE IF KEEPING THIS
	CLAUSE]
	e
	w
	O n l
	y

  I2 Highly-qualified human resources: persons who perform research and technology transfer activities, regardless of their qualifications. They are mainly full professors, associated professors, permanent researchers, post-Doc post-doctoral scholarships and PhD students. It includes employees dedicated to activities in support of research, institutions and companies. Since this variable could be interpreted such as an indicator of technology transfer activities, we it was investigated if it could alter the efficiency scores and we ran a DEA model has been estimated without considering it.The results are the same for almost all the DMUs so we decided to keep it in the

	such as, certification, tests and measurements, patenting and commercialisation, model. [NOT SURE WHETHER YOU ARE SAYING IT WOULDN'T MATTER IF
	financial assistance, partners search, technologies monitoring, events organisation, IT WASN'T IN THE MODEL?]
	editing of technical-scientific "spreading publications".
	I3 Other personnel: administrative personnel that should ensure the efficient working As discussed in the previous section the choice of input and output variables is
	of human resources devoted to research and technology transfer activities. coherent compatible with the most recent literature. The final sample contains 75
	application of the DEA technique. departments, for 4 years; overall, there are 300 observations, revealing considerable
	University departments are main actors in Lombardy innovation system; they are 147 out of 364 structures mapped, which balance the presence of Knowledge Business Intelligence Services, which count for 151. The map is completed with 13 Hospital F o r Outputs O1 Revenues from financed activities: activities financed with regional or national calls for bids. This variable denotes the ability of the department to compete with variation (Table 1). F o r <Table 1> about here
	and medical research centres, 34 Public Research Organisations and 19 company R&D units. R e v other research organisations in getting public-funded research, thus measuring the R amount of research that meet requirements of national and regional Research and Innovation R&I [in full] strategies. O2 Revenues from financed activities from abroad: internationally funded activities e v
	i e w accomplished activities financed [do you mean internationally funded activities?] that i e show the capacity to draw on international funding, such as European projects, resulting by working with third countries organisations [not clear what this is w
	We built our The dataset was built by considering those university departments that O n l y referring to?]. It reflects department down stream internationalisation [not clear what this means] and it is a sign of the research activities that are acknowledged at international level. O n l O3 Revenue from orders: measures how far research activities carried out in a department are funded by companies are committed directly by companies [not clear y
	provided information for the whole period 2004-2007 (units for which there are not what this means] or other institutions.
	missing data), by selecting the following variables: O4 Yearly number of publications: yearly publications in academic international
	journals articles signed by department human resources on the international scientific
	Inputs press [not clear what this means, do you mean publication in international journals?].
	I1 Laboratories: the areas (measured in square meters) devoted to laboratories This is the traditional and widespread variable used to measure the output of research
	belonging to or accessible by the department. This data provides a dimensional data activities.
	related to the most distinctive and characterising areas equipped for scientific
	research. [Not clear what this means]

The Lombardy innovation system maps 364 structures of which 147 are university departments, 151 are knowledge business intelligence services, 13 are hospital and medical research centres, 34 are public research organisations and 19 are company research and development units. In order to better point out the capillarity of these structures within the territory, the geographical analysis shows a concentration in the Province of Milan and a little less in Bergamo, Brescia, Pavia and Varese, which is explained by the presence of universities in those cities, all of them accredited with their single departments. [??] O5 Doctorates in cooperation with external bodies: sheds light on the capability of departments in attracting funds for doctoral students investments for education finalised to research [not clear what this means] and to establish cooperation with

  Table2reports the descriptive statistics about the efficiency scores derived with reference to the year 2007 (the annex reports the specific scores for each department).

	NOT JOURNAL STYLE. INSTEAD SAY WHAT THE DATA SHOWS AND
	REFER TO 'EVIDENCE' (VIZ TABLE) IN PARENTHESES. As explained in the
	methodological section, we calculated CRS efficiency and VRS efficiency, as well as
	the "scale" efficiency computed as CRS/VRS scores (it is important to recall here that
	the model is output-oriented). AVOID TOO MUCH REPETITION AS ARTICLE
	TOO LONG.

<Table 2> about here

The research efficiency of university departments in the Lombardy region looks quite [?? IS IT OR ISN'T IT HIGH, THIS IS TOO VAGUE] is high (CRS efficiency = 0.75, VRS efficiency = 0.78 (0.75 in terms of PLEASE REPHRASE TO AVOID "in terms of" CRS efficiency, 0.78 in terms of PLEASE REPHRASE TO AVOID "in terms of" VRS efficiency); actually, the high average efficiency is coherent

Table 1 .

 1 Inputs/outputs descriptive statistics

	F o r
	R
	e v
	i e
	w
	O n l
	y

research will follow both including new data (e.g. referring to teaching activities and research quality) and considering other geographical regions.

Table 3 .

 3 Malmquist index 2004Malmquist index -2007 -descriptive statistics -descriptive statistics

	Department

Table 4 .

 4 Factors affecting efficiency(Kruskal-Wallis analysis)

	Effects of Metropolitan area (Milan)
	H (observed value)

1 Evaluating

the efficiency of research in academic departments: an empirical analysis in an Italian Region AUTHOR Abstract.

  Table2reports the descriptive statistics about the efficiency scores derived with reference to the year 2007 (the annex reports the specific scores for each department).

	<Table 2> about here
	The research efficiency of university departments in the Lombardy region is high
	(CRS efficiency = 0.75, VRS efficiency = 0.78). What is worth pointing out is a
	strong heterogeneity across departments: indeed, the standard deviation is >0.2.
	F o r
	R
	e v
	i e
	w
	O n l
	y

  Table 4).

	<Table 4> about here
	4.2.1 First external factor: are departments located in Milan metropolitan area
	(regional capital) more efficient than other departments?
	F o r
	R
	e v
	i e
	w
	O n l
	y

Table 1 .

 1 Inputs/outputs descriptive statistics

	Attention will be devoted to this

Table 3 .

 3 Malmquist index 2004Malmquist index -2007 -descriptive statistics -descriptive statistics

	Department

Table 4 .

 4 Factors affecting efficiency(Kruskal-Wallis analysis)

	Effects of Metropolitan area (Milan)
	H (observed value)

1 Evaluating

the efficiency of research in academic departments: an empirical analysis in an Italian Region Tommaso

  Agasisti 1,α , Antonio Dal Bianco β , Paolo Landoni α , Alessandro Sala β , Mario Salerno α,Φ α Politecnico di Milano, Department of Management, Economics and Industrial Engineering β IRER -Istituto Regionale di Ricerca della Lombardia Φ Politecnico di Torino, Department DISPEA -Production Systems and Business EconomicsAbstract.This paper investigates the efficiency of university departments on science, technology and medicine in an Italian Region (Lombardy). The aim of the paper is twofold: (i) to analyse the changes in productivity in recent years (from 2004 to 2007); and (ii) to detect factors that are potentially affecting efficiency. The research benefited from a new and unique dataset (called QuESTIO) developed by the Lombardy Regional Government. Using facilities and academic staff as inputs and research grants and publications as outputs, the research activity of academic departments was modelled. The methodological approach for computing efficiency scores is Data Envelopment Analysis. Malmquist indexes have been used to measure changes in productivity, while Kruskal-Wallis tests were employed to study the potential determinants of efficiency. The main results were that in the period considered, the academic departments improved their efficiency but, at the same time, the efficiency frontier worsened. Moreover, external and measurable factors (such as, scientific sector, proportion of tenured staff, location) have a limited impact in explaining efficiency differentials. Policy implications of the results are discussed.

	F o r
	R
	e v
		i e
	Keywords.	w
	University research, efficiency, research policies, Data Envelopment Analysis, Malmquist index, higher education policy. O n l
		y

  studied the efficiency of economics departments in 29 Australian universities. Inputs were teaching and research staff, outputs were graduates and publications (for teaching and research respectively). As in other studies listed below, here the focus was both on teaching and research performance, even if the analysis were conducted at sub-unit level (that is at departmental level instead of institutional level). The specific objective of theMadden et al. (1997) study was to verify whether policies promoted by governments impacted on productivity; they confirmed that there was actually an increase in productive efficiency.

	F o r
	R
	Thursby (2000, p. 400) analysed the efficiency of the Economics Research Departments in the United States. He defined efficiency as follows: e v
	a department is deemed technically efficient if, when compared to departments i e with similar level of inputs, it could produce greater research outputs without increasing its inputs usage, or equivalently, it is one which, compared to w
	departments with similar levels of outputs, could produce the current levels of outputs with fewer inputs. O n l Thursby used publication, citation data and number of Ph.D.s as outputs and the number of researchers and professors, graduate students, federal grants, library y
	acquisitions and a dummy for private ownership as inputs. He found that the average
	level of efficiency in the United States is high (67% of economics departments appear
	as efficient).
	Moreno and Tadepalli (2002) focused on 42 academic units distributed across seven
	colleges in the United States. They used staff, financial resources and proxies for
	structures (for example, building space) as inputs and number of students, full-time

equivalent enrolments and grant awards as outputs. In discussing their results, they showed that efficiency scores do not illustrate information about effectiveness: that is, they argued that a qualitative dimension is necessary for this purpose. Lastly, as their

Table 1 .

 1 Inputs/outputs descriptive statistics

	o r
	R
	e v
	i e
	w
	O n l
	y

Table 2 .

 2 Efficiency scores (DEA on 2007 data) -descriptive statistics

	CRS Efficiency	VRS Efficiency Scale Efficiency

Table 3 .

 3 Malmquist index 2004Malmquist index -2007 -descriptive statistics -descriptive statistics Source: authors' elaborations on QuESTIO data, Lombardy Region and IRER.

	Department

Table 4 .

 4 Factors affecting efficiency(Kruskal-Wallis analysis)

	Effects of Metropolitan area (Milan)
	H (observed value)

Higher Education Quarterly

Corresponding author: Politecnico di Milano, Department of Management, Economics and Industrial Engineering, P.za Leonardo da Vinci, 32,

Milano (Italy). T. +39.02.2399.3963, e. tommaso.agasisti@polimi.it.

25

Johnes, G., Johnes, J., (1993), Measuring the Research Performance of UK Economics Departments: an Application of Data Envelopment Analysis, Oxford Economic Papers, 45, pp.332-347. Johnes, G., Johnes, J., (1995), "Research Funding and Performance in UK University Departments of Economics: A Frontier Analysis", Economics of Education Review, 14(3), 301-314. Johnes, J., ( 2004), Efficiency Measurement, in Johnes, G., Johnes, J., (eds), International Handbook on the Economics of Education, Edward Elgar Publishing Ltd, Cheltenham. Johnes, J., (2006), Data Envelopment Analysis and its Application to the Measurement of Efficiency in Higher Education, Economics of Education Review, n.

The Evolution of Industrial Districts and Regional Networks: The Case of the Biotechnology Region Munich/Martinsried, Journal of Management and Governance, 3, 309-338. Levin, H.M., (1991), Raising productivity in higher education, Journal of Higher Education, 62(3), 242-262. Madden, G., Savage, S., Kemp, S., (1997), Measuring public sector efficiency: a study of economics departments at Australian universities, Education Economics, 5(2), 153-167. Marinova, D., Newman, P., (2008), The changing research funding regime in Australia and academic productivity, Mathematics and Computers in Simulation 78(2-3), 283-291. McMillan, M.L., Datta, D., (1998), The Relative Efficiencies of Canadian Universities: A DEA Perspective, Canadian Public Policy, 24(4), 485-511. Moreno, A.A., Tadepali, R., (2002), Assessing Academic Department Efficiency at a Public University, Managerial and Decision Economics, 23, 385-397. Ng, Y.C., Li, S.K., (2000), Measuring the Research Performance of Chinese Higher Education Institutions: An Application of Data Envelopment Analysis, Education Economics, 8(2), 139-156. Organisation for Economic Co-operation and Development (OECD), (2002), Proposed Standard Practise for Surveys of Research and Experimental Development -Frascati Manual, sixth edition. Organisation for Economic Co-operation and Development (OECD), (2003), Education policy analysis. Paris: OECD. Pastor, J.T., Ruiz, J.L, Sirvent, I., (2002), A Statistical Test for Nested Radial Models, Operations Research, vol. 50, n. 4, pp.728-735. Pesenti, R., Ukovich, W., (1996), Evaluating Academic Activities Using DEA, Internal Report, DEEI, Università di Trieste. Porter M.E. (1998) Clusters and the New Economics of Competition, Harvard Business School Press.

An interesting issue is to analyse the related literature about the Italian universities' research. To the best of the authors' knowledge, there are only two specific contributions that specifically analyse the efficiency of universities in producing research outputs (Pesenti & Ukovich, 1996;Rizzi, 1999) and just one attempting to focus on teaching production (Agasisti & Dal Bianco, 2009).

The two contributions at departmental level (Pesenti & Ukovich, 1996;Rizzi, 1999) used a similar set of variables and focused both on teaching and research outputs. The main scope for these contributions was to provide information and suggestions for allocating resources within a university.

Contributions about the efficiency of Italian higher education have been conducted at institution-level (thus, not considering separately the structures devoted to teaching and research). This is because the strategy and the mission of universities and the balance between teaching and research are defined at institutional level (Bonaccorsi & Daraio, 2007, p. 22).

Among these contributions at an institutional level Agasisti & Dal Bianco (2006) studied the efficiency of 58 Italian public universities by using, as inputs, number of staff, financial resources and students and, as outputs, graduates and research grants.

They suggested the use of efficiency analysis for policy purposes, especially to evaluate the differences in performance between efficient and inefficient units. Later, Agasisti & Salerno (2007) analysed the cost efficiency of another sample of Italian universities by adding some qualitative measures for inputs and outputs. They concluded that a redistribution of inputs across universities could lead to an overall improvement of efficiency in the Italian higher education sector. Bonaccorsi et al. (2006) used a robust non-parametric technique (developed by Daraio & Simar, 2005) to take in account external conditions affecting the productivity of universities. The inputs and outputs adopted are quite similar to previous studies (that is, staff numbers, expenses and spaces as inputs and publications and graduates as outputs) but they also included some "external" factors: university size, number of departments within the university, percentage of private funding, number of curricula activated. They concluded that economies of scale and scope are not relevant factors in explaining (in)efficiency.

To sum up, the literature seems to suggest that, to analyse research efficiency of universities, that is, the efficiency in the production of research outputs:

An interesting issue is to analyse the related literature about the Italian universities' research. To the best of the authors' knowledge, there are only two specific contributions that specifically analyse the efficiency of universities in producing research outputs (Pesenti & Ukovich, 1996;Rizzi, 1999) and just one attempting to focus on teaching production (Agasisti & Dal Bianco, 2009).

The two contributions at departmental level (Pesenti & Ukovich, 1996;Rizzi, 1999) used a similar set of variables and focused both on teaching and research outputs. The main scope for these contributions was to provide information and suggestions for allocating resources within a university.

Contributions about the efficiency of Italian higher education have been conducted at institution-level (thus, not considering separately the structures devoted to teaching and research). This is because the strategy and the mission of universities and the balance between teaching and research are defined at institutional level (Bonaccorsi & Daraio, 2007, p. 22).

Among these contributions at an institutional level Agasisti & Dal Bianco (2006) studied the efficiency of 58 Italian public universities by using, as inputs, number of staff, financial resources and students and, as outputs, graduates and research grants.

They suggested the use of efficiency analysis for policy purposes, especially to evaluate the differences in performance between efficient and inefficient units. Later, Agasisti & Salerno (2007) analysed the cost efficiency of another sample of Italian universities by adding some qualitative measures for inputs and outputs. They concluded that a redistribution of inputs across universities could lead to an overall improvement of efficiency in the Italian higher education sector. Bonaccorsi et al. (2006) used a robust non-parametric technique (developed by Daraio & Simar, 2005) to take in account external conditions affecting the productivity of universities. The inputs and outputs adopted are quite similar to previous studies (that is, staff numbers, expenses and spaces as inputs and publications and graduates as outputs) but they also included some "external" factors: university size, number of departments within the university, percentage of private funding, number of curricula activated. They concluded that economies of scale and scope are not relevant factors in explaining (in)efficiency.

To sum up, the literature seems to suggest that, to analyse research efficiency of universities, that is, the efficiency in the production of research outputs:

What is "technology" in a regional research system, when research performance is defined as the ability to attract financial resources and to publish in academic journals? Technology could be defined as the bundle of policies that should help in improving research efficiency, for example, information technologies, research grants, support for publishing research results or to attract financial resources from international grants or companies. This fact raises two reflections. The first is that maybe the universities (and, consequently, the research units) look not at the regional incentives but at the national ones (as the Ministry is still the main financer of the universities' activities) and at international ones (the research reputation strongly depends upon the ability to publish in international journals and to attract international grants). If this is the case, a question arises about the opportunity for a regional government to be involved in this sector. A second (possibly alternative) reflection is that the policies adopted for boosting research productivity were not adequately designed. To properly design a policy in this field the necessary first step is to understand what affect the academic productivity. In this context, Faria (2002) suggests that policies for fostering the overall productivity of the research sector are:

(i) competition in the labour market among researchers (for example, through career or salary incentives) and (ii) increasing research funds. Did the policies of Lombardy Region respect these elements? Are there other elements that affect research productivity? These are interesting starting point for further research in this field.

As a result, however, each academic unit in 2007, on average, is in the same position relative to the frontier as in 2004. The vast majority of the units actually improved their productivity in the period ("efficiency change" >1) but the "frontier shift" was <1: it means that the performance differential is much less dispersed in 2007 but at the same time it happens because the frontier is not improved: the best performers did not improve, or even worsened their efficiency. The result is a "research system" in the region that performs better on average but without having excellence.

DEA efficiency scores, year 2007: the external factors affecting efficiency

This section explores what affects a department's performance. Several factors were tested using the Kruskal-Wallis test (Table 4). <Table 4> about here