Dirk Holste
email: dirk.holste@ait.ac.at

Thomas Scherngell
email: thomas.scherngell@ait.ac.at

Ivana Roche
email: ivana.roche@inist.fr

Marianne Hörlesberger
email: marianne.hoerlesberger@ait.ac.at

Dominique Besagni

Marie-Elisabeth Züger

Pascal Cuxac
email: pascal.cuxac@inist.fr

Edgar L Schiebel
email: edgar.schiebel@ait.ac.at

Claire François

Maria-Elisabeth Züger

Clair Francoise
email: claire.francoise@inist.fr

Capturing frontier research in grant proposals and initial analysis of the comparison between model vs. peer review

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Peers are central to the research community and scientific system at all stages of the publication or professional cycle [START_REF] Wouters | Citation cycles and peer review cycles[END_REF]. Peer-review serves as an essential mechanism for resource allocation and quality control [START_REF] Bornmann | Scientific peer review[END_REF], with input in both ex-ante reviews (e.g., at the funding stage, deciding what proposed research deserves to be funded through national or regional agencies or funding institutions) and in ex-post reviews (e.g., at the dissemination stage, deciding what conducted research deserves to be published). Peers are given and take on the challenge to determine the "best-fitting" scientific research in accord with a journal"s status or funding agency"s strategy. Journals and grant schemes" objectives are often not aligned and consequently there are no standardized and easily transferable practices between ex-ante and ex-post reviews. While peer-review is widely accepted and actively supported by the scientific community, they are not free form criticism on a number of long-standing issues (Roy, 1985;[START_REF] Chubin | Peerless science: peer review and U.S. science policy[END_REF][START_REF] Chubin | Grants Peer Review in Theory and Practice[END_REF]. Because of its central role the monitoring of peer-review processes is important to continuously reveal to what extent goals are actually accomplished through review processes and decisions [START_REF] Hojat | Impartial judgment by the "gatekeepers" of science: Fallibility & accountability in the peer review process[END_REF], Sweizer & Collen 1994, Bornmann & Daniel 2008[START_REF] Marsh | Improving the peer-review process for grant applications -Reliability, validity, bias, and generalizability[END_REF]. Given a large number of project evaluations and needed resources for peer-review, the systematic use of quantitative methods to either support or evaluate the decision-making is witnessing increasing attention to cope with science output and efficiency (e.g., van den Besselaar & Leydesdorff 2009;[START_REF] Van Noorden | A profusion of measures[END_REF]. Advantages of bibliometric respectively scientometricbased methods are manifest in their objectivity, definition with of high precision, reliability, efficiency, and automation, while disadvantages are in limits of interpretation, applicability, confounding factors, and predictive validity [START_REF] Adam | Citation analysis: The counting house[END_REF][START_REF] Van Noorden | A profusion of measures[END_REF].

While a number of studies have targeted peer-review in project funding decisions (see, e.g., [START_REF] Bornmann | A Meta-evaluation of Scientific Research Proposals: Different Ways of Comparing Rejected to Awarded Applications[END_REF][START_REF] Juznic | Scientometric indicators: peer-review, bibliometric methods and conflict of interest[END_REF], this paper"s primary interest is the extent to which research proposal comply with attributes of frontier research and the influence of these attributes on the selection of awarded grants. To this end, it looks at the scientometric evaluation of research project proposals.

From a grant point of view it focuses on proposals submitted to the European Research Council (ERC) in the scientific domains "Physics & Engineering" (PE) and "Life Sciences" (LS). There are ten (nine) main research fields in PE (LS) and about 170 (100) subfields. The third domain "Social Sciences & Humanities (SSH)" is excluded as it is expected to differ in terms of publishing, citation behaviour, and other features from those observed in PE and LS (e.g., national/regional orientation, less publications in form of articles, different theoretical "development rate", number of authors, non-scholarly publications), which make it less assessable for approaches developed for natural and the life sciences [START_REF] Nederhof | Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A review[END_REF][START_REF] Juznic | Scientometric indicators: peer-review, bibliometric methods and conflict of interest[END_REF]. Researchers can compete for Starting or Advanced Grants [START_REF] Antonoyianakis | The European Research Council Takes Flight[END_REF] to support pioneering, far-reaching research that shows high risk/high impact potential, breaks established disciplinary boundaries, or to explore new productive lines of scientific enquiry, methodology or techniques.

From a methodological point of view it specifically takes into account data sought informative about aspects of "frontier research" (EC, 2005) extractable from research proposals that have been either successful or non-successful in obtaining a grant. The ERC"s understanding of frontier research and its strategic importance for the funding scheme provide suitable conditions for combining lexical and other types of analysis combined with statistical modelling in the field of scientometric evaluation [START_REF] Yoon | Development and application of a keyword-based knowledge map for effective research[END_REF].

The remainder of the paper is structured as follows. After briefly introducing the peer-review system of the ERC and criteria for frontier research, we first introduce indicators for quantifying individual aspects of frontier research using text-analytic methods and the tools of citation scientometrics (e.g., [START_REF] Roche | Identification and characterisation of technological topics in the field of Molecular Biology[END_REF][START_REF] Schiebel | An advanced diffusion model to identify emergent research issues: the case of optoelectronic devices[END_REF]). Subsequently we introduce a statistical discrete choice model to estimate the decision probability for a proposal to be accepted on the basis of measured attributes of frontier research. Finally we study in a proof-of-concept approach the influence of attributes on the decision probability and conduct an initial analysis of the comparison between indicator-based scientometric evaluation and empirical review process. A discussion and outlook of the approach ends the paper.

Overall concept

Key attributes of frontier research under the grant scheme of the ERC

The first European research funding body targets research at the highest level of excellence in any scientific discipline. It supports investigator-driven projects aiming at broadening the scientific and technological knowledge without regard for established disciplinary boundaries through open and direct competition (ERC, 2010). Because the clear distinction between basic and applied science has become less distinct and emerging areas of science and technology often embrace elements of both, a High-Level Expert Group set up by the ERC used the term "frontier research" to denote research that reaches beyond horizons of existing knowledge by being intrinsically risky endeavours without regard for established disciplinary boundaries. Table 1 cites key attributes of frontier research according to the Group"s report (EC, 2005), on the basis of which indicators will be defined.

The selection of research proposals is based on peer-review. The ERC has established a process which is to identify scientific excellence of frontier research as the sole evaluation criterion for funding decisions (ERC, 2010). Internationally renowned scientists and scholars constitute two sets of review panels (for the Starting respectively Advanced Grants), each of which subdivided into 25 individual panels that cover the entire range of disciplines and fall into in the domains PE, LS, and SH. Each panel is composed of about a dozen members and headed by a chair. If further expertise is required, external reviewers may be consulted by providing assessments on a proposal-by-proposal basis.

Grant Application

• Starting (SG) or advanced grant (AG)

• Main panel (= major discipline)

Criteria-1

Scientometric and content-based indicators

Step-2 evaluation correspondence

Modelling approach for capturing aspects of frontier research

Figure 1 shows our approach for capturing aspects of frontier research in grant proposals to exploit key aspects frontier research (cf. Table 1). It conceptualizes and defines for each attribute a corresponding indicator. While each indicator has a clear description, numerical procedure for quantification and offers an interpretation, the faithful representation of frontier research requires the combination of all indicators to accommodate levels of pair-wise and higher-order balances and counterbalances between different aspects. In the present approach the indicators are joined in form of a statistical model.

In computing indicators, an initial step identifies from a corpus of grant application relevant scientometric respectively bibliometric data (e.g., research field, publications, citations, patents) and content data (e.g. text-strings, keywords) bearing relevance to frontier research, extracts and subjects them to data mining. In a subsequent step, indicators are computed and subjected to the model for comparison between peer-review panel and statistical model outcomes. Finally, model analysis and validation refine in a last step the performance of the model"s usability. The following sections describe indicators, the model and proof of concept demonstration in more detail.

Indicators for frontier research

The indicators TIMELINESS and RISK are derived from citation analysis. TIMELINESS is based on the simple assumption that the time (publication year) distribution of cited proposal references is a proxy for the novelty of research. The more recent references are (e.g. on average), the more likely the work is at the cutting edge of science. TIMELINESS computes for every reference of a proposal the relative difference in years between its publication date and the year of the application. References of the proposal are considered appropriate because not only do they relate directly to the project but constitute the knowledge base on which the proposal is built. Citation studies of obsolescence are conducted from two different perspectives. A diachronous obsolescence examining citations received by a scientific publication within a time period; and a synchronous obsolescence examining references cited in a select set of documents at one point of time [START_REF] Gupta | Analysis of distribution of the age of citations in theoretical population genetics[END_REF]. Here we use the latter perspective where the set of documents is a single proposal. After identifying references and extracting publication dates in actual texts, TIMELINESS can be calculated from the set of values. In this study we use the statistical median to characterize the distribution by a single number. The indicator RISK is used as a proxy for the "individual risk" of the principal investigator in carrying out the proposed research. In addition to references of a proposal (defining set I) it makes use of external reference information (with respect to the proposal). It compiles references of research papers (set II) previously published by the applicant. Comparing the applicant"s references in set I vs. set II, the overlap between sets is used to compare the proposed research direction with respect to past research. The underlying assumption is that the lower the overlap between sets I and II is, the more it is indicative of a change from previous pursued research (and hence the more independent of previous research directions resp. risk-affine). Computationally, the indicator is defined by the correlation coefficient. The indicators SIMILARITY and INTERDISCIPLINARITY are derived from lexical analysis. The indicator SIMILARITY is based on lexical analysis and used as a proxy to infer the "novelty" of a proposal. The core concept has two main steps. 1) The construction of a "publication landscape" via a cluster map derived from scientific and technological information (including research publications, excluding proposals). The landscape is created at two time steps to characterize its level of change over time and identify resp. rank clusters with dynamic growth. 2) Each proposal is "embedded" in the landscape to compute a SIMILARITY value depending on both distance and rank of nearest clusters. The underlying assumption is that the closer a proposal is to clusters of dynamic growth, the more novel it is. Computationally, SIMILARITY is based on indexing keywords. To this end, the bibliographic database PASCAL is used, which provides a broad multidisciplinary coverage of about 20 million records. Each PASCAL record is indexed, either manually by scientific experts or automatically based on content analysis, with both keywords and thematic categories. Raw data are extracted from PASCAL (for international scientific and technological literature) by employing a query derived from the description of ERC main research fields (15 in 2007, since then expanded to 10 fields in PE and 9 fields in LS). A non-hierarchical clustering algorithm and principal component analysis is used to compute a 2-D cluster map [START_REF] Lelu | Hypertext paradigm in the field of information retrieval: A neural approach[END_REF]Lelu, 1993) of "similar references" (on the basis of related keywords). Algorithms are implemented in the software system STANALYST (Polanco et al., 2001). Subsequently diachronic cluster analysis is used to study the evolution of the publication landscape across time windows (T1, T2). The most recent time window (T2) is the year in which proposals were submitted. Structural alterations of clusters between T1 and T2 are identified and analyzed by human scientific experts (Roche et al., 2008). Techniques of association rule extraction are applied to facilitate the cluster analysis, using fuzzy association rules [START_REF] Han | Data Mining: Concepts and Techniques[END_REF]Hand et al., 2001;[START_REF] Mahgoub | A text mining technique using association rules extraction[END_REF]. There are two objectives. 1) Determining which clusters carry novel topics and to rank clusters by their "novelty index" (a measure of the relationships between clusters from T1 and T2 build on association rules). 2) Evaluating the novelty of proposals by their similarity with respect to clusters with a high rank [START_REF] Roche | Detecting domain dynamics: Association Rule Extraction and diachronic clustering techniques in support of expertise, submitted to Technology Analysis & Strategic Management Roy R[END_REF]. The indicator INTERDISCIPLINARITY is used as a proxy to infer self-consistently the presence and proportions of characteristic terminology associated with individual ERC main research fields, thereby revealing the intra or inter-field character of a proposal. It is build upon the previously successfully tested approach [START_REF] Schiebel | An advanced diffusion model to identify emergent research issues: the case of optoelectronic devices[END_REF]) that the frequency of occurrence and distribution of research field specific keywords of scientific documents can classify and characterize research fields. While the core of the approach has been retained, the computation has been adopted and fine-tuned to the grant scheme under study.

Empirical keyword distributions are computed from proposal abstract information. For each proposal, the single pre-assigned ("main") research field of a proposal is compared with a set of research fields based on the keyword frequency distribution across all proposals and 19 research fields. To this end, each keyword is labeled according to its statistical frequency of occurrence across fields, filters are applied to distinguish relevant from irrelevant (i.e. field unspecific) keywords, and the concentration of keywords with their newly assigned ("home") research field is assessed to calculate an index for the inter-field concentration of a proposal. The underlying assumption is that the larger the proportion of inter-field keywords, the more interdisciplinary is a proposal. The term "pasteuresqueness" is coined in reference to the definition of Pasteur"s Quadrant [START_REF] Stokes | Pasteur's Quadrant. Basic Science and Technological Innovation[END_REF], which describes scientific research or methods that seek both fundamental understanding and social benefit. Guided by the Pasteur Quadrant, the indicator PASTEURESQUENESS serves as a proxy for the applicability of expected results of each proposal. It is based on patent counts and journal classification (ratio of applied vs. theoretical) of applicant publications. Input data are obtained from proposals and external information sources (e.g. bibliographic databases).

On the level of major research domains (PE, LS, SSH), we noted above that we excluded the six SSH research fields because they are less assessable for our approach for a number of reasons. It is clear that such and similar differences exist between the 10 subfields in 10 PE and 9 subfields in LS (a "field effect" with respect to the science dynamics of different research questions, time scales, methodologies, community sizes, publication and citation rates, interconnectivity, social patterns, etc.) In principle each indicator can be computed form some field and the model can be computed and compared for each field with the review decision. Form a statistical point of view, however, the size of the current data sample allows meaningful comparisons only by pooling data across PE and LS subfields. Given larger data samples, ongoing analysis can study the dependencies on specific subfields with the here developed approach.

Scientometric-statistical modelling

Modelling the influence of indicators of frontier research on the decision probability

From our conceptual background, we are interested in whether different dimensions of frontier research, captured by five indicators TIMELINESS, RISK, SIMILARITY, INTERDISCIPLINARITY and PASTEURESQUENES for frontier research, are a statistically significant determinant influencing a research proposal submitted to the ERC to be accepted or rejected (cf. [START_REF] Holste | A concept for inferring "frontier research[END_REF]. We statistical specify a statistical model that relates different exogenous factorsinvolving indicators for frontier research-to the probability of a proposal to be accepted or rejected, under control of additional factors that may influence the acceptance probability.

We address this task by relying on methods from econometrics. We denote the set of project proposals by Y i (i = 1,…,n = 198) and define a binary dependent variable that is set to one if a proposal is accepted (and zero otherwise), given by 1 0 otherw ise.

i proposal is accepted Y     (1)
In econometric terms we are dealing with a limited dependent variable, referring to situations where the dependent variable assumes discrete alternatives rather than a continuous measure of activity [START_REF] Greene | Econometric analysis, Fifth Edition[END_REF]. For model specifications we borrow from the wide-spread class of Discrete Choice Models (DCM), which are based on the unobservable utility obtained from a choice among alternatives (Train 2009). Here it is the choice of a reviewer (respectively review panel) to accept or reject a project proposal.

We present our modelling approach by defining

( ) ( ) ( ) ( ) ( ) ( ) ( ) k N R P I C i i i i i i  X X X X X X (2)
where X i is the joint vector of k (k = 1, ..., K) exogenous factors that may influence the decision probability of a proposal to be accepted, i.e. Pr(Y i = 1). It comprises different vectors of variables, each of them representing a specific aspect of frontier research, namely SIMILARITY, RISK, PASTEURESQUENESS, INTERDISCIPLINARITY and TIMELINESS as well as other intervening effects that are captured in the control variables vector. We construct our basic model by ()

() P r( 1) ( , ) P r( 0) 1 ( , ) k i i k i i YF Y F     X X   (3)
where β is the estimated k-by-1 parameter vector reflecting the impact of changes in X i on the probability Pr(Y i = 1), and F(.) denotes the respective cumulative distribution function, which has to be chosen. It is common practice to use logistic regression models, where F(.) is substituted with the logistic distribution function Λ(.) so that the resulting Logit model reads as

() () () exp P r( 1) ( , ) 1 + exp k k i i i k i Y     X X X    (4)
where X is a set of k frontier research and k control variables.

Empirical data and initial analysis based on a statistical sample of starting grants

The dependent variables are modelled by using a data sample from about 2500 proposals submitted as ERC Starting Grants in the year 2009 (StG2009). The data sample consists of 198 proposals out of about one-third of applicants providing their consent to making their data available for academic research on ERC grants. The sample of StG2009 are composed of 41 successful proposals that have been selected for funding, while 157 of the proposals have been rejected, i.e. we have 41 cases for which Y i = 1 and 157 cases for which Y i = 0. Table 2 presents selected descriptive statistics as a prelude to the model analysis that follows.

Note that the variables GENDER and ORGANISATION TYPE are dummy variables. The statistics suggests that for INTERDISCIPLINARITY, SIMILARITY and TIMELINESS we can assume a normal distribution, while for the RISK and PASTEURESQUENESS normality cannot be assumed due to the considerable number of zeros such that the standard deviation is higher than the mean. that holds the information of how each variable influences the proposal decision probability (cf. Equation ( 4)). The estimated parameters provide statistical evidence in the context of the guiding research questions. 1) Do different attributes of frontier research extracted from proposals influence the decision probability? 2) Are these effects statistically related to each other? One well-known interpretation can be conducted in the form of probability odds. From Equation (4) it follows directly that () P r( 1 ) ex p ( ). 1 P r( 1)

k i i i ik ik Y Y X X    X  (5)
Here exp() is the effect of the independent variables on the odds, including indicators for frontier research and control variables. That is how much a change of a specific variable affects the probability for a proposal to be accepted, given all other variables are kept constant (see, e.g., Greene, 2008). The parameter estimation is based on standard Maximum-Likelihood techniques (cf. [START_REF] Greene | Econometric analysis, Fifth Edition[END_REF] for further details on the estimation procedure).

Table 3 presents the parameter estimates produced by Maximum-Likelihood estimation. The second column presents a model version using five indicators for frontier research, while the third column presents results of the full model including all control variables. (Asymptotic standard errors are given in parentheses.) The results point to interesting mechanisms that could play a role in the ERC review process. 1) The parameter estimates are sufficiently robust. They do only change slightly when the control variables are added in the full model. 2) The model produces significant estimates for INTERDISCIPLINARITY and SIMILARITY, i.e. it suggests that the review process accounts for these attributes of frontier research in their decision-making. 3) Parameter estimates for the remaining attributes, that is TIMELINESS, RISK and PASTEURESQUENESS, are not statistically significant in either model version. The model suggests that for the study under consideration these attributes are not playing a significant role in the review process.

The term exp() is the marginal effect. It shows how a change in a specific exogenous factor affects the probability for a proposal to be accepted, given all other variables are kept constant. We can thus characterize significant effects in more detail. For example: An increase of the INTERDISCIPLINARITY of a proposal by 1% increases the likelihood for acceptance by a factor of 1.13 (holding all other variables constant); similarly an increase of the SIMILARITY of a proposal by 1% increases the likelihood for acceptance by 1.84 (holding all other variables constant).

Concerning control variables we find that an increase of the R&D EXPENDITURE of a host country by 1% increases the likelihood for acceptance by 1.12 (holding all other variables constant); applicants applying to a university (instead of a research organisation) increase the likelihood for acceptance by 2.45; while an increase of the RANKING SCORE of the host university by 1% increases the likelihood for acceptance even by 4.31 (holding all other variables constant). These results hint at interesting and important patterns of the review process, with implication for review process evaluation.

Validation of the model based on a statistical sample of starting grants

One can address the validity of the model specification from a statistical perspective as well as the model robustness of the parameter estimates produced by Maximum-Likelhood estimation procedures through statistical model tests. The above model has been tested using a number of standard tests for robustness and validation (e.g., testing the link function between the dependent and the independent variables as well as the behavior of the residuals) and was found to be valid.

Table 3 presents selected data on the conducted diagnostic test. The Likelihood-Ratio test is statistically significant for either model. They confirm that independent variables increase the log-likelihood of the model, i.e. they significantly statistically explain the variance of the dependent variable. In addition, the full model fits better than partial model (for frontier research only), given all model diagnostic data presented in Table 3. The statistically insignificant Hosmer-Lemeshow Goodness of Fit test confirms that the logistic link function was the right choice to statistically explain the relationship between the dependent and independent variables (cf. [START_REF] Greene | Econometric analysis, Fifth Edition[END_REF]. This is underpinned when plotting the predicted probabilities vs. the independent variables for INTERDISCIPLINARITY and SIMILARITY (data not shown). The variance of predicted probabilities and residuals also underlines the increased fit of the full model. Finally the pseudo R-squared measures show that the amount of explained variance by independent variables is markedly high and that the explained variance increases from the partial model to the full model. The multicollinearity condition number yields a value of 15.43 for the partial model and a value of 26.48 for the full model. We note that if the condition number is larger than 30, a model is considered to have significant multicollinearity (Chatterje, Hadi and Price 2000). That is estimates would then be considered biased due to the violation of the assumption that the explanatory variables are uncorrelated. This is confirmed by calculating mean Variance Inflation Factors (VIFs). We find that VIFs equal to 1.02 for the partial model and equal to 1.28 for the full model, from which we infer that the estimation and made inferences are not subject to intercorrelation problems [START_REF] Greene | Econometric analysis, Fifth Edition[END_REF].

We computed acceptance probabilities for each proposal using the obtained parameter estimates from the full model, which enables in-depth analysis of proposals. The initial results are already insightful. 1) Among the top-20 probabilities, we find 4/20 wrong predictions, i.e. four nonsuccessful proposals. 2) Between ranks 21 and 30, we find alterations between successful and non-successful proposals, i.e. indicative of tight decision-making whether a proposal is accepted or rejected. 3) Below ranks 30 and up to rank 198, we find 20/169 wrong model predictions.

Discussion

The above concept aims at advancing the development of quantitative methods for determining and examining the relationship between peer-review and decisions about research grant allocation in terms of attributes of frontier research. The model introduced in this paper utilizes information present in research proposals and purposefully builds on references and lexical analyses as well as econometric modelling to address the influence of frontier research on the decision probability of submitted proposals.

The essence of the study is to implement the conceptualized indicators for frontier research in a statistical model, enabling the rigorous exploration of different attributes of frontier research.

Here we developed the indicators SIMILARITY, RISK, PASTEURESQUENESS, INTERDISCIPLINARITY and TIMELINESS, revealed statistically significant determinant influencing research proposals to be accepted or rejected, and studies how they are statistically related to each other. In our proofof-concept approach, we use a data sample of 198 research proposals submitted to as ERC Starting Grants of the year 2009. We employed a discrete choice modelling perspective, specified in form of a logistic regression model, to quantify whether the review process selects proposals that address frontier research theme according to the conceptualization of frontier research presented above.

The empirical analysis convincingly demonstrates the benefit of the approach taken in this paper, both in terms of a first proof of the indicator concept as well as in terms of the modelling approach and obtained results with statistical reliability. On the one hand, the results suggests that (under control of additional effects that may affect decision probability) the attributed of frontier research SIMILARITY and INTERDISCIPLINARITY influence the decision probability for a proposal to be selected, whereas SIMILARITY is the more important attribute. On the other hand, the reviewer process is not seen as being able to select proposals taking into account RISK, PASTEURESQUENESS or TIMELINESS. From the perspective of a grant agency these initial results bear promises for tactical and strategic implications derived from scientometric evaluation. The presented model has focused on the ERC grant scheme but could be more broadly applicable depending on the mission, review process, attributes and correspondence of indicators for other grant schemes.

Ultimately the concept shall advance the methodology to allow a grant agency to support the monitoring of the operation of the peer-review process from a scientometric perspective. In this context several features may increase the robustness of the initial results. For instance adding more control variables possibly influencing the decision probability, e.g., the number of publications or citations of the applicant; or enlarging the data sample using proposals from StG2009 and other years to control for time effects and address field and subfield effects.

Figure 1 .

 1 Figure 1. Selected steps of the ERC review process, indicators and the model.

  Our independent model variables include five indicators and five control variables (cf. Holste et al. 2012 for information of the computation): k = 1 INTERDISCIPLINARITY of a proposal in terms of its distribution of keywords over different ERC panels k = 2 SIMILARITY of a proposal to emerging research fields in terms of its terminological content k = 3 PASTEURESQUENESS of a proposal in terms of the number of patents granted k = 4 RISK of a proposal in terms of similarity between citations given in the proposal and the applicant"s citation behavior before 2008 k = 5 TIMELINESS of a proposal in terms of the mean age of the cited references in the proposal. The control variables are derived from different data sources: k = 6 R&D EXPENDITURES as percentage of GDP of the host country k = 7 GENDER of the applicant k = 8 ORGANIZATION TYPE of the host institution (university or research organization) k = 9 GDP of the host country k = 10 UNIVERSITY RANKING score of the host institution (Leiden university ranking).

Table 1 .

 1 Relation between ERC descriptions of frontier research, key attributes, indicators and the selected approach to operationalize the extraction of attributes.

	Frontier research	Key attribute Indicator	Approach
	"(…) stands at the forefront of	Novelty of the	TIMELINESS	Backward cited
	creating new knowledge and	proposed	SIMILARITY	references;
	developing new understanding. Those involved are responsible for	research		Diachronic cluster analysis based on
	fundamental discoveries and			textual
	advances in theoretical and			information
	empirical understanding (...)"
	"(…) is an intrinsically risky	Risk of the	RISK	Originality of the
	endeavour. In the new and most	investigator		proposed research
	exciting research areas (...)	through		based on
	Researchers must be bold and take	establishing		reference
	risks. The task of funding agencies	scientific		information of the
	is confined to supporting the best	independence		proposal and
	researchers with the most exciting	and/or taking		principal
	ideas, rather than trying to identify priorities."	research field on a new		investigator
	"(…) Therefore, there is a much	Applicability	PASTEURESQUENESS	Applicability of
	closer and more intimate	(entrepreneurial		the expected
	connection between the resulting	principal		results
	science and technology, with few	investigator;
	of the barriers that arise when basic	proposed
	research and applied research are carried out separately."	research)
	"(…) pursues questions	Science of	INTERDISCIPLINARITY Diversity
	irrespective of established	interdisciplinar		reflected of the
	disciplinary boundaries. It may	y nature		proposal on
	well involve multi-, inter-or trans-			related panels
	disciplinary research that brings			other than the
	together researchers from different			"home" panel
	disciplinary backgrounds (...)"			based on textual
				information
	Source: definition: EC (2005); indicator: own data.

Table 2 .

 2 Selected descriptive statistics of model variables.

		Min	Max		Mean	Standard
							deviation
	INTERDISCIPLINARITY	0		100		68.69	12.59
	SIMILARITY	0		4.84		1.35	1.25
	PASTEURESQUENESS	0			13		0.61	1.64
	RISK	0		0.62		0.11	0.15
	TIMELINESS	0		59.66		8.14	6.04
	R&D EXPENDITURES	0		2.43		1.43	0.56
	GENDER	0			1		0.24	0 .42
	ORGANISATION TYPE: UNIVERSITY	0			1		0.23	0 .42
	GDP	22.53 3,060.31 1,471.80	899.85
	UNIVERSITY RANKING	0 .41		1.77		1.04	0.29
	We are interested in estimating the parameter vector	(	(1)	, ...,	( )	)

K   

Table 3 .

 3 Parameter estimates of the discrete choice model.

	Frontier	Full
	research only	model

Table 3 .

 3 Selected model diagnostics tests.

	Frontier	Full
	research only	model

Legend: *significant at the 0.01 % level.

This work was supported by the Ideas specific programme of the EU"s FP7 Framework Programme for Research and Technological Development (project reference no.

240765).

Acknowledgments

The project is supported by the Ideas specific programme of the EU"s FP7 Framework Programme for Research and Technological Development (project reference no. 240765). The authors gratefully acknowledge the contributions made by Christoph Pollak.

Authors' contributions

Dirk Holste (DH) and Thomas Scherngell (TS) conceived the study. TS and DH designed the experiments. TS performed the numerical experiments. DH, Ivana Roche, Marianne Hörlesberger, Dominique Besagni and Maria-Elisabeth Züger contributed data and analyzed the data in exploratory analyses. DH and TS wrote the paper. All authors read and approved the final manuscript.


Notation

Notation 2.1 (primed and unprimed variable) When the value of a variable v in two execution states is considered, the value of v in the first state, called the current state, is written v, and its value in the second state is written v . For instance, the action consisting in increasing the value of v by 1 is specified by the postcondition v = v + 1.

Agent Type Specification

Simplified Definition 2.1 (Agent Type) An agent type t is described by a name (name t ), a set of internal variables (V arI t ), a set of surface variables (V arS t ), an invariant (i t ), and a behabiour (b t ). In this definition, an internal variable is a variable that only the owner agent can see and modify (compare it to a private attribute in the object model); a surface variable is a variable that only the owner agent can modify, but that can be seen by the other agents (compare it to a private attribute with a public getter method); an invariant is a predicate defined on the internal and surface variables of the agent type and that must always be true for every agent of the given type; and the behaviour of an agent is specified by a Goal Decomposition Tree, defined later in this section.

Simplified Definition 2.2 (Action) An action a is specified by a name (name a ), a precondition (pre a ), a postcondition (post a ), an ns flag (ns a ) and a gpf (gpf a ). The precondition is a predicate specifying when the action is enabled, the postcondition specifies what that action does (x = x -1 for instance expresses that the action decreases the value of x by 1), the ns flag has the value NS (necessarily Satifiable) if the action is guaranteed to always succeed, and NNS if the action may fail, and the gpf, the Guaranted Property in case of Failure, is a predicate specifying what is however guaranteed to be true if the action fails.

Definition 1 (Goal Decomposition Tree (GDT)). A Goal Decomposition Tree describes the behaviour of the agents of a given type. Each node of this tree is a GDT goal. The tree structure is defined thanks to the decomposition of each GDT goal into subgoals.

Definition 2 (GDT goal).

A GDT goal g is described by a name (name g ), a satisfaction condition (sc g ), a gpf (gpf g ), a decomposition, an ns flag (ns g ) and a laziness flag (l g ). The satisfaction condition is a predicate specifying what the goal must establish, the gpf is a predicate specifying what is guaranted to be established if the execution of the goal fails, the ns flag specifies whether the goal always succeed or not, and the laziness flag specifies whether the goal decomposition is executed when the satisfaction condition of the goal is already true when the goal is considered.

Please notice that in this article, in order to simplify the formulae and their understanding by the reader, we only consider non-lazy goals.

Definition 3 (Goal decomposition).

A GDT goal is either a leaf goal or an intermediate goal. In the latter case, the goal is decomposed into one or several subgoals, thanks to a decomposition operator. A list of decomposition operators can be found in [START_REF] Mermet | Specifying, verifying and implementing a MAS: A case study[END_REF].

Among others, we can introduce the following decomposition operators: SeqAnd operator, but environment variables can be locked during the whole execution of the parent goal decomposition. • case: The case operator is similar to the switch statement of most imperative languages: it decomposes a goal into several subgoals, with a choice condition associated to each subgoal. When the parent goal must be achieved, one of the subgoals whose choice condition is true is executed. • iter: This operator decomposes a goal into one subgoal. The achievement of this subgoal will be attempted as long as the parent goal is not achieved.

Properties proven by the method

The GDT4MAS method allows to prove several kinds of properties. We first prove invariant and liveness properties, at the agent-type level and at the system-level. We recall here that invariant properties are properties that must be always true, and that liveness properties are properties that must eventually be true. Moreover, the proofsystem of the method verifies that goal decompositions are valid. In this article, we focus on the proof of decompositions and of invariant properties. This is the topic of the next section.

3 Previous proof system

Principles

The main principles of the previous proof system consists in the following steps:

• A context is inferred for each node, in a top-down manner, and from left to right when oriented operators are used (like SeqAnd and SeqOr);

• A gpf, Guaranted property in case of failure, is inferred for each NNS goal, in a bottom-up manner; • A proof schema is used for each decomposition operator, in order to generate a proof obligation, that is to say a first-order formula, whose proof can be attempted by any first-order logic theorem prover.

Consider the GDT presented in figure 1. Between states b and c, other agents may act on the environment. So, the solution we have chosen is a projection of formulae on internal variables. This lead us to the following context propagation rules and proof schemas for the SeqAnd Operator, when a goal n is decomposed into n 1 SeqAnd n 2 .

Context Propagation rule

C n1 = C n (1)
C n2 = ((sc n1 ) ri ) r (2)
In these formulae:

• sc n1 is the satisfaction condition of goal n 1 ;

• F ri is the projection of formula F on the primed internal variables, preserving all the properties entailed by F on the primed internal variables. For instance, if

F ≡ (x = 2 ∧ y = 3 ∧ x > y ∧ y = x + 1
) with x and internal variable and y an environment variable, we have F ri ≡ (x = 2 ∧ y = 3 ∧ x > 3). • F r is the projection of a formula F onto the right, that is to say on primed variables, where primes have been removed. For instance, if F ≡ (x = 2 ∧ y = 3 ∧ x > 3), we have F r ≡ (x > 3).

Guaranted Property in case of Failure

In the GDT4MAS model, the achievement of an agent goal may fail (it can a consequence of the behaviour of another agent in the system for instance). When a goal execution fails, the situation reached is however not completely chaotic; certain properties are guaranted to be true. For instance, if an agent has to open a door and it fails, it is not magically teleported into another room. So, we associate to each goal that may fail (NNS goal: Non-Necessarily Satisfiable goal) a Guaranted Property in case of Failure (or GPF). This property is inferred from leaf goals thanks to GPF propagation schemas.

Proof schema

When we defined the proof schema for the SeqAnd operator, we considered that the proof obligation should contain three types of variables:

• unprimed variables, corresponding to the value of variables in state a on figure 2; • primed variables, corresponding to the value of variables in state d on figure 2;

• temporary variables, corresponding to the value of variables in state c on figure 2. These variables are subscripted by tmp and are computed from the satisfaction condition of the first subgoal, after a projection on the internal variables in order to take into account the fact that other agents may have modified environment variables between states b and c.

Thus, we demonstrated that the following proof schema was valid:

i ∧ i A ∧ C n ∧ T tmp (sc n1 ) ri ∧ T tmp (sc n2 ) → sc n (3)
Where:

• i is the invariant of the environment;

• i A is the invariant of the agent;

• C n is the context of node n;

• T tmp (F) is the transformation of formula F where each primed variable becomes unprimed and subscripted by tmp. For instance, T tmp (x = y + 1) ≡ x = y tmp + 1; • T tmp (F) is the transformation of formula F where each unprimed variable becomes subscripted by tmp. For instance, T tmp (x = y + 1) ≡ x tmp = y + 1;

Limits

Projections

The main limit of the approach defined above is that true projections, as defined in section 3.1.1, cannot be computed automatically. A simplified version can be com-puted, by removing each term of the conjunctive normal form containing another variable that those concerning the projection. For instance, if a projection of the formula F ≡ (x = 2 ∧ y = 4 ∧ x = z + 2 ∧ z > 3) on x is required, we obtain F x ≡ (x = 2). However, this formula is weaker that what we would like to obtain (In particular, the fact that x is greater than 5 is lost).

Weak contexts

As shown in equation 2, The context inferred for a goal is quite weak: it only depends on the execution of the previous goal: its satisfaction condition if the operator is a Seqand, or its guaranted property in case of failure if the operator is a SeqOr. But more information could be preserved from the context in which this previous goal is executed. For instance, if node n is decomposed into n 1 SeqAndn 2 , if the context of node n 1 is x = y, if x and y are internal variables and if the satisfaction condition of node n 1 is sc n1 ≡ x = x + 1 ∧ y = y + 1, we know that x = y when node n 2 is considered. But this cannot be automatically inferred by the contexte propagation rules.

Weak proof schemas

In the proof schema given in equation 3, some important hypotheses miss. For instance, even if environment variables can be modified between states b and c in figure 2, the invariant of the environment is preserved. But this is not specified in this formula.

General observations

Most of the weaknesses highlighted above can be bypassed by re-inforcing satisfaction conditions of goals. However, this is not a satisfying solution, because it requires more work to the developper, and it makes the specification more complicated. Thus, it appeared necessary to re-inforce the proof mechanism to solve these drawbacks.

The new proof system

To define the new proof system, we need new notations, that are introduced in the following section.

Predicate transformers

Notation 4.1 (At) Let f a predicate. f [i] is a predicate where each non-subscripted variable in f is subscripted by i.

Example:

(x = y 0 )[1] ≡ (x 1 = y 0 ). Notation 4.2 (Between) Let f a predicate. f i→j is a predicate derived from f
where each unprimed and unsubscripted variable is subscripted by i and each primed variable becomes unprimed and subscripted by j.

Example:

(y < x ∧ x = x 0 ) 1→2 ≡ (y 2 < x 1 ∧ x 2 = x 0 ).
Notation 4.3 (Temporal switch) Let f a predicate. f →i is predicate derived from f where each subscript is increased by i.

Example:

(x = x 1 ∧ y 2 = x 1 ) →-2 ≡ (x = x -1 ∧ y 0 = x -1 ).
Notation 4.4 (Priming) Let f a predicate. If f contains at least one primed variable, then pr(f ) = f . Otherwise, pr(f ) is the predicate derived from f where each unsubscripted variable is primed.

Examples: pr((x = x 0 )) ≡ (x = x 0 ) and pr((x = x )) ≡ (x = x ).

Notation 4.5 (Stability) Let t and agent type with two internal variables via, vib and one surface variable vs (internal and surface variables are described in the next section). Then, when one agent a of this type is considered stab i→j is the predicate via i = via j ∧ vib i = vib j ∧ vs i = vs j .

Notation 4.6 (Untemporalization) Let f a predicate. f × x is the formula f in which all subscripts of value x are removed.

Example:

(x 1 = x 2 ) × 1 ≡ x = x 2 .
Notation 4.7 (Invariant) Let A an agent situated in an environment E. We write:

• i A the invariant associated to the internal variables of the agent;

• i E the invariant associated to the environment variables;

• i EA the conjunction of i A and i E .

Context inference

In a context formula, it may be necessary to refer to the value of a variable in a previous state. In that case, the variable is subscripted by a negative integer. The value in the current state is represented by the variable name neither subscripted nor primed. For instance, consider the GDT presented in figure 1, and suppose the context of the root goal is x = y.

In the state in which the right subgoal is considered (with a bold outline in figure 2), we know that:

• in state a (numbered -2), from the parent goal, x and y are equals; So: x -2 = y -2 ; • between states a and b (numbered -1), the value of x is increased, whereas the value of y is preserved, and so : x -1 = x -2 + 1 ∧ y -1 = y -2 ; • between state b and the state in which the right subgoal is considered, if x and y are internal variables (and thus, cannot be modified by other agents), we have:

x = x -1 ∧ y = y -1 .
So, the context of the right subgoal is:

ctx = (x -2 = y -2 ) ∧ (x -1 = x -2 + 1 ∧ y -1 = y -2 ) (x = x -1 ∧ y = y -1 )
Please notice that this allows to deduce that in the state in which the right subgoal is considered (state c), we have x = y + 1.

Finally, the new context inference rules are the following:

   C N1 = C N C N2 = (C N1 ∧ sc N1 ) 0→1 ∧ stab 1→2 ∧ i EA [1] ∧ i EA [2] →-2 × 0 (4)
Using such rules, contexts are guaranted to use neither primed variables nor variables with positive subscripts. Indeed, if its true for C N :

• it is trivially true for C N1 ; • it is true for C N2 because: -(C N1 ∧ sc N1 )
0→1 contains variables with negative subscripts from C N1 , and variables subscripted by 0 and 1; -stab 1→2 contains variables subscripted by 1 and 2; -i EA [START_REF] Bordini | Model-checking AgentSpeak[END_REF] contains variables subscripted by 1; -i EA [START_REF]Clear-Sy: B for free[END_REF] contains variables subscripted by 2; -From the previous facts, the formula

(C N1 ∧ sc N1 ) 0→1 ∧ stab 1→2 ∧ i EA [1] ∧ i EA [2]
→-2 contains variables with negative or null subscripts; -and so, C N2 contains variables with negative subscripts or unprimed variables without subscripts.

Proof schema

In this new proof mechanism, the context of the right subgoal, in the case of the SeqAnd operator, contains the essential properties that are guaranted to be true when the right subgoal is considered. So, as the semantics of the SeqAnd operator is that, when the right subgoal is executed and succeed, the parent goal must be achieved, the proof schema associated to the SeqAnd operator only consists in verifying that when the right subgoal succeeds in its context, then the parent goal also succeeds. This leads to the following proof schema:

C N2 [0] ∧ sc 0→1 N2 ∧ i EA [1] → sc -2→1 N (5)
In this formula, we can notice that the values of variables in state a of In order to give to the reader an intuitive meaning for this proof schema, it can be explained as follows:

• We consider the case where the second subgoal, N 2 , is executed, which occurs when its context C N2 is verified. We assign to the state the number 0; • We only consider the case where this subgoal succeed, reaching another state numbered 1, hence the formula sc 0→1 N2 ;

• We also know that in this new state, the environment variable and the agent variable are true. So we have the hypothesis i EA [START_REF] Bordini | Model-checking AgentSpeak[END_REF]; • Finally, if all these hypotheses are true, the satisfaction condition of the parent goal must be established between the state in which the parent goal execution has begun (numbered -2) and the state in which the execution of the second subgoal ends (numbered 1).

Guaranted Property in case of Failure

In the previous proof system, GPF were inferred in a bottom-up manner from leaf goals. This characteristics presented at lease two drawbacks:

• The GDT must be completely specified to be proven. Indeed, leaf goals must be known to infer GPF of intermediate goals, and these GPFs are required to prove the correctness of the GDT, as they are used in some proof schemas; • The GPF inferred may be very complicated and may hold many non-necessary hypotheses, that may reduce the success rate of automatic provers.

So, in the new proof system, it is asked to the developper to explicitely give GPF of each node, as he has to do for satisfaction condition. This is not a harder work than specifying satisfaction condition, and it makes the system quite more compositional. However, it must be checked that the given GPF are entailed by the behaviour specified. Thus, we must give GPF inference rules and we have toverify that the GPF given by the developper is entailed by the GPF inferred from the GDT.

For the SeqAnd operator, the parent goal may fail either when the first subgoal fails (in that case, its gpf is verified in the context of the execution of the first subgoal) or when the second subgoal fails (the gpf of this subgoal is then verified in the context of the execution of this seconde subgoal). So we have:

infgpf N = C N1 ∧ gpf N1 ∨ C N2 [0] ∧ gpf N2 [0] × -2 (6)
We can notice that in the formula corresponding to the inferred gpf of a goal, unprimed variables correspond to the values of the variables when the execution of the goal begins, and primed variables correspond to the values of the variables when the execution of the goal ends.

We have now to prove that the specified gpf is correct. Thus, for each NNS goal, we must establish that when the decomposition fails (that it to say, the inferred gpf is true), either the parent goal is achieved or its gpf is true. Hence the following proof schema:

infgpf N → (sc N ∨ gpf N ) (7)

Related works

Several works deal with the formal specification of multi-agent systems, but just a few consider the formal verification of their specification. Moreover, most of these systems use model-checking [START_REF] Bordini | Model-checking AgentSpeak[END_REF][START_REF] Raimondi | Verification of multiagent systems via orderd binary decision diagrams: an algorithm and its implementation[END_REF][START_REF] Kacprzak | Verification of multiagent systems via unbounded model checking[END_REF]. The most recent and promising work in this area is surely the Agent Infrastructure Layer (AIL) [START_REF] Dennis | Model Checking Agent Programming Languages[END_REF] and its connection with AJPF (Agent JPF), an adaptation of the Java Path Finder model-checker. AIL is a kind of intermediate language that can be used to give a formal semantics to most BDI agent languages (such as AgentSpeak or MetateM). JPF is however not a very efficient model-checker. Although there are best model-checkers such as SPIN [START_REF] Holzmann | The Model Checker SPIN[END_REF], this technique is not well-suited to massive multi-agent systems, where the state-space is too huge, especially because of the great interleaving possibilities between agent actions, and as a consequence, only systems with few agents manipulating boolean concepts can be proven. On the other hand, the system we propose make possible the automatic verification of huge systems, manipulating complex data using an any existing first-order theroem prover (it has been tested with PVS [9] and krt [START_REF]Clear-Sy: B for free[END_REF]).

Conclusion

The previous proof system for GDT4MAS agents was of course valid, but it was difficult to perform proofs with intuitive goal specifications: the satisfaction conditions that the developper had to give had to be really proof-oriented, and using the model was not really feasible by a developper who did not know all the details of the proof process. With the new principles presented in this article, the specification task is more independant from the proof process. Of course, the developper must be well-heeled with the predicate logic, but it it not necessary for him to understand how the proof works. Using the method is thus easier. Moreover, the number of proofs that can be performed is increased.

Fig. 1

 1 Fig. 1 Simple GDT

Fig. 2

 2 Fig. 2 Execution trace

  figure 2 are represented by the variables subscripted by -2, values in state b are represented by the variables subscripted by -1, values in state c are represented by variables subscripted by 0 and values in state d are represented by variables subscripted by 1.

•

  SeqOr: Sequential Or. It decomposes the parent goal into several subgoals N i . Subgoals are considered from the the left to the right. If the considered subgoal succeeds, the parent goal is achieved and the execution of the decomposition is ended. But if it fails, the next subgoal is considered. If the last subgoal is reached and fails, the satisfaction condition of the parent goal must be evaluated to know if it is achieved or not.• SeqAnd: Sequential And. It decomposes the parent goal into several subgoals N i . Subgoals are considered from the left to the right. If the considered subgoal succeeds, the next one is considered. If the last subgoal is considered and succeeds, the parent goal is achieved. But if it fails, the satisfaction condition of the parent goal must be evaluated to determine whether the parent goal is achieved or not. • SyncSeqOr and SyncSeqAnd: These operators are similar to the SeqOr and


Marianne Hörlesberger
email: marianne.hoerlesberger@ait.ac.at

Ivana Roche
email: ivana.roche@inist.fr

Christine Louala
email: christine.louala@inist.fr

Nathalie Antonot

Beatrix Wepner

SEARCHING FOR THEORETICAL CONCEPTS IN A SPECIFIC SCIENTIFIC DISCIPLINE: AN APPROACH BASED ON TERMINOLOGICAL FLOWS

Keywords: Tribology, Mechanism of wear, Friction, Lubrication, Bibliometrics, Theory, Experiments

Tribology is a very interdisciplinary field. Tribology, lubrication and surface technology are key technologies and with utmost importance for all branches in industrial production. Innovations in these technologies require the best possible fundamental understanding of the complex processes taking place at the interfaces of a tribological system. Therefore the XTribology centre in AC2T Research GmbH is confronted with the challenges of a highly interdisciplinary field. In previous works, we developed an approach based on the analysis of the evolution of a specific scientific discipline by means of studying its terminology extracted from related specialized international literature. In this work we investigate the very important subfield of Tribology "the mechanism of wear". The scientific basis is still explored. Besides general bibliometric aspects the "theories" and its connection to experiments are of specific interest.

Introduction

Tribology is the science and engineering of interacting surfaces in relative motion, including the study and application of the principles of friction, lubrication and wear. Tribology, lubrication and surface technology are key technologies and with utmost importance for all branches in industrial production. Innovations in these technologies require the best possible fundamental understanding of the complex processes taking place at the interfaces of a tribological system. Therefore the XTribology centre in AC 2 T Research GmbH is confronted with the challenges of a highly interdisciplinary field.

In previous works, we developed an approach based on the analysis of the evolution of a specific scientific discipline by means of studying its terminology extracted from related specialized international literature. After the identification of the constituent sub-fields of the discipline, our methodology operates a global characterisation of their related terminology allowing comparing their evolution and studying their relationships defined in terms of terminology exchanges. The results are presented under the shape of a web server to facilitate their assessment by scientific experts.

In this work we investigate the very important subfield of Tribology "the mechanism of wear". The scientific basis is still explored. There is still to be done much in the research for wear and its bridges to its applications, which entails a high potential of innovation. The work is guided by the following research questions:

 Where is the focus of research in "mechanism of wear" currently on?  What are the contemporary and established theories for describing the "mechanism of wear"?  Which bridges can be identified between the experimental methods and the theories?

From the methodology point of view, the in-depth analysis of the results made available on a Tribology-devoted web server was able, in a general way, to contribute to a better understanding of the evolution of the studied core discipline and its relationships in a multidisciplinary context. In this work, we are particularly interested on the development of an analytical approach allowing identifying the links privileged between the Tribology and the sub-fields supplying its needed theoretical background.

In the first part of this contribution a bibliometric analysis of the field "Machine components. Friction, wear, lubrication" is presented, where the search strategy, general aspects of the publication activities, and connections to the activities in Austria and AC 2 T Research GmbH is discussed.

The second part deals with a detailed investigation of the content of the data set with a specific focus on theories and its connection to experiments.

Bibliometric analysis of the "Machine components. Friction, wear, lubrication" field

We focus on the international scientific literature published from 2001 to nowadays, dealing with the field "Machine components. Friction, wear, lubrication" and indexed in the PASCAL bibliographic database.

This information source provides broad multidisciplinary coverage of scientific publications and contains at the present time about 20 million of bibliographic records from the analysis of the scientific and technical international literature published predominantly in journals and conference proceedings. 1545-858x" ou sn="0301-679x" ou sn="0036-8792" ou sn="0724-3472" ou sn="1045-7828" ou sn="0742-4787" ou sn="0024-7154" ou sn="0043-1648" ou sn="1040-2004") et (dp=2001 ou dp=2002 ou dp=2003 ou dp=2004 ou dp=2005 ou dp=2006 ou dp=2007 ou dp=2008 ou dp=2009 ou dp=2010 ou dp=2011)et(cc=001d12d*)

The obtained result is a corpus of 9,454 records representing the worldwide production of the field "Machine components. Friction, wear, lubrication" in the PASCAL database for the considered period. This corpus is analysed by means of bibliometric analysis tools in order to highlight some characteristics of the worldwide production: annual productivity, journals, authors, affiliation.

In parallel, we examine the position of the Austrian production in the international scientific landscape and, more finely, we focus on the AC 2 T literature characteristics in the Austrian context. In this study, the Austrian production is determined from the worldwide corpus by searching for the bibliographic references containing at least one author whose country of affiliation is Austria. Also, the AC 2 T production is identified by a second filtering retaining exclusively the bibliographic references having at least one occurrence of this institute in their list of affiliation addresses.

Annual productivity

In the considered period (2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011), the international production is represented by 9,454 records, the Austrian one by 87 records and the AC 2 T one by 34 records. The Austrian production represents 1% of the worldwide one. The AC 2 T production corresponds to 0.4% of the international one and 39% of the national production.

In the figures 1, 2 and 3, we present the annual distribution of, respectively, the worldwide, the Austrian, and the AC 2 T productions. A first observation of the Figures 1, 2 and 3 shows a significant decrease of the production in 2004. This artefact is due to a modification of the PASCAL database coverage and not to a real drop off of the field production. On the other hand, let us remark that the annual production of the 2011 year is not yet complete and, by consequence, it looks anormally weak in Figures 1, 2 and3. This is the reason we do not take into account the values of these years in the following analysis of the annual productivity of the field "Machine components. Friction, wear, lubrication".

While international literature presents a quite regular production since 2005, we can observe a clear progress of the Austrian production since 2007. The same observation can be made for the AC 2 T which shows a spectacular progression since 2008. Particularly, we can also notice that in 2010 the AC 2 T production represents more than the half of the Austrian one.

Publishing types

Classically, the diffusion of research results is made either by their publication in journals or by their presentation to peers in conferences. In the field "Machine components. Friction, wear, lubrication" the part of communications presented in conferences is quite high: 27%, 43% and 53 %, respectively, for international, Austrian and AC 2 T productions. This would lead us to affirm that this scientific community considers very important to exchange on their outcomes in the direct interactive context that conferences bring. The particularly high rate of AC 2 T could be the sign of the important attachment of this institute to this type of exchanges.

Interdisciplinarity of the field

The data set is extracted from the PASCAL database that is specifically adapted to point out the interdisciplinarity of a field. Indeed, its records benefit from an indexing by both keywords and multiple thematic categories of a disciplinary classification scheme assigned to each individual publication, either manually by scientific experts or automatically, based on a content analysis. This category multiplicity allows us to identify the interactions between different disciplines by considering that, if a reference gets different thematic categories, this means that the content of the related document deals with different disciplines.

In the figures 4, 5 and 6, we present respectively the repartition of the international, the Austrian, and the AC 2 T productions in the field "Machine components. Friction, wear, lubrication" by considering their bibliographic references dealing also with other disciplines. They are, in order to facilitate the reading, organized in 13 different disciplinary sets that we detail in table 1. We can observe that the international literature in the field "Machine components. Friction, wear, lubrication" deals especially with "Mechanics of solids", as well as with Engineering sciences like "Energy, Electronics and Polymers", and finally with "Metals, metallurgy" and "Drives".

Austrian production focuses for a great part on "Mechanics of solids", and then on "Metals, metallurgy" and "Drives", showing that studies are conducted in the field of fundamental sciences as well as in more applied sciences. The AC 2 T publications concentrate mainly on "Metals, metallurgy", and to a lesser extent on "Mechanics of solids" and on "Drives", pointing out a more applied research oriented toward Engineering sciences.

Affiliations

Bibliometric analyses launched on countries of affiliation appearing in publications of both Austria and the AC 2 T in the field "Machine components. Friction, wear, lubrication" show important scientific collaborations with Germany and with India. On the national level, 18.4 % of publications are written together with German scientists while 6.9 % of the analysed records concern collaborations with India. For the AC 2 T, the trend is inverted in the sense that its largest number of collaborations occurs with Indian laboratories in 17.6 % of records. Germany follows with 11.8 % or publications.

Furthermore, a detailed analysis of the Austrian affiliations in the obtained corpus, confirms the very important position of the Austrian Center of Competence for Tribology in the "Machine components. Friction, wear, lubrication" field. Another important actor is the Department of Physical Metallurgy and Materials Testing in the University of Leoben, showing that metallurgy seems to be a significant area of tribology application fields in Austria. This confirms the precedent observation made in the interdisciplinarity study.

Journals

In the tables 2, 3 and 4, we present the top journals of, respectively, the worldwide, the Austrian, and the AC 2 T productions. The first thing which we can bring to light when analysing the three tables is the important position of the journal in German language, Tribologie und Schmierungstechnik, in both the Austrian and the AC 2 T corpus. Indeed, in these two cases, it ranks second while it is in fifth place in the international literature.

The other observation we can make about the national production is that Austrian researchers, although being interested in the fundamental aspects of the field "Machine components. Friction, wear, lubrication", also focuses on particular applications, as evidenced by the presence of the periodicals "Vehicle System Dynamics" and "Thin Solid films".

Paper language

In the tables 5, 6 and 7, we present the most frequent languages used in scientific publications in the field "Machine components. Friction, wear, lubrication" of, respectively, the worldwide, the Austrian, and the AC 2 T productions. At a first glance, the more the geographic zone is limited, the more the percent of German language in tribology scientific publications becomes important.

For comparison, if we consider the global scientific literature in the same period, we notice that English represents 93% of the paper languages, followed by French with 6%. German comes third with 0.7%. This analysis confirms the trend observed in the journal ranking.

Can we therefore deduce that the tribology scientific field is an important research area in German speaker countries?

The results obtained in this work by means of bibliometric tools in the field "Machine components. Friction, wear, lubrication" show an increase of the Austrian research activity in this domain.

The interest of Austria for applied fields of tribology, in particular for metallurgy, is highlighted by the analysis of the interdisciplinarity of the field "Machine components. Friction, wear, lubrication" as well as the journal distributions.

Moreover, we evidenced the growing proportion of the AC 2 T annual production in the Austrian landscape in the last years. This point was confirmed by the study of the Austrian affiliations and trends to suggest that the Austrian Center of Competence for Tribology is at the forefront of Austrian research in the field "Machine components. Friction, wear, lubrication".

The content in connection to the term "theory"

For a more detailed analysis we consider all terms1 in the frequency interval of [20; 999]. The frequency of a term is given by its occurrence in an article. It is counted once if it occurs at least once in an article. The interval [20; 999].was chosen because of the following considerations.

Terms with a higher frequency are much more general such as "tribology" with a frequency of 8,006, or "experimental study" of a frequency of 4,276, or "wear" with a frequency of 1,527 for example. Terms with a low frequency deal possibly with new ideas or aspects far away from an established research topic, or are spelling mistakes.

The most visible terms of the considered field "Machine components. Friction, wear, lubrication" are presented in the following Table 8 (12 terms with the highest frequency in the considered interval): The term "theory" Now we investigate especially the term "theory" and its co-terms.

The term "theory" occurs in 588 articles (either in the title, in the abstract, or as a descriptor/keyword), which means in 6.22% of the data set. If we consider only the descriptor field, we get 154 articles with "theory" as a descriptor/keyword, which is approximately 1.63% of the considered data set. Hence we can say that "theory" seems to play a subordinate role in this technology field.

Remarkable is the fact that the term "theory" occurs only till 2004 in this data set (see figure 7), whereas "theoretical study" also can be found in 2011. The term "theory" concurs with 138 different terms (out of 1,149 descriptors in the considered data set.

For a detailed analysis the software BibTechMon (Bibliometric Technology Monitoring developed in the department Technology Management of the AIT Austrian Institute of Technology) is applied. This software is based on co-object analysis. In case we consider for instance the keywords because of their common occurrence in articles a node stands for a keyword. Two nodes (keywords) share an edge if they occur in the same article. The strength of an edge is given by a similarity coefficient like the Jaccard coefficient. The "size" of a node is given by the frequency of a keyword, which means the number of articles a considered keyword occurs at least once. This software helps especially in the analysis of the data. Furthermore it presents the network graphs of e.g. keywords.

Figure shows the term "theory" and its co-terms (the coloured ones). The term "theory" is more on the edge of the network in the north east part. This means that this term does not play a central role in the considered data set. Terms in the centre of the network have often a connection to many other terms and therefore articles. Terms more in the centre occurs in many articles of the considered data set and deal with the main aspects of the investigated data. In our case the term "theory" is on the edge, but it is obvious that "theory" is connected to terms in the centre, as figure 7 shows. The term with the highest co-frequency to "theory" is "experiments", as Table shows. This fact is remarkable, because experts working in this field and familiar with "experiments" think that there are not so many theories which could support the experimental results. We will investigate this problem in more details. The term "theory" in titles and abstracts

If we search for the term "theory" in titles or abstracts we 389 articles or 4.11% of the publications. That means that it is not necessary also in the field "descriptor". 30 articles with these conditions are published in 2011. They deal with the following kinds of theories: Experts in this field will observe that these articles and also the mentioned theories above connect experiments and theories. Nevertheless the question is if there is a specific theory / model for describing a rounded technological problem.

 cyclic
Compared with the whole corpus of the data (9,454 records all in all) we see that "theory" not covered well, is not discussed or developed in this field of investigation. Either there are fewer theories in this field or they are covered with another terminology. Therefore we touch briefly a further term.

The term "modelling"

"Modelling" occurs in 51 articles.

Conclusions

We learn from this study that there is a slight decrease in the productivity worldwide, whereas Austria and especially AC 2 T show an increase in the publication activities in the field of "Machine components. Friction, wear, lubrication".

The international literature in the field "Machine components. Friction, wear, lubrication" deals especially with "Mechanics of solids", as well as with Engineering sciences like "Energy, Electronics and Polymers", and finally with "Metals, metallurgy" and "Drives".

Austrian production focuses for a great part on "Mechanics of solids", and then on "Metals, metallurgy" and "Drives", showing that studies are conducted in the field of fundamental sciences as well as in more applied sciences. The AC 2 T publications concentrate mainly on "Metals, metallurgy", and to a lesser extent on "Mechanics of solids" and on "Drives", pointing out a more applied research oriented toward Engineering sciences.

Which role does "theory" play in this field? The analysis concerning "theory" is poor. The term "theory" occurs in 588 articles (either in the title, in the abstract, or as a descriptor/keyword), which means in 6.22% of the data set. If we consider only the descriptor field, we get 154 articles with "theory" as a descriptor/keyword, which is approximately 1.63% of the considered data set. Hence we can say that "theory" seems to play a subordinate role in this technology

Figure 1 .

 1 Figure 1. Distribution by year (2001-2011) of the international production in the field "Machine components. Friction, wear, lubrication"

Figure 2 .

 2 Figure 2. Distribution by year (2001-2011) of the Austrian production in the field "Machine components. Friction, wear, lubrication"

Figure 3 .

 3 Figure 3. Distribution by year (2001-2011) of the AC 2 T production in the field "Machine components. Friction, wear, lubrication"

Figure 4 .Figure 5 .Figure 6 .

 456 Figure 4. Repartition of the international production in the 13 disciplines in relation with the field "Machine components.Friction, wear, lubrication"

Figure 7 .

 7 Figure 7. Number of articles with the keyword "theory" over all considered years. Looking for specific theories in this data set (with "theory" in the descriptor field) we get the following theories and titles:  A simple elastic model is used to describe the elastic deformation of the bearing liner in the article The effects of surface irregularities on the performance characteristics of flexible finite journal bearings lubricated with couple stress fluids (2002)  Approximation theory in o Soot Characterisation and Diesel Engine Wear (2004) o Steady motion of a drop along a liquid interface (2003) o Efficient parallel numerical solver for the elastohydrodynamic Reynolds-Hertz problem (2001)  Constraint theory in o Optimization of set-up conditions for stability of the centerless grinding process (2004) o Multidirectional compliance and constraint for improved robotic deburring. Part 1: Improved positioning (2001  Control Theory in Design and implementation of command and friction feedforward control for CNC motion controllers (2004) o Designing automotive gear oils for the new millennium (2002)  Drainage theory in A review of drainage and spontaneous rupture in free standing thin films with tangentially immobile interfaces (2003)  Fuzzy theory in Fuzzy synthetic method for life assessment of power transformer (2004)  Graph theory in Analysis of tool and workpiece interaction in diamond turning using graphical analysis of acoustic emission (2003)  In the article The thickness of a marangoni-driven thin liquid film emerging from a meniscus (2002) the resulting reduced model … recovers the result of the older theory.

Figure 8 .

 8 Figure 8. The term "theory" and its co-terms in colour in the network graph.

Figure 9 .

 9 Figure 9. The term "theory" and "experiments" in the network graph at the edge at north east. Few examples of titles published in 2004 (there are 32 articles in 2004, and these are the youngest ones) which connect experiments and theory are the following once:  Mechanism of Boundary Film Formation from n-Hexadecane  Coated gearings and bearings optimised for environmentally acceptable lubricants  High-pressure rheology of eight synthetic lubricants based on phase diagrams  The tribological properties of several silahydrocarbons for use in space mechanisms

Figure 10 .

 10 Figure 10. The term "modelling" in the data set over the considered time span.But as well as "modelling" and "empirical model" are on the edge of the network graph. Hence these topics are side issues. (empirical model has a frequency of 26).

Figure 11 .

 11 Figure 11. The term "modelling" and "empirical model" in the data set over the considered time span.

Table 1 . List of the 13 disciplines in relation with "Machine components. Friction, wear, lubrication"

 1

	ID number Discipline	Percentage of records
		International	Austria	AC	2 T
	1 Mathematics	0.36	0.00		0.00
	2 General physics. Condensed matter. Materials science	5.89	0.00		0.00
	3 Mechanics of solids	17.54	22.99		14.71
	4 Surfaces and interfaces	0.60	2.30		0.00
	5 Chemistry	0.75	0.00		0.00
	6 Energy. Electronics. Polymers	14.45	4.60		2.94
	7 Metals, metallurgy	13.08	11.49		20.59
	8 General mechanical engineering. Industrial metrology	5.03	3.45		5.88
	9 Drives	11.55	10.34		8.82
	10 Engines. Pumps. Steel design	2.51	3.45		2.94
	11 Precision engineering	1.11	0.00		0.00
	12 Universe science	0.12	0.00		0.00
	13 Life science	2.48	2.30		5.88

General physics. Condensed matter. Materials science 3 Mechanics of solids 4 Surfaces and interfaces 5 Chemistry 6 Energy. Electronics. Polymers 7 Metals, metallurgy 8 General mechanical engineering. Industrial metrology 9 Drives 10 Engines. Pumps. Steel design 11 Precision engineering 12 Universe science 13 Life science

Table 2 . The six top journals of the worldwide production in the field "Machine components. Friction, wear, lubrication"

 2

	Title	Percentage of records
	Wear	36.67
	Tribology international	15.98
	Journal of tribology	10.17
	Tribology transactions	7.20
	Tribologie und Schmierungstechnik	5.13
	Tribology & lubrication technology	4.20

Table 3 . The six top journals of the Austrian production in the field "Machine components. Friction, wear, lubrication"

 3

	Title	Percentage of records
	Wear	32.18
	Tribologie und Schmierungstechnik	31.03
	Tribology international	24.14
	Vehicle System Dynamics	3.45
	Journal of tribology	2.30
	Thin solid films	2.30

Table 4 . The four journals of the AC 2 T production in the field "Machine components. Friction, wear, lubrication" Title Percentage of records

 4

	Tribology international	47.06
	Tribologie und Schmierungstechnik	38.24
	Wear	11.77
	Tribology transactions	2.94

Table 5 . The most frequent paper languages of the worldwide production in the field "Machine components. Friction, wear, lubrication"

 5

	Language	Number of records	Percent
	English	8887	94,003
	German	456	4,823
	French	101	1,068
	Spanisch	8	0,085
	Japanese	3	0,032

Table 6 . The most frequent paper languages of the Austrian production in the field "Machine components. Friction, wear, lubrication"

 6

	Language	Number of records	Percent
	English	69	79,31
	German	18	20,69

Table 7 . The most frequent paper languages of the AC 2 T production in the field "Machine components. Friction, wear, lubrication"

 7

	Language	Number of records	Percent
	English	26	76,471
	German	8	23,529

Table 8 . Most visible terms in the considered field.

 8

	Keyword	Frequency
	Rough surface	992
	Lubrication	947
	Scanning electron microscopy	923
	Lubricating oil	914
	Roughness	908
	Dry friction	889
	Metal	823
	Abrasive wear	813
	Wear rate	792
	Sliding friction	727
	Sliding wear	704
	Contact stress	700
	Which terms occur the first time in 2011? These terms indicate the current research topics or
	emerging topics.

Table 9 . Terms with a frequency higher than 1 and the first time in 2011.

 9

	Documentation
	Extract
	Lead sulfide
	Railway industry
	Recovery
	Springback

o Surface Defect Generation and Recovery in Cold Rolling of Stainless Steel Strips o Gap control for near-dry EDM milling with lead angle  Springback o Elastoplastic Stress Analysis and Residual Stresses in Cylindrical Bar Under Combined Bending and Torsion o Evaluation of lubricants for electrically-assisted forming

Table 10 . Co-terms to « theory » with the highest co-frequency.

 10

	co-term	co-occurence
	Experiments	72
	Lubrication	49
	Lubricants	41
	Wear of materials	19
	Viscosity	18
	Additives	17
	Insulating oil	17
	Lubricating oils	16
	Oxidation	12
	Abrasives	11

  These theories are either applied, or the described methodologies are based on these theories.

	 extreme-value theory;  FASTSIM (simplified theory of rolling contact);  fractal theory;  Group theory;  Hertz Contact Theory;  Hertz theory;  Hopf bifurcation theory;  Irreversible thermodynamic theory;  isomorphic replacement theory in mineralogy and petrology;  Jakobsson-Floberg-Olsson cavitation theory;
	Johnson-Kendall-Roberts (JKR) theory;
	 Kalker s non-Hertzian rolling contact theory;  linear wear theory;  lubrication theory;  M-estimation theory;  rolling contact theory;  rubber friction theory;  stokes microcontinuum theory;  theory of scale;  three-dimensional theory of elasticity in conjunction with Archard's generalized wear
	equation;
	There are six articles with the term "theory" in the titles in 2011:  Development of a Centrifugal Oil Lubricator for Long-Term Lubrication of Spacecraft
	Attitude Control Systems-Design and Theory (University of South Australia Mawsonlakes
	Campus)
	 RCF and wear in theory and practice-The influence of rail grade on wear and RCF
	(Voestalpine Schienen GmbH, Austria; Austrian Academy of Sciences, Austria; University of
	Leoben, Austria; RailMeasurement Ltd, Germany; National Research Council Ottawa,
	Canada; University Clayton, Australie)
	 A Review of Elasto-Hydrodynamic Lubrication Theory (SKF Engineering & Research Centre
	Nieuwegein; Universite de Lyon, France)
	 Thermal Fly-Height Control Slider Instability and Dynamics at Touchdown: Explanations
	Using Nonlinear Systems Theory (University of California,Berkeley, CA, USA)
	 Characteristics of micro-gas journal bearings based on kinetic theory (National Cheng Kung
	University, Taiwan; National Kaohsiung University of Applied Sciences, Taiwan)
	 A Greenwood & Williamson theory for line contact (University Cambridge, UK; Politecnico
	plasticity theory; di Bari, Italy
	 density functional theory;  dynamics theory of the vehicle;  Erigen s micropolar theory;  Euler s belt theory;

Term or keyword or descriptor denote main aspects of an article, and describe the matter of the article very generally.

field. The experts in Tribology will give their comments till the conference is running so that the benefit of such an analysis can be assessed.

Bibliometric methods give a good overview to a specific field generally. But if you would like to know specific details it is necessary to read the article, what is evident. However bibliometric methods help to go into details easily.


Ivana Roche
email: ivana.roche@inist.fr

Nathalie Vedovotto
email: nathalie.vedovotto@inist.fr

Claire François
email: claire.francois@inist.fr

Dominique Besagni
email: dominique.besagni@inist.fr

Marianne Hörlesberger
email: marianne.hoerlesberger@ait.ac.at

Dirk Holste
email: dirk.holste@ait.ac.at

Edgar L Schiebel
email: edgar.schiebel@ait.ac.at

Pascal Cuxac
email: pascal.cuxac@inist.fr

Assessment of the applied orientation of a researcher's production: An informetric approach based on a content analysis 1

Assessment of the applied orientation of a researcher's production: An informetric approach based on a content analysis

Extended abstract

One way of making the distinction between pure and applied research was introduced by Donald Stokes [START_REF] Stokes | Pasteur's Quadrant -Basic Science and Technological Innovation[END_REF] who defined a two dimensions chart, "the Pasteur's Quadrant". It is a label given to a class of scientific research developments that both seek fundamental understanding of scientific problems, and, at the same time, seek to be eventually beneficial to society. The works of Louis Pasteur, a French chemist and physicist, pioneer of the microbiology, are thought to exemplify this type of method, which bridges the gap between "basic" and "applied" research. The Pasteur's Quadrant characterizes three distinct classes of research:

-pure basic research, illustrated by the work of Niels Bohr, early 20th century atomic Danish physicist; -pure applied research, exemplified by the work of Thomas Edison, North-American inventor and businessman; -use-inspired basic research, described as "Pasteur's Quadrant".

A classic solution to determine the applied orientation of the works of a researcher is to search for the possible patents in the development of which he(she) took part (e.g. [START_REF] Glänzel | Patents cited in the scientific literature: An exploratory study of "reverse" citation relations[END_REF][START_REF] Moed | Handbook of quantitative science and technology research: The use of publication and patent statistics in studies of S&T systems[END_REF][START_REF] Glänzel | Publication activity, citation impact and bi-directional links between publications and patents in biotechnology[END_REF]. Usually, the submission of a patent is the outcome of an initiative of technology transfer and can be indeed considered as the practical fulfilment of research works presenting determinedly applied characteristics. Another possible approach consists on directly examining the researcher's works published in the S&T (scientific and technological) literature and characterizing their content in applied or fundamental.

In the context of a previous European project (Holste, 2012), we modelled this two criteria by means of indicators calculated as:

-the enumeration of the patents in which the researcher has contributed. This indicator gets integer values included in the interval [0,  [. Unfortunately, the number of patents is often very low, which involves a lack of accuracy of the related indicator.

1 The authors wish to acknowledge the support of the IDEAS specific programme of the EU's 7th Framework

Programme for Research and Technological Development in which context this work is partially accomplishing (DBF project -reference n° 240765 -within the Coordination and Support Actions).

Page 2 of 3 -the ratio of the researcher publications appearing in journals which content is categorized as applied. This indicator gets real values between 0 and 1. For both indicators we guessed that the higher the value, the more asserted is the applicability of the researcher works.

Even if such an approach could sound pragmatic, it conceals a few weaknesses in particular because of the journals categorization step. Indeed, by using a binary categorization "applied or fundamental", it seems a priori easy to automatically transpose the journal's category to all the articles that are published in it. A first difficulty appears when we have to determine the criteria defining this binary categorization of the S&T journals. If works have been done supplying hierarchical tree classifications, more or less detailed, of the scientific domains, the problem persists when we wish to determine which ones can be determinedly considered applied or fundamental.

In addition, the category of a journal can be not unique but variable, according to the scientific domain of each researcher that publishes his(her) work in it. Let us consider, for instance, the Biology domain as a priori fundamental. All the journals classed in this domain get then the category "fundamental", as well as all the articles published in them. Yet this does not always correspond to reality. Indeed, if this remains true for the biologists' publications in these journals, that of an IT specialist who would bring a development software to Biology should receive the "applied" category.

And so in this work, instead of examining directly the researcher's production, we propose to assess its applied orientation by analyzing the S&T literature citing his(her) publications. The citation expresses its exploitation, in different ways and at different degrees of importance, and we consider that this corpus of bibliographic records referring to the researcher's production represents a real and pragmatic information source about the utilization of his(her) former works by the scientific community in new developments. The content analysis approach applied to this corpus gives us the means to appreciate the degree of application of the researcher's scientific production. This way, we can detect potentially applicable works whose results could be integrated in more applied issues. So, in a first step, we extract from a bibliographic database the list of the researcher's publications in S&T literature. This list enables us to determine the set of publications citing at least one of the extracted publications. This corpus can be considered as an image of the scientific landscape of citing papers that are based on the past work of this researcher.

Then a data mining step, based on NLP (natural language processing) techniques, is used to obtain an assisted indexing of the records by combining them with keywords. A clustering is then applied to this enriched corpus. The software tool uses a non-supervised and non-hierarchical clustering algorithm, the axial K-means, inspired by Kohonen's self-organizing maps formalism. This method considers the keywords as indicators of the content of bibliographic records, which in their turn are considered as indicators of the research themes. This first step is followed by a principal component analysis leading to a 2D-mapping of the clusters. Thematic networks emerge from the relations between clusters and, according to a geographical metaphor, build a map of the corpus research landscape.

At this stage, a scientific expert performs an analysis of the clusters in terms of their content and of their position and relations in the map. For this analysis, he(she) must adopt a particular point of view, in order to evaluate whether the content of a corpus is essentially applied or essentially fundamental: he(she) looks therefore at the content of each cluster by considering the bibliographic records title and keywords, in order to evaluate how much the content of the clusters can be considered as applied. We call this the level of "applicateness" of the cluster. Finally, we weight the "applicateness" value of each cluster with the help of a parameter coming from the clustering results, in order to take into account the global context of the corpus research landscape represented by the map of thematic clusters and, in particular, by analyzing the links between them. The combination of the weighted "applicateness" of each cluster gives us the level of the applied orientation of the researcher's production.

In this paper, we will firstly detail the approach used to evaluate the applied orientation of a researcher's production. Secondly, we will describe the data processing, and finally we will, on one hand, present and discuss the obtained results for a few case studies and, on the other hand, look towards the future by proposing possible improvements of our methodology.