Kenneth Benoit

Michael Laver

Christine Arnold
email: c.arnold@fsw.vu.nl

Paul Pennings
email: pjm.pennings@fsw.vu.nl

Madeleine O Hosli
email: hosli@fsw.leidenuniv.nl

Measuring National Delegate Positions at the Convention on the Future of Europe Using Computerized Word Scoring

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The Convention on the Future of Europe, held in Laeken to negotiate and eventually draft a constitution for the European Union (EU), involved political actors from many countries, national parties, European Parliament party groups and representatives of interest groups. Each actor held preferences towards the negotiation outcomes, which they presumably pursued by articulating their positions during the Convention. Measuring these preferences is of high relevance both for political science in general terms and for estimating the course and determinants of European integration more specifically. Fortunately, Convention participants left a written record of their negotiations, articulating these in a manner that was meticulously recorded. Because of the vast quantity of these texts, however -involving a total of 6474 contributions from 197 distinct speakers from 28 different countries -it is timeconsuming, costly and difficult to extract useful information from these documents. Recent methodological advances in computerized content analysis, however, offer new ways to deal with such huge volumes of text. The recently developed computerized word-scoring technique [START_REF] Laver | Extracting Policy Positions from Political Texts Using Words as Data[END_REF] is one such method, and has already been applied to the analysis of party manifestos [START_REF] Laver | Extracting Policy Positions from Political Texts Using Words as Data[END_REF][START_REF] Benoit | Estimating Irish Party Policy Positions Using Computer Wordscoring: The 2002 Election[END_REF] and parliamentary speeches [START_REF] Laver | Locating TDs in Policy Spaces Using Computer Word-Scoring[END_REF]Giannetti and Laver, forthcoming). Driven by relative word frequencies, the word-scoring method breaks down each text into words, treating these words as data to be analysed statistically. Such techniques offer particularly large payoffs when the research task involves analysing very large numbers of texts in a systematic way, as is precisely the case in relation to negotiations during the Laeken Convention.

In this article we apply the word-scoring method to analyse the thousands of Convention texts, with the goal of estimating the political positions of national party representatives participating in the Convention. In order to assess the validity of the word-score estimates, we compare national party word-score estimates from the Laeken negotiations with expert survey scores of European national parties from the 2003 expert survey reported in Benoit and Laver (forthcoming). The exercise thus serves as a good test for wordscoring in a substantive application where the sheer volume of texts makes other methods of text analysis impossible or infeasible. Our measure of success will be the degree of statistical correlation we can observe between the word-score estimates and the independent expert survey estimates of the positions of national parties on the same policy dimensions.

The paper proceeds as follows. First, we briefly describe the nature of the Convention on the Future of Europe and the negotiation process on the EU Constitution for which we seek to measure actor preferences. We also set forth our basic theoretical assumptions, namely that actors hold and pursue political outcome preferences that they articulate through negotiations, and that these preferences will be manifest in their negotiating texts. Second, we describe our data, consisting of the Convention texts and the set of expert survey estimates of party positions used to validate the word-score estimates. Third, we briefly describe the computerized word-scoring methodology, and present in detail how it was applied to the Convention texts. Next, we present and discuss the results of the word-scoring and compare them with the set of expert survey estimates. Finally, we discuss the overall achievements of this research and draw conclusions for our ability to measure political actor preferences from their texts.

The Convention on the Future of Europe

During the Intergovernmental Conference held in Nice in 2000, the need was recognized to address substantial EU institutional and constitutional matters that were considered to have been left over from previous rounds of Treaty reform. In a 'Declaration on the Future of Europe', annexed as Declaration No. 23 to the Treaty of Nice (2001), EU member states agreed to initiate a deeper and wider debate within Europe. At an informal meeting of the European Council in Ghent in October 2001, general guidelines were agreed for the establishment of a Convention that would foster a constitutional debate on the future of the EU.

The momentum for a constitutional Convention continued in December 2001, when the European Council in Laeken laid out more precisely not only the goals and the objectives of such a Convention, but also the procedural details of when it should convene and how it should be constituted. According to the Laeken Declaration (2001), the following key issues were expected to be discussed during the Convention: the division of competences between the Union and its member states, the simplification of the Union's legislative instruments, the maintenance of the inter-institutional balance, an improvement to the efficacy of the decision-making procedures, the definition of the EU's role in an increasingly global environment, and the constitutionalization of the Treaties.

The European Council at Laeken appointed Valéry Giscard d'Estaing as chairman of the Convention, and Giuliano Amato and Jean-Luc Dehaene as vice-chairmen. Additionally, the Convention was constituted by the following members: 15 representatives of the heads of state or government (one from each member state), 30 representatives of the national parliaments of the member states (two from each member state), 16 members of the European Parliament (EP), 2 representatives of the European Commission, 13 government representatives of the accession and candidate countries (one from each country) and 26 representatives of the national parliaments of the accession and candidate countries (two from each country). Participating as observers were 3 representatives of the Economic and Social Committee, 6 representatives of the Committee of the Regions, 3 representatives of the social partners, and 1 person representing the European Ombudsman. In addition to the 105 full members, each member of the Convention, with the exception of the three chairmen, was allowed to appoint one alternate, bringing the total number of members to 204. Furthermore, during the course of the deliberations, many governments replaced previously appointed members with new ones, bringing the total number of members participating in the Convention at one point or another to a total of 275. These members collectively produced a substantive record of the contentious deliberations during the Convention in the form of documents, amendments and speeches, which were all made available publicly.

Underlying our attempt to estimate the policy preferences of Convention participants is not only the fact that actors' positions are recorded in the publicly available texts, but also the expectation that these texts will contain information about the sincere policy preferences of national political actors. Existing research on cleavage theory supports the idea that policy differences will divide parties not just at the national level, but also at the European level [START_REF] Marks | The Past in the Present: A Cleavage Theory of Party Response to European Integration[END_REF][START_REF] Hooghe | Does Left/Right Structure Party Positions on European Integration?[END_REF][START_REF] Gabel | Defining the EU Political Space[END_REF][START_REF] Pennings | The Dimensionality of the EU Policy Space: The European Elections of 1999[END_REF][START_REF] Thomson | Actor Alignments in European Union Decision Making[END_REF]. As the fundamental issues of the Convention dealt with the institutions of Europe, furthermore, we expect that different national party stances regarding the strength and scope of European powers will be directly reflected in national position-taking at the Convention [START_REF] Pennings | The EU Constitution and Positions on Governance[END_REF]. We also expect, although possibly to a lesser extent, general left-right policy differences to be evident in Convention texts. Existing theories indicate that, when actors evaluate their preferences on formal rules to be incorporated into a constitution, they do so by calculating the extent to which these will favour or hamper the realization of their most preferred policy goals [START_REF] Collignon | The European Republic: Reflections on the Economy of a Future Constitution[END_REF]. We therefore expect a correspondence between independently measured positions of national political parties on European and general left-right issues and positions taken at the Convention. The test applied here is whether the computerized content method of word-scoring can reliably extract this correspondence from the body of texts produced during the Convention.

Data Convention texts

The data used in this paper come from two principal sources. The first is the set of Convention texts, which were made publicly available on the official Convention website (http://european-convention.eu.int). We were able to access all the documents that were submitted by the Convention participants. 1 Since the Convention completed its work on 10 July 2003, we harvested the full set of the documents in September 2003 to make sure that our database could be presumed to be complete. As might be expected, the corpus of documents collected required cleaning up before it could be used. First, the names of the actors came in many variations, partly because of alternative spellings, differences in electronic character sets and national conventions. We wrote a canonicalization procedure which used disambiguating heuristics to match the 1018 name variations found in the database to the 275 known actors. From the same website, we also obtained information on the actors' nationalities and their national party affiliations. Of the total of 275 actors, we were able to match 197 to a national party. The remaining actors mostly either belonged to the Ministry of Foreign Affairs or were members of the diplomatic corps.

Second, in the meta-information in the Convention database, the language of the document was more often than not identified incorrectly. Therefore, we ignored the information about the language contained in the original database and instead used 'TextCat': an n-gram-based language identification tool that uses rank-order statistics to compare the prevalence of short letter sequences [START_REF] Cavnar | N-Gram-Based Text Categorization[END_REF][START_REF] Dunning | Statistical Identification of Language[END_REF][START_REF] Sibun | Language Identification: Examining the Issues[END_REF]. 2 This helped us to discover that the 6474 texts came in seven different languages (English, French, German, Italian, Spanish, Portuguese and Dutch; see Table 1). Because a clear majority of texts (60%) were in English, our analysis began by selecting only the 3893 English texts for further processing. This selection process was necessary because the word-scores method requires that all texts being analysed come from the same lexical universe, which means they must all be in the same language. In appendix A 3 we present evidence that the sample of authors providing English, singleauthored texts does not contain actors whose positions differ substantially from those in the entire collection of texts.

Concatenations of these files by national party were then processed for use by word scores. We concatenated the files because we view the natural text unit for analysis as being all of the text submitted by particular national party speakers, and we expect all of the speeches to represent a particular political viewpoint at the Convention. 4 Another choice to be made concerned how to handle texts submitted to the Convention by multiple authors. As can be seen in Table 2, of the English-language texts, some 64% were singleauthored, with the rest being handed in by two authors or more. For simplicity, we selected the 2493 single-authored texts for further analysis.

The texts came in six basic variations, as can be seen in Table 3. Substantively, the most important type was amendments, and these also represent the vast majority of documents submitted. In fact, the total of 2136 proposed amendments represent almost 86% of all English-language texts submitted by a single author. Given the fact that the overarching rationale for the Convention was to simplify and to constitutionalize the Treaties, it is not surprising that we should find that member states frequently, and strategically, made use of suggested amendments to express their policy preferences. Many of the amendments, in fact, had a fair amount of original text added by the actor submitting the document, in which a detailed explanation was given of why a certain Treaty article needed to be changed. Additional types of documents submitted to the Convention were 'contributions' (186) and 'speeches' (138).

Together, these two categories represent 13% of all English-language documents submitted by a single author.

Expert surveys

As a set of external reference points for the policy positions of the national parties whose Convention texts we examine in this paper, we use an expert survey of party policy positions conducted by Kenneth [START_REF] Benoit | Estimating Irish Party Policy Positions Using Computer Wordscoring: The 2002 Election[END_REF]. This survey, reported in Benoit and Laver (forthcoming), extended the [START_REF] Laver | Party and Policy Competition[END_REF] expert survey methodology to all of the 25 EU member states. In each of the countries surveyed, respondents were sent (by regular post or by e-mail) a questionnaire on the main political parties in their country, plus a list of approximately 12 policy dimensions on which they were asked to locate these parties. Respondents consisted of academic political experts, selected in each country from lists provided by national political science associations. In an effort to reach more respondents, Benoit and Laver translated the survey (originally in English) into the native language of 16 of the 25 EU countries, resulting in native languages (including English) being used in 18 of the 25 countries. Details on response rates and question items are provided in appendix A to this paper, as well as being described in full in Benoit and Laver (forthcoming).

The expert surveys characterized policy dimensions a priori, and asked respondents to locate political parties on a scale ranging from one extreme of this dimension to the other. Following the Laver-Hunt methodology, the Benoit-Laver expert survey used 1 to 20 point scales, where 1 generally corresponds to the 'left' position and 20 to the 'right'. In the context of the EU, 1 meant a pro-EU stance and 20 meant an anti-EU position. For this paper, we have relied mainly on a subset of these questions that relate specifically to the EU. These questions were the following:

EU Accountability

Promotes the direct accountability of the EU to citizens via institutions such as the European Parliament. ( 1 The expert survey also included a direct measure of left-right positioning, asking experts to locate parties on a general left-right scale as well as on a set of substantive policy scales. After providing an average of 10 other policy dimensions per country, the questionnaire further asked:

Left-Right

Please locate each party on a general left-right dimension, taking all aspects of party policy into account. Left (1). Right (20).

In what follows we will use the estimates of national party positions from these dimensions to provide an external validity check on the results of computerized word-scoring of each national party's Convention texts. If we can demonstrate a positive and significant correlation between the word-score estimates of a party's Convention position and the estimates of that national party's policy position from the completely independent expert survey, then this will be evidence that word scores can be successfully applied to measure the positions of political actors in forums such as the Convention on the Future of Europe.

Methodology: Computerized word-scoring

Having identified the Convention texts we will analyse here, we now set out to estimate the policy positions of the authors who wrote them. To do this, we analyse the texts using the computerized 'word-scoring' technique set out in [START_REF] Laver | Extracting Policy Positions from Political Texts Using Words as Data[END_REF], which contains a full description of what is involved, together with web links to downloadable software that will perform all of the data analysis required. 6 Briefly, the word-scoring technique is a method for estimating the position of 'virgin' (unknown) texts on a priori policy dimensions. Essentially, it does this by statistically comparing the patterns of word frequencies in the virgin texts under investigation with the patterns of word frequencies in a set of 'reference' texts from well-known sources. Once the analyst has access to external estimates of the positions of the reference texts on the a priori dimensions under investigation, as we have with our expert survey findings, or indeed is confident in being able to assume these, the word-scoring technique proceeds as follows.

First, the reference texts are analysed in order to calculate a matrix of the relative frequencies of all words in the word universe of the set of reference texts. This in turn allows the calculation of a matrix of key conditional probabilities, each element of which is the probability that a reader is reading reference text r given that he or she is reading word w. For any given a priori policy dimension for which the positions of the authors of each reference text can be either estimated or confidently assumed, this allows the calculation of a vector of 'word scores', each element of which is in effect an estimated policy position of text r, given that the reader is reading word w. The vector of word scores for any a priori policy dimension is thus a function of the policy positions and the patterns of relative word frequencies in the set of reference texts. Having calculated the vector of word scores for a given dimension from the reference texts, the analyst is now in a position to investigate 'virgin' texts, where no information whatsoever is available about the policy positions expressed in them. This is very simply achieved. The pattern of relative word frequencies in each virgin text is observed and this, combined with the vector of word scores for the dimension under investigation, allows the analyst to estimate the position of each virgin text on this dimension.

Because this technique is purely statistical, it has three great advantages over more traditional methods of text analysis. It requires no substantive judgement calls during the process of data analysis and is thus perfectly replicable; it operates in any language, not needing predefined coding dictionaries; and it generates an estimate of the uncertainty associated with any estimated policy position. The analyst's crucial expert role when using the word-scoring technique is at the research design stage -in identifying appropriate reference texts and in picking good estimates or assumptions about the positions of these texts on the a priori dimensions under investigation. If the reference texts are inappropriate or if their estimated or assumed policy positions are misleading, then this will produce misleading estimates of the positions of the virgin texts. The key, therefore, is to build on a solid foundation of well-chosen reference texts, with solid estimates or assumptions about policy positions.

Our selection of reference texts involves identifying a set of extreme pro-EU texts on the one hand, and a set of extreme anti-EU texts, on the other. Since our unit of estimation will be national political parties, we can use the Benoit and Laver (forthcoming) expert survey results to identify extreme parties on the four policy dimensions. Our method was therefore kept deliberately simple and data driven. For each dimension under investigation, a cluster of parties occupying the extreme positions on the expert survey dimensions was identified by using the 10th and 90th percentiles of dimension scores as cut-offs. On the a priori scale of 1 to 20, going from pro-to anti-EU, any text associated with a national political party scoring in the 10th or lower percentile of 'extremism' was used to create the pro-EU reference text. Likewise, any text associated with a national political party scoring in the 90th or higher percentile of extremism was used to create the anti-EU reference text. All such texts were concatenated into two sets of reference texts: one representing a pro-EU position and one representing an anti-EU position. 7 For each of our policy dimensions examined, appendix C of this paper provides full details on which speakers, parties and countries provided the texts to create these 'extreme' reference texts.

Reference texts were assigned scores on a metric of -1.0 to +1.0, with the pro-EU reference text receiving the -1.0 score, and the anti-EU text receiving +1.0. We then scored the words for each of the policy dimensions under investigation using the different reference texts. 8 We created two sets of scored words for each of these: one using the pair of concatenated reference texts (one on each extreme) consisting of amendments, and another pair of reference texts consisting of other (non-amendment) text types. Following the creation of the word scores, we then applied this to virgin texts contributed by national parties, paired similarly into amendment and non-amendment texts. Virgin texts consisted of concatenated texts of all speakers representing the same national political party. For example, for the EU Authority Dimension, there were three different speakers from the UK Conservative Party: Heathcoat-Amory, Kirkhope and Stockton (see appendix C). These speakers' texts were concatenated into two sets of texts to be scored, depending on text type: respectively 76, 134 and 65 proposed amendments from the three speakers were used to create a single concatenated text for the UK Conservative Party for amendments, and the respectively 6, 3 and 3 other types of text were used to create the 'other' text type of concatenated text. 9 Because of the possible concerns about different text types providing different lexical orientations, we separated virgin texts into two types: proposed amendments and all other texts. As stated previously, we expect amendments to contain more directed information about actors' policy preferences, given their focused purpose and more standard content relative to the much looser possibilities for other text types, which included contributions, speeches, press releases and miscellaneous other documents.

Our final step was then to compare the scalar estimates for each virgin text with the expert survey means for each national party, analysing amendments and other text types separately. We used both graphical and statistical methods for this evaluation, with statistics being simple OLS regression analysis. We do not expect to predict party scores perfectly -for a number of reasons -but we consider it a success for the word-scores technique if we observe a statistically significant, positive correlation between the word-score estimates and the expert survey estimates. In addition to cross-national tests comparing word-scores and expert survey estimates for all national parties, we also compared word-scores and expert survey rankings within each country.

Results

Pooled comparisons

If word scores can correctly identify the positions of national political parties as recorded in their Convention texts, then we would expect word-score estimates of national party texts to show a significant, positive relationship with the expert survey estimates that provide external and independent measurement of national party positions. Our first test thus pooled all national parties and examined cross-national correlations between the word-score and expert survey estimates. Figures 1234portray the associations for each of our four policy dimensions: EU Accountability, EU Authority, EU Security and general Left-Right preferences for socioeconomic policy. In each graph, the expert survey estimates (ranging from 1 to 20) are plotted against the raw wordscore estimates, set originally in a (-1, 1) interval, but bunched more narrowly towards an interval of approximately (-0.5, 0.5) because of overlapping words. The solid line in each plot shows the (OLS) linear relationship, and the shaded region depicts the associated 95% confidence interval.

We see immediately from Figures 1-4 that in each case there is indeed a positive and significant relationship between the national party texts and the associated word scores. The panel on the left shows the results for amendment texts, and the panel on the right for all other texts. In the case of the three EU policy dimensions, our expectation that amendment texts would show a better word-scores fit is confirmed, whereas the linear-fitting models for the Left-Right dimension are essentially the same between amendments and other text types. The poorer fit of the Left-Right word scores to the expert survey estimates (Figure 4) also suggests that there is less direct information about this policy dimension in the Convention texts than exists for EU-related policy.

The linear fits and confidence regions suggest a strong association between the virgin text scores and the quite independent expert survey estimates of the positions of the authors of the virgin texts, something we can examine further by looking at the OLS regression results presented for each dimension in Table 4. From this table we see numerical versions of the relationships portrayed in Figures 1234. All coefficients are positive and statistically significant, with the strongest coefficient estimates coming from the amendment texts for the EU Security and EU Accountability questions, both estimated at .0207. This coefficient means that a movement from the minimum to the maximum of the expert survey scale -from 1 to 20 -is associated with a .41 change in the word-score value, which is a substantively significant shift considering the range for all word-scores estimates is within the (-0.45, 0.45) interval. The overall fit of each model as measured by the R 2 values for each dimension is also remarkably high, ranging from .56 for amendment texts on the EU Security dimension, to .47 and .44 for EU Accountability and EU Authority respectively. As expected, the linear-fitting models for proposed amendment texts were better than for other text types for every EU dimension.

Word-score estimates for Left-Right positions also show a positive, statistically significant relationship, although the slopes were of a lower magnitude (.0168 for both text types) than for the typical corresponding EU dimension. Linear fit as measured by the R 2 values was also typically lower, with expert survey values explaining only about 36% and 32% of the variation in word-score estimates for amendments and other text types, respectively. This suggests either less information about Left-Right policy positions in the Convention texts, or alternatively that national party positions on substantive policy matters may be somehow modified when represented in supranational bodies. The latter possibility is something we return to briefly in our concluding remarks.

Our overall assessment of these results is positive: the strong statistical associations between the word-score estimates of Convention texts and completely independent expert survey estimates of national party policy positions serve as compelling evidence that, on the whole, the computerized word-scoring method has the ability to measure policy positions of political actors in a cross-national context. For the most part in discussions on the future EU Constitution, representatives of national political parties held preferences very similar to those one would expect them to hold based on their policy stances expressed in the national context. There are, however, some notable outliers, which it would be interesting to investigate further. For instance, in the case of the PDS (Party of Democratic Socialism) in Germany, the party was able to send only one representative, namely Yvonne Kaufmann. It would perhaps be instructive to explore the political substance behind these outliers by looking into the type of working group in which each delegate chose to be particularly active.

Within-country comparisons

Since estimates of party policy positions may not necessarily be directly comparable across different national contexts, we have also compared withincountry rankings of the two independent estimates. For each country, we compared the ranking from pro-to anti-EU (or left to right) for the parties for which both word-scores and expert survey estimates existed, and summarized the correspondence in rankings using Spearman's rho, the ordinal equivalent of Pearson's R and, like Pearson's coefficient, ranging from -1.0 to +1.0.

Table 5 summarizes the results of the within-country rank associations. For each dimension analysed in this paper, we have provided a frequency distribution for ranges of Spearman's coefficient, listing the number of countries scoring in this ranking. The total number of countries compared indicates how many different countries were involved in the overall frequency breakdown, ranging from 16 countries for the Left-Right dimension to 10 for EU Accountability. The country median rho reports the median Spearman coefficient for the countries compared, indicating the typical value. For amendment texts, these values were exceptionally high, ranging from 1.0 for EU Authority and the Left-Right scores, to 0.85 and 0.75 for EU Accountability and EU Security, respectively. This indicates that the typical ranking of national party amendment texts by word scores and by the expert surveys is correlated between 0.75 and perfectly, depending on the policy dimension. Similar median correlations for the other text types were also high, ranging from 0.50 for the Left-Right dimension to 1.0 for EU Accountability.

Of course, different numbers of parties were being compared in different countries, and this affects the probability of rank correspondence occurring by chance. 10 To provide a sense of the number of parties being compared, we report the mean number of parties used for comparison. In most countries, about 3 parties were compared, with a range of 2 to 5 parties.

Our strong results here on the correspondence of intra-country party rankings generated by word scores and independent expert surveys reinforce our findings from the cross-national comparison. It is also remarkable that the median Spearman coefficient for 16 country rankings compared for the Left-Right dimension was (for amendments) 1.0, indicating a high degree of correspondence in the word scores and expert survey rankings of this substantive policy dimension. Overall, word scores provide a remarkably accurate way to reproduce national country rankings from pro-to anti-EU and from left to right, as compared with and validated by external and independent expert survey estimates.

Conclusion

Substantively, the Laeken process that led to a draft EU Constitution is of considerable importance for the EU's future development. A large and diverse set of actors, from many different countries and starting with a varied set of policy preferences, came together to negotiate a common document. Many of these actors left traces on the historical record, in the form of texts that were input into the negotiation process, and that we can take to contain information about their policy preferences. Analysing this complex body of texts in a systematic way presents some formidable methodological challenges. It is to these challenges that this paper has been addressed. The hope is that, if we can find appropriate techniques for analysing these texts, then we can find systematic ways to interrogate this potentially very valuable data source about the preferences of the actors involved in negotiating the future of Europe. Statistical techniques of computerized content analysis such as word scores offer particularly large payoffs when the research task involves analysing very large numbers of texts in a systematic way, as is precisely the case in relation to negotiations during the Laeken Convention.

This article thus sought to investigate the potential of computerized word-scoring in analysing the large number of texts associated with this complex international negotiation. As a methodological contribution, our paper set out to explore whether the initial positive evaluations of computerized word-scoring could be replicated in the context of international negotiations. The results are resoundingly positive, and suggest not only that extraordinary gains in terms of time, effort and expense are possible using the word-scoring method of computerized content analysis, but also that this method is capable of extracting valid measures from negotiating texts such as those generated from the Laeken Convention.

As a substantive contribution, our analysis explored whether systematic order could be brought to the empirical analysis of the diverse set of Laeken texts. The acid test in all of this was to see whether computerized wordscoring of the Laeken Convention texts could replicate the judgements of country specialists in relation to national party positions on four key policy dimensions -three related specifically to the EU (on accountability, authority and security) and a more general left-right dimension. Our analysis in this paper suggests that information on the policy preferences of the authors of the Convention texts, retrieved using computerized word-scoring, can be cross-validated against information on the policy positions of the same authors as estimated using independent expert surveys. In general crossnational comparisons, strong positive results were observed for word scores, particularly when analysing proposed amendments, with independently measured expert survey scores explaining between 44% and 56% of the variance in the word-score estimates for the EU dimensions of policy.

Our results clearly show that word-scoring of the Laeken documents was able to retrieve the left-right positioning of national political parties, although with greater prediction error than for their policy positions on various aspects of the EU. Expert survey estimates of left-right positioning explained only 36% of the variation in word-score estimates (for amendment text types). This suggests two possibilities. One is that the Convention texts lacked sufficient textual information about left-right policy positions of the parties -because the negotiations concerned matters that were not systematically structured in left-right terms. The other is that the parties revealed their 'true' preferences on the EU in the Laeken amendments that they proposed but did not reveal their left-right positions in such a systematic way -or indeed felt differently about left and right in the EU context.

The core purpose of the present paper has been methodological, its aim being to assess the value of computerized word-scoring in this complex and data-rich international context. If we accept that word-scoring does have some value in this context, then the way opens up for it to put to substantive use. It also suggests the strong future gains to be realized through applying computerized content analysis techniques such as word scores to fields where traditional hand-coded content analysis is infeasible and unreliable. For example, word scores could be used to trace potential movements of the policy positions of particular actors during the course of the negotiations, testing the conjecture that these positions moved closer together as the actors moved towards agreeing a common text. Alternatively, they could be used, within individual countries, to estimate the policy positions of non-party political actors, such as trade unions and interest groups, whose policy texts could be treated as virgin texts and scored on the same policy dimensions as part of an exploration of domestic constraints on the international bargaining process. Our initial findings are encouraging and suggest that such applications could indeed be attempted in the expectation of valid results.

Notes

This research was supported by the European Commission Fifth Framework (project number SERD-2002-00061) and by the Institute for International Integration Studies, Trinity College, Dublin. We thank Ben Crum for his advice on the positions taken by Convention delegates, and Christian Grose, Gail McElroy and two anonymous reviewers for comments.

1 To 'spider' automatically through the Convention website and to harvest all the documents with the corresponding meta-information, we wrote customized scripts in the computer language Perl. 2 We converted all the documents to ISO-8859-1/ASCII using 'pdftotext' in a Linux environment and ran the language identification tool through all of them. 3 The Appendices can be found on the EUP webpage. As indicated by Table 1, the largest non-English text set was in French. These texts tended to come primarily from Belgian and French sources, although Belgian and French sources also recorded substantial English-language texts: 273 English versus 985 French texts for France, and 303 English texts versus 1001 French texts for Belgium. 4 It would of course be possible to analyse each text separately, to see (for instance) whether national party positions changed over time as the Convention proceeded. This analysis would be significantly more complex than the one attempted here, however, given the thousands of texts involved. Many individual texts are also too short to permit reliable analysis. 5 Two closely related questions were also coded as EU Authority: (Ireland only)

Favours a more powerful and centralized EU (1). Opposes a more powerful and centralized EU (20). (France only) Opposes an expanded and stronger EU (1). Favours an expanded and stronger EU (20). 6 A full replication data set for this paper, including log files with full results of the word-scoring, is available from URL: http://www.politics.tcd.ie/ wordscores/Laeken.html. 7 In fact, two reference texts were created for each extreme, one for proposed amendments and one for all other text types. Both text types were given identical reference scores. 8 One difference between the analysis of the EU dimensions and the usual Left-Right dimension is that expert survey estimates for the EU dimensions were available only for the 15 EU member states existing prior to 1 May 2004. The 10 accession countries were asked only a question about EU joining, and this question was not posed to the experts from 15 non-accession states.

Because the accession states provided relatively few texts, there is not sufficient variation or information about EU accession to support a wordscores analysis of that dimension on the basis of the Convention documents. 9 Because the UK Conservative Party was also identified as an anti-EU reference party for this dimension (see appendix C), these two concatenated texts for the UK Conservative Party were also used in creating the anti-EU reference texts (one for amendments and one for other text types) for the EU Authority dimension. The choice to analyse reference texts individually again as virgin texts allows us to place reference text parties relative to other virgin text parties. Because raw virgin text scores are computed independently from one another, the decision to include reference texts in no way affects the estimates for non-reference virgin text parties. 10 For instance, two rankings of a pair of parties have a .5 probability of perfect correspondence by chance. The number of parties compared in each country mainly differs widely because, for each dimension and each text type, different parties had both submitted texts and been scored on the expert survey dimensions.

  ) Promotes the indirect accountability of the EU to citizens via their own national governments. (20) EU Authority 5 Favours increasing the range of areas in which the EU can set policy. (1) Favours reducing the range of areas in which the EU can set policy. (20) EU Security Favours <Country> involvement in European security and peacekeeping missions. (1) Opposes any <Country> involvement in European military affairs. (20)

Figure 1 Figure 2 Figure 3 Figure 4

 1234 Figure 1 Raw word-score estimates compared with expert survey estimates for national parties: EU Accountability dimension. Note: Extremism is 10th, 90th percentile, 200-word minimum. The party names and abbreviations are listed at URL: http://www.politics.tcd.ie/ppmd/.

Table 1

 1 Texts by language

	Language	Number of texts	%
	Dutch	14	0.22
	English	3893	60.11
	French	1425	22.01
	German	719	11.11
	Italian	190	2.93
	Portuguese	115	1.78
	Spanish	118	1.82
	Total	6474	100.00

Table 2

 2 Authorship of English-language texts

	Number of authors	Number of texts	%
	1	2493	64.04
	2	437	11.23
	3	120	3.08
	4	223	5.73
	5	127	3.26
	6	216	5.55
	> 6	277	7.11
	Total	3893	100.00

Table 3

 3 Text type for English, single-authored texts

	Text type	Frequency	%
	Proposed amendments	2136	85.8
	Contributions	186	7.5
	Speeches	138	5.5
	Documents	17	0.7
	Press and information	10	0.4
	Related documents	3	0.1
	Other	3	0.1
	Total	2493	100.00

Table 4

 4 Regressions of raw word-score estimates on expert survey positions of national parties

	Text type

Table 5

 5 Summary of within-country word-scores results: Rank-order correlations of word-scores rankings versus expert survey rankings within countries

		EU Accountability	EU Authority	EU Security	Left-Right
	Spearman's R	Freq.	%	Freq.	%	Freq.	%	Freq.	%
	Amendments
	-1.00--0.50	1	10.0	1	7.7	3	25.0
	-0.50-0.00							1	6.3
	0.00-0.25
	0.25-0.50							1	6.3
	0.50-0.75	2	20.0	2	15.4	3	25.0	3	18.8
	0.75-0.95	3	30.0	3	23.1	2	16.7	2	12.5
	1.00	4	40.0	7	53.9	4	33.3	9	56.3
	Total countries	10		13		12		16
	Country median rho	0.85		1.00		0.75		1.00
	Mean parties compared	3.1		3.1		3.2		3.0
	Other texts
	-1.00--0.50	2	18.2	3	23.1	1	8.3	5	31.3
	-0.50-0.00	2	18.2			2	16.7
	0.00-0.25			1	7.7			1	6.3
	0.25-0.50
	0.50-0.75	1	9.1	2	15.4	2	16.7	4	25.0
	0.75-0.95	0	0	1	7.7	2	16.7	1	6.3
	1.00	6	54.6	6	46.2	5	41.7	5	31.3
	Total countries	11		13		12		16
	Country median rho	1.00		0.80		0.85		0.50
	Mean parties compared	3.1		3.1		3.1		2.8

Benoit, Laver, Arnold, Pennings and Hosli Measuring National Delegate Positions

European Union Politics 6(3)

About the authors


Simon Hix
email: s.hix@lse.ac.uk

Christophe Crombez
email: christophe.crombez@econ.kuleuven.ac.be

Extracting Ideal Point Estimates from Actors' Preferences in the EU Constitutional Negotiations

This article uses the Domestic Structures and European Integration (DOSEI) data set of actors' preferences on 65 issues in the 2003-4 Intergovernmental Conference to extract the underlying preferences of the governments, the Commission and the European Parliament on the main dimensions of conflict in the European Union's constitutional negotiations. The analysis starts by comparing the ideal point estimates produced by three 'inductive' techniques: exploratory factor analysis, NOMINATE and Optimal Classification. The results are a series of ideal point estimates that do not correlate well with some simple a priori assumptions about key actors' positions on the reform of the EU. The analysis then proceeds with a 'mixed' deductive/inductive method, in which responses to the survey questions relating to the two exogenous dimensions of constitutional design in a multi-level polity (the 'vertical' and 'horizontal' allocation of power) are used to generate ideal point estimates on these two dimensions. The result is a more intuitive set of ideal point estimates for the 28 main actors in the negotiations.

Introduction: Towards a 'mixed method' of ideal point estimation

There are multiple ways of inducing ideal point estimates from survey data. One of the most common methods is factor analysis, which uses the correspondence between actors' preferences on different issues to extract the underlying dimensions of preferences. A different method is to recode each preference on a given question as a binary 'item response', and then apply a nominal scaling method to extract ideal point estimates, such as NOMINATE or Optimal Classification. Nevertheless, these techniques are essentially 'exploratory', or inductive, since no a priori information about the expected relationship between certain issues is used to structure the extraction of dimensions from the data. The danger of such methods is that the extracted ideal point estimates may not match intuitive expectations, because responses on a small number of key issues can be 'polluted' by random responses on a larger number of irrelevant issues.

Alternatively, a purely deductive method could be used. For instance, a small set of exogenous issue dimensions could be identified, and the actors' positions on the issues that relate to these dimensions could be added together to create ideal point estimates for the actors on these dimensions. This is the method used by the Americans for Democratic Action (ADA) to create 'scores' for members of the US Congress from their voting behaviour on the issues the ADA cares about in American politics. 1 A problem of such purely deductive methods, however, is that they do not allow actors' positions on different issues to be weighted by how far these issues correlate with the underlying dimension in the calculation of ideal point estimates.

Consequently, 'mixed' methods employ a combination of deduction and induction. On the deductive side, a small number of expected dimensions and the specific issues relating to these dimensions can be identified. Then, on the inductive side, in the construction of ideal point estimates, the actors' positions on these issues can be weighted according to how the positions on these issues relate to the main extracted dimension. 'Confirmatory' factor analysis does something similar, by allowing for a set of expected dimensions to be identified, and then seeing how far the issue preferences accord with the expected dimensionality of the issue space.

In this article we try to identify the location of the EU governments, the Commission and the European Parliament on the main dimensions of conflict in the 2003-4 Intergovernmental Conference (IGC) negotiations on the draft constitutional Treaty. We use the positions of these actors on 65 issues in the IGC, as measured in the Domestic Structures and European Integration (DOSEI) data set. We then identify two exogenous dimensions of constitutional design, which can be expected to be central conflicts in the design of an EU Constitution: the allocation of power between the centre and the states (which we call the 'vertical dimension'), and the allocation of power between the central institutions (which we call the 'horizontal dimension').

In the third section we compare the results produced by applying three 'inductive' techniques: exploratory factor analysis, NOMINATE and Optimal Classification. We then employ a mixed method similar to confirmatory factor analysis -but, whereas confirmatory factor analysis aims to identify the 'best fit' for describing the dimensionality of a space, our method is designed to extract ideal point estimates on our two exogenous dimensions (Bentler, 1990). After briefly comparing the ideal point estimates produced in the previous sections, we offer a short conclusion. Lijphart (1999), inter alia, identifies two essential dimensions of the design of democracies, each relating to who has the power to make and enact policy (Powell, 2000; Taagepera, 2003). First, a vertical dimension relates to the degree of centralization of policy competences. At one extreme, all major areas of public policy could fall within the remit of the central institutions, as in a unitary system of government (such as The Netherlands). At the other extreme, the powers of the central institutions could be restricted to a small set of policy competences while the states maintain exclusive rights over many important policy areas and a strong constitutional court has the power to police competence conflicts, as in a decentralized federal system of government (such as the United States).

Vertical and horizontal dimensions of constitutional design

Second, a horizontal dimension relates to how far a particular political majority is able to enact its policy preferences, for example by forming the executive and controlling a majority in the legislature. At one extreme, a polity can have a single agenda-setter and veto-player, as in the classic 'Westminster' model of single-party government, executive dominance of the legislature and a weak second chamber. At the other extreme, a polity can be designed to have multiple veto-players, either through proportional representation and coalition governments (particularly where a grand coalition is prevalent), or through a separation of powers between the executive (President) and the legislature (Congress), or through 'strong bicameralism', or any combination of these factors (e.g. Tsebelis, 1995, 2002).

These designs determine how easy or difficult it will be to make and change policies. In a centralized/majoritarian model (such as the United Kingdom), the central executive has enormous power to change the policy status quo. Conversely, in a decentralized/consensual model (such as Switzerland), the policy competences of the central government are limited, and the executive must construct a large majority to change any policy within its remit. In a centralized/consensual model (such as Finland), the central government has the potential to change any policy, but is constrained by the need to construct an oversized coalition to support policy change. Or, in a decentralized/majoritarian model (such as Canada), the central government can change the policy status quo relatively easily, but in only a limited set of policy areas.

Because decentralized or consensual institutions represent constraints on the power of the central government, constitutional design preferences on these two dimensions often correlate. On one side, an actor who would like to allow for an electoral majority to enact radical policy change (such as a socialist party) is likely to advocate a centralized/majoritarian constitutional design. On the other side, an actor who would like to limit the power of government as much as possible (such as a free market libertarian) is likely to support a decentralized/consensual constitutional design. Nevertheless, the intermediate constitutional mixes may also be consistent with underlying policy preferences. For example, an actor may prefer policy change, but may expect that he or she is more likely to secure change in his or her preferred direction at the local level than at the central level, and so advocate a decentralized/majoritarian model. Equally, an actor may not want policy change, but might feel constrained at the domestic level by policy externalities from neighbouring states, and so prefer a centralized/consensual model.

These four basic constitutional design preferences are present in the debate about the EU Constitution, as Figure 1 illustrates. In the centralized/ majoritarian corner is the classic 'Euro-federalism' model, where major policy

Horizontal allocation of power

Consensus government

Majoritarian government

Centralized

Union of nation-states (Norman, 2003).

In the opposite decentralized/consensus government corner is the classic 'intergovernmentalist' model, where only a limited set of policy competences would be allocated to the European level. The governments in the Council would maintain a monopoly on decision-making power over these policies, decided predominantly by unanimity. The European Parliament would be weak. This position has been held most consistently by the United Kingdom. Despite Tony Blair's self-proclaimed pro-European credentials, the British government representatives in the Convention and IGC favoured new constraints on the exercise of policy competences at the EU level (where national parliaments could raise a 'flag' if they were concerned about a breach of the subsidiarity rules), declared a number of 'red line' areas where unanimity voting in the Council would have to be maintained, and opposed any major extension of the powers of the European Parliament (for example over the budget).

In the centralized/consensus corner is a mixed model of government for the EU, which is perhaps best characterized as 'centralized intergovernmentalism'. This position has been held most consistently by France, at least since the Hague Summit in 1969, after Charles de Gaulle was no longer the French President. For instance, in the Convention and IGC, Jacques Chirac argued for a 'Union of Nation-States', where the policy competences of the EU would be broadly defined (including macroeconomic management and space exploration!), but decision-making power would reside in the Council and the European Council and the Commission and the Parliament would remain weak.

Finally, in the decentralized/majoritarian corner is a model of the EU that can be described as 'decentralized federalism'. This position is hard to identify in the debates on the various treaty reforms since the mid 1980s. This is because the negotiations on the Single European Act, the Maastricht Treaty, the Amsterdam Treaty and the Nice Treaty were more about what policy competences should be added to those already existing at the EU level rather than what policy powers should be handed back to the member states. Nevertheless, in the negotiations on the constitutional Treaty, several of the new member states seemed to favour this sort of design for the EU, with support for a powerful Commission and Parliament against the Council (which is likely to be dominated by the big member states), but new restrictions on the policy competences of the EU, for example with powers for national parliaments to challenge breaches of a new catalogue of competences.

In other words, it is reasonable to assume that the governments' preferences on the specific issues in the IGC negotiations reflected underlying preferences on these two basic dimensions of constitutional design. We shall first investigate whether this was the case with exploratory factor analysis before applying a mixed deductive/inductive method.

Three inductive techniques

The input data for our analysis are the positions of the 26 EU governments (the 25 governments that were there at the start of the negotiations plus the new Spanish government elected in March 2004) together with the Commission and the European Parliament on 65 issues in the IGC, as measured in the DOSEI data set. The data were collected through face-to-face interviews with several actors in each of the EU government administrations, the Commission and the European Parliament. Each issue in the survey represented an ordinal three-point or five-point (Likert) scale, and interviewees were asked to locate the position of their government/EU institution at one point on each of the 65 scales. 2 To extract ideal point estimates for these actors we start by applying three inductive scaling techniques: exploratory factor analysis, NOMINATE and Optimal Classification. Exploratory factor analysis is a widely used technique, which can be applied to these sorts of ordinal response data without any recoding. In contrast, NOMINATE and Optimal Classification were originally developed for the study of roll-call voting in parliaments, and are therefore designed for binary, Yes/No, responses. Hence, to apply these methods to our survey data, the data needed to be recoded into a series of binary response items. For example, an issue where actors are located at five possible positions on a scale (1, 2, 3, 4 and 5, say) is transformed into five separate binary items (1Y/1N, 2Y/2N, 3Y/3N, 4Y/4N, 5Y/5N). Recoding the data in this way transformed the 65 scales into 156 binary response items.

NOMINATE utilizes a parametric and stochastic model, and recovers information about individual legislator and roll-call vote characteristics by exploiting the assumption of the probabilistic spatial model that some errors are more likely than others. The method also assumes that errors are distributed according to a logistic function, and that errors are independent and identically distributed across both legislators and votes. Developed originally by Keith Poole and Howard Rosenthal (1997) for the study of voting in the United States Congress, NOMINATE has rapidly become one of the standard techniques for estimating actors' ideal points in parliaments and assemblies across the world (Voeten, 2000; Hix, 2001; Schonhardt-Bailey, 2003).

Keith Poole (2000) subsequently developed Optimal Classification as an alternative to NOMINATE. This method does not rely on assumptions about errors to derive ideal point estimates from binary response data. Instead, ideal points are derived by minimizing classification errors. A classification error for an actor on an issue occurs when the legislator's ideal point is such that his or her vote is inconsistent with the cutting-line on the issue (the line that splits the Yes camp from the No camp). The advantage is that no single error or group of errors can bias the overall ideal point estimates. On the downside, though, using a non-parametric method allows only a rank order of actors' ideal points to be recovered. Nevertheless, as Rosenthal and Voeten (2004) show in the case of the French National Assembly, Optimal Classification can produce more reliable ideal point estimates than NOMINATE in environments where voting/preference expression errors are not randomly distributed, for whatever reason.

One problem with both NOMINATE and Optimal Classification, however, is that the substantive meaning of the dimensions produced by these methods is difficult to interpret. With parliamentary roll-call voting data, the usual method is to look at the angle of the 'cutting-line' of some of the key votes. For example, if the cutting-line of most socioeconomic votes crosses the first dimension at approximately 90 degrees, then the first dimension is assumed to represent a set of left-right preferences. These methods produce cutting-line angles for each of the recoded response items from the survey. However, because these items are so specific, as a result of the recoding, it is virtually impossible to interpret the meaning of these angles. Hence, the only way to interpret the substantive content of the dimensions produced by NOMINATE and Optimal Classification in the analysis is to look at the relative locations of the actors on each dimension, and apply some exogenous knowledge about the underlying preferences and interests of the actors.

NOMINATE and Optimal Classification have not been widely applied to survey data. However, preferences expressed in surveys are not inherently different from preferences expressed in parliamentary voting. Hence, it is reasonable to assume that results produced by NOMINATE and Optimal Classification can be compared with results produced by exploratory factor analysis.

Exploratory factor analysis

Starting with factor analysis, we applied principal components factor analysis, with varimax rotation to maximize the degree of fit on the first extracted factor. The first result to note is the high dimensionality of the issue space revealed by this method. Specifically, the first factor explained 11.2% of the variance, the second factor an additional 8.4%, the third factor an additional 7.1%, and the fourth factor an additional 6.6%. Altogether, it took 13 factors to explain more than 75% of the variance in preferences on the 65 issues in the IGC negotiations.

Table 1 shows the loadings for the actors' preferences on each issue on the first four factors extracted using this method. 3 The 10 issues that loaded highest on the first factor were (in declining order of loading on this factor): majority voting on tax harmonization; majority voting on foreign policy; the organization of EU defence; majority voting on defence policy; enhanced cooperation; migration and asylum policies; majority voting on social security; majority voting on social policy; the design of qualified majority voting (QMV); and the initiation of legislation by the European Parliament (which loaded negatively). These were all highly sensitive and divisive issues in the negotiations, where actors seemed to take sides according to whether they favoured a centralized/majoritarian model or a decentralized/consensual model. The former group favoured a common European foreign, defence and migration policy, more majority voting, and more powers for the European Parliament, whereas the latter group favoured the opposite. Nevertheless, several other high-profile issues do not load on this factor, such as the allocation of policy competences between the EU and the member states, whether there should be a single chair of the European Council, or how far the co-decision procedure should be extended.

The second factor also seems to have an identifiable substantive meaning, as five of the 'level of competence' issues load highest on this factor: whether structural and cohesion, agriculture, environment, health, and education policies should be exercised at the EU or national levels. But the third and fourth factors are harder to interpret. The two issues that load highest on the third factor -subsidiarity and the election of the Council President -do not seem to have any necessary connection, as the first issue is central to the vertical allocation of power in the EU polity and the second issue is central to the horizontal allocation of power. Similarly, the four issues that load highest on the fourth factor seem completely unrelated: the Charter of Fundamental Rights; majority voting on structural and cohesion policies; the external representation of the EU; and the budgetary rights of the European Parliament. Despite the seemingly logical meaning of the first two factors, the ideal point estimates produced by these dimensions do not seem to accord with a priori intuitions about the relative location of the key protagonists in the IGC. Using the loadings of all the issues on the first two factors to generate ideal points produces a set of actor locations as described in Figure 2. The preference-outliers -Czech Republic, Latvia, Slovakia, Malta, Cyprus, Greece, Italy and Portugal -seem rather strange, because none of these states was reported in the press as having particularly difficult preferences to accommodate. Conversely, the actors whom we described as classically representing extreme positions on either the vertical or the horizontal dimension of design (the United Kingdom, France, Belgium or the European Commission), and the two member states that almost derailed the negotiations because of their preference for the Nice status quo on voting weights in the Council (Spain and Poland), are all centrally located in this set of ideal point estimates.

NOMINATE and Optimal Classification

Applying NOMINATE to the 156 recoded binary issues also reveals a multidimensional issue space. The set of ideal points on the first dimension produced by NOMINATE correctly predicts 77.7% of responses by the 28 actors, and the second dimension correctly predicts an additional 3.9% of responses. 4 These sound like relatively high figures. But, the percentage of responses explained under NOMINATE cannot be compared directly with the percentage of variance explained by factor analysis. Moreover, these percentages are in fact rather low. For example, the first dimension of NOMINATE explains more than 85% of vote decisions in the US Congress and European Parliament and more than 90% of decisions in the United Nations General Assembly. In other words, this NOMINATE result supports the result from the exploratory factor analysis -that inductive techniques suggest a high degree of dimensionality in the IGC issue space.

Figure 3 shows the ideal point estimates produced by the application of NOMINATE to the IGC issue data. As we discussed, unlike factor analysis, it is difficult to interpret the substantive meaning of each of the dimensions produced by NOMINATE. Nevertheless, the location of Belgium, Germany and the two supranational institutions (the Commission and the European Parliament) at the extreme of the first dimension suggests that this dimension might relate to general attitudes of actors towards European integration. Also, this dimension may have something to do with the battle between big states and small states, because all the large states except Poland are towards the right end of this dimension.

Nevertheless, several actors' positions seem rather surprising. For example, the United Kingdom is located relatively close to the supranational institutions on both dimensions and a long way from actors that one would think have similar preferences on EU reforms, such as Denmark, Sweden or Ireland. Similarly, France is close to the supranational institutions on the first dimension, and close to highly integrationist states such as Belgium and Italy on both dimensions. Spain and Poland also appear in rather centrist positions on both dimensions, as they did in the figure produced by the exploratory factor analysis. Optimal Classification does somewhat better than NOMINATE in the extent of actors' preferences classified correctly by two dimensions -although one must be careful in comparing the variance reduction statistics of these two methods. Using Optimal Classification, the first two dimensions of ideal point estimates classify 86.4% of actors' preferences on the 156 response items in the recoded survey data. 5 Nevertheless, this is a comparatively low figure compared with the application of Optimal Classification to other parliaments, where over 90% of decisions (or preferences) are routinely classified correctly by two dimensions using this method (Poole, 2000). In other words, even Optimal Classification, which, unlike the other two methods, primarily aims to maximize the number of decisions classified correctly, still fails to capture about 14% of actors' preferences in two dimensions.

Figure 4 shows the ideal point estimates produced by the application of Optimal Classification to the IGC issue data. Not surprisingly, given the similar underlying metrics of Optimal Classification and NOMINATE, this some interesting differences in the relative locations of some of the actors. In the Optimal Classification figure, the Commission and European Parliament are not at the extreme on the first dimension; Germany, Belgium, Slovakia and Greece appear to have more extreme positions. Also, Poland occupies a relatively extreme position (although the first Spanish government does not), and France and Germany appear closer together, which could perhaps be expected given the attempts by these two states to coordinate their positions in the IGC negotiations. Three of the small new member states -Latvia, Lithuania and Malta -appear to have extremely similar positions. Nevertheless, the positions of the United Kingdom and France, close to the centre of the figure, do not fit our a priori assumptions. Again, as with NOMINATE, it is difficult to say much about the likely substantive content of the dimensions produced by Optimal Classification by simply 'eye-balling' the figure.

In sum, the application of these three inductive techniques to the preferences of actors in the IGC clearly illustrates that ideal point estimation is as much about art as about science. There are some similarities between the estimates produced by the three methods. But, there are also some striking differences, such as the position of the United Kingdom in the three figures -towards the bottom left in Figure 2, towards the bottom right in Figure 3, and almost exactly in the middle in Figure 4. Also, none of the three sets of inductively derived ideal point estimates accords well with our prior theoretical assumptions about the likely shape of the issue space in the IGC negotiations.

One possible reason for the rather strange set of ideal point estimates generated by these methods is that a large number of issues/items in the data set have a relatively low salience for many of the actors. All these methods, however, treat every preference equally. Hence, even if an issue is largely irrelevant, the location of actors on this issue is still used in the generation of the ideal points. This is not a problem if only a small number of issues is of low salience. But, where there is a high number of low-salience issues, as is the likely case here, the accumulated 'noise' from these issues will drown out the effect of the high-salience issues in the construction of the ideal point scores. As a result, a more deductive method is likely to generate more meaningful ideal points.

Mixed deduction/induction: Factor analysis of two separated dimensions

Building on the theoretical discussion of the likely underlying dimensions of constitutional design, we can separate the issues in the IGC into issues that relate to the (vertical) allocation of power between the EU and the member states and issues that relate to the (horizontal) allocation of power between the member states (in the Council) and the supranational institutions (the Commission, European Parliament and European Court of Justice).

The vertical dimension of constitutional design clearly covers issues related to the definition and policing of the competences of the EU (the Charter of Fundamental Rights, subsidiarity, the objectives of the Union, the religious reference, external representation and whether a particular policy competence should be allocated to the EU or the national level), and issues that define the extent of EU action in a particular policy field (the Stability and Growth Pact, defence policy, external borders and migration and asylum). Similarly, the freedom of states either to pursue further integration (enhanced cooperation) or to leave the EU (right to withdraw) also relates to this dimension, because it is about the rights of states vis-à-vis the centre. Finally, the battery of issues relating to majority voting in the Council should also be included as part of the vertical dimension (as well as the horizontal dimension) because the movement from unanimity to majority voting is the equivalent of shifting a policy competence from the national level to the European level, because a member state could veto any move from its domestic policy status quo under unanimity voting.

The horizontal dimension, in contrast, covers the nature of representation in, and appointment of, the European institutions (the design of QMV in the Council, the nature of the Presidency of the Council, the election of the Council President, the appointment of a Foreign Minister, the number of Commissioners, the appointment of the Commission President and the Commissioners, and the role of the European Parliament in the investiture of the Commission). The horizontal dimension also covers the powers of the European Court of Justice and the powers of the European Parliament (the policy areas covered by the co-decision procedure and the budgetary powers of the Parliament) and who has the right to initiate legislation (the Commission, the Parliament, the Council, national parliaments or EU citizens). Finally, the extension of qualified majority voting in the Council should also be included in the horizontal dimension because the move from unanimity to QMV in a particular policy area increases the agenda-setting power of the Commission and Parliament relative to the member states, and increases the likelihood that a policy status quo can be changed. In other words, the 12 issues in the data that relate to the extension of qualified majority voting are the only issues that we think clearly relate to both the vertical allocation of power (between the member states and the EU institutions) as well as the horizontal allocation of power (in terms of the ability to block or change policies at the European level).

Table 2 presents the results of an exploratory factor analysis of the preferences of the 26 governments and the two EU institutions on the 36 issues identified as relating to the vertical dimension of constitutional design, again using the principal components factor method and varimax rotation. 6 The first factor explains 18.4% of the variance in preferences on these issues and the second factor explains a further 14.0%. Hence, these two factors together explain almost one-third of the variance in preferences on this subset of issues.

Moreover, the four issues that load highest on the first factor of the vertical dimension subset of issues were all highly salient issues for one or more member state, and relate to the protection of national sovereignty against a shift of policy-making power to the European level: majority voting on tax harmonization and social security (which were two of the United Kingdom's 'red-line' issues); whether macroeconomic policy should be allocated to the national or European level, and whether the Growth and Stability Pact should be made more flexible (France and Germany wanted more flexibility in return for more EU policy competences on economy policy, against the opposite view of many of the smaller member states).

The five issues that load highest on the second factor of this subset of issues were also relatively salient in the negotiations: the allocation of competences to the national or European level of agricultural, structural and cohesion, health and environmental policies, and the extent of European-level rules on the control of the EU's external borders.

Table 3 presents the results of an exploratory factor analysis of the preferences of the actors on the 35 issues identified as relating to the horizontal dimension of constitutional design. 7 This time, the first factor explains 19.4% of the variance in preferences and the second factor explains a further 10.1%. So, the first two factors together explain almost 30% of the total variance in preferences on this subset of issues.

Six of the seven issues with the highest loadings on the first factor on this horizontal dimension subset of issues relate to increasing the powers of the European Parliament over sensitive areas of EU policy: the extension of the co-decision procedure to defence policy, common foreign policy, social security, tax harmonization, economic policy, and monetary policy. The seventh issue, the extension of majority voting on social security, is the only issue that loads highly on the first factor of the horizontal dimension subset of issues as well as the first factor of the vertical dimension subset of issues.

The second factor on the horizontal dimension captures divisions on the extension of majority voting, where the issues with the highest loadings are majority voting on social security, common foreign policy, defence policy and tax harmonization, and the design of QMV in the Council. Two other issues also show up highly on this second factor: whether the initiative for legislative acts should be taken away from the European Parliament, and whether the co-decision procedure should continue to be applied to internal market legislation.

In other words, this mixed deductive/inductive method captures the substantive content of the two exogenous dimensions of constitutional design quite well, as many of the key issues on each dimension load highly on the respective first factors. Moreover, as Figure 5 shows, the extracted ideal point estimates from the factor loadings of all the issues on the first factor of each dimension accord closely with intuitive expectations about the location of the key actors on the vertical and horizontal allocation of power in the EU.

Specifically, the United Kingdom is revealed as having a classic 'intergovernmentalist' set of preferences -at the decentralization end of the vertical dimension, and towards the consensus side of the horizontal dimension. The position of France also accords with our expectations: pro-centralization of policy competences, but in favour of intergovernmental institutions at the European level. Against these two, Belgium, the European Parliament and the Commission all reveal classic 'Euro-federalist' preferences, with a centralization of power and majoritarian institutions. Interestingly, Slovakia appears to be the only member state with the fourth ideal-typical combination of preferences: favouring a decentralization of policy competences but majoritarian institutions in the EU. As regards other actors, the location of the two Spanish governments captures the shift in position from the Aznar to the Zapatero government, from a classic intergovernmentalist position to a considerably more pro-integrationist stance (on the vertical dimension) and a more moderate view on the allocation of powers to the Commission and the European Parliament (on the horizontal dimension).

Correlation between the ideal point estimates

To compare the ideal point estimates produced by the inductive and mixed methods, Table 4 shows the correlations between the ideal points on the first two dimensions 'discovered' by each method. The table in the appendix contains the actual ideal point estimates for the 28 actors on each of these 10 dimensions. 8 The results illustrate that the methods produced quite different sets of estimates. Interestingly, the estimates on each of the first dimensions produced by the three inductive methods are highly correlated. Also, the inductive methods did pick up some of the variations identified in the two sets of mixed dimensions. For example, the ideal points from the first factor derived from the horizontal subset of issues are highly correlated with the first dimension of NOMINATE coordinates and the first dimension of Optimal Classification coordinates. The ideal points from the second factor derived from the vertical subset of issues are also highly correlated with the second factor scores from the exploratory factor analysis. Nevertheless, no other set of coordinates is correlated with the set of ideal points on the first factor from the vertical subset of questions -the other main dimension produced by our mixed method.

Conclusion

In this article we studied the preferences of the EU governments and the supranational institutions on the principal dimensions of conflict during the negotiations on an EU Constitution. We used the DOSEI data set and applied four alternative methods for deriving ideal point estimates for these actors on the main dimensions of conflict in the IGC negotiations. First, we applied three different inductive methods. These methods all revealed a comparatively high number of underlying dimensions in the issue space. Moreover, the actors' locations produced by these dimensions do not correspond well with our prior intuitions about the likely location of some of the key actors. In addition, the substantive content of the dimensions produced by these methods is hard to interpret.

Hence, we applied a mixed deductive/inductive factor analysis to see if we could produce a more intuitive set of estimates. Here, we consider two a priori dimensions in the constitutional debate, one related to the vertical allocation of power between the EU and the member states and the other related to the horizontal allocation of power between the EU institutions. We then applied factor analysis on the subset of issues that we identified as relating to these two dimensions. We found that this method explains significantly more variation than a purely inductive factor analysis. Moreover, the derived ideal point estimates from this method corresponded more closely to our prior intuitions about the likely location of some of the key actors and the substantive content of the main dimensions of competition in the IGC negotiations than did the results of the three inductive methods.

Obviously our analysis is not the definitive word on the measurement of preferences in the constitutional debate on the EU, but rather illustrates that any estimates produced are likely to be highly determined by the instrument one chooses to derive them. In terms of further research, we would recommend comparing our results produced by a variety of scaling methods with other, more exogenous, estimates of actors' ideal preferences in the constitutional debate, such as expert placement of actors on the key dimensions in EU politics (Ray, 1999; Marks and Wilson, 2000), the coding of parties' manifestos in recent national and European elections (Budge et al. 2001; Gabel and Hix, 2002), and the application of word scores to debates and texts in the Convention and the IGC (Laver et al., 2003). The comparison and combination of ideal point estimates derived from different techniques are likely to lead to considerably more reliable estimates of the positions of the key actors on the main underlying dimensions in the EU constitutional debate.

Notes 1

See the Americans for Democratic Action website, URL: http://www. adaction.org/votingrecords.htm. Each year the ADA selects 20 votes it considers the most important during that session, and each Congressperson receives 5 points if he/she voted the 'right way' according to the ADA, and does not receive 5 points if he/she voted against the ADA or was absent. The highest possible total is consequently 100. 2

We use the version of the data set where missing responses are entered as equivalent to 'indifference' on an issue.

Figure 2

 2 Figure2Ideal point estimates using exploratory factor analysis.Note: SP1 is the Aznar government of Spain, and SP2 is the Zapatero government.

Figure 3

 3 Figure 3 Ideal point estimates using NOMINATE.

Figure 4

 4 Figure 4Ideal point estimates using Optimal Classification.

FactorFigure 5

 5 Figure 5Ideal point estimates using a 'mixed' method.

Table 1

 1 Factor loadings using the full set of issues

	Factor

Table 1 Continued

 1

	Factor

Table 2

 2 Factor loadings on the expected vertical dimension

	Factor

Table 3

 3 Factor loadings on the expected horizontal dimension

	Factor

Table 4

 4 Correlations between ideal point estimates

		Expl. FA Expl. FA					Vertical Vertical Horizontal Horizontal
		(full-set) (full-set) NOM. NOM. OC	OC	subset subset subset	subset
		Factor 1 Factor 2 Dim. 1 Dim. 2 Dim. 1 Dim. 2 Factor 1 Factor 2 Factor 1	Factor 2
	Exploratory factor analysis (full set) factor 1 1.000
	Exploratory factor analysis (full set) factor 2	.000	1.000
	NOMINATE dimension 1	.548** -.086	1.000
	NOMINATE dimension 2	.095	.299	.043	1.000
	Optimal Classification dimension 1	.480**	.016	.941** .189 1.000
	Optimal Classification dimension 2	-.019	.473*	-.001	.916** .195 1.000
	Vertical subset factor 1	.215	-.122	.112	.016	.122 -.023 1.000
	Vertical subset factor 2	-.032	.843** -.035	.325	.027	.362	.000	1.000
	Horizontal subset factor 1	-.019	.050	.489** -.058	.528** -.044	.136	.159	1.000
	Horizontal subset factor 2	-.864**	.155	-.481** -.111 -.426* .043 -.382*	.071	.000	1.000
	Note: Persson correlation coefficients.
	** significant at .01 level (2-tailed); * significant at .05 level (2-tailed).

European Union Politics 6(3)

3

Four issues were dropped in the analysis owing to zero variance on these issues: (15a) Initiative for legislative acts: Commission; (15d) Initiative for legislative acts: National parliaments; (18a.1) Majority voting: Agriculture; and (18a.4) Majority voting: Internal market. 4

We set NOMINATE to generate estimates on two dimensions. Setting a higher number of dimensions did not change the results significantly. 5

We set Optimal Classification to generate estimates on two dimensions. Setting a higher number of dimensions did not change the results significantly. 6

Two issues were dropped because of zero variance: (18a. The appendix can be found on the EUP website.