Enrique Henestroza Anguiano

Marie Candito
email: marie.candito@linguist.jussieu.fr

Parse Correction with Specialized Models for Difficult Attachment Types

This paper develops a framework for syntactic dependency parse correction. Dependencies in an input parse tree are revised by selecting, for a given dependent, the best governor from within a small set of candidates. We use a discriminative linear ranking model to select the best governor from a group of candidates for a dependent, and our model includes a rich feature set that encodes syntactic structure in the input parse tree. The parse correction framework is parser-agnostic, and can correct attachments using either a generic model or specialized models tailored to difficult attachment types like coordination and pp-attachment. Our experiments show that parse correction, combining a generic model with specialized models for difficult attachment types, can successfully improve the quality of predicted parse trees output by several representative state-of-the-art dependency parsers for French.

Introduction

In syntactic dependency parse correction, attachments in an input parse tree are revised by selecting, for a given dependent, the best governor from within a small set of candidates. The motivation behind parse correction is that attachment decisions, especially traditionally difficult ones like pp-attachment and coordination, may require substantial contextual information in order to be made accurately. Because syntactic dependency parsers predict the parse tree for an entire sentence, they may not be able to take into account sufficient context when making attachment decisions, due to computational complexity. Assuming nonetheless that a predicted parse tree is mostly accurate, parse correction can revise difficult attachments by using the predicted tree's syntactic structure to restrict the set of candidate governors and extract a rich set of features to help select among them. Parse correction is also appealing because it is parser-agnostic: it can be trained to correct the output of any dependency parser.

In Section 2 we discuss work related to parse correction, pp-attachment and coordination resolution. In Section 3 we discuss dependency structure and various statistical dependency parsing approaches. In Section 4 we introduce the parse correction framework, and Section 5 describes the features and learning model used in our implementation. In Section 6 we present experiments in which parse correction revises the predicted parse trees of four state-of-the-art dependency parsers for French. We provide concluding remarks in Section 7.

Related Work

Previous research directly concerning parse correction includes that of [START_REF] Attardi | Tree revision learning for dependency parsing[END_REF], working on English and Swedish, who use an approach that considers a fixed set of revision rules: each rule describes movements in the parse tree leading from a dependent's original governor to a new governor, and a classifier is trained to select the correct revision rule for a given dependent. One drawback of this approach is that the classes lack semantic coherence: a sequence of movements does not necessarily have the same meaning across differ-ent syntactic trees. [START_REF] Hall | Corrective modeling for non-projective dependency parsing[END_REF], working on Czech, define a neighborhood of candidate governors centered around the original governor of a dependent, and a Maximum Entropy model determines the probability of each candidate-dependent attachment. We follow primarily from their work in our use of neighborhoods to delimit the set of candidate governors. Our main contributions are: specialized corrective models for difficult attachment types (coordination and pp-attachment) in addition to a general corrective model; more sophisticated features, feature combinations, and feature selection; and a ranking model trained directly to select the true governor from among a set of candidates.

There has also been other work on techniques similar to parse correction. [START_REF] Attardi | Reverse revision and linear tree combination for dependency parsing[END_REF] investigate reverse revision: a left-to-right transition-based model is first used to parse a sentence, then a right-to-left transition-based model is run with additional features taken from the left-toright model's predicted parse. This approach leads to improved parsing results on a number of languages. While their approach is similar to parse correction in that it uses a predicted parse to inform a subsequent processing step, this information is used to improve a second parser rather than a model for correcting errors. [START_REF] Mcdonald | Online learning of approximate dependency parsing algorithms[END_REF] consider a method for recovering non-projective attachments from a graph representation of a sentence, in which an optimal projective parse tree has been identified. The parse tree's edges are allowed to be rearranged in ways that introduce non-projectivity in order to increase its overall score. This rearrangement approach resembles parse correction because it is a second step that can revise attachments made in the first step, but it differs in a number of ways: it is dependent on a graph-based parsing approach, it does not model errors made by the parser, and it can only output non-projective variants of the predicted parse tree.

As a process that revises the output of a syntactic parser, parse reranking is also similar to parse correction. A well-studied subject (e.g. the work of [START_REF] Charniak | Coarse-to-fine nbest parsing and MaxEnt discriminative reranking[END_REF] and of [START_REF] Collins | Discriminative reranking for natural language parsing[END_REF]), parse reranking is concerned with the reordering of n-best ranked parse trees output by a syntactic parser. Parse correction has a number of advantages compared to reranking: it can be used with parsers that do not output n-best ranked parses, it can be easily restricted to specific attachment types, and its output space of parse trees is not limited to those appearing in an n-best list. However, parse reranking has the advantage of selecting the globally optimal parse for a sentence from an nbest list, while parse correction makes only locally optimal revisions in the predicted parse for a sentence.

Difficult Attachment Types

Research on pp-attachment traditionally formulates the problem in isolation, as in the work of [START_REF] Pantel | An unsupervised approach to prepositional phrase attachment using contextually similar words[END_REF] and of [START_REF] Olteanu | PP-attachment disambiguation using large context[END_REF]. Examples consist of tuples of the form (v, n 1 , p, n 2 ), where either v or n 1 is the true governor of the pp comprising p and n 2 , and the task is to choose between v and n 1 . Recently, [START_REF] Atterer | Prepositional phrase attachment without oracles[END_REF] have criticized this formulation as unrealistic because it uses an oracle to select candidate governors, and they find that successful approaches for the isolated problem perform no better than stateof-the-art parsers on pp-attachment when evaluated on full sentences. With parse correction, candidate governors are identified automatically with no (v, n 1 , p, n 2 ) restriction, and for several representative parsers we find that parse correction improves pp-attachment performance.

Research on coordination resolution has also often formulated the problem in isolation. [START_REF] Resnik | Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language[END_REF] uses semantic similarity to resolve nounphrase coordination of the form (n 1 , cc, n 2 , n 3 ), where the coordinating conjunction cc coordinates either the heads n 1 and n 2 or the heads n 1 and n 3 . The same criticism as the one made by [START_REF] Atterer | Prepositional phrase attachment without oracles[END_REF] for pp-attachment might be applied to this approach to coordination resolution. In another formulation, the input consists of a raw sentence, and coordination structure is then detected and disambiguated using discriminative learning models [START_REF] Shimbo | A discriminative learning model for coordinate conjunctions[END_REF] or coordination-specific parsers [START_REF] Hara | Coordinate structure analysis with global structural constraints and alignment-based local features[END_REF]. Finally, other work has focused on introducing specialized features for coordination into existing syntactic parsing models [START_REF] Hogan | Coordinate noun phrase disambiguation in a generative parsing model[END_REF]. Our approach is novel with respect to previous work by directly modeling the correction of coordination errors made by general-purpose dependency parsers.

Dependency Parsing

Dependency syntax involves the representation of syntactic information for a sentence in the form a directed graph, whose edges encode word-to-word relationships. An edge from a governor to a dependent indicates, roughly, that the presence of the dependent is syntactically legitimated by the governor. An important property of dependency syntax is that each word, except for the root of the sentence, has exactly one governor; dependency syntax is thus represented by trees. Figure 1 shows an example of an unlabeled dependency tree. 1 For languages like English or French, most sentences can be represented with a projective dependency tree: for any edge from word g to word d, g dominates any intervening word between g and d.

Dependency trees are appealing syntactic representations, closer than constituency trees to the semantic representations useful for NLP applications. This is true even with the projectivity requirement, which occasionally creates syntax-semantics mismatches. Dependency trees have recently seen a surge of interest, particularly with the introduction of supervised models for dependency parsing using linear classifiers. Such parsers fall into two main categories: transition-based parsing and graphbased parsing. Additionally, an alternative method for obtaining the dependency parse for a sentence is to parse the sentence with a constituency-based parser and then use an automatic process to convert the output into dependency structure.

1 Edges are generally labeled with the surface grammatical function that the dependent bears with respect to its governor. In this paper we focus on unlabeled dependency parsing, setting aside labeling as a separate task.

Transition-Based Parsing

In transition-based dependency parsing, whose seminal works are that of [START_REF] Yamada | Statistical dependency analysis with support vector machines[END_REF] and [START_REF] Nivre | An efficient algorithm for projective dependency parsing[END_REF], the parsing process applies a sequence of incremental actions, which typically manipulate a buffer position in the sentence and a stack for built sub-structures. Actions are of the type "read word from buffer", "build a dependency from node on top of the stack to node that begins the buffer", etc. In a greedy version of this process, the action to apply at each step is deterministically chosen to be the best-scoring action according to a classifier, which is trained on a dependency treebank converted into sequences of actions. The strengths of this framework are O(n) time complexity and a lack of restrictions on the locality of features. A major drawback is its greedy behavior: it can potentially make difficult attachment decisions early in the processing of a sentence, without being able to reconsider them when more information becomes available. Beamed versions of the algorithm [START_REF] Johansson | Investigating multilingual dependency parsing[END_REF] partially address this problem, but still do not provide a global optimization for selecting the output parse tree.

Graph-Based Parsing

In graph-based dependency parsing, whose seminal work is that of [START_REF] Mcdonald | Online large-margin training of dependency parsers[END_REF], the parsing process selects the globally optimal parse tree from a graph containing attachments (directed edges) between each pair of words (nodes) in a sentence. It finds the k-best scoring parse trees, both during training and at parse time, where the score of a tree is the sum of the scores of its factors (consisting of one or more linked edges). While large factors are desirable for capturing sophisticated linguistic constraints, they come at the cost of time complexity: for the projective case, adaptations of Eisner's algorithm [START_REF] Eisner | Three new probabilistic models for dependency parsing: An exploration[END_REF] are O(n 3 ) for 1-edge factors [START_REF] Mcdonald | Online large-margin training of dependency parsers[END_REF] or sibling 2-edge factors [START_REF] Mcdonald | Online learning of approximate dependency parsing algorithms[END_REF], and O(n 4 ) for general 2-edge factors [START_REF] Carreras | Experiments with a higher-order projective dependency parser[END_REF] or 3-edge factors [START_REF] Koo | Efficient third-order dependency parsers[END_REF].

Constituency-Based Parsing

Beyond the two main approaches to dependency parsing, there is also the approach of constituency-based parsing followed by a conversion step to dependency structure. We use the three-step parsing architecture previously tested for French by Candito et al. (2010a): (i) A constituency parse tree is output by the BerkeleyParser, which has been trained to learn a probabilistic context-free grammar with latent annotations [START_REF] Petrov | Learning accurate, compact, and interpretable tree annotation[END_REF] that has parsing time complexity O(n3 ) [START_REF] Matsuzaki | Probabilistic CFG with latent annotations[END_REF]; (ii) A functional role labeler using a Maximum Entropy model adds functional annotations to links between a verb and its dependents; (iii) Constituency trees are automatically converted into projective dependency trees, with remaining unlabeled dependencies assigned labels using a rule-based approach.

Baseline Parsers

In this paper, we use the following baseline parsers: MaltParser [START_REF] Nivre | Malt-Parser: A language-independent system for datadriven dependency parsing[END_REF] for transition-based parsing; MSTParser [START_REF] Mcdonald | Online large-margin training of dependency parsers[END_REF] (with sibling 2-edge factors) and BohnetParser [START_REF] Bohnet | Very high accuracy and fast dependency parsing is not a contradiction[END_REF] (with general 2-edge factors) for graph-based parsing; and BerkeleyParser [START_REF] Petrov | Learning accurate, compact, and interpretable tree annotation[END_REF] for constituency-based parsing.

For MaltParser and MSTParser, we use the best settings from a benchmarking of parsers for French (Candito et al., 2010b), except that we remove unsupervised word clusters as features. The parsing models are thus trained using features including predicted part-of-speech tags, lemmas and morphological features. For BohnetParser, we trained a new model using these same predicted features. For BerkelyParser, which was included in the benchmarking experiments, we trained a model using the so-called "desinflection" process that addresses data sparseness due to morphological variation: both at training and parsing time, terminal symbols are word forms in which redundant morphological suffixes are removed, provided the original part-ofspeech ambiguities are kept (Candito et al., 2010b).

All models are trained on the French Treebank (FTB) [START_REF] Abeillé | Enriching a French treebank[END_REF], consisting of 12,351 sentences from the Le Monde newspaper, either "desinflected" for the BerkeleyParser, or converted to projective dependency trees (Candito et al., 2010a) for the three dependency-native parsers. 2 For 2 The projectivity constraint is linguistically valid for most French parses: the authors report < 2% non-projective edges in a hand-corrected subset of the converted FTB. the dependency-native models, features include predicted part-of-speech (POS) tags from the MElt tagger [START_REF] Denis | Coupling an annotated corpus and a morphosyntactic lexicon for state-of-the-art POS tagging with less human effort[END_REF], as well as predicted lemmas and morphological features from the Lefff lexicon [START_REF] Sagot | The Lefff, a freely available, accurate and large-coverage lexicon for French[END_REF]. These models constitute the state-of-the-art for French dependency parsing: unlabeled attachment scores (UAS) on the FTB test set are 89.78% for MaltParser, 91.04% for MSTParser, 91.78% for BohnetParser, and 90.73% for Berkeley-Parser.

Parse Correction

The parse correction algorithm is a post-processing step to dependency parsing, where attachments from the predicted parse tree of a sentence are corrected by considering alternative candidate governors for each dependent. This process can be useful for attachments made too early in transition-based parsing, or with features that are too local in MST-based parsing.

The input is the predicted parse T of a sentence. From T a set D of dependent nodes are chosen for attachment correction. For each d ∈ D in left-toright sentence order, a set C d of candidate governors from T is identified, and then the highest scoring c ∈ C d , using a function S(c, d, T ), is assigned as the new governor of d in T . Pseudo-code for parse correction is shown in Figure 2. 3

Choosing Dependents

Various criteria may be used to choose the set D of dependents to correct. In the work of [START_REF] Hall | Corrective modeling for non-projective dependency parsing[END_REF] and of [START_REF] Attardi | Tree revision learning for dependency parsing[END_REF], D contains all nodes in the input parse tree. However, one advantage of parse correction is its ability to focus on specific attachment types, so an additional criterion for choosing dependents is to look separately at those dependents that correspond to difficult attachment types.

Analyzing errors made by the dependency parsers introduced in Section 3 on the development set of the FTB, we observe that two major sources of error across different parsers are coordination and ppattachment. Coordination accounts for around 10% of incorrect attachments and has an error rate ranging from 30 -40%, while pp-attachment accounts for around 30% of incorrect attachments and has an error rate of around 15%.

In this paper, we pay special attention to coordination and pp-attachment. Given the FTB annotation scheme, coordination can be corrected by changing the governor (first conjunct) of the coordinating conjunction that governs the second conjunct, and ppattachment can be corrected by changing the governor of the preposition that heads the pp. 4 We thus train specialized corrective models for when the dependents are coordinating conjunctions and prepositions, in addition to a generic corrective model that can be applied to any dependent.5

Identifying Candidate Governors

The set of candidate governors C d for a dependent d can be chosen in different ways. One method is to let every other node in T be a candidate governor for d. However, parser error analysis has shown that errors often occur in local contexts. [START_REF] Hall | Corrective modeling for non-projective dependency parsing[END_REF] define a neighborhood as a set of nodes N m (d) around the original predicted governor c o of d, where N m (d) includes all nodes in the parse tree T within graph distance m of d that pass through c o . They find that around 2/3 of the incorrect attachments in the output of Czech parses can be corrected by selecting the best governor from within N 3 (d). Similarly, in oracle experiments reported in section 6, we find that around 1/2 of coordination and pp-attachments in the output of French parses can be corrected by selecting the best governor from within N 3 (d). We thus use neighborhoods to delimit the set of candidate governors.

While one can simply assign C d ← N m (d), we add additional restrictions. First, in order to preserve projectivity within T , we keep in C d only those c such that the update T {gov(d) ← c} would result in a projective tree. 6 Additionally, we discard candidates with certain POS categories that are very unlikely to be governors: clitics and punctuation are always discarded, while determiners are discarded if the dependent is a preposition.

Scoring Candidate Governors

A new governor ĉ for a dependent d is predicted by selecting the highest scoring candidate c ∈ C d according to a function S(c, d, T ), which takes into account features over c, d, and the parse tree T . We use a linear model for our scoring function, which allows for relatively fast training and prediction. Our scoring function uses a weight vector w ∈ F, where F is the feature space for dependents we wish to correct (either generic, or specialized for prepositions or for coordinating conjunction), as well as the mapping Φ : C×D×T → F from combinations of candidate c ∈ C, dependent d ∈ D, and parse tree T ∈ T, to vectors in the feature space F. The scoring function returns the inner product of w and Φ(c, d, T ):

S(c, d, T ) = w • Φ(c, d, T ) (1)

Algorithm Complexity

The time complexity of our algorithm is O(n) in the length n of the input sentence, which is consistent with past work on parse correction by [START_REF] Hall | Corrective modeling for non-projective dependency parsing[END_REF] and by [START_REF] Attardi | Tree revision learning for dependency parsing[END_REF].

Attachments for up to n dependents in a sentence are deterministically corrected in one pass. For each such dependent d, the algorithm uses a linear model to select a new governor after extracting features for a local set of candidate governors C d , whose size does not dependent on n in the average case. 7 Locality in candidate governor identification and feature extraction preserves linear time complexity in the overall algorithm.

Model Learning

We now discuss our training setup, features, and learning approach for obtaining the weight vector w.

Training Setup

The parse correction training set pairs gold parse trees with corresponding predicted parse trees output by a syntactic parser, and it is obtained using a jackknifing procedure to automatically parse the gold-annotated training section of a dependency treebank with a syntactic dependency parser. We extract separate training sets for each type of dependent we wish to correct (generic, prepositions, coordinating conjunctions). Given p, then for each token d we wish to correct in a sentence in the training section, we note its true governor g d in the gold parse tree of the sentence, identify a set of candidate governors C d in the predicted parse T , and get feature vectors {Φ(c, d, T ) : c ∈ C d }.

Feature Space

In order to learn an effective scoring function, we use a rich feature space F that encodes syntactic context surrounding a candidate-dependent pair (c, d) within a parse tree T . Our primary features are indicator functions for realizations of linguistic or treebased feature classes. 8 From these primary features we generate more complex feature combinations of length up to P , which are then added to F. Each combo represents a set of one or more primary features, and is an indicator function that fires if and only if all of its members do.

Primary Feature Classes

The primary feature classes we use are listed below, grouped into categories corresponding to their use in different corrective models (d obj is the object of the dependent, c gov is the governor of the candidate, and c d-1 and c d+1 are the closest dependents of c linearly to the left and right, respectively, of d). -If c is linearly after d, then whether there exists a plausible rival candidate to the left of d (implemented as whether there is a noun or adjective linearly before d, without any intervening finite verb)

Generic features (always included

Feature Selection

Feature combos allow our models to effectively sidestep linearity constraints, at the cost of an exponential increase in the size of the feature space F. In order to accommodate combos, we use feature selection to help reduce the resulting space.

Our first feature selection technique is to apply a frequency threshold: if a feature or a combo appears less than K times among instances in our training set, we remove it from F. In addition to making the feature space more tractable, frequency thresholding makes our scoring function less reliant on rare features and combos.

Following frequency thresholding, we employ an additional technique using conditional entropy (CE) that we term CE-reduction. Let Y be a random variable for whether or not an attachment is true, and let A be a random variable for different combos that can appear in an attachment. We calculate the CE of a combo a with respect to Y as follows,

H(Y |A = a) = - y∈Y p(y|a) log p(y|a) (2)
where the probability p(y|a) is approximated from the training set as f req(a, y)/f req(a), with example balancing used here to account for more false attachments (Y = 0) than true ones (Y = 1) in our training set. Having calculated the CE of each combo, we remove from F those combos for which a subset combo (or feature) exists with equal or lesser CE. This eliminates any overly specific combo a when the extra features encoded in a, compared to some subset b, do not help a explain Y any better than b.

Ranking Model

The ranking setting for learning is used when a model needs to discriminate between mutually exclusive candidates that vary from instance to instance. This is typically used in parse reranking [START_REF] Charniak | Coarse-to-fine nbest parsing and MaxEnt discriminative reranking[END_REF], where for each sentence the model must select the correct parse from within an n-best list. [START_REF] Denis | A ranking approach to pronoun resolution[END_REF] INPUT: Aggressiveness C, rounds R.

INITIALIZE: w 0 ← (0, ..., 0), w avg ← (0, ..., 0) REPEAT: R times also show that ranking outperforms a binary classification approach to pronoun resolution (using a Maximum Entropy model), where for each pronominal anaphor the model must select the correct antecedent among candidates in a text. 9In our ranking approach to parse correction (PA-Ranking), the weight vector is trained to select the true governor from a set of candidates C d for a dependent d. The training set X is defined such that the t th instance is a collection of feature vectors { x t,c = Φ(c, d t , T t ) : c ∈ C dt }, where C dt is the candidate set for the dependent d t within the predicted parse T t , and the class is the true governor g t . Instances in which g t ∈ C dt are discarded.

LOOP: For t = 1, 2, . . . , |X| • Get feature vectors { x t,c : c ∈ C dt } • Get true governor g t ∈ C dt • Let h t = argmax c∈C d t -{gt} ( w t-1 • x t,c ) • Let m t = ( w t-1 • x t,gt ) -( w t-1 • x t,ht ) IF: m t < 1 • Let τ t = min C , 1-mt xt,g t -x t,h t 2 • Set w t ← w t-1 + τ t ( x t,gt -x t,ht ) ELSE: • Set w t ← w t-1 • Set w avg ← w avg + w t • Set w 0 ← w |X| OUTPUT: w avg /(R • |X|)
PA-Ranking training is carried out using a variation of the Passive-Aggressive algorithm [START_REF] Crammer | Online passive-aggressive algorithms[END_REF], which has been adapted to the ranking setting, implemented using the Polka library. 10 For each training iteration t, the margin is defined as

m t = ( w t-1 • x t,gt ) -( w t-1 • x t,ht )
, where h t is the highest scoring incorrect candidate. The algorithm is passive because an update to the weight vector is made if and only if m t < 1, either for incorrect predictions (m t < 0) or for correct predictions with insufficient margin (0 ≤ m t < 1). The new weight vector w t is as close as possible to w t-1 , subject to the aggressive constraint that the new margin be greater than 1. We use weight averaging, so the final output w avg is the average over the weight vectors after each training step. Pseudo-code for the training algorithm is shown in Figure 3. The rounds parameter R determines the number of times to run through the training set, and the aggressiveness parameter C sets an upper limit on the update magnitude.

Experiments

We present experiments where we applied parse correction to the output of four state-of-the-art dependency parsers for French. We conducted our evaluation on the FTB using the standard training, velopment (dev), and test splits (containing 9,881, 1,235 and 1,235 sentences, respectively). To train our parse correction models, we generated specialized training sets corresponding to each parser by doing 10-fold jackknifing on the FTB training set (cf. Section 5.1). Each parser was run on the FTB dev and test sets, providing baseline unlabeled attachment score (UAS) results and output parse trees to be corrected.

Oracles and Neighborhood Size

To determine candidate neighborhood size, we considered an oracle scoring function that always selects the true governor of a dependent if it appears in the set of candidate governors, and otherwise selects the predicted governor. Results for this oracle on the dev set are shown in Table 1. The baseline corresponds to m=1, where the oracle just selects the predicted governor. Incrementing m to 2 and to 3 resulted in substantial gains in oracle UAS, but further incrementing m to 4 resulted in a relatively small additional gain. We found that average candidate set size increases about linearly in m, so we decided to use m=3 in order to have a high UAS upper bound without adding candidates that are very unlikely to be true governors. We also compared the oracle for parse correction with an oracle for parse reranking, in which the parse with the highest UAS for a sentence is selected from the top-100 parses output by MSTParser. We found that for MSTParser, the oracle for parse correction using neighborhood size m=3 (95.6% UAS) is comparable to the oracle for parse reranking using the top-100 parses (95.4% UAS). This is an encouraging result, showing that parse correction is capable of the same improvement as parse reranking without needing to process an n-best list of parses.

Feature Space Parameters

For the feature space F, we performed a grid search to find good values for the parameters K (frequency threshold), P (combo length), and CE-reduction. We found that P =3 with CE-reduction allowed for the most compactness without sacrificing correction performance, for all of our corrective models. Additionally, K=2 worked well for the coordinating conjunction models, while K=10 worked well for the preposition and generic models. CE-reduction proved useful in greatly reducing the feature space without lowering correction performance: it reduced the size of the coordinating conjunction models from 400k to 65k features each, the preposition models from 400k to 75k features each, and the generic models from 800k to 200k features each.

Corrective Configurations

For our evaluation of parse correction, we compared two different configurations: generic (corrects all dependents using the generic model) and specialized (corrects coordinating conjunctions and prepositions using their respective specialized models, and corrects other dependents using the generic model).

The PA-Ranking aggressiveness parameter C was set to 1 for our experiments, while the rounds parameter R was tuned separately for each corrective model using the dev set. For our final tests, we applied each combination of parser + corrective configuration by sequentially revising all dependents in the output parse that had a relevant POS tag given the corrective configuration. In the FTB test set, this amounted to an evaluation over 5,706 preposition tokens, 801 coordinating conjunction tokens, and 31,404 overall (non-punctuation) tokens.11

Results

Final results for the test set are shown in Table 2.

The overall UAS of each parser (except Bohnet-Parser) was significantly improved under both corrective configurations. 12 The specialized configura-tion performed as well as, and in most cases better than, the generic configuration, indicating the usefulness of specialized models and features for difficult attachment types. Interestingly, the lower the baseline parser's UAS, the larger the overall improvement from parse correction under the specialized configuration: MaltParser had the lowest baseline and the highest error reduction (6.8%), BerkeleyParser had the second-lowest baseline and the second-highest error reduction (5.4%), MST-Parser had the third-lowest baseline and the thirdhighest error reduction (3.6%), and BohnetParser had the highest baseline and the lowest error reduction (1.2%). It may be that the additional errors made by a low-baseline parser, compared to a high-baseline parser, involve relatively simpler attachments that parse correction can better model. Parse correction achieved significant improvements for coordination resolution under the specialized configuration for each parser. MaltParser and MSTParser had very low baseline coordinating conjunction UAS (around 60%), while Berke-leyParser and BohnetParser had higher baselines (around 70%). The highest error reduction was achieved by MSTParser (19.0%), followed by Malt-Parser (10.4%), BerkeleyParser (10.1%), and finally BohnetParser (7.5%). The result for MSTParser was surprising: although it had the second-highest baseline overall UAS, it shared the lowest baseline coordinating conjunction UAS and had the highest error reduction with parse correction. An explanation for this result is that the annotation scheme for coordination structure in the dependency FTB has the first conjunct governing the coordinating conjunction, which governs the second conjunct. Since MSTParser is limited to sibling 2-edge factors (cf. section 3), it is unable to jointly consider a full coordination structure. BohnetParser, which uses general 2-edge factors, can consider full coordination structures and consequently has a much higher baseline coordinating conjunction UAS than MSTParser.

Parse correction achieved significant but modest improvements in pp-attachment performance under the specialized configuration for MaltParser and BerkeleyParser. However, parse correction did not significantly improve pp-attachment performance for MSTParser or BohnetParser, the two parsers that had the highest baseline preposition UAS (around

86%

). These results are a bit disappointing, but they suggest that there may be a performance ceiling for pp-attachment beyond which rich lexical information (syntactic and semantic) or full sentence contexts are needed. For English, the average human performance on pp-attachment for the (v, n 1 , p, n 2 ) problem formulation is just 88.2% when given only the four head-words, but increases to 93.2% when given the full sentence [START_REF] Ratnaparkhi | A maximum entropy model for prepositional phrase attachment[END_REF].

If similar levels of human performance exist for French, additional sources of information may be needed to improve pp-attachment performance.

In addition to evaluating UAS improvements for parse correction, we took a closer look at the best corrective configuration (specialized) and analyzed the types of attachment modifications made (Table 3). In most cases there were around 2-3 times as many error-correcting modifications (w→c) as error-creating modifications (c→w), and the overall % of tokens modified was very low overall (around 1-2%). Parse correction is thus conservative in the number of modifications made, and rather accurate when it does decide to modify an attachment.

Finally, we compared the running times of the four parsers, as well as that of parse correction, on the test set using a 2.66 GHz Intel Core 2 Duo machine. BerkeleyParser took 600s, BohnetParser took 450s using both cores (800s using a single core), MaltParser took 45s, and MSTParser took 1000s. A rough version of parse correction in the specialized configuration took around 200s (for each parser). An interesting result is that parse correction improves MaltParser the most while retaining an overall time complexity of O(n), compared to O(n 3 ) or higher for the other parsers. This suggests that linear-time transition-based parsing and parse correction could combine to form an attractive system that improves parsing performance while retaining high speed.

Conclusion

We have developed a parse correction framework for syntactic dependency parsing that uses specialized models for difficult attachment types. Candidate governors for a given dependent are identified in a neighborhood around the predicted governor, and a scoring function selects the best governor. We used discriminative linear ranking models with features encoding syntactic context, and we tested parse correction on coordination, pp-attachment, and generic dependencies in the outputs of four representative statistical dependency parsers for French. Parse correction achieved improvements in unlabeled attachment score for three out of the four parsers, with MaltParser seeing the greatest improvement. Since both MaltParser and parse correction run in O(n) time, a combined system could prove useful in situations where high parsing speed is required.

Future work on parse correction might focus on developing specialized models for other difficult attachment types, such as verb-phrase attachment (verb dependents account for around 15% of incorrect attachments across all four parsers). Also, selectional preferences and subcategorization frames (from hand-built resources or extracted using distributional methods) could make for useful features in the pp-attachment corrective model; we suspect that richer lexical information is needed in order to increase the currently modest improvements achieved by parse correction on pp-attachment.

Figure 1 :

 1 Figure 1: An unlabeled dependency tree for: Elle ouvrit la porte avec la clé. (She opened the door with the key).

Figure 2 :

 2 Figure 2: The parse correction algorithm.

-

  ) -POS, lemma, and number of dependents of c -POS and dependency label of c d-1 -POS and dependency label of c d+1 -POS of c gov -POS and lemma of d -POS of d obj and whether d obj has a determiner -Whether c is the predicted governor of d -Binned linear distance between c and d -Linear direction of c with respect to d -POS sequence for nodes on path from c to d -Graph distance between c and d -Whether there is punctuation between c and d Features exclusive to coordination Whether d would coordinate two conjuncts that: Have the same POS -Have the same word form -Have number agreement -Are both nouns with the same cardinality -Are both proper nouns or both common nouns -Are both prepositions with the same word form -Are both prepositions with object of same POS Features exclusive to pp-attachment -Whether d immediately follows a punctuation -Whether d heads a pp likely to be the agent of a passive verb -If c is a coordinating conjunction, then whether c would coordinate two prepositions with the same word form, and whether there is at least one open-category word linearly between c and d (in which case c is an unlikely governor)

Figure 3 :

 3 Figure 3: Averaged PA-Ranking training algorithm.

Table 1 :

 1 Parse correction oracle UAS (%) for different neighborhood sizes, by dependent type (coordinating conjunctions, prepositions, or all dependents). Also, a reranking oracle for MSTParser using the top-100 parses.

					Size (m)
			Base	2	3	4
		Coords 67.2	76.5	82.8	84.8
	Berkeley	Preps	82.9	88.5	92.2	93.2
		Overall 90.1	94.0	96.0	96.5
		Coords 70.1	80.6	85.6	87.7
	Bohnet	Preps	85.4	89.4	93.4	94.5
		Overall 91.2	94.4	96.1	96.6
		Coords 60.9	72.2	78.2	80.5
	Malt	Preps	82.6	88.1	92.6	93.7
		Overall 89.3	93.2	95.1	95.8
		Coords 63.6	73.7	80.7	84.4
	MST	Preps	84.7	89.4	93.4	94.4
		Overall 90.2	93.7	95.6	96.2
	MST	Overall Reranking top-100 parses: 95.4

Table 2 :

 2 Coordinating conjunction, preposition, and overall UAS (%) by corrective configuration on the test set. Significant improvements over the baseline starred.

		Corrective		UAS (%)
		Configuration Coords Preps	Overall
		Baseline	68.3	83.8	90.73
	Berkeley	Generic	69.4	84.9*	91.13*
		Specialized	71.5*	85.1*	91.23*
		Baseline	70.5	86.1	91.78
	Bohnet	Generic	71.2	86.4	91.88
		Specialized	72.7*	86.2	91.88
		Baseline	59.8	83.2	89.78
	Malt	Generic	63.2*	84.5*	90.39*
		Specialized	64.0*	85.0*	90.47*
		Baseline	60.5	85.9	91.04
	MST	Generic	64.2*	86.2	91.25*
		Specialized	68.0*	86.2	91.36*

Table 3 :

 3 Breakdown of modifications made under the specialized configuration for each parser, by dependent type. w→c is wrong-to-correct, c→w is correct-towrong, w→w is wrong-to-wrong, and Mods is the percentage of tokens modified.

				Modification Type
			w→c c→w w→w Mods
		Coords	40	14	33	10.9 %
	Berkeley	Preps	118	39	41	3.5 %
		Overall	228	67	104	1.3 %
		Coords	32	15	33	10.0 %
	Bohnet	Preps	52	46	32	2.3 %
		Overall	150	121	130	1.1 %
		Coords	55	21	56	16.5 %
	Malt	Preps	149	50	76	4.8 %
		Overall	390	172	293	2.4 %
		Coords	80	20	51	18.9 %
	MST	Preps	64	45	26	2.4 %
		Overall	183	88	117	1.1 %

Contrary to[START_REF] Hall | Corrective modeling for non-projective dependency parsing[END_REF], our iterative algorithm (along with the fact that C d never includes nodes that are dominated by d) ensures that corrected structures are trees, so it does not require additional processing to eliminate cycles and preserve connectivity.

The FTB handles pp-attachment in a typical fashion, but coordination may be handled differently by other schemes (e.g. the coordinating conjunction governs both conjuncts).

In our experiments, we never revise punctuation and clitic dependents. Since punctuation attachments mostly carry little meaning, they are often annotated inconsistently and ignored in parsing evaluations (including ours). Clitics are not revised because they have a very low attachment error rate (2%).

We also keep candidates that would lead to a non-projective tree, as long as it would be projective if we ignored punctuation. This relaxation of the projectivity constraint leads to better oracle scores while retaining the key linguistic properties of projectivity.

Degenerate parse trees (e.g. flat trees) could lead to cases where |C d |=n, but for linguistically coherent parse trees |C d | is rather O(k m ), where k is the average -arity of syntactic parse trees and m is the neighborhood distance used.

For instance, there is a binary feature that is 1 if feature class "POS of c" takes on the value "verb", and 0 otherwise.

considered a binary training approach to parse correction in which the model is trained to independently classify candidates as true or false governors, as used by[START_REF] Hall | Corrective modeling for non-projective dependency parsing[END_REF]. However, we found that this approach performs no better (and often worse) than the ranking approach, and is less appropriate from a modeling standpoint.

Since the MElt tagger and BerkeleyParser POS tagging accuracies were around 97%, the sets of tokens considered for revision differed slightly from the sets of tokens (with gold POS tags) used to calculate UAS

scores.12 We used McNemar's Chi-squared test with p = 0.05 for all significance tests.

Acknowledgments

We would like to thank Pascal Denis for his help using the Polka library, and Alexis Nasr for his advice and comments. This work was partially funded by the ANR project Sequoia ANR-08-EMER-013.


th International Symposium on Applied Isotope Geochemistry 1 Tarragona, Spain 19 th -23 rd September 2011

A METHOD FOR U-Th ISOTOPIC ANALYSIS OF SALINE AND GEOTHERMAL WATERS Christophe Innocent

BRGM/MMA/ISO, 3 avenue Claude Guillemin, BP 6009, 45060 Orléans cedex 2, France c.innocent@brgm.fr U-Th activity ratios are widely used to constrain the chronology of recent (i.e.; Quaternary) events and processes. Considering natural waters, these activity ratios may help to derive groundwater residence times in an aquifer, or to investigate water-rock interaction processes. For this purpose, large quantities of water (at least 15 l) are necessary, especially when the analysis of Th isotopic composition is required.

Classically, the isolation of U and Th from the water (previously acidified to prevent the development of bacterial flora, as well as the adsorption of cations on the walls of the container) includes a preliminary step of coprecipitation at pH ≥ 7 of transition metals, using a Fe carrier. However, in the case of saline and geothermal waters, some other elements may also precipitate with transition metals, resulting in an important residue. Furthermore, in case of Sirich water, hydrated silica also precipitates as a gel that cannot further redissolve unless using large quantities of hydrofluoric acid. This problem could be theoretically overcome if using soda (instead of ammonia) as base to get a basic pH. In this case, water-soluble water glass (Na 2 SiO 3 ) is formed that prevents silica precipitation.

Alternatively, U and Th may be isolated from natural water using a chelating resin [START_REF] Greenberg | Trace element analysis of natural water samples by neutron activation analysis with chelating resin[END_REF]. A procedure allowing for efficient chemical extraction of U and Th has been performed at BRGM Orléans. The water sample is collected in a jerrycan equipped with a tap, then the pH of the water is adjusted to a value close to 3.5 (± 0.5). A column filled with Chelex 100 ® resin, previously adjusted at the same pH (~ 3.5), is directly fitted on the tap to allow for a continuous flow of the water through the column. As no additionnal overpressure system is involved, this operation may take some time, but the experience shows that the process is generally achieved within a few days in the case of high water volumes. This oepration can be processed either in a cleanlab or even directly in the field depending on pollution requirements (and also on the time !). At such a pH, U and Th are quantitatively retained on the chelating resin, whereas most cations, like alkalis and most of the alkaline earths are removed. In addition, silica is not retained under such conditions. Finally, U and Th are eluted, with some transition metals, using dilute nitric acid. A further step of chemical isolation of U and Th involves a specific UTEVA ® resin. Such a resin is designed for U elution, but it may be also utilized for Th [START_REF] Douville | Rapid and accurate U-Th dating of ancient carbonates using inductively coupled plasma-quadrupole massspectrometry[END_REF]. Both elements are retained on the resin in dilute (2N or 3N) nitric acid, and U may be eluted either with very dilute nitric acid or dilute (1N) hydrochloric acid. Now, the high affinity of Th for hydrochloric acid allows to elute it in such a media, whatever its normality. Hence, it is possible to collect specifically both U and Th in the same fraction using dilute (1N) HCl if these two elements are wanted to be further analyzed together, as is the case for the standard MC-ICP/MS U-Th analysis technique utilized at BRGM. This method has been successfully tested on a sample of the geothermal waters from the Bouillante area (French Guadalupe), whose Si content averages 500 mg/l [START_REF] Sanjuan | Champ géothermique de Bouillante (Guadeloupe). Synthèse des travaux réalisés en géochimie avant[END_REF]. A second test carried out on the high-temperature geothermal fluids from Iceland has also allowed to remove silica efficiently. In this latter case, an extended UTEVA ® purification step may be required, depending on the chemical characteristics of the fluids. Finally, this method has revealed very successful for saline, low temperature geothermal waters from the Vendée area (France). U-Th results obtained on these waters are presented and discussed.


Damien Besancenot
email: besancenot.damien@univ-paris13.fr

Radu Vranceanu
email: vranceanu@essec.fr.

Experimental evidence on the "insidious" illiquidity risk

Keywords: Coordination game, Illiquidity risk, Threshold strategy, Experimental economics. JEL Classification: D81, C92, C72, G20

This paper brings experimental evidence on investors' behavior subject to an "illiquidity" constraint, where the success of a risky project depends on the participation of a minimum number of investors. The experiment is set up as a frameless coordination game that replicates the investment context. Results confirm the insidious nature of the illiquidity risk: as long as a first illiquidity default does not occur, investors do not seem able to fully internalize it. After several defaults, agents manage to coordinate on a default probability above which they refuse to participate to the project. This default probability is lower than the default probability of the first illiquidity default.

Introduction

Once again, during the Great Recession of 2007-2009, illiquidity proved to be one major factor of economic instability [START_REF] Pedersen | When everyone runs for the exit[END_REF]Brunermeyer, 2009;[START_REF] Miller | Leverage and asset bubbles: averting Armagedon with Chapter 11[END_REF]. In a short lapse of time, in the last four months of 2008, traditional buyers of many asset backed securities and bank commercial paper, all vanished. Taking one representative quote from The Economist of February 10th, 2010, "Many of those clobbered in the crisis, including Bearn Sterns, Northern Rock and AIG -were struck down by a sudden lack of cash or funding sources, not because they run out of capital."1 One important lesson from the last recession is that researchers must provide a much better understanding of liquidity crises than they hold today. [START_REF] Morris | Illiquidity component of credit risk[END_REF] notice that the credit risk of any institution has three different measures. They define the insolvency risk as the conditional probability of default due to deterioration of asset quality if there is no run by short term creditors in the first place. The total credit risk is the unconditional probability of default, either because of short-term creditor run or long term asset insolvency. Illiquidity risk is the difference between the two, i.e., the probability of a run on an otherwise solvent institution. 2In general economists explain illiquidity as an inefficient equilibrium outcome, where, in absence of a coordination mechanism, investors' cannot coordinate their actions on the high equilibrium.

The basic rationale was introduced by Summers (2000, p.7):

A crude but simple game, related to Douglas [START_REF] Diamond | Bank runs, liquidity, and deposit insurance[END_REF] celebrated analysis of bank runs, illustrates some of the issues involved here. Imagine that everyone who has invested $10 with me can expect to earn $1, assuming that I stay solvent. Suppose that if I go bankrupt, investors who remain lose their whole $10 investment, but that an investor who withdraws today neither gains nor loses.

What would you do? Each individual judgment would presumably depend on one's assessment of my prospects, but this in turn depends on the collective judgment of all of the investors. Suppose, first, that my foreign reserves, ability to mobilize resources, and economic strength are so limited that if any investor withdraws I will go bankrupt. It would be a Nash equilibrium (indeed, a Pareto-dominant one) for everyone to remain, but (I expect) not an attainable one. Someone would reason that someone else would decide to be cautious and withdraw, or at least that someone would reason that someone would reason that someone would withdraw, and so forth. This phenomenon, which Douglas Hofstadter has labeled "reverberant doubt," would likely lead to large-scale withdrawals, and I would go bankrupt. It would not be a close-run thing. John Maynard Keynes's beauty contest captures a similar idea.

Hence the theory of global (or coordination) games appears to provide a good background for understanding illiquidity. In these games, the individual payoff to some action is an increasing function in the number of agents that undertake that action [START_REF] Carlsson | Global games and equilibrium selection[END_REF][START_REF] Morris | Unique equilibrium in a model of self-fulfilling currency attacks[END_REF][START_REF] Fischbacher | z-Tree: Zurich toolbox for ready-made economic experiments[END_REF].3 [START_REF] Bryant | A simple rational expectations Keynes-type model[END_REF] shows that in a complete information set-up these strategic complementarities result in multiple equilibria with self-fulfilling beliefs. The polar situations where either all players undertake the highest action (the Pareto dominant or payoff dominant equilibrium) or the lowest action (the highest security or minimax equilibrium) are intuitively appealing but cannot be justified on pure deductive grounds. To remove this indeterminacy, [START_REF] Van Huyck | Tacit coordination games, strategic uncertainty and coordination failure[END_REF] have run an experiment of a basic coordination game with complete information and find that, after repeating the game for several times, the security equilibrium takes over the other equilibria. 4Rochet and Vives ( 2004) and [START_REF] Morris | Illiquidity component of credit risk[END_REF] apply the theory of coordination games to banking crises. The main contribution of these papers is to combine the analysis of illiquidity with that of insolvency. The former can arise if banks cannot roll over their short term debt due to coordination problems, while insolvency is defined as the riskiness of their long-term investment projects, conditional of not being illiquid at the first stage. 5 Schotter and Yorulmmazer (2004) and [START_REF] Madiès | An experimental exploration of self-fulfilling banking panics: Their occurrence, persistence and prevention[END_REF] have provided experimental analyses of classical bank runs; they focus on depositors' decision to pull out their money once that a bank becomes insolvent.

To the best of our knowledge, the relationship between the intrinsic risk of an asset (insolvency) and illiquidity has not been tested in laboratory experiments. In this paper, we report results of a simple experiment aiming to directly assess the contribution of illiquidity to the total risk of an investment project. The project can deliver either a positive gain or nothing, with a known and predetermined probability. Without illiquidity, this "insolvency" or "intrinsic" probability of default is the only risk that investors should take into account. However, for a large and indivisible investment project, investors must pool resources in order to implement it. If the number of investors is too small, the project can only fail, what we refer to as the "illiquidity risk". If the number of investors is large enough, the project is implemented and will succeed or not, depending on the intrinsic probability of success.

To take an example, suppose that a famous biologist wants to implement a research programme on a new molecule with huge market value for drug companies. He wants to run this project by creating a company, and sells one year bonds for 2000 euros each, with a face value of 3000. In order to able to carry out its research, he needs a molecular microscope that costs one million euros. So, if there are less than 500 lenders, he cannot buy this heavy equipment, the research cannot be engaged, and his nascent company will fail and lenders will lose their investment. If the microscope is bought, then the gain can be positive (if he succeeds) or zero (if we assume, not very realistically, that a second hand microscope worth nothing).

To test if whether investors (lenders) are able to internalize this "illiquidity risk", we set up an experiment as a pure coordination game. While the connection with the financial market is straightforward, we avoid using words such as "loan, debt, default" that have their own emotional loading. Basically, subjects can pay 20 cents in order to participate to a lottery that brings them 30 cents with a probability q and nothing with probability (1q). This intrinsic risk (1q) is common knowledge. If this lottery is interpreted as an investment into a risky project, in case of default of the borrower (with probability 1q), the lender loses his investment, in the opposite case (with probability q), the lender makes a net profit of 10 cents.

Students were invited to play two games. In the first game, referred to as the "no-interaction game", the outcome of the investment depends on the individual risk only (with no interaction between individuals' decisions). In the second game, referred to as "the illiquidity game", the positive gain is delivered only if at least x% of the subjects have participated to the investment.

This participation constraint is inspired by the investment story, where the success of a joint project depends on the participation of a large enough number of individual investors. Hence, an investor must not only take into account the individual chances that his investment will bring him the high payoff, but also the would-be decisions of the other investors.

The same group of subjects was asked to play the illiquidity game for several times, for varying levels of the intrinsic risk. We observe for what level of the displayed individual risk they coordinate their actions either on the "invest" or "do not invest" strategy. Actually, the experiment aims to elicit "threshold strategies" on which players do coordinate their decisions, in the line with the experiment implemented by Heinemann et al. (2004). 6 The experiment provides several interesting insights. Firstly, we observe that the minimax equilibrium -where no investor participates to the market and the investment is not implemented -never occurs.

[Appendix 1 provides a theoretical explanation for this outcome]. Secondly, there is a default threshold for the risk probability (1q) on which agents tend to coordinate their actions. Thirdly, this default threshold is smaller in presence of the illiquidity risk than without the illiquidity risk, but agents manage to coordinate on this threshold only after several defaults.

We comment on the policy implications of these results in the conclusion section.

For sure, as any experimental research, this paper has its own limits. At difference with a standard global game, in our experiment the individual's payoff does not depend on some order statistics of the group (minimum, maximum, average) and the percentage of investors that triggers illiquidity is exogenously given. The advantage of this simple experiment is that it can disentangle the illiquidity risk (stemming from poor coordination) from the total risk of the investment. Further research might test the game for varying participation thresholds, or make this participation threshold endogenous.

The paper is organized as follows. The next section presents the experimental design. Section 3 provides the results of the experiment. Section 4 presents our conclusion.

Experimental design

All the subjects were recruited by the ESSEC Experimental Laboratory from the student population of the ESSEC Business School, who answered to a call for paid decision experiments. The experimental design was presented via computer interface and all interactions were computerized. The programme was written in Z-tree [START_REF] Fischbacher | z-Tree: Zurich toolbox for ready-made economic experiments[END_REF]. Instructions were read aloud.

Throughout the sessions students were not allowed to communicate and could not see the others'

screen. The core study builds on a "within-subjects" design, but we also tested the illiquidity game alone. In the within-subjects set-up, three sessions were conducted in May (9 students),

June (15 students) and November 2010 (16 students). [see Appendix 3 for the instructions].

There is a basic lottery L defined as an elementary two-payoff gamble:

Lottery L :

½ 30 cents with a probability q 0 cent with a probability 1q

In the first game students were invited to play repeatedly the lottery L. At each round, they receive 20 cents before making their choice (play or not the lottery); loses and gains were also material. The individual levels of risk, i.e., the probability (1q) of losing the 20 cents bet, was announced and was varied discretely, rising from 0.05 to 0.70 with a step of 0.05 over a total of 14 rounds. After each round the student who chooses to participate is informed about the actual outcome (win/loss). In this paper, we will refer to this game as the "no interaction game".

Students where then asked to play for several times a second game, referred to as "the illiquidity game". As in the former setup, at every round students get 20 cents. They can use them to participate to the lottery L (with a known probability q of winning the positive payoff). Yet this time an additional constraint is brought into the picture: if less than 50% of the players participate to the lottery at that round, they will receive nothing whatever the outcome of the individual draw. 7 Hence, the students must take into account not only the individual risk, but also the risk related to the behavior of the others students in the group.

The screen inviting the student to participate to the game recalls the possible payoffs, indicates the "individual risk" (1q) of the lottery at that round and the percent of students who played the lottery at the previous round.

Once that all players have taken their decision (to play the game or not), the computer displays what was the percentage of participants at that round, and, if the student played the game, which was the outcome of the game (0 or 30 cents).

In this game, the level of individual risk (i.e., the loss probability 1q) varies according to a simple rule (Figure 2). The game starts at t = 0 with a small default risk, (1q) = 0.15. As long as no illiquidity default occurs the risk is increased by 0.05 from one round to another. If an illiquidity default occurs, the same level of risk is submitted once again. Then, if illiquidity arises, the level of risk is reduced by 0.05 at the next round; if else, it is increased by 0.05. The experiment stops after 31 rounds.

At the end of the game the software indicates the total gain of every student. A short satisfaction survey is implemented. These experiments last for 45 minutes in average. Students earned in average 13.25 euros from playing the game; the amount includes 5 euros for showing up.

In order to control for the carry over effect specific to the within-subjects design, we run two additional sessions, where we asked students to perform only the illiquidity game, with the same rules as before. These sessions were organized in April 6 and May 18, 2011, with respectively 10 and 12 subjects.

yes (1-q) illiquidity (1-q) no yes (1-q) -0.05 illiquidity no
(1-q) + 0.05

(1-q) + 0.05 t=0 : 1-q = 0.05

Figure 1: The risk adjustment rule 3 Results

Data from the within-subjects design

We report firstly the result of the within-subjects set of experiments, where the same group of students first played the "no interaction game", and then the "illiquidity game".

Table 1 displays the summary of the results of the "no interaction game": the outcome of an individual decision depends only on the risk (1q); there is no interaction between students' decisions. The first line indicates the loss probability (1q). The next lines indicates for each probability the number of students who accept to play the basic lottery.

(1q) = 0.05 0.10 0.15 0.20 0.25 0. We notice that less than 50% of the students played the lottery when the objective risk (1q) first exceeded 0.55 in the first and third group, and 0.45 in the second group.8

Table 2 presents the main results of the "illiquidity game" (students' decisions are now interconnected -the individual gain is paid if at least 50% of the students participate). The data indicate the announced loss probability (1q) that triggered the first illiquidity default (i.e. less than 50% of the students lend), then the loss probability when more than 50% of the students decided to lend again (return-to-market); the same sequence is repeated three more times. The number between brackets gives the number of students who accepted to play the lottery for that "threshold" probability. In Appendix 2 we present graphs that show the evolution of the group decision during the experience (for each group). (Between brackets, the number of students who play the lottery)

The first default occurs for (1p) = 0.40 in groups 1 and 3, and 0.45 in the second group.

Once that illiquidity occurs, students adopt a security attitude, and refuse to lend in a coordinated way until the individual risk is further reduced (below 0.25 for groups 1 and 3, 0.35 for group 2).

After a few defaults (not more than three), students manage to coordinate their strategies: a large majority of players refuse to play when a critical threshold is reached (0.35 in the first group, 0.40 in the second and 0.30 in the third). The number of persons that coordinate their decision around this critical value increases throughout the rounds; after three default rounds, almost all the population in the sample coordinate their decision at that level. The fourth default threshold is lower than the first default threshold in the illiquidity game for all the three sessions.

The Wilcoxon signed ranks test performed for 40 students shows that the threshold reduction is statistically significant at pvalue < 0.01 (there are 27 negative ranks, 1 positive rank and 12 ties).

When comparing data from Table 1 with Table 2, it turns out that the first default threshold in the "illiquidity game" is lower than in the "no interaction game" for the groups 1 and 3; it is the same for the group 2. In order to check whether the reduction is statistically significant over the total sample, we run a Wilcoxon signed ranks test. We must remove from the analysis 7 individuals over 40 whose behavior was inconsistent. 9 The assumption according to which the threshold in the illiquidity game is lower than in the elementary decision can be accepted for a pvalue < 0.01 (there are 19 negative ranks, 9 positive ranks and 8 ties).

Figure 3 summarizes these findings. The first set of bars build on data from Table 1 relative to the "no-interaction game"; they indicate for each group the smallest risk probability for which less than 50% of the students decided participate to the lottery (similar to the illiquidity scenario in the second game). The next sets of bars build on data from Table 2, pertaining to the "illiquidity game". For each session, we select the critical thresholds (1q) when the first illiquidity default Figure 2: Critical thresholds in the "no interaction game" and the "illiquidity game": a withinsubjects design occurs, the critical thresholds for which students decide to lend again after the 3 rd default, and the critical thresholds at the fourth default.

We can notice that after several illiquidity defaults, the level of risk above which a majority of students refuse to play the lottery (i.e. the illiquidity threshold (1q) * ) is much lower than in the individual choice setup, down from [0.45 -0.55] to [0.30 -0.40].

Data from the between-subjects design

To check whether this pattern holds in a bewteen-subjects framework, we carried out the experiment twice again in April and May 2011, and asked students to play only the illiquidity game. For sure, in this case there is no perception of change in the rules that could prompt them to be more careful when moving from one game to the other. Results are close to the former within-subjects setup. As shown in Table 4 and the last two graphs displayed in Appendix 2, after a first illiquidity default (that occurs for a high level of risk), investors adjust downwards the acceptable limit. (Between brackets, the number of students who play the lottery)

Coordination around a critical threshold is achieved around a risk probability 0.45, lower than the probability of the first default (see Figure 4).

If we compare now the probability of the first default in the second wave of experiments with the probability where less than 50% participate in the elementary game in the first wave of experiments, the difference is not significant. Without the carry-over effect specific to the withinsubjects design, students seem unable to internalize the illiquidity risk: the first default in the illiquidity game is at the same risk level where less than 50% participate to the lottery in the elementary game. Once that default occurs, the learning process is engaged, and they end up coordinating around a lower default threshold, as they did in the within-subjects design. This default coordination threshold seems to be higher in this last wave of experiments than in the within-subjects experiments (check data in column "4 th default" in Tables 3 and4).

A pooled data analysis

A simple pooled data analysis allows us to emphasize as a general pattern the systematic downwards revision in the illiquidity default thresholds. For so doing, we report for each experiment the probabilities that triggered the first, second, third, fourth and fifth default; the order number n of the default is similar here to the time dimension in standard pooled data analysis.

The series of the threshold default probabilities for each of the within-subjects experiments (May 26, June 9 and November 3, 2010 -data from Table 2) are denoted respectively by QRW S1, QRW S2, QRW S3 ; the series of the default threshold for the between-subject design (April 6

and May 18, 2011 -data from Table 4) are denoted by QRBS1 and QRBS2. The five series are displayed in Figure 5, where the horizontal axis corresponds to the order number of the default (first, second, etc.) and the vertical axis indicates the threshold risk (1-q) for which the illiquidity default occurred (less than 50% of the students participate to the lottery).

We then run the regression:

QR jn = C + F j + an + bn 2 + jn (1)
where j stands for the experiment [WS1, WS2, WS3, BS1, BS2], C is the overall constant, F j is the fixed effect for each experience (a dummy variable), n is the order number or the default, a and b are coefficients and jn is an error term. The quadratic form is interesting since it allows for a dampening effect, representative of the convergence put forward in the descriptive statistics.

The OLS estimation of Equation ( 1) are displayed in As expected, the coefficient a is negative in both models: players revise downwards their default thresholds. In model 2, the small and positive coefficient of the quadratic term is also consistent with our expectations: after several successive defaults, the strength of the adjustment is declining.

From model 1, we notice that from the first to the fourth default, in average the default threshold would vary by -0.0525, i.e. 11% below the average first default threshold. Model 2 would suggest that after three more defaults, the default threshold would vary by -0.0735, i.e., 14% below the average first default threshold.

The analysis of the fixed effects also brings some interesting insights. Indeed, as we have already seen, the default thresholds in between-subjects experiments (BS1 and BS2) are systematically higher than in within-subjects experiments (WS1, WS2, WS3). The main contribution of this paper is to provide an experimental framework able to replicate the essential features of institutional credit risk, where default of the borrower can stem from both an intrinsic risk specific to the project, and illiquidity or poor coordination of the investors.

Conclusion and some policy implications

Our simple laboratory experiment has confirmed the insidious nature of the illiquidity risk: as long as a first illiquidity default does not happen, subjects do not seem able to fully internalize it:

the first default threshold in the illiquidity game is systematically higher than the fourth default threshold. If in real life investors can neither fully internalize the illiquidity risk, it is not surprising that there are not many signals that can call policymakers' attention on the accumulation of such imbalances. Indeed, illiquidity crises often come as a surprise. This phenomenon was extremely prominent during the Great Recession, when in 2008 buyers of many financial assets vanished in a short lapse of time and many financial markets, such as the market for CDOs, literally collapsed.

Our Lab experiment has shown that subjects become aware of the illiquidity risk after several defaults; they tend to adopt a threshold strategy that allows them to coordinate on a level of risk above which they refuse to participate to the market. In the experiment, it took a small number of defaults to reach this lower default threshold. In policy terms, for all financial markets where liquidity matters, the measure of what a "sustainable risk level" is should be adjusted downwards.10 Many asset-pricing models build their evaluation on the asset's past performance;

in the light of our analysis, if one asset has never been shunned by investors, then the price predicted by these models will have an upward bias due to an underestimated illiquidity risk.

Finally, immediately after a first illiquidity default, subjects in our experiment adopted a security stance, and refused to lend until the objective risk declined in a substantial way (without falling to zero). This second effect can be responsible of some crisis hysteresis. Before the Great Recession, banks were financing many high risk projects. Actually, many observers claim that this "race for risk" is a deep underlying factor of the crisis [START_REF] Besancenot | Banks' risk race: A signaling explanation[END_REF]). Yet after the crisis, the same banks were criticized on ground that they were too cautious, and did not lend enough. Our analysis points out that this extremely prudent attitude is the normal behavioral response after an illiquidity-driven crisis. Banks need to test the coordination mechanism before they start lending again.

5 Appendix 1. Theory: equilibrium of the illiquidity game

In this Appendix, we analyze the equilibrium of the game from a theoretical perspective. We consider an imperfect information game where a group of n identical risk-neutral players (where n is an odd number) are invited to play a lottery L that can pay:

½ 30 cents with a probability q 0 cent with a probability 1q

The probability q is public information. The cost of entering this lottery is 20 cents.

For sure, if no other motive of default is present, identical risk-neutral individuals would participate to the lottery if the winning probability exceeds a threshold q, implicitly defined by equation:

20 = 30q. (2)
If q < q = 2/3, a risk-neutral, rational player will not participate to the lottery.

In order to bring into the picture the risk of illiquidity, we impose the exogenous rule according to which if less than x% of the subjects participate to the lottery, they all get nothing whatever the outcome of the individual draw.

While in this game all individuals know the intrinsic risk (1q), they might not have the same perception of the illiquidity risk, which depends on the decision of all other players. Thus, given q, each player must decide whether to play the lottery (strategy y) or to refrain from playing (strategy n) depending on his own assessment the actions of the others. In such a game, the optimal strategy s of an individual i consists in choosing a reservation probability q * i and play the game if the actual probability is larger than this critical threshold (and vice-versa), i.e.:

s i = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ y, if q ≥ q * i n, if q < q * i (3)
According to his subjective expectation of the others' behavior, each individual i will choose his specific reservation probability q * i .

Let us order now the n players according to their q * i , from the lowest to the highest value. We define as the "pivot player" the player m such that the number of players with q * > q * m represent x% of the total number of subjects. For an objective risk q, if the m individual plays y (q > q * m ), then illiquidity does not arise; to the opposite, if he plays n, then more than x% of the players do the same, and illiquidity occurs. It must be noticed that, when taking their individual decision, players should consider only the decision of the pivot player, and not the individual decision of all players. As a consequence, the outcome of this game should not depend on the number of players.11

We denote by F i () the player's i subjective c.d.f of q * m .

For a given probability of getting the positive gain in the lottery, q, the gain from participating to the illiquidity game U i (y) is obtained by multiplying the positive payoff in the lottery (30) by the individual probability of winning (q) and by the probability that the pivot player does play the game, i.e. that q * m is lower than q; this probability is defined for the individual i as

Pr i [q * m < q] = F i (q). U i (y) = 30qF i (q) (4)
We notice that U i (y) is a monotonous increasing function in q with U i (y) = 0 for q = 0 and U i (y) = 30 for q = 1.

The reservation probability q * i turns out to be the value of q such that an individual i is indifferent between the two strategies, i.e. U i (n) = U i (y), where in this game U i (n) = 20.

Formally, q * i is the implicit solution to:

20 = 30qF i (q). (5)
Figure 1 represents the left-hand terms in Eq. ( 2) and Eq. ( 5) as two functions of q. At point A we get the critical default probability in the elementary setting (no interaction), at point A 0 the critical default probability in the illiquidity game. Even in the simplified context of this theoretical analysis (homogenous, risk neutral players), in presence of the liquidity constraint players require a higher winning probability (or a lower default probability) in order to accept the lottery. Should investors have different preferences towards risk, it would be more difficult to foresee the behavior of the pivot player (the p.d.f. of F would become wider and flatter), and the minimum wining probability required to participate would rise. 1 0 20 30 30qF i (q) q i * Winning probability q i 30q  A' q

Payoff

Figure 5: Critical winning thersholds for individual i Furthermore, with homogenous, fully rational players the first occurrence of illiquidity default would reveal the true value of the threshold q * . After the firs default the distribution F i () would degenerate and players should coordinate their decision around this value (i.e., the threshold strategy, to use the concept submitted by [START_REF] Heinemann | The theory of global games on test: Experimental analysis of coordination games with public and private information[END_REF]).

The experiment presented in the core text aims at analyzing how a group of "flesh and blood" decision makers behave within the former framework. The core issue is whether players do internalize the coordination risk, and, if they do, how fast. For sure, human subjects are not "homogenous"; they have their own psychological characteristics and biases (cognitive skills, various risk aversion, overconfidence/over prudence); in this case, the illiquidity default threshold would emerge at the term of a learning process.

Appendix 2

The figures below indicate the evolution of illiquidity defaults depending on the level of risk. The computer chooses the level of risk according to the rule presented in Figure 2. Screens 3-4-5. Would you pay 20 cents to participate to a lottery that brings you 30 cents with a probability of 60% (50%, 40%) and 0 cents with a probability of 40% (50%, 60%) ? Yes/No.

THE ELEMENTARY GAME

Screen 6. From now on the experiment will be paid. You receive the money at the beginning of each game and the decisions are material. 12 Original instructions in French.

THE ILLIQUIDITY GAME

Screen 9. In this game there is an additional risk stemming from group behavior.

As before, you get 20 cents at each round. You can use them to buy the right to play a lottery that delivers either 30 cents or 0 cents. As before, this external probability of the zero payoff is indicated at each round.

At difference with the previous game:

-if at least [50%] of the players in the group participate, the loss / win outcome depends only on this external probability.

-if less than [50%] of the players participate to the lottery, the gain is [zero] for all players who participated to the lottery.

Slide 10. Examples

Example 1. There are 20 subjects. The displayed probability of loss is [1q]. For this level of risk, 6 players participate, 14 do not. Less than [50%] participate.

Outcome: All those who participate get zero, all those who did not keep the 20 cents.

Example 2. There are 20 subjects. The displayed probability of loss is [1q]. For this level of risk, 18 players participate, 2 do not. More than [50%] participate.

Outcome: Those who did not participate keep the 20 cents. Those who participate get 30 cents with a probability [q] and zero with a probability [1q]. If "less" -your gain is zero. If "more": you earn 30 cents (or you get zero).

Screen 12. Your total gain is [A] euros. Thank you for your participation to this experiment.

Figure 3 :

 3 Figure 3: Critical thresholds in the "illiquidity game".

Figure 4 :

 4 Figure 4: Order-n default thersholds in five experiments

  In a world where financial markets would further deepen and broaden, the lack of coordination of lenders or investors might well generalize as a factor of risk. The Asian crisis of 1997, the Argentinean default of 2001, the 2007-2009 Great Recession are representative of the dramatic consequences of sudden illiquidity upspring.

  Welcome to the ESSEC Experimental Laboratory. This experiment aims at studying the interaction between two risks and their consequences on the individuals' decisions. There will be two stages, one warming-up non-paid stage, and a paid stage. Pay attention please: the game progresses when the last player take his/her decision.THE WARMING-UP GAMEScreen 2. Lotteries. It costs you 20 cents to participate to a lottery. The lottery will allow you either to gain 30 cents or nothing.

Screen 7 .

 7 You will receive 20 cents at each round of the game. You can use them to buy the right to play a lottery. If you play the lottery, you lose and the gain is zero or you can win 30 cents. The loss probability will be displayed on the screen. This is the true probability used by the computer to pick the outcome. If you answer Yes to the question "do you accept to play the lottery" the participation fee is charged and you will benefit of the outcome of the lottery. Payment rule: if you do not play, the 20 cents are kept on your account. If you play and win, your account is credited of 30 cents. If you play an loose, your account is credited of 0 cents. Screens 8 and next. Number of the actual round [1 to 14]. The loss probability is [1q]. 13 "Do you accept to play this lottery ?" Yes/No. Screens 8' and next. Outcome of the round. You decided not to play; your gain is [20]. OR You decided to play; you gain is [0 or 30].

  Screens 11 and next. Number of the actual round [1 to 31]. The loss probability is [1q]. 14 At the former round, the percentage of subjects that participated to the lottery was of [y%]. "Do you accept to play the lottery ?" Yes/No. Screens 11' and next. Outcome of the lottery. The loss probability was [1q]. You decided to participate (not to participate). The percentage of players is [z%], i.e. more (less) than [50%].

Table 1 .

 1 Number of students that choose to play the lottery in the no-interaction game

	30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70

  1 st default Lend again 2 nd default Lend again 3 rd default Lend again 4 th default Lend again

	May 26, 2010
	N=9	0.40 (4)	0.25 (7)	0.40 (0)	0.25 (6)	0.35 (1)	0.25 (6)	0.35 (1)	x
	June 9, 2010
	N=15 0.45 (4)	0.35 (10)	0.40 (6)	0.30 (12)	0.40 (1)	0.30 (13)	0.40 (0)	0.30 (12)
	Nov. 3, 2010
	N=16 0.40 (7)	0.25 (8)	0.35 (6)	0.25 (10)	0.30 (7)	0.25 (15)	0.30 (2)	0.25 (14)

Table 2 .

 2 Illiquidity and return-to-market thresholds (1q).

Table 4

 4 1 st default Lend again 2 nd default Lend again 3 rd default Lend again 4 th default Lend again

	April 6, 2011
	N=10 0.50 (2)	0.35 (6)	0.45 (2)	0.40 (6)	0.45 (2)	0.40 (6)	0.45 (1)	0.40 (6)
	May 18, 2011
	N=12 0.55 (1)	0.35 (7)	0.45 (1)	0.35 (8)	0.45 (2)	0.35 (11)	0.45 (2)	-

. Illiquidity and no-illiquidity thresholds (1q).

Table 5 .

 5 The dependent variable is the illiquidity default threshold QR jn . Model 1 is a special case of model 2, where we omit the

	quadratic term.
	Variable	MODEL 1 MODEL 2
	C	0.4581***	0.517***
	n	-0.0175*** -0.0700***
	n^2	-	0.0091***
	Fixed effects (cross section)
	WS1-F	-0.0394	-0.0356
	WS2-F	0.0043	0.0023
	WS3-F	-0.0756	-0.0776
	BS1-F	0.0543	0.0523
	BS2-F	0.0606	0.064319
	Nb obs. (unbalanced) 23	23
	R 2 -Adjusted	0.857	0.922
	Proba(F-stat)	0.00	0.00
	(***) indicates significance at 1%.

Table 5 .

 5 Output of the regression model

Governments also can be victims of sudden and unexpected investors' run. See[START_REF] Summers | International financial crises: Causes, prevention, and cures[END_REF] and[START_REF] Reinhart | This Time is Different. Eight Centuries of Financial Folly[END_REF].

See also[START_REF] Besancenot | Financial instability under a flexible exchange rate[END_REF] for a similar decomposition of total risk into insolvency and illiquidity.

The core logic of these n-player games can be inferred from the classical two player 'stag hunt' game. SeeCamerer (2003) for an analysis of its equilibria and empirical tests.

[START_REF] Berninghaus | Coordination and information: Recent experimental evidence[END_REF] consider the same game under different information treatments. See also[START_REF] Devetag | Coordination and information in critical mass games: An experimental study[END_REF]. A related problem is the "dying seminar game" introduced by[START_REF] Schelling | Micromotives and Macrobehavior[END_REF], and analyzed in an experimental setting bySemeshenko et al. (2009).

The finance literature also brings a significant contribution to understanding the illiquidity risk, defined as a situation where suddenly the various arbitrageurs refuse to buy assets. For instance,Acharya and Pedersen (2005) present a simple theoretical model that helps explain how asset prices are affected by liquidity risk and commonality in liquidity. The spiral of panic-driven asset liquidation, specific to the 2008 turmoil, was analyzed by[START_REF] Pedersen | When everyone runs for the exit[END_REF]. See also[START_REF] Besancenot | Default on sustainable public debt: Illiquidity suspect convicted[END_REF] and[START_REF] Besancenot | Financial instability under a flexible exchange rate[END_REF] for another model of interaction between insolvency risk and illiquidity risk (without coordination difficulties).

These scholars have designed an experiment of a currency speculative attack model, where chances that devaluation occurs depend on the number of traders that attack the currency.

If 50% or more of the players participate to the lottery, individuals who participated receive the actual outcome of the lottery (such as drew by the computer with the announced probability).

Since in this lottery, risk neutral individuals should stop lending for a risk above 0.33, these results indicate that in our setup some individuals might have a preference for risk.

These subjects did not participate to the lottery in the "no interaction game for" a small risk, then participated for a higher risk, than did not for a even higher risk. Is is therefore impossible to determine their reservation risk.

In turn, a higher risk premium, would push out of the market other lenders. This feed-back mechanism is central to many of the theoretical papers mentioned in the Introduction.

If players can implement some form of coordination, probably the number players should matter.

From one round to another, 1q will be increased by 0.05, from 0.05 to 0.70.

From one round to another, this probability will vary starting from 0.15 according to the rule described in Section 2.

experiment was first implemented with the help of Jean-Pierre Choulet from ESSEC Business School. This version was developed by Delphine