S Blassiau

A Bunsell

A Thionnet

MICROMECHANICS, DAMAGE AND LIFE PREDICTION OF CARBON FIBRE COMPOSITE PRESSURE VESSELS

A study by 3D finite element analysis has revealed the changes in the stress field around broken fibres in cfrp by considering the evolution of representative elementary volumes of the composite for both elastic and viscoelastic matrices and the effects of debonding. The viscoelastic behaviour of the matrix does not change the mechanism of load transfer but introduces time as a parameter. At a constant load, the relaxation of the matrix increases the coefficients of load transfer around broken fibres and causes new failures of fibres. This analysis relates the acoustic emission detected during steady loading to fibre failures. In this way a model to predict the lifetime of pressure vessels has been developed.

INTRODUCTION

The use of filament wound pressure vessels is hindered by the lack of effective means of determining the continuing reliability of the structure whilst in service. All pressure vessels require periodic testing and proof testing and, in service, techniques have been developed for metal structures. Some of these techniques require the vessel to be pressurised to above (usually 1.5 times) the maximum pressure in service. These techniques are not however applicable to fibre reinforced composite pressure vessels, as the nature of damage accumulation is different. Composite pressure vessels do not fail by simple crack propagation but, in the absence of stress raisers or localised damage, by the accumulation of a more diffuse and global type of damage, of which the most important is the failure of fibres. The overload test is therefore discredited as it leads to an increase in overall damage. There is no technique available which allows an evaluation of minimum residual lifetimes of composite pressure vessels to be made. Nevertheless the use of natural gas as a fuel for buses and trucks is increasing rapidly as it is environmentally friendly and of low cost. Such a pressure vessel, made of high strength carbon fibres in an epoxy matrix (cfrp), is shown in Figure 1. This study has used the acoustic emission technique in order to monitor the accumulation of damage.

Acoustic Emission (A.E.) has previously been shown to be a useful technique for monitoring damage in composites and, by localization, for identifying concentrated areas of damage. From earlier studies of cfrp, in the form of both plates and pressure vessels, and taking into account the shape of the stress fields within the filament wound pressure vessel, it has been possible to make an analogy between the accumulation of damage in these structures and in unidirectional cfrp, as, for both, the fibres are subjected to uniaxial loading. The present study has confirmed the observation that the number of emissions monitored versus time, under a constant load, obeys an empirical law which allows the calculation of damage accumulation as a function of time. An analysis of the fibre arrangement in composite pressure vessels shows that the damage monitored is due to fibre failure. In the absence of stress concentrations the fibre failures are distributed randomly in the composite structure.

The continuing accumulation of fibre failures under constant load is due to the effects of the viscoelastic properties of the matrix coupled to carbon fibres, which show a wide scatter in their strengths. The elastic nature of carbon fibres suggests that, after an initial strain, no further evolution of the composite, loaded parallel to the fibres, should be observed. As a consequence no emissions should be detected. However the experimental results show that damage continues to accumulate. This is interpreted as being due to local relaxation of the matrix around fibres breaks producing a local overload of the neighbouring intact fibres. This local increase in load, combined with the scatter in fibre strengths, leads to an increased probability of additional fibre failures. This process has been investigated in the present study.

A local finite element analysis, of the region around a fibre break, of the mechanisms governing composite damage, has shown that even a simple viscoelastic law describing the behaviour of the bulk epoxy resin leads to an increasing effective extra load on the neighbouring intact fibres, as a function of time. The calculations were carried out on different representative 3D unit cells (cf. Figure 2), corresponding to different states of composite damage. The results of the calculations agree well with the results found by different analytical methods of the shear-lag type. This agreement supports the conclusions arrived at in this study which could not be obtained by other methods, in particular because they result from a better definition of the local state of the stress fields in the vicinity of the broken fibres. The effects of the interfacial decohesion along part of the broken fibre near the break have been shown and the overload coefficients obtained without decohesion agree with those determined analytically by Hedgepeth and Von Dyke [START_REF] Hedgepeth | Local Stress Concentrations in Imperfect Filamentary Composite Materials[END_REF] and more recently by Landis [START_REF] Landis | An improved Shear-Lag Model for Broken Fibers in composite Materials[END_REF]. Moreover, we can compare the effect of the fibre volume fraction on the ineffective length of the broken fibre. With this numerical approach, we can establish an accurate definition of the local load sharing versus time and along the fibre. In addition, this method can take into account all possible behaviour laws for the resin. In this way it is possible to determine the accumulation curve of fibre failures as a function of time induced by matrix relaxation around fibre breaks from results obtained using the statistics of carbon fibre failure and knowing the approximate number of fibre failures distributed randomly in the composite on initial loading at t=0 for a period of steady loading. The forms of the experimental curves obtained by acoustic emission on unidirectional composites and filament wound vessels and the theoretical curve obtained from the finite element calculation are identical.

The study has shown that the experimental A.E. curve representing the evolution of composite damage under a steady given load, can allow a minimal lifetime of the pressure vessel to be determined. In this way a limiting damage curve can be defined over the envisaged lifetime of the pressure vessel, which determines a master curve. A comparison between this master curve and the rate of damage accumulation of any other pressure vessel of the same type reveals if it will become unstable before or after the desired lifetime. It is proposed that the in service control of carbon fibre composite pressure vessels of the type used on many buses and other vehicles should be based on this approach.

MATERIALS

Plate and filament wound carbon fibre reinforced epoxy specimens have been tested. The plates were unnotched, rectangular, 270 x 15 x 0.9 mm unidirectional cfrp specimens with a fibre volume fraction of 60 %. The fibres in these specimens were aligned parallel to the long axis and had to be broken for the composite to fail. These specimens were subjected to tensile loading. Pressure vessels made of carbon fibres, filament wound onto an aluminium liner, which served both as a mandrel and provided a gas proof seal, have also been tested and comparisons made between the behaviours of the two. A typical pressure vessel as shown in After manufacture, the pressure vessels had been subjected to hydraulic pressure tests of 300 atmospheres (~30 MPa), which corresponded to 1.5 times the operating pressure.

In this study, damage occurring in the two types of specimens was monitored with a MISTRAS 2001 acoustic emission system which comprised two R15 piezoelectric transducer with a resonant frequency of 30KHz, and two preamplifiers with fixed gains of 40 dB. The use of two detection channels permitted the location of the events.

The pressurization of the vessels was carried out with the assistance of a hydraulic pump operating up to a maximum pressure of 1000 bars (~100 MPa). The vessels were pressurised with water.

THEORY

The development of damage, in and ultimate failure of, a carbon fibre unidirectional or filament wound composite structure, in which the fibres are arranged so as to transfer maximum strength and rigidity to the structure, depends on the failure of the fibres. The failure probability as a function of applied stress on the fibres can be determined experimentally and is described by Weibull statistics, which gives the probability of fibre failure as :

P = 1 -exp {-∫[(σ -σ u )/σ o ] m V ( 1
)
Where P is the failure probability of a fibre of volume V, σ is the applied stress, σ u the stress failure threshold and σ o and m material constants. The factor m is known as the Weibull modulus and describes the dispersion of fibre strength.

Loading the composite in the direction of the fibres produces fibre breaks which are randomly distributed in the volume of the composite, in the absence of any stress concentration, as there is a wide dispersion of fibres strengths Each fibre break is isolated from other breaks by the shear of the matrix which produces a load transfer along the fibre so that at a critical distance from the break the fibre is able to continue to support the load [START_REF] Rosen | Tensile failure of fibrous composites[END_REF]. This means that each fibre break creates a damage zone around the break, the limit of which is determined by the shear properties of the matrix and the Young's modulus of the fibre. In addition and importantly, there is a debonded length between the fibre and the matrix around the break which increases the load transfer length. The intact fibres neighbouring a broken fibre experience an increase in stress over a short length of each fibre as the total load supported by the plane passing through the fibre break has to remain unchanged. Shear stress relaxation of the matrix means that, with time, the lengths of the damage zones and the lengths of neighbouring intact fibres which experience increased stresses, become longer.

As the probability of failure increases with fibre length, for a given stress, additional fibres fail which can be detected by acoustic emission and the damage zone increases in area. When the damage zones begin to interact damage accumulation accelerates and this represents the onset of instability which is taken as the definition of the end of useful life of the pressure vessel.

Relaxation of the shear stress leads to an evolution of the size of the damage sites and an accumulation of fibre failures, detected as acoustic emission events, which has been shown to obey a law of the following form :

( ) n t A dt dN τ + = (3)
where N is the number of acoustic emission events,

• t is the time •

A is a factor which depends only on the applied stress • τ is a time constant • and n is a factor less than unity.

By defining an upper damage threshold in terms of a maximum number of acoustic emission events which is acceptable it now becomes possible to use equation (3) to create master curves for structures which have exactly the lifetimes required, as shown in the curve of acoustic emission as a function of time, in Figure 2. If a unidirectional specimen or a pressure vessel is expected to survive for more than the twenty years considered with the master curve the acoustic activity detected from the structure under predetermined steady loading conditions must lie on a curve under the appropriate master curve. It is not feasible to obtain all the curve of a pressure vessel however at any given time its rate of acoustic activity must be lower than that of the master curve. In this way a comparison of the activity during a short period with the activity predicted by the master curve allows an evaluation of the residual lifetime of the pressure vessel to be made. If the number of emissions during the test is less than that given by the master curve the vessel is acceptable. If not it should either be changed or subjected to further examination.

EXPERIMENTAL RESULTS

Unidirectional and pressure vessels have been taken to failure in order to obtain the value of the maximum threshold damage level. The numbers of events for each type of specimen differed, as would be expected due to their differing sizes and shapes, however for each type of specimen the total number of events to failure showed a dispersion of less than 10%. In order to verify the assumptions of the model, unidirectional specimens were subjected to steady loads of 70 to 95% of failure load for periods of twelve hours. Figure 3 shows that, in all cases, the numbers of events varied as the logarithm of time and that the higher the applied load the greater the number of events which were recorded. The curve shown for the highest applied stress shows a jump in numbers of events which occurred when a packet of fibres was observed to fail at the edge of the specimen. This event induced a redistribution of the load in the specimen and an increase in acoustic activity. This observation reveals how the evolution of a failure zone can lead to the increased rate of damage accumulation of the remaining intact material. Tests were carried out on pressure vessels at different pressures but as the maximum service pressure was 200 atmospheres (20 Mpa) particular attention was paid to vessels pressurised to this level. Figure 4 shows a typical curve obtained with a pressure vessel.

It can be seen that the results obtained from both types of specimen are very similar.

THEORETICAL MODELLING

Since the aim is to explain the damage mechanics of unidirectional composites, the scale of the study that will reveal the beginning and the coalescence of these mechanisms must distinguish between the fibres and the matrix at the mesoscopic scale. At this scale, observations (by optical microscopic) of sections of the composite part of the pressure vessels show that the fibre distribution in the matrix is relatively regular. Consequently, the description of the R.E.V. (Representative Elementary Volume) which is the elementary periodic cell (or, more simply, the elementary cell) can be defined. Figure 5 shows a section of the composite and the distribution of the fibres and matrix. In places the fibres can be seen to be arranged in an hexagonal pattern whereas elsewhere there are resin rich areas. The relationship between the complexity of the cell geometry and the need to represent accurately the real material led to a square distribution of the fibres being adopted for the model.

Square pattern

Hexagon pattern Epoxy Matri

Carbon fibres

Figure 5 : The fibres can be seen to be organised in a square or hexagonal pattern in parts of the composite but there are also resin rich areas. An average square pattern has been modelled using finite element analysis.

The geometrical definition of the R.E.V. is important. Actually, studies [START_REF] Hedgepeth | Local Stress Concentrations in Imperfect Filamentary Composite Materials[END_REF][START_REF] Nedele | Three dimensional finite analysis of the stress concentration at a single fibre break[END_REF][START_REF] Lifshitz | Time dependent longitudinal strength of unidirectional fibrous composites[END_REF][START_REF] Van Den Heuvel | Failure phenomena in fibre-reinforced composites. Part 6: a finite element study of stress concentrations in unidirectional carbon fibre reinforced epoxy composites[END_REF] on the influence of one broken fibre on its surroundings have shown that the perturbation caused to the stress field is limited to the nearest neighbouring intact fibres: the intact fibres surrounding the broken fibre play a shielding role where other fibres further away from the break are concerned. Beyond the nearest third intact fibre the failure has almost no effect on the stress field. For this reason several R.E.V.s have been chosen to represent the different composite damage. Figure 6 shows an example of a typical R.E.V. used to represent the composite damage (the precision on these R.E.V. will be discussed in a subsequent article) For these calculations, the viscous nature of the matrix is represented by a linear viscoelastic model. The chosen model corresponds, in the one-dimensional case, to the model known as a Zener solid, in which the stress field is described as where γ is the relaxation tensor. C0 is the initial rigidity tensor and c the infinite rigidity tensor. In order to have a basis of comparison, a linear elastic analysis has been carried out followed by a viscoelastic analysis.

ε γ ε σ γ σ ∞ + = + C C & & 0
For the analysis of results, two coefficients K r and L c are used to evaluate the evolution of the stress in the fibres neighbouring the broken fibre and in the broken fibre as a function of time and debonded length.

Kr is equal to the average of the axial stress in the fibre between two sections of the fibre defined by the finite element mesh with respect to the value at that point before fibre failure.

L c defines the inefficient length. The value of L c is determined when the load transfer coefficient in the broken fibre is equal to 0.9.

MODELLING RESULTS AND DISCUSSION

a. Failure without debonded length Figure 9 represents the evolution of the load transfer coefficient in the intact fibre nearest to the broken fibre for both an elastic and viscoelastic matrix. This Figure, in spite of the differences of behaviour, shows that the mechanisms of load transfer are identical.

At first, the coefficients determined for the elastic case with this cell, give a good estimation of the value of the overloading in the nearest fibres. The coefficients evaluated by this finite element method give coherent values (Figure 8) which are more reliable than those obtained by analytical methods. The evaluated coefficients K r are much lower than those obtained by the Shear Lag method. With the variation of fibre volume fraction (Figure 8), the coefficient of load transfer is changed and as Van den Heuvel [START_REF] Lifshitz | Time dependent longitudinal strength of unidirectional fibrous composites[END_REF] noticed, the inefficient length decreases proportionally with an increase in fibre volume fraction. The length Lc is respectively 92µm, 68µm and 34µm for a volume fraction of 40%, 60% and 77%. It should be noted that the value of K r in the plane of failure is independent of fibre volume fraction since Kr is constant. It is only along the length of the fibre that the influence of the fibre volume fraction changes the coefficient.

Figure 8: Evolution of load transfer coefficients as a function of fibre volume fraction

The viscoelastic matrix introduces time as a supplementary dimension. The introduction of time in the model enables the change from the static state to the fluid state to be described. At the beginning of the creep, the Kr coefficients agree with the coefficients obtained in the elastic model. The only distinction is the slope of the K r curve that is more gradual for the viscoelastic matrix. For long term creep, compared to the elastic case, a notable modification of Kr and load transfer can be noticed. For the nearest fibre to the failure, independently of all damage phenomena, an increase of load transfer can be observed. So, in the broken fibre, the inefficient length grows between 0 and 172000s from 108 to 120µm. This observation justifies the requirements of our damage model based on the work of Lipschitz and Rotem [START_REF] Van Den Heuvel | Failure phenomena in fibre-reinforced composites. Part 6: a finite element study of stress concentrations in unidirectional carbon fibre reinforced epoxy composites[END_REF] which explains the possibility of differed failure of unidirectional composites. The introduction of a debonded length (Figure 10) does not change the mechanics of load transfer but entails a important modification of load transfer coefficients from the broken fibre to the nearest intact fibres. A phenomenon, cited recently by Van den Heuvel [START_REF] Lifshitz | Time dependent longitudinal strength of unidirectional fibrous composites[END_REF], of growth and decrease of Kr with the rise of debonded length is observed with this model. A peak in the load transfer is located at the beginning of the zone of perfect adhesion. The debonded length increases proportionally with the inefficient length. The solution is to make a multi scale process that could predict the structure lifetime from the data on the load transfer and on the probability of carbon fibre failure at the mesoscopic scale. The calculation made on the different cells gives, for macroscopic stresses, microscopic responses that can link the two scales.

The multi scale process coupled with a finite element calculation can simulate a tensile test of a unidirectional composite and predict the tensile failure and follow the fibre failures. The curve obtained is similar to the acoustic emission curve when a tensile test is being monitored by acoustic emission.

Most importantly the model allows the behaviour and failure of the composite to be predicted under steady loads, since, for every cell at the mesoscopic scale, the stress field changes with time and causes the degradation of the rigidity of the element, at the macroscopic scale, which in turn can cause the failure of the structure.

APPLICATION TO PRESSURE VESSELS

Using equation 3 this curve can be extrapolated over a period of twenty years and compared to a master curve giving an exact lifetime of twenty years, as shown in Figure 2. It is assumed that control tests have to be carried out after manufacture and then every three or five years. It can be seen that at each control the gradient of the experimental curve is less than that of the master curve for the case of a pressure vessel which is acceptable for further service. The values of A and τ in equation 3 for pressure vessels subjected to 200 atmospheres (~20 MPa) (were obtained from the experimental curve and this allowed the master curve to be calculated.

It now becomes possible to draw curves which can be used in a practical tests. The numbers of emissions expected from the master curve depend on the length of time the control test is conducted and this can be chosen so as to accommodate the needs of the bus operator. Figure 2 shows curves for a control period of 24 hours corresponding to pressure vessels which have been in service for different periods of time.

Figure 10 shows the numbers of events recorded for one such test over a period of 24 months. The master curve used presents a coefficient of security superior to 10 and predicts the acoustic behaviour of the vessels for 20 years in service. For the different control of 24 hours, the experimental points are under the master curve and shows perfect mechanical stability.

DISCUSSION

The monitoring of acoustic emission has shown that carbon fibre reinforced composites continue to accumulate failure events even when they are subjected to a constant load, or pressure in the case of a pressure vessel. The numerical investigations have justified the influence of the matrix relaxation and the effect of debonded length on the coefficients of load transfer. The relaxation mechanism control the delayed failure of neighbouring intact fibres around the broken fibres. This leads to an increasing influence of the break and an overload effect on the intact fibres. A local increase in stress on intact fibres inevitably causes further breaks, as the strength of fibres shows a wide scatter. Acoustic emission is a technique which allows this damage to be followed. The regular rate of emission which is detected reflects the effects of shear stress relaxation around fibre breaks which leads to further failures. The accumulation of damage can therefore be modelled to give the accumulation of damage as a function of time. For a given series, of supposedly identical pressure vessels, initial tests can identify the maximum acoustic emission which is acceptable and so master curves can be drawn for pressure vessels with just the lifetimes required. Pressure vessels at manufacture or in service can be controlled periodically by a comparison between their actual acoustic emission count in a given period and the count predicted by the master curve for the same period. If the actual count is less than the limit given by the master curve the pressure vessel is deemed acceptable for further service. If not it should be withdrawn from service.

The technique which is proposed allows the natural overall ageing of carbon fibre pressure vessels to be determined and minimum residual lifetimes calculated. The acoustic emission technique can also be used to locate localised damage which may occur for example due to an impact.

CONCLUSION

The numerical studies on the influence of broken fibres on the neighbouring intact fibres in a viscoelastic matrix justifies the monitoring of pressure vessels under constant pressure by A.E. so as to determine their level of accumulated damage due to in-service use. By a Multiscale process, a tensile test of a mesh, representing a unidirectional cfrp specimen, can be simulated, showing the accumulation of breaks similar to experimental results obtained by monitoring acoustic emissions from real specimens.

Thanks to this study, a non destructive evaluation technique has been developed for carbon fibre composite pressure vessels. The technique uses the acoustic emission technique to monitor damage accumulation during periods of fixed pressurisation. A model based on the relaxation of the resin around fibre breaks has been described which accounts for the delayed failure of fibres which can eventually lead to failure of the pressure vessel. The number of counts recorded in a given period is compared to those predicted by master curves and, depending on whether the counts are below or above the master curve for the pressure applied, the vessel is deemed acceptable for further service or not.

Figure 1 :

 1 Figure 1: A typical carbon fibre pressure vessel

Figure 2 :

 2 Figure 2 : The accumulated AE events curve is compared to a master curve calculated for a life of 20 years. If the composite structure is acceptable for further use its curve should be lower.

Figure 3 :

 3 Figure 3 :Acoustic events recorded at different steady stress levels on unidirectional specimens.

Figure 4 :

 4 Figure 4: Comparison between the experimental and theoretical curve of acoustic emission for different types of pressure vessel.

Figure 6 :

 6 Figure 6: Example of a Representative Elementary Volume of a damage state and geometry of the Periodic Elementary Cell (the red circles and R represent the broken fibres)

ForFigure 9 :

 9 Figure 9: Comparison of the evolution of load transfer coefficient in the adjacent fibre and in the broken fibre for elastic and viscoelastic matrices b. Failure with debonded length

ForFigure 10 :

 10 Figure 10: Evolution of load transfer coefficients for three debonded lengths Nevertheless, this possibility of an evolution of the load transfer associated with the stochastic data of the carbon fibre failure can explain the possibility of differed failure of composite structures loaded in creep under the theoretical tensile failure load.

Figure 10 :

 10 Figure 10: Comparison of the master curve and experimental results showing the stability of pressure vessels.

ACKNOWLEDGEMENTS

This study has, in part, been financed by the Gaz de France. An International Patent on the technique described has been made.


Maria Evangelia Chatzimina
email: hatziminamaria@gmail.com

Cyril Grouin

Pierre Zweigenbaum

Use of unsupervised word classes for entity recognition: Application to the detection of disorders in clinical reports

Keywords: Clinical Texts, Natural Language Processing, Unsupervised Word Classes

published or not. The documents may come

Use of unsupervised word classes for entity recognition:

Application to the detection of disorders in clinical reports

Introduction

Clinical texts and biomedical literature are important sources of medical knowledge. The exponentially growing amount of biomedical sources in recent years requires dedicated tools to process this domain [START_REF] Meystre | Extracting information from textual documents in the electronic health record: a review of recent research[END_REF]. Named entity recognition is the task that identifies an entity's boundaries within text and assigns the entity to their corresponding class or category. It is usually one of the basic steps applied when extracting information from texts. Supervised learning based on annotated corpora, typically using sequence learning algorithms such as Conditional Random Fields [START_REF] Lafferty | Conditional Random Fields: Probabilistic models for segmenting and labeling sequence data[END_REF][START_REF] Sutton | An introduction to Conditional Random Fields for relational learning[END_REF], is now a standard method to detect entities. Unannotated corpora are larger than annotated corpora, making them useful for unsupervised methods to learn word classes and help supervised methods perform entity detection [START_REF] Turian | Word representations: a simple and general method for semisupervised learning[END_REF]. The main objective of this work is to study the use of unsupervised word classes in medical entity recognition, and more specifically to test different methods and variants of unsupervised word class construction which use more initial information thanks to syntactic analysis. The remainder of the paper is organized as follows. Section 2. provides a brief description of previous work related to our study. Section 3. describes the corpora we take as a testbed for our experiments. Section 4. describes the design and implementation of our study. Section 5. presents and discusses experimental results. We conclude in Section 6.

Related Work

In an early attempt to help supervised learning with the use of supervised word classes, [START_REF] Brown | Class-based n-gram models of natural language[END_REF]'s clustering algorithm was used to detect standard named entities [START_REF] Miller | Name tagging with word clusters and discriminative training[END_REF]. The combined method achieved a 25% reduction in error on a standard named-entity problem. In further experiments, [START_REF] Turian | Word representations: a simple and general method for semisupervised learning[END_REF] also tested the word embeddings obtained by neural language models [START_REF] Collobert | A unified architecture for natural language processing: deep neural networks with multitask learning[END_REF][START_REF] Mnih | A scalable hierarchical distributed language model[END_REF] but observed that Brown clusters were superior when helping chunking or named entity recognition. Using only one type of word representation, Brown clusters had the highest F-measure on the test set compared to [START_REF] Collobert | A unified architecture for natural language processing: deep neural networks with multitask learning[END_REF]'s embeddings and the hierarchical log-bilinear model of [START_REF] Mnih | A scalable hierarchical distributed language model[END_REF], and even outperformed the use of gazetteers by 1 point of Fmeasure. Combinations of these unsupervised word representations gained close to another point of F-measure. In the biomedical domain, Brown clusters have been used on top of domain-specific, knowledge-based semantic classes such as the UMLS semantic types [START_REF] Bodenreider | The Unified Medical Language System (UMLS): Integrating biomedical terminology[END_REF]) for medical entity recognition and relation detection (de Bruijn et al., 2011). The specific contribution of Brown clusters to this top-performing system in the i2b2 2010 challenge [START_REF] Uzuner | 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text[END_REF] amounted to 0.14 points of F-measure for entity recognition. [START_REF] Jonnalagadda | Enhancing clinical concept extraction with distributional semantics[END_REF] computed a thesaurus of the 20 distributionally most similar words for each input word and obtained a 2pt increase in F-measure over the same data set. [START_REF] Tang | Recognizing clinical entities in hospital discharge summaries using Structural Support Vector Machines with word representation features[END_REF] tested Brown clusters and also the distributionally most similar words as features to help medical entity recognition on the i2b2 2010 data set. They obtained increases of 0.40-0.56pt F-measure in a system which performs at the same level as that of [START_REF] De Bruijn | Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010[END_REF], i.e., starting from a stronger baseline than [START_REF] Jonnalagadda | Enhancing clinical concept extraction with distributional semantics[END_REF]. [START_REF] Jonnalagadda | Using empirically constructed lexical resources for named entity recognition[END_REF] used distributional representations of words built with the Semantic Vectors package of [START_REF] Widdows | The semantic vectors package: New algorithms and public tools for distributional semantics[END_REF] to create word clusters, a 'quasi-lexicon', and a thesaurus as above (20 nearest neighbors): the thesaurus improves entity recognition on the i2b2 2010 data set by 1.6pt F-measure, whereas the quasi-lexicon and clusters obtain smaller gains. They also observed that a combination of distributional word classes (quasi-lexicon and thesaurus) could replace UMLS-based word classes with no loss (and even an increase) of performance. The present work also aims to assess the relative contribution of unsupervised word classes to medical entity detection, compared to knowledge-based semantic classes such as provided by the UMLS. Besides, it tests new word classes which include syntactic information in their construction, which to our knowledge has not been done previously.

Corpora

We used two types of corpora:

• The annotated corpora from the ShARe/CLEF eHealth Evaluation Lab [START_REF] Suominen | Overview of the ShARe/CLEF eHealth evaluation lab[END_REF]. The datasets consist of de-identified clinical free-text notes from the MIMIC II database1 [START_REF] Saeed | Multiparameter Intelligent Monitoring in Intensive Care II: a publicaccess Intensive Care Unit database[END_REF] of Intensive Care Unit (ICU) data. The clinical reports were given with stand-off annotations of disorder mention spans and UMLS concept unique identifiers; we do not consider the latter in the present mention detection task, which corresponds to Task 1A of CLEF eHealth.

• A larger unannotated corpus obtained from the MIMIC-II database besides the CLEF eHealth challenge. It contains about 18,000 discharge summaries. For supervised medical entity detection, we reuse a pipeline of components [START_REF] Bodnari | A supervised named-entity extraction system for medical text[END_REF] prepared for the CLEF eHealth challenge. The features produced by this pipeline included knowledge-based semantic classes, but no datadriven classes. Supervised classification was performed with the Wapiti CRF toolkit [START_REF] Lavergne | Practical very large scale CRFs[END_REF]. 2 The baseline system (later called nosem) uses the following features:

• Lexical features: token, lemma.

• Morphological features: (i) token containing only upper case letters, (ii) token is a digit, (iii) is capitalized and (iv) is a punctuation.

• Syntactic features: part of speech information extracted with the cTAKES system [START_REF] Savova | Mayo clinical Text Analysis and Knowledge System (cTAKES): architecture, component evaluation and applications[END_REF].

• Document structure features: document type and section type, extracted with a rule-based method which identifies occurrences of section names in the text [START_REF] Bodnari | A supervised named-entity extraction system for medical text[END_REF].

For most types, a feature is computed based on the previous position in the sentence, current position and next position (for unigram features), and based on the previous+current positions or current+next positions (for bigram features). This is also the case for the semantic classes defined below.

Figure 1 provides an overview of the pipeline used to produce semantic features, which we detail below. Table 2 shows an excerpt from the corpus in a simplified tabular form, processed by our pipeline. For each token (first column), a set of attributes has been computed. We only show a small subset of these attributes: lemma, partof-speech, syntactic chunk, the concept unique identifier (CUI) from the UMLS, section from the document (herein, "History of Present Illness"), the cluster ID produced using Brown's algorithm (for a given configuration). We also show the gold standard word label and the label predicted by our system.

Knowledge-based semantic classes

Semantic classes for words were obtained from two humancurated resources [START_REF] Bodnari | A supervised named-entity extraction system for medical text[END_REF].

The UMLS. The Unified Medical Language System (Bodenreider, 2004) contains a large repository of biomedical terms with associated semantic types. We produced for each word eleven attributes based on various ways to access this resource.

The Wikipedia. The Wikipedia contains an increasing number of articles relevant to the medical domain. We used Wikipedia category data to collect a lexicon with nine semantic groups, which was used to produce two attributes for each word.

Unsupervised, data-driven word classes

Unsupervised learning induces word representations from unannotated corpora. We explore here the use of Brown clusters [START_REF] Brown | Class-based n-gram models of natural language[END_REF] in their usual form as well as a novel application of this method to dependency relations (Section 4.3.1.), then categories obtained through training a probabilistic context-free parser with latent categories (PCFG-LA, (Petrov et al., 2006), Section 4.3.3.). In all cases, unsupervised learning results in a lexicon which associates words to classes; this lexicon is then applied to the training or test corpus to add attributes to the input words for supervised entity detection.

Brown clusters

Brown clusters optimize a class bigram language model over the corpus on which they are computed, here the unannotated corpus of discharge summaries. We created Brown clusters using Liang (2005)'s implementation of [START_REF] Brown | Class-based n-gram models of natural language[END_REF]'s clustering algorithm, requesting 320 and 1000 clusters (these provided the best results in preliminary experiments) combined to five thresholds (2 1...5 ) on the minimum number of word occurrences. The 10 resulting clusterings were used as attributes to create CRF features for each input word: for each clustering, the identifier of the cluster which contains this word (or #NA if the word is not present in the clusters) is added as an attribute. An example of cluster is that named 01101111011111, computed with the unannotated corpus, which contains the following words: LLL, RLL, RUL, bibasilar, dense, diffuse, expiratory, faint, linear, multifocal, nodular, partial, patchy, retrocardiac, scattered, trace, etc., many of which occur as modifiers in disorder expressions in the training corpus: patchy opacities, retrocardiac opacification, expiratory wheeze, inspiratory wheeze, etc. Among these, RUL (right upper lobe) does not occur as a term in the UMLS. In the sentence "77 year old man with questionable RUL pneumonia," the word pneumonia is present as a disorder in the UMLS, but RUL is unknown and as a result the CRF only labels pneumonia as a disorder, missing RUL. When using Brown clusters, the feature for cluster 01101111011111 obtains a high score for being part of a disorder, and RUL is therefore labeled as part of the disorder too, thus obtaining the correct complete entity.

Dependency-based Brown clusters

In previous approaches, the Brown clusters were computed over bigrams in an input string of words. In the present work, we tested whether computing clusters based on syntactic dependencies instead of word bigrams might improve entity detection. The unannotated corpus was parsed with [START_REF] Mcclosky | Self-training for biomedical parsing[END_REF]'s self-trained parser and the output was converted into Stanford reduced dependency format [START_REF] De Marneffe | Stanford typed dependencies manual[END_REF]. Rather than trying to change the Brown algorithm, we tried to find a representation of the syntactic dependencies that would be amenable to processing directly by the existing algorithm and implementation. In this purpose, each dependency was represented as a pseudo-sentence, including the two related tokens (governor then dependent) and possibly the grammatical relation. This means that each initial sentence is replaced with a series of pseudo-sentences, one per dependency relation. We tested the following representations of dependency relations: (i) token-relation-token, (ii) relation-token-token, (iii) token-token-relation, and (iv) relation conjoined with the first token (relation-token), second token conjoined with the relation (token-relation) and two tokens without the relation (token-token). Experiments evidenced that the pair of tokens (token-token), without the relation, obtained the best results, so for the sake of space we only consider this setting in this paper. As for the clusters in Section 4.3.1., for each input word, dependency-based Brown clusters contribute 10 attributes generated by the two numbers of clusters and the five frequency thresholds.

Latent syntactic categories

We also tested whether the latent categories obtained by a PCFG-LA parser [START_REF] Petrov | Learning accurate, compact, and interpretable tree annotation[END_REF] can be used as (syntactico-)semantic categories. Parsers based on Probabilistic Context-Free Grammar with Latent Categories (PCFG-LA parsers) learn latent categories when trained on a treebank. These latent categories are obtained by splitting the syntactic categories present in the treebank in a way which optimizes parsing accuracy. We use these categories to create additional word attributes.

Default grammar. The unannotated corpus was parsed with the Berkeley PCFG-LA parser [START_REF] Petrov | Learning accurate, compact, and interpretable tree annotation[END_REF] using its default grammar. Latent categories for the corpus words were then extracted and used to build a lexicon with 'semantic' categories. However, the default English grammar of the Berkeley parser is not based on medical texts.

Specific grammar. We hypothesized that a better performance in disorder entity recognition could be achieved by using a grammar adapted to the specific domain and genre of our target corpus. We therefore specifically trained a grammar for this corpus. We assumed that the Charniak-McClosky parser [START_REF] Mcclosky | Reranking and self-training for parser adaptation[END_REF], which itself is self-trained on biomedical texts [START_REF] Mcclosky | Self-training for biomedical parsing[END_REF], should have a better performance on these texts than the vanilla Berkeley parser. We therefore prepared a training treebank by extracting 10% of the unannotated corpus parsed with the Charniak-McClosky parser, then used it with no manual correction to train a specific grammar for the Berkeley parser. This specific grammar was used by the Berkeley parser to parse the larger corpus.

As an example, one of the latent categories obtained by splitting the noun (NN) category in the corpus is NN-7, which contains (among other) the following words: Albuterol, postop, Insulin, Pantoprazole, Lopressor, platelet, oxygen, Furosemide, Glyburide, Prednisone, Atenolol, ejection, Levofloxacin, Toprol, most of which are drug names.

Building a lexicon of latent categories is slightly more complex than for Brown clusters. This is because depending on the context in which it occurs, a word in a sentence may receive a variety of latent categories. For each word, we collected all the categories it received in the automatically parsed corpus, and ordered them by descending number of occurrences. We only kept the categories with a number of occurrences greater than a threshold T , and selected the top N categories (possibly less if some or all of them did not pass the threshold). These ordered N categories were used to add N attributes to the word representation (again, #NA was used as a null value if less than N categories remained).

Based on initial experiments we kept two combinations of T and N : (T = 50, N = 2) and (T = 100, N = 1). Therefore, 3 attributes were created with this method.

Design of experiments

Given the above-mentioned sets of attributes, the goals of our experiments were to answer the following questions:

• What is the performance of knowledge-based semantic classes (i.e., classes based on the UMLS or on the Wikipedia) with respect to the baseline system with no semantic classes (Section 5.1.);

• In a setting where no knowledge-based semantic classes would be available, what is the performance of data-driven semantic classes (i.e., word-based Brown clusters, dependency-based Brown clusters, and latent syntactic categories); besides, do syntax-aware semantic classes bring an advantage over non-syntaxaware semantic classes (Section 5.2.);

• What is the relative performance of knowledge-based and data-driven classes (Section 5.3.).

All experiments were performed on the task of DISOR-DER entity detection, using ten-fold cross-validation on the CLEF eHealth 2013 training set, with unsupervised attributes computed on the unannotated MIMIC-II corpus (see Section 3.). The results displayed in the tables are average precision, recall, and F-measure over these ten folds.

Results and discussion

Baseline system and knowledge-based semantic classes

Table 3 shows the results obtained with a baseline system which uses no semantic classes (NOSEM), with a system which only uses one type of knowledge-based semantic class (Wikipedia or UMLS), or with a combination of these. Our baseline system emphasizes precision over recall, a characteristic often found in CRFs. The use of only UMLS attributes comes a few Fmeasure points (4pt) short of the baseline system NOSEM, which shows the importance of this knowledge source. Wikipedia-contributed attributes have a good precision too, but much lower coverage than UMLS attributes: this shows that Wikipedia is a resource of good quality which is worth considering even in a specialized domain, but that a large, dedicated resource such as the UMLS is best suited to the needs of the domain. Adding Wikipedia attributes on top

Baseline system and data-driven word classes

Table 4 shows the performance obtained by the same baseline system (NOSEM), by a system which only uses one type of data-driven word classes, or with a combination of these. Data-driven word classes alone. Latent categories alone obtain a better F-measure than Wikipedia alone because their higher recall more than compensates for a lower precision. Retraining the parser on a part of the domain-specific corpus results in categories that gain 2.5pt F-measure. Brown clusters alone outperform UMLS alone and even the baseline NOSEM attributes: in our setting, distributional analysis alone performs better than a combination of lexical, morphological, syntactic and document structure features. It does so by a strong increase in recall, at the expense of a decrease in precision. This is the inverse of the preference for precision of our baseline system. Dependencybased Brown clusters also bring an improvement of 0.7pt F-measure over bigram-based Brown clusters. PCFG-LA latent categories are obtained by splitting syntactic categories into more specific subcategories. We may thus compare them to using only part-of-speech (POS) information. A system trained with only part-of-speech information (not displayed in Table 4) obtains (P=49.58, R=22.16, F=30.63), to be compared to the rows for LA or LA-spec which obtain much higher results. We believe that the more specific categories, added to the description of each word by three categories instead of only one POS, account for this large difference.

Combination of data-driven word classes. When combined, Brown clusters and Latent categories further gain 2pt F-measure, which shows that they capture different types of information. Further adding the NOSEM attributes gains another 2pt F-measure. All in all, this falls short of NOSEM plus knowledge-based classes by 2pt F-measure.

On the one hand, we can say that rich knowledge-based classes such as obtained by the UMLS (and to a lesser extent Wikipedia) still do a better job of helping the detection of disorders, both in terms of precision and recall. On the other hand, we also observe that the huge human effort invested to build these resources only gains 2pt in our setting. Finally, we note that the combination of unsupervised word classes with the baseline NOSEM features boosts recall while leaving precision untouched.

Combination of baseline and data-driven word classes.

In combination with the NOSEM attributes, dependencybased Brown clusters bring a positive or negative contribution depending on the presence or absence of LA / LAspec attributes. In contrast, the specific grammar for latent categories improves the results compared to the default grammar in most of the cases. Nevertheless, in all cases combining Brown clusters with PCFG latent categories improves over either of them. the present setting where we can pool two kinds of rich knowledge-based classes, the data-driven classes only bring a moderate additional improvement. In contrast, adding knowledge-based classes (UMLS + Wikipedia) to the best combination of data-driven classes with NOSEM boosts the F-measure by 3.4pt F-measure: this underlines the importance of these classes when they are available.

Baseline system and all word classes

Ablation studies. Table 6 reproduces the best result and some of the rows of Table 5 to show ablation experiments. The most contributing set of attributes in this final configuration is the UMLS (3pt F-measure), followed by NOSEM (1.8pt) and word-based Brown clusters (1.0pt).

The remaining two sets of attributes (Wikipedia and LAspec) contribute much less to the final F-measure (0.5pt and 0.2pt). Besides ablation, we also examined the effect of the dependency-based variant of the Brown clusters: they increase the precision of the system, but decrease its recall, which leads to a slightly lower F-measure. We also confirm that using a specific grammar instead of the default grammar for the PCFG-LA parser improves the results, with a 0.5pt gain in F-measure (in fact, using the PCFG-LA word categories with the default grammar is worse than not using them at all).

Coverage of word classes

An important factor in understanding the systems is to measure the coverage of the attributes on the corpus. By this we mean the proportion of occurrences of tokens in the corpus for which a non-null value is provided by a lexicon for a given attribute. Knowledge-based semantic classes and data-driven semantic classes are each represented by a set of attributes: 2 for Wikipedia, 11 for UMLS, 10 for Brown clusters and 3 for PCFG-LA categories. In principle, data-driven classes are susceptible to obtain full coverage of the words in the annotated corpus, since they are induced from a much larger unannotated corpus of the same origin. Their actual coverage depends on the thresholds imposed in the implemented methods. Nevertheless, for Brown clusters, even with our highest threshold of 32 occurrences in the unannotated corpus, between 98% and 99% of the word occurrences in the annotated corpus are present in a cluster. Dependency-based Brown clusters have slightly lower coverage due to the use of reduced dependencies, which result for instance in the suppression of prepositions and their replacement with reduced dependency names such as prep of. The thresholds imposed on latent categories were higher (50 and 100) and may explain the slightly lower coverage of these categories. This coverage remains high, with maximums of 94-95%. In summary, data-driven word classes have a higher coverage of the corpus than knowledge-based semantic classes. Their contribution to the performance of the system, when added to knowledge-based classes, acts through an increase in recall. But because they are noisier, they also decrease precision. Since the system based on NOSEM plus knowledge-based classes lacks recall, they improve its Fmeasure.

Limitations

This study has the following limitations. First, it considers only one corpus in one domain. Similar studies should be performed on a variety of corpora to check the generalization of these observations. Second, this dataset deals with only one type of entity (disorders). The impact of the various types of word classes that we studied in this paper could also vary depending on the number of entity types present in an annotated corpus and their nature. Besides, concerning syntax-informed word classes, i.e., dependency-based Brown clusters and PCFG-LA word categories, we note that syntactic analysis was performed on texts where sentence splitting was often incorrect, leading to some erroneous parses. Improved sentence splitting might lead to a more accurate comparison of methods. We expect however that the induced changes should remain marginal.

Finally, the ablation studies in Table 6 show that although the UMLS has a much smaller coverage of the full annotated corpus, it does provide the largest contribution to recall (the first row of the table, which shows results without UMLS, has the lowest recall), followed by Brown clusters (third row in the table). Measuring coverage over the target entities and their vicinity, instead of over the full corpus, might therefore be more relevant.

Conclusion

We have performed systematic experiments to study the contribution of both knowledge-based and unsupervised word classes to disorder recognition, using the CLEF eHealth 2013 challenge data. They showed that knowledge-based word classes obtained from the UMLS and Wikipedia drastically boost (+7.3pt F-measure) a baseline system which has no semantic classes (NOSEM), with a main contribution of the UMLS (6.7pt). We also observed that in the absence of knowledge-based classes, unsupervised word classes also improve the NOSEM system, albeit by a smaller margin (5.1pt), with a main contribution by Brown clusters.

We proposed two novel methods to compute word classes based on syntactic information: dependency-based Brown clusters and PCFG-LA word categories. Our experiments with dependency-based Brown clusters were not conclusive. We also observed that PCFG-LA categories brought improvements when based on a grammar that has been trained on a similar corpus. When added to the knowledgebased classes, unsupervised word classes brought a moder-ate additional improvement (1.3pt), highlighting the importance of the knowledge-based classes in our context. This study should be extended to new data sets with multiple and different types of entities, and other domains or specialties. Other methods could also be tested to compute word classes based on syntactic representations, such as modifying Brown's algorithm to model syntactic dependencies in a more principled way. Neural network 'word embeddings' (e.g., [START_REF] Collobert | A unified architecture for natural language processing: deep neural networks with multitask learning[END_REF][START_REF] Mnih | A scalable hierarchical distributed language model[END_REF][START_REF] Mikolov | Linguistic regularities in continuous space word representations[END_REF]) are also a popular way to derive data-driven word representations and word classes which add yet another class of methods to try to match the power of knowledge-based semantic classes.

Figure 1 :

 1 Figure 1: Global overview of semantic feature production

Table 1

 1 provides statistics on the three corpora.

	Type	Set	# reports # words
	Annotated	Training	200	94 k
	Annotated	Test	100	88 k
	Unannotated		18,000	27 M
	Table 1: Statistics on corpora
	4. Word classes for supervised medical
		entity detection
	4.1. Baseline system features

Table 2 :

 2

	Form	Lemma	POS Chunk CUI	Section Cluster ID	Gold standard Prediction
	The	the	DT	B-NP #NA	HPI	B-110100	O	O
	patient	patient	NN	I-NP #NA	HPI	B-110001	O	O
	is	be	VBZ B-VP #NA	HPI	B-01110010	O	O
	a	a	DT	O	#NA	HPI	B-010000	O	O
	40	40	CD	B-NP #NA	HPI	B-111101011100	O	O
	-	-	HYPH I-NP #NA	HPI	B-10010	O	O
	year	year	NN	I-NP #NA	HPI	B-1001111101	O	O
	-	-	HYPH I-NP #NA	HPI	B-10010	O	O
	old	old	JJ	I-NP #NA	HPI	B-0101110110	O	O
	female	female	NN	I-NP B-C0015780 HPI	B-011000000	O	O
	with	with	IN	B-PP #NA	HPI	B-001111000	O	O
	complaints complaint NNS B-NP B-C0277786 HPI	B-011011111100111 O	O
	of	of	IN	B-PP #NA	HPI	B-001110	O	O
	headache headache NN	B-NP B-C0018681 HPI	B-01101001011	B-Disorder	B-Disorder
	and	and	CC	I-NP #NA	HPI	B-00010	O	O
	dizziness dizziness NN	I-NP B-C0012833 HPI	B-0110100100	B-Disorder	B-Disorder
	.	.	.	O	#NA	HPI	B-1110	O	O

Excerpt from the annotated corpus processed by our pipeline. POS = Part-of-Speech; CUI = Concept Unique Identifier; Reference = expected category; Prediction = predicted category

Table 6 :

 6 Ablation studies with respect to the best configuration (δ is the loss incurred by removing one set of attributes). See Table5for the other abbreviations.

		Feature set		P	R	F -δ
	N W U Br Br-dep LA LA-spec
	N W	Br		LA-spec 85.04 74.40 79.36 3.0
	W U Br		LA-spec 84.29 77.20 80.59 1.8
	N W U		LA-spec 88.13 75.69 81.44 1.0
	N	U Br		LA-spec 87.02 77.31 81.88 0.5
	N W U Br	LA		87.33 77.15 81.92 0.5
	N W U Br			87.49 77.48 82.18 0.2
	N W U	Br-dep	LA-spec 88.59 76.71 82.22 0.2
	N W U Br		LA-spec 87.55 77.80 82.39 0.0
	Sets of features Minimum Median Maximum
	Wikipedia	3.17	(NA)	4.80
	UMLS		2.90	16.40	21.59
	LA-spec		63.16	86.40	94.08
	LA			83.60	91.90	95.53
	Br-dep		74.50	96.59	97.65
	Br			97.91	98.87	99.30

Table 7 :

 7 Coverage (in %) of the corpus by knowledge-based and data-driven attributes (see Table5for abbreviations)

  Table 7 provides coverage information for each set of attributes: the lowest, median and highest value obtained by an attribute in this set. Since Wikipedia only has two attributes, it only has two values which are displayed in the table as minimum and maximum values.The rows in the table are sorted by maximum value in ascending order. We observe that the medical lexicon that we built from Wikipedia has the lowest coverage, which explains its low recall. The UMLS essentially contains medical terms, most of which are built around a head noun. These properties explain the moderate coverage of UMLS attributes.

http://mimic.physionet.org/

http://wapiti.limsi.fr/


PhD LIMSI Cyril Grouin

LIMSI at CEGS N-GRID 2016 NLP Shared-Tasks: Track 1.A De-Identification of Unseen Clinical Texts

  L'archive ouverte pluridisciplinaire

Introduction

Track 1.A of the CEGS N-GRID 2016 NLP Shared-Tasks consisted in automatically de-identifying unseen clinical texts. The organizers gave access to 600 unannotated clinical texts on June 6th. Participants were expected to submit their outputs no later than June 10th. The categories of de-identification were based on those defined during the i2b2/UTHealth 2014 NLP Challenge on de-identification of clinical texts [START_REF] Stubbs | Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/UTHealth shared task track 1[END_REF]. Nevertheless, clinical texts were different in the two shared-tasks in terms of content and text pre-processing (tokenization and end-of-line formatting).

Methods

Since we participated in the i2b2/UTHealth 2014 NLP Challenge, and because categories of de-identification were similar, we simply reused the system we designed [START_REF] Grouin | Clinical records de-identification using CRF and rule-based approaches[END_REF] and applied on the i2b2/UTHealth 2014 NLP Challenge data. Our approach relies on Wapiti [START_REF] Lavergne | Practical very large scale CRFs[END_REF], a CRF-based system, and post-processing rules.

CRF system We used the following features: token, length of the token, typographic case of the token, presence of punctuation mark and digit in the token, part-of-speech of the token provided by the Tree Tagger [START_REF] Schmid | Probabilistic part-of-speech tagging using decision trees[END_REF], identification of the token in a list of trigger words (Dr, M.D., Mrs, Name, yo, y/o, avenue, st, street), and cluster ID from an automatic unsupervised clustering of all tokens of the training corpus into 320 clusters using Brown's algorithm [START_REF] Peter F Brown | Class-based n-gram models of natural language[END_REF]. We also defined bigrams of features for token, typographic case, and POS. An automatic feature selection was performed (l1 regularization).

The CRF model was trained on 2014 data to predict 21 sub-categories of de-identification. As in 2014, we merged some of the sub-categories (in italics) into generic categories (in upper case) to produce the final outputs: CONTACT (email, fax, IP address, phone, URL); NAME (doctor, patient, username); DATE (date); AGE (age); PROFESSION (profession), ID (ID num, medical record); LOCATION (city, country, hospital, organization, other, state, street, zip).

Post-processing We reused the 77 post-processing rules we defined in 2014 to correct the CRF output. These post-processing rules deal with the following specific cases: identification of multi-word expressions in lexicon (city, country, profession and organization name), identification of multi-token sequences (phone number, date, patient and doctor name introduced or followed by a trigger word, etc.), deletion of elements out of the scope of the PHI (only numerical values for ages, no title for medical doctor names).

Additionally, we produced lists of PHIs from the 2014 i2b2/UTHealth training dataset. In the full post-processing stage, tokens were also searched within these lists of known PHIs while in the limited post-processing stage, we did not use them.

Design of experiments For our participation to the CEGS N-GRID 2016 Shared-Tasks, we reused the three configurations we defined while participating in the 2014 i2b2/UTHealth NLP Challenge:

• Run #1: CRF model from 2014 and full post-processing rules • Run #2: CRF model from 2014 only (no post-processing rules)

• Run #3: CRF model from 2014 and limited post-processing rules

Results

Table 1 presents the results we achieved on the test dataset for each one of our three submissions.

Category

Run

Discussion

We achieved our best results with the third run (F=0.5842). The second run achieved the worst results (no post processing rules) and the first run obtained slightly lower results than the third one due to the fact that the lists used in the post-processing rules were based on annotations from 2014. In 2014, we obtained our best results using the first run (R=0.7332, P=0.8937, F=0.8055), closely followed by results from the third run.

Since clinical texts and formatting characteristics were different between 2014 and 2016, the model we applied did not suit the clinical texts of 2016. The differences between the two corpora were of two types. First, one can identify three types of formats in 2014 clinical texts while 2016 texts only refer to the last type: fixed-width columns, double spacing between each line of text, single spacing between each line of text. Second, a tokenization issue occurred in 2016 texts: a high proportion of end-of-lines were removed, producing run-in words, i.e., sequences of two tokens where it may happen that only one token must be de-identified (e.g., "11/20/2073CPT"). Since our approach needs a perfect tokenization, such an issue bore a strong impact on the success of our system.

Conclusion

Simply reusing a CRF model trained on one type of clinical text on a new type of clinical text without any adaptation produced lower results than on the original texts (loss of 22 F-measure points between 2014 and 2016). Nevertheless, this experiment illustrates that the type of system we designed, which belongs to the general class of supervised machine learning systems, must be tuned to the target data in order to obtain useful results.

Table 1 :

 1 Official results on the test dataset (token level: R=Recall, P=Precision, F=F-measure)

			#1 (full post-proc)	Run #2 (no post-proc)	Run #3 (limited post-proc)
		R	P	F	R	P	F	R	P	F
	All	0.4901 0.5439 0.5156 0.4192 0.6657 0.5145 0.4899 0.7234 0.5842
	CONTACT	0.4304 0.2750 0.3356 0.1130 0.5909 0.1898 0.4304 0.2750 0.3356
	NAME	0.6042 0.4035 0.4838 0.5989 0.5454 0.5709 0.6042 0.6931 0.6456
	DATE	0.5951 0.7389 0.6592 0.5226 0.6860 0.5933 0.5951 0.8189 0.6893
	AGE	0.4362 0.8924 0.5860 0.3855 0.9493 0.5483 0.4362 0.8924 0.5860
	PROFESSION 0.3434 0.4675 0.3960 0.1007 0.9540 0.1822 0.3414 0.4848 0.4007
	ID	0.0345 1.0000 0.0667 0.0345 1.0000 0.0667 0.0345 1.0000 0.0667
	OTHER	0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
	LOCATION	0.3712 0.4295 0.3982 0.3153 0.6685 0.4285 0.3712 0.6874 0.4821


PhD LIMSI Cyril Grouin

LIMSI at CEGS N-GRID 2016 NLP Shared-Tasks: Track 1.B De-Identification of Clinical Texts at Character and Token Levels

Keywords: City, Country, Doctor, Hospital, Organization, Patient, Profession, State, Age, Date, Fax, License, Medical Record, Phone, Zip

HAL is

Introduction

Track 1.B of the CEGS N-GRID 2016 NLP Shared-Tasks consisted in automatically de-identifying clinical texts. The training dataset is composed of 600 texts annotated into 7 main categories of PHI (AGE, CONTACT, DATE, ID, LOCATION, NAME, PROFESSION). The test dataset is composed of 400 unannotated texts. A tokenization issue occurred in both corpora: a high proportion of end-of-lines were removed, producing run-in words, i.e., sequences of two tokens where it may happen that only one token must be de-identified (e.g., "11/20/2073CPT"). Our approach relies on Wapiti [START_REF] Lavergne | Practical very large scale CRFs[END_REF], a CRF-based system. Since a tokenization issue occurred in all texts, we decided to process the clinical texts using two types of text segmentation (see Figure 1). We produced both character-based models (predictions for each character) and token-based models (predictions for each token, using a tokenization based on white spaces). For character-based models, we produced both a generic model to predict all categories of PHI and specific models to predict nominative categories * only or numerical categories † only. We used the following features:

Methods

Methods

Corpus

• Character-based CRF models: character, unigrams of the six previous and the six next characters, token the character belongs to, unigrams of previous and next tokens, character is a punctuation, character is a digit;

• Token-based CRF models: token, unigrams of two previous and next surrounding tokens, presence of punctuation mark and digit in the token, typographic case of the token, and cluster ID from an automatic unsupervised clustering of all tokens of the training corpus into 360 clusters using Brown's algorithm [START_REF] Peter F Brown | Class-based n-gram models of natural language[END_REF].

We defined three configurations for this participation (see Figure 1):

• Run #1: one generic character-based CRF model. The aim is to process all categories using a single model.

• Run #2: combination of two specific character-based CRF models, a nominative model and a numerical model. The aim is to process PHIs of the nominative and numerical categories more accurately using specific models.

• Run #3: combination of two generic CRF models, a character-based model and a token-based model. The aim is to maximize the benefit of each type of text segmentation to deal with the initial tokenization issue. In the combination process, in case of overlap between predictions made by the two models (e.g., the sequence "March 2090Spiritual/Religion" by the token-based model vs. the sequence "March 2090" by the characterbased model), we gave priority to predictions produced by the character-based model due to its better accuracy.

Results

Table 1 presents the results we achieved on the test dataset for each one of our three configurations.

Category

Discussion

We achieved our best results when combining character and token-based CRF models (global F-measure of 0.8193, run #3). This configuration allows us to obtain the highest values of F-measure and recall for almost all PHI categories. The first configuration (character-based model only) allows us to obtain the highest values of precision for almost all categories. Nevertheless, the global F-measure (0.7973) as well as the F-measure values of all categories are lower (except for CONTACT) than those obtained on the third configuration. Finally, the combination of nominative and numerical character-based models did not improve the results, except for PROFESSION where the recall is the highest.

Nevertheless, for long sequences, token-based models outperform character-based models. As an example, the sequence "construction worker" is correctly identified using the token-based model while the character-based model only identifies the first element of the sequence ("construction"), the sequence being too long for this kind of model. In this case, the priority given to predictions from the character-based model during the combination process would produce an incorrect output. We estimate a voting procedure may improve the combination process.

Conclusion

To cope with tokenization issues in the clinical texts of the corpus, we produced both character-based CRF models (to identify frontiers of sequences with more accuracy) and token-based CRF models (to process long sequences) to automatically de-identify clinical texts. The combination of predictions made by character and token-based models, with priority given to predictions from the character-based model in case of overlapping sequences, allowed us to achieve our best results (global recall of 0.7487 and F-measure of 0.8193). There is room for improvement, especially for the NAME category for which we achieved a recall of 0.6392 using our best configuration. New features must be identified to process the false positives (e.g., last names written in lower case: "Dr. denton" and composed names).

Figure 1 :

 1 Figure 1: Type of segmentation (token/character-based), type of models (generic/specific), and official submissions

Table 1 :

 1 Official results on the test dataset (token level: R=Recall, P=Precision, F=F-measure). Best results are in bold

		Run #1 (character)	Run #2 (numer+nominat)	Run #3 (character+token)
		R	P	F	R	P	F	R	P	F
	All	0.7090 0.9106 0.7973 0.7211 0.8915 0.7973 0.7487 0.9048 0.8193
	CONTACT	0.8560 0.8652 0.8606 0.7893 0.8630 0.8245 0.8587 0.8610 0.8598
	NAME	0.5620 0.9240 0.6989 0.5965 0.9167 0.7227 0.6392 0.9121 0.7516
	DATE	0.9510 0.9797 0.9651 0.9488 0.9402 0.9445 0.9585 0.9765 0.9674
	AGE	0.7952 0.8593 0.8260 0.8139 0.8394 0.8264 0.8366 0.8462 0.8414
	PROFESSION 0.5182 0.8966 0.6568 0.5912 0.8120 0.6842 0.5643 0.8914 0.6911
	ID	0.3958 0.7037 0.5067 0.3542 0.9444 0.5152 0.4167 0.7143 0.5263
	OTHER	0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
	LOCATION	0.4967 0.8061 0.6146 0.4983 0.8374 0.6248 0.5551 0.8133 0.6598


Cyril Grouin
email: cyril.grouin@limsi.fr

Simplification de schémas d'annotation : un aller sans retour ?

Keywords: Entités nommées, schéma d'annotation, simplification Named entities, annotation scheme, simplification

Dans cet article, nous comparons l'impact de la simplification d'un schéma d'annotation sur un système de repérage d'entités nommées (REN). Une simplification consiste à rassembler les types d'entités nommées (EN) sous deux types génériques (personne et lieu), l'autre revient à mieux définir chaque type d'EN. Nous observons une amélioration des résultats sur les deux versions simplifiées. Nous étudions également la possibilité de retrouver le niveau de détail des types d'EN du schéma d'origine à partir des versions simplifiées. L'utilisation de règles de conversion permet de recouvrer les types d'EN d'origine, mais il reste une forme d'ambiguïté contextuelle qu'il est impossible de lever au moyen de règles.

Introduction

L'annotation de corpus est un processus long et coûteux, mais utile pour construire des resources, réaliser des analyses linguistiques, entraîner des modèles d'apprentissage statistique, ou pour évaluer les sorties de systèmes. [START_REF] Leech | Corpus annotation schemes[END_REF] rappelle qu'un processus d'annotation comprend plusieurs étapes : écriture et mises à jour du guide d'annotation, entraînement des annotateurs humains, double annotation et phases d'adjudication. Plusieurs solutions existent pour réduire le coût, le temps et améliorer la qualité des annotations : pré-annotation automatique [START_REF] Dandapat | Complex linguistic annotation -no easy way out ! a case from Bangla and Hindi POS labeling tasks[END_REF][START_REF] Fort | Influence of pre-annotation on POS-tagged corpus development[END_REF], apprentissage actif par [START_REF] Ganchev | Semi-automated named entity annotation[END_REF] sur du repérage d'entités nommées (REN) et par [START_REF] Voutilainen | Improving corpus annotation productivity : a method and experiment with interactive tagging[END_REF] et [START_REF] Yimam | Automatic annotation suggestions and custom annotation layers in WebAnno[END_REF] sur de l'étiquetage en parties du discours, annotation agile [START_REF] Grover | Agile corpus annotation in practice : an overview of manual and automatic annotation of CVs[END_REF], ou encore propagation contrôlée d'annotations existantes [START_REF] Grouin | Controlled propagation of concept annotations in textual corpora[END_REF].

Alors que ces solutions préservent le schéma d'annotation, une autre piste de simplification du processus d'annotation consiste à modifier le schéma par fusion de types ou redéfinition. Wilson & Thomas (1997, p. 54) rappellent que les schémas sémantiques relèvent d'un compromis entre le souhait de refléter l'organisation des mots dans l'esprit humain, et la nécessité d'avoir des annotations utiles pour les chercheurs, cette utilité étant guidée par la tâche. Si un schéma d'annotation n'est jamais défini en fonction de possibilités techniques, il est possible de tenir compte des propriétés des données (distribution, ambiguïté, etc.) pour améliorer la qualité des annotations. Un équilibre doit être trouvé entre simplification et possibilité de recouvrer le niveau de détail du schéma d'origine.

Dans cet article, nous évaluons l'impact de la complexité d'un schéma d'annotation sur les performances d'un système de reconnaissance d'entités nommées. Depuis un schéma d'annotation complexe (nombre élevé de catégories, ambiguïtés entre catégories, prise en compte du contexte pour décider d'annoter et déterminer l'étiquette à utiliser, etc.), nous réduisons la complexité en fusionnant des types existants ou en proposant de nouvelles définitions pour les types difficiles à traiter. Nous évaluons les résultats obtenus par des modèles entraînés sur ces versions, et la possibilité de retrouver les types du schéma d'origine après simplification.

État de l'art

Travailler sur les schémas d'annotation peut viser un objectif d'interopérabilité [START_REF] Blache | Annotation schemes, annotation tools and the question of interoperability : from typed feature structures to XML schemas[END_REF] ou d'analyse de l'impact d'un schéma sur des outils du TAL appliqués sur ces annotations, en particulier pour démontrer l'impact des propriétés du corpus et des différences d'annotations (définitions, nombre de types, portions annotées, coordination d'entités). Plusieurs travaux ont déterminé le seuil au-delà duquel un schéma d'annotation devenait trop complexe.

Sur une tâche d'analyse de dépendances, [START_REF] Mille | How does the granularity of an annotation scheme influence dependency parsing performance ?[END_REF] ont graduellement ajouté des relations grammaticales au schéma d'annotation du Penn Treebank (15, 31, 44 et 60 relations) et ont comparé les performances sur quatre analyseurs de dépendances. Ils observent qu'un schéma d'annotation linguistiquement plus riche n'implique pas forcément une baisse d'exactitude (moins de 0,9) entre 15 et 44 relations alors que les différences sont plus marquées avec le schéma composé de 60 relations.

En reconnaissance d'entités nommées, [START_REF] Shmanina | Impact of corpus diversity and complexity on NER performance[END_REF] font l'observation inverse. Les auteurs ont comparé les performances de Banner [START_REF] Leaman | Banner : an executable survey of advances in biomedical named entity recognition[END_REF] sur un corpus annoté en suivant deux schémas d'annotations des maladies. Avec une validation croisée 10-plis, l'outil obtient de moins bons résultats sur le corpus annoté avec le schéma d'annotation complexe (F-mesure de 0,7365) qu'avec un schéma d'annotation plus simple (F-mesure de 0,9164).

Corpus et méthodes

Nos expériences reposent sur le corpus produit pour la tâche de désidentification de la campagne d'évaluation i2b2/UTHealth 2014 NLP Challenge [START_REF] Stubbs | Automated systems for the de-identification of longitudinal clinical narratives : Overview of 2014 i2b2/UTHealth shared task track 1[END_REF]. Ce corpus comprend 1304 documents cliniques rédigés en anglais (de 2 à 5 par patient), pour un total de 296 patients différents. Les annotations couvrent 7 types principaux (Age, Contact, Date, ID, Location, Name, et Profession) et 25 sous-types (doctor, patient, username pour Name ; street, state, city, zip, hospital, organization pour Location). Ces annotations ont ensuite été remplacées en corpus par des données fictives réalistes.

Nous nous intéressons à deux principaux types d'entités nommées : les noms de personnes en raison de leur complexité sémantique (différence entre nom et prénom, ou entre un médecin et un patient), et les noms de lieu en raison de leur composition d'éléments de types différents, en particulier sur les adresses et noms d'hôpitaux qui sont complexes à traiter pour des systèmes de REN lorsque ces noms font référence à des personnes ou à d'autres lieux.

Nos expériences reposent sur trois versions seulement d'un schéma d'annotation (voir Figure 1) :

-la version d'origine (v1) composée de différences sémantiques (distinction médecin/patient) qui peuvent être ambiguës puisque ces entités sont de type "nom de personne" composées de prénoms et de noms (1) Ms.

© person § ¦ ¤ ¥ patient § ¦ ¤ ¥ first Michelle § ¦ ¤ ¥ last Klein
was seen in general neurology clinic today following her recent admission for complex migraine. Dr.

person

© doctor § ¦ ¤ ¥ first Remigio L. § ¦ ¤ ¥ last Allison
was present for all salient aspects of the history and physical exam.

(2) Internal Medicine. ---.737 .836 .784 ---Hosp. . précision,Fmesure) Doc. .859 .929 .893 .848 .800 .823 .788 .673 .726 Patient .846 .863 .854 .713 .896 .794 .473 .447 .460 Hosp. .655 .830 .732 .591 .611 .601 .459 .634 .532 Street .838 .826 .832 .596 .853 .701 .588 .370 .455 City .633 .788 .702 .617 .593 .605 .481 .418 .447 State .790 .802 .796 .795 .853 .823 .816 .856 .836 Zip .914 .928 .921 .900 .984 .940 .900 .984 .940 TOUS .800 .881 .839 .747 .774 .760 .640 .607 .623 TABLE 3: Rappel,précision, et F-mesure des prédictions de NeuroNER avec application de posttraitement. Les meilleurs résultats sont en gras v1 (origine) v2 (simple) v3 (régulier) R P F R P F R P F . 859 .929 .893 .950 .834 .888 .879 .702 .780 .846 .863 .854 .735 .896 .808 .507 .471 .488 .655 .830 .732 .830 .724 .774 .753 .747 .750 .838 .826 .832 .721 .961 .824 .794 .480 .598 .633 .788 .702 .761 .638 .694 .883 .581 .701 .790 .802 .796 .842 .856 .849 .953 .924 .938 .914 .928 .921 .943 .971 .957 .957 .971 .964 .800 .881 .839 .860 .815 .837 .664 .783 .718 TABLE 4: Rappel, précision, et F-mesure des conversions sur les entités nommées de référence. Les meilleurs résultats sont en gras d'annotation plus régulier, nous obtenons des résultats plus faibles pour les types d'EN utilisés dans plusieurs contextes (F=0.691 vs. 0.702 pour les villes, F=0.773 vs. 0.796 pour les États) par rapport au schéma d'origine. Puisque les systèmes statistiques sont sensibles aux variations contextuelles, une plus grande variété de contextes dans lesquelles apparaissent des EN de ces types a un impact négatif sur les performances du système. De manière non intentionnelle, nous avons remplacé l'ambiguïté de définition par une ambiguïté de contexte. La conversion des types d'EN depuis les deux versions simplifiées se révèle complexe dans la mesure où elle revient à réintroduire de l'ambigüité dans un jeu d'annotations simplifiées.

© loc § ¦ ¤ ¥ street § ¦ ¤ ¥ number 86 § ¦ ¤ ¥ city Paris £ ¢ ¡ type Rd loc © city § ¦ ¤ ¥ city Washington , © loc § ¦ ¤ ¥ state £ ¢ ¡ state DC © loc § ¦ ¤ ¥ zip § ¦ ¤ ¥

Conclusion

Dans cet article, nous avons comparé l'impact de deux versions d'un schéma d'annotations sur un système de REN, une version simplifiant les types d'EN en deux types consensuels (nom et lieu) et une version proposant des définitions régulières. Dans les deux cas, cette simplification améliore les résultats. Nous avons étudié la possibilité de recouvrer les types d'EN du schéma d'origine à partir des versions simplifiées. Les règles de conversion ne suffisent pas pour retrouver le niveau de détail du schéma d'annotation d'origine. Si l'intérêt d'une simplification est réel, nous estimons que d'autres solutions doivent être identifiées pour retrouver les types d'EN d'origine. Dans le cadre de travaux futurs, nous envisageons de reproduire ces expériences au moyen de CRF de chaîne linéaire, ainsi que l'application de méthodes symboliques. La comparaison des résultats obtenus avec ces approches permettra de vérifier l'impact de la simplification de schémas d'annotation avec d'autres approches.

  FIGURE 3: Évolution de la F-mesure sur le test par itération pour les versions d'origine (v1), simplifiée (v2) ou régulière (v3)

  du repérage d'entités nommées selon le schéma d'annotation utilisé pour créer le modèle 5 Discussion Concernant le repérage d'entités nommées, la figure 3 met en évidence deux conclusions : (i) un schéma d'annotation complexe (v1 avec 7 types) obtient de moins bons résultats que la version simplifiée (v2 avec 2 types) : F=0.839 vs. F=0.865 ; (ii) un schéma plus régulier (v3 avec 8 types) permet à NeuroNER d'obtenir plus rapidement de meilleurs résultats (la courbe verte augmente plus rapidement que les autres). Les conclusions que nous pouvons tirer du tableau 2 sont plus contrastées. Comme attendu, la distinction entre patient et médecin produit de moins bons résultats (F=0.854 and 0.893 respectivement) que celle entre prénom et nom (F=0.903 and 0.922), ce qui prouve sa complexité pour un système statistique. Malgré la réduction de l'ambiguïté dans le schéma Cat. v1 (origine) v2 (simple) v3 (régulier) R P F R P F R P F

TABLE 1 :

 1 1 . Cette version se fonde sur des types d'entités reposant sur des définitions humaines, ces types étant conçus pour être directement exploitables en sortie (ici, une différence entre médecins et patients) ; -une version simplifiée (v2) composées de types d'entités globaux (personnes et lieux) qui font généralement consensus dans la communauté du REN ; -une version plus régulière (v3) pour laquelle la fonction d'un élément ne détermine pas la catégorie :Les trois versions du schéma d'annotation sont présentées sur la phrase 1 pour les noms de personnes et la phrase 2 pour les lieux. Les boîtes extérieures (en rouge) renvoient à la v2, les boîtes intermédiaires (en bleu) à la v1, et les boîtes intérieures (en vert) à la v3.

					Entraînement	Test
				Type	(521 fichiers)	(514 fichiers)
					v1	v2	v3	v1	v2	v3
				Person	-2809	-	-2791	-
				Doctor 1932	-	-1913	-	-
	Schéma d'annotation d'origine (v1) • médecin	Patient	879	-	-879	-	-
	• patient • hôpital • adresse (numéro, nom et type de voirie)	First Last	--	-1661 -2663	--	-1670 -2586
	• ville • État (nom, code)			Code	-	-308	-	-217
	• code postal			Location	-1695	-	-1610	-
	simplification	suppression de	Hospital 928	-	-875	-	-
	de catégorie	l'ambiguïté	Address 144	-	-136	-	-
		Version plus régulière (v3)	City	259	-702 264	-721
	Version simplifiée (v2)	• prénom • nom • code (initiales, code clinique)	State Zip	222 139	-243 190 --140	-205 --
	• personne • lieu	• numéro (numéro de rue, code postal) • ville	Number	-	-280	-	-273
		• État (nom, code) • type (pour les rues et hôpitaux) • nom complémentaire (hôpital)	Name Type	--	-692 -507	--	-624 -467
	FIGURE 1: Évolution du schéma d'annotation,
	de la version d'origine (v1) vers les versions
	simplifiée (v2) et plus régulière (v3)

-les prénoms, noms de familles, de villes et d'États sont toujours étiquetés comme 'first', 'last', 'city' et 'state', même s'ils sont utilisés comme composants d'une adresse ou d'un nom d'hôpital (86 Paris Rd ; 89 James Street ; Cooley Dickinson Hospital ; Johnsonville Family Clinic ; Southwest Texas Medical Center ; Zucker Hillside Hospital) ; -tous les chiffres sont étiquetés 'number' (y compris dans les adresses et codes postaux) ; -les initiales de médecins et les codes internes aux hôpitaux (XD ; MAL60 ; lc855 ; ullmann) sont rassemblés sous l'étiquette 'code'. Nombre d'annotations par type dans les corpus d'entraînement et de test pour chaque version du schéma d'annotation

  Méthodes Pour chaque version du schéma d'annotation, nous avons identifié les entités nommées grâce à un système d'apprentissage statistique, puis appliqué des règles de conversion pour retrouver les types d'entités nommées de la version d'origine. Le schéma 2 résume les différentes étapes suivies.Nous avons utilisé l'outil NeuroNER fondé sur des réseaux de neurones récurrents (bi-LSTM) pour identifier les entités nommées. Cet outil prend en entrée un corpus annoté au format BRAT ou ConLL et produit une sortie annotée en entités nommées. Nous avons conservé la configuration d'origine de l'outil et renvoyons à[START_REF] Dernoncourt | NeuroNER : an easy-to-use program for named-entity recognition based on neural networks[END_REF] pour de plus amples détails sur l'architecture du réseau de neurones.La tokénisation a été réalisée avec l'outil spaCy 4 pour l'anglais, nous avons repris les plongements lexicaux entraînés avec GloVe[START_REF] Pennington | GloVe : global vectors for word representation[END_REF] fournis avec l'outil NeuroNER (200 dimensions pour les plongements lexicaux et 15 dimensions pour les plongements de tokens), l'optimisation est fondée sur un SGD (stochastic gradient descent), et nous avons conservé le seuil d'apprentissage par défaut de 0.005. La construction du modèle a été réalisée en utilisant 8 CPU (<12 minutes par itération).Nous avons créé des règles de conversion pour retrouver les types d'entités nommées de la version d'origine du schéma d'annotation depuis les sorties produites en versions simplifiée (v2) et régulière (v3). Les règles se fondent sur des déclencheurs et une analyse du contexte. Elles visent uniquement à convertir les types d'EN, sans chercher à identifier de nouvelles entités.La figure 3 présente l'évolution de la F-mesure de NeuroNER sur le test, pour chaque version du schéma. Le tableau 2 présente les résultats par catégorie à l'issue de la construction du modèle. L'apprentissage prend fin lorsque les dix dernières itérations n'ont présenté aucune amélioration. Le tableau 3 présente les résultats par application de NeuroNER et des règles de conversion des types d'EN produits par NeuroNER. Puisque les types produits sur la version d'origine du schéma d'annotation (v1) constituent déjà les types d'EN attendus, les résultats sur la v1 sont identiques entre les tableaux 2 et 3. La tableau 4 présente les résultats lorsque la conversion est appliquée sur les entités nommées de référence. Cette évaluation met en évidence la qualité des règles de conversion indépendamment de l'identification des entités nommées par NeuroNER.

		Regroupement de catégories	Corpus annoté schéma simplifié	Apprentissage NeuroNER	Annotations corpus de test	règles de conversion	Annotations schéma d'origine
			(v2)	(v2)	(v2)		(v1)
		Distinction
	Corpus annoté schéma d'origine (v1)	noms/prénoms et découpage adresses	Corpus annoté schéma régulier (v3)	(v3) Apprentissage NeuroNER	(v3) Annotations corpus de test	règles de conversion	(v1) Annotations schéma d'origine
			Apprentissage	Annotations
			NeuroNER	corpus de test
			(v1)	(v1)
	FIGURE 2: Étapes suivies : conversion du schéma d'annotation d'origine en versions simplifiées (v2)
	et régulière (v3), construction de modèles statistiques pour ces trois versions, application des modèles
	sur le corpus de test, et conversion des annotations v2 et v3 vers le schéma d'origine
						' ;
	-pour produire la v3, nous avons appliqué des règles définies empiriquement pour réintroduire
	les prénoms et noms de famille à la place de la distinction 'Patient' et 'Doctor' 2 , et scindé
	les adresses en 'Number', 'Name', et 'Type' 3 . Un contrôle manuel de ces modifications
	automatiques a été réalisé en deux temps (4h12 pour la première vérification et 1h06 pour la
	deuxième).
	Le tableau 1 présente le nombre total d'annotations par type dans chaque version du schéma d'anno-
	tation sur les corpus d'entraînement (521 fichiers) et de test (514 fichiers).

number 20006 . Afin de produire les versions 2 (simplifiée) et 3 (plus régulière), nous avons converti la version d'origine du schéma d'annotation au moyen des heuristiques suivantes : -pour produire la v2, nous avons rassemblé les catégories 'Doctor' et 'Patient' sous 'Person', et de manière similaire pour 'Street', 'City', 'State' et 'Zip' sous 'Location2. Dans les documents traités, le prénom précède toujours le nom (« Michelle Klein ») sauf si le premier élément est suivi d'une virgule, auquel cas le nom précède le prénom (« Glenn, Olivia »). Les règles visent à identifier la frontière entre prénom et nom en fonction du nombre de tokens dans la portion et en tenant compte de la présence d'initiales (« Remigio L. Allison »). 3. Ce découpage repose sur des listes (hôpitaux, États), des déclencheurs (types d'hôpitaux et de voiries) et des règles (éléments numériques).

Il n'est pas possible de savoir si Paul Martin est un médecin ou un patient sans analyser le contexte (sauf en cas de médecin ou de patient connu).

Depuis la version simplifiée (v2), nous avons produit des règles de conversion uniquement pour traiter les cas les plus fréquents (soit une dizaine d'heures de travail). Pour les cas non couverts par nos règles, nous avons utilisé des valeurs par défaut, même si cela engendre une baisse de précision pour les types d'entités utilisés comme valeurs par défaut. Ces valeurs reposent sur les types les plus utilisés en corpus (soit 'Doctor' pour un nom de personne et 'Hospital' pour un nom de lieu). Sur les noms de personnes, nous utilisons les indices de conversion suivants :-vers 'Doctor' : déclencheurs en contexte gauche 'Att :' 'Attending :' 'CC :' 'Dr' 'Dr.' 'Fellow :' 'Intern :' 'MD :' 'RefMD :', droit 'MD' 'M.D.', et expressions introductrices 'as per by' 'Dear' 'follow-up X months' , etc. -vers 'Patient' : déclencheurs 'Mr' 'Mr.' 'Mrs' 'Mrs.' 'Ms' 'Ms.' et expressions introductrices 'Patient :' 'Pt :' 'Patient Name :' 'HPI :' 'RE :' 'Impression :' 'to see' 'of seeing' 'I saw''w/ sister' 'your patient', etc. Concernant les lieux, notre approche repose sur des règles simples pour retrouver les types d'origines : séquence de 5 chiffres pour 'Zip', liste d'États américains pour 'State', une chaîne de caractères commençant par une majuscule suivie d'un État et d'un code postal pour 'City', la combinaison d'un numéro, d'un nom et d'un type de voirie ('Avenue' 'Circle' 'Ct' 'Drive' 'Dr' 'Lane' 'Place' 'Road' 'Street' 'STREET' 'St' 'Terrace' 'Way') pour 'Street'. Depuis la version régulière (v3), nous avons rassemblé dans la même portion les noms et prénoms des personnes et avons repris les mêmes déclencheurs et phrases introductives que ci-dessus. 4. https://spacy.io

Remerciements

Ce travail a été réalisé dans le cadre du groupe de travail « Sécurité des Données Textuelles » du Labex DigiCosme (projet ANR-11-LABEX-0045-DIGICOSME) financé par l'ANR au travers du programme "Investissement d'Avenir" Idex Paris-Saclay (ANR-11-IDEX-0003-02).