Nancy Ide
email: ide@cs.vassar.edu

Laurent Romary
email: romary@loria.fr

International Standard for a Linguistic Annotation Framework

This paper describes the Linguistic Annotation Framework under development within ISO TC37 SC4 WG1. The Linguistic Annotation Framework is intended to serve as a basis for harmonizing existing language resources as well as developing new ones.

Introduction

Language resources are bodies of electronic language data used to support research and applications in the area of natural language processing. Typically, such data are enhanced (annotated) with linguistic information such as morpho-syntactic categories, syntactic or discourse structure, coreference information, etc.; or two or more bodies may be aligned for correspondences (e.g., parallel translations, speech signal and transcription).

Over the past 15-20 years, increasingly large bodies of language resources have been created and annotated by the language engineering community. Certain fundamental representation principles have been widely adopted, such as the use of stand-off annotation [START_REF] Ide | Corpus Encoding Standard[END_REF], use of XML, etc., and several attempts to provide generalized annotation mechanisms and formats have been developed (e.g., XCES (Ide, et al., 2000), annotation graphs [START_REF] Bird | A formal framework for linguistic annotation[END_REF]). However, it remains the case that annotation formats often vary considerably from resource to resource, often to satisfy constraints imposed by particular processing software. The language processing community has recognized that commonality and interoperability are increasingly imperative to enable sharing, merging, and comparison of language resources.

To provide an infra-structure and framework for language resource development and use, the International Organization for Standardization (ISO) has formed a sub-committee (SC4) under Technical Committee 37 (TC37, Terminology and Other Language Resources) devoted to Language Resource Management. The objective of ISO/TC 37/SC 4 is to prepare international standards and guidelines for effective language resource management in applications in the multilingual information society. To this end, the committee is developing principles and methods for creating, coding, processing and managing language resources, such as written corpora, lexical corpora, speech corpora, dictionary compiling and classification schemes. The focus of the work is on data modeling, markup, data exchange and the evaluation of language resources other than terminologies (which have already been treated in other subcommittees of ISO/TC 37). The worldwide use of ISO/TC 37/SC 4 standards should improve information management within industrial, technical and scientific environments, and increase efficiency in computer-supported language communication.

At present, language professionals and standardization experts are not sufficiently aware of the standardization efforts being undertaken by ISO/TC 37/SC 4. Promoting awareness of future activities and rising problems, therefore, is crucial for the success of the committee, and will be required to ensure widespread adoption of the standards it develops. An even more critical factor for the success of the committee's work is to involve, from the outset, as many and as broad a range of potential users of the standards as possible.

Within ISO/TC 37/SC 4, a working group (WG1) has been established to develop a Linguistic Annotation Framework (LAF) that can serve as a basis for harmonizing existing language resources as well as developing new ones. In order to ensure that the framework is developed based on the input and consensus of the research community, a group of experts 1 was convened on November 21-22, 2002, at Pont-à-Mousson, France, to lay out the overall structure of the framework. Based on the determinations of the experts at the workshop, the general outlines of the Linguistic Annotation Framework have been defined. In this paper, we describe the LAF design as it has been developed so far, and solicit the input of other members of the community to inform its further development.

Background and rationale

The standardization of principles and methods for the collection, processing and presentation of language resources requires a distinct type of activity. Basic standards must be produced with wideranging applications in view. In the area of language resources, these standards should provide various technical committees of ISO, IEC and other standardizing bodies with the groundwork for building more precise standards for language resource management. 2 The need for harmonization of representation formats for different kinds of linguistic information is critical, as resources and information are more and more frequently merged, compared, or otherwise utilized in common systems. This is perhaps most obvious for processing multi-modal information, which must support the fusion of multimodal in- puts and represent the combined and integrated contributions of different types of input (e.g., a spoken utterance combined with gesture and facial expression), and enable multimodal output (see, for example, [START_REF] Bunt | Towards Multimodal Content Representation[END_REF]. However, language processing applications of any kind require the integration of varieties of linguistic information, which, in today's environment, come from potentially diverse sources. We can therefore expect use and integration of, for example, syntactic, morphological, discourse, etc. information for multiple languages, as well as information structures like domain models and ontologies.

We are aware that standardization is a difficult business, and that many members of the targeted communities are skeptical about imposing any sort of standards at all. There are two major arguments against the idea of standardization for language resources. First, the diversity of theoretical approaches to, in particular, the annotation of various linguistic phenomena suggests that standardization is at least impractical, if not impossible. Second, it is feared that vast amounts of existing data and processing software, which may have taken years of effort and considerable funding to develop, will be rendered obsolete by the acceptance of new standards by the community. Recognizing the validity of both of these concerns, WG1 does not seek to establish a single, definitive annotation scheme or format. Rather, the goal is to provide a framework for linguistic annotation of language resources that can serve as a reference or pivot for different annotation schemes, and which will enable their merging and/or comparison. To this end, the work of WG1 includes the following:

• analysis of the full range of annotation types and existing schemes, to identify the fundamental structural principles and content categories;

• instantiation of an abstract format capable of capturing the structure and content of linguistic annotations, based on the analysis in (1);

• establishment of a mechanism for formal definition of a set of reference content categories which can be used "off the shelf" or serve as a point of departure for precise definition of new or modified categories.

• provision of both a set of guidelines and principles for developing new annotation schemes and concrete mechanisms for their implementation, for those who wish to use them.

By situating all of the standards development squarely in the framework of XML and related standards such as RDF, OWL, etc., we hope to ensure not only that the standards developed by the committee provide for compatibility with established and widely accepted web-based technologies, but also that transduction from legacy formats into XML formats conformant to the new standards is feasible.

3 General requirements for a linguistic annotation framework

Usage scenarios

Natural language processing (NLP) applications can be applied to create annotations for linguistic data by analyzing text, speech, and data representing other modalities to determine specific linguistic attributes and associate them with the segments of that data to which they apply. NLP applications also use linguistic annotations to facilitate language understanding and generation. Development of a standard linguistic annotation framework must proceed by considering both of these "views" on linguistic annotation, and integrating the two to ensure maximal inter-operability.

Annotation of linguistic data may involve multiple annotation steps, for example, morpho-syntactic tagging, syntactic analysis, entity and event recognition, semantic annotation, co-reference resolution, discourse structure analysis, etc. Annotation at higher linguistic levels typically relies on annotations at lower levels-that is, information at lower linguistic levels serves as input in the determination of higher-level annotation categories, so that annotation can be viewed as an incremental process. Depending on the application intended to use the annotations, lower-level annotations may or may not be preserved in a persistent format. That is, the output of the annotation software may consist solely of higher-level annotations, even though lower-level analysis has been performed. Note that many application programs-e.g., information extraction software-perform the analysis required for annotation of various linguistic features and utilize it internally to deliver the desired result, without preserving the annotation information.

The need to support annotations in the context of the Semantic Web is one of the most important considerations for development of the Linguistic Annotation Framework. Annotated corpora are, at present, primarily static entities used mainly for training annotation software, as well as for corpus linguistics and lexicography (which rely on annotated corpora to study language use). However, the advent of the Semantic Web and the development of supporting technologies will significantly alter the ways in which annotations are used and preserved in the future. In the context of the Semantic Web, annotations for a variety of (at least) higherlevel linguistic and communicative features will be preserved in web-accessible form and used by software agents and other analytic software for inferencing and retrieval. This demands that the Linguistic Annotation Framework not only relies on web technologies (e.g., RDF, OWL) for representing annotations, but also that "layers' of annotations for the full range of annotation types (including named entities, time, space, and event annotation, annotation for gesture, facial expression, etc.) are at the same time separable (so that agents and other analytic software can access only those annotation types that are required for the purpose, and mergeable (so that two or more annotation types can be combined where necessary). They may also need to be dynamic, in the sense that new and/or modified information can be added as necessary.

Another increasingly important concern for LAF development is the handling of streamed data, wherein the processor analyzes input as it is encountered in a linear, time-bound sequence. Streamed data can be text, video, and audio, or might be a stream of sensor readings, satellite images, etc. This dictates that annotations to be attached to the data may be (temporarily) partial, especially where long-distance dependencies between seen and unseen segments of the data exist.

Requirements

To serve the goals of creation and use of linguistic annotation discussed above, we identify the fol-lowing general requirements for a linguistic annotation framework:

Expressive adequacy. The framework must provide means to represent all varieties of linguistic information (and possibly also other types of information). This includes representing the full range of information from the very general to information at the finest level of granularity.

Media independence.

The framework must handle all potential media types, including text, audio, video, image, etc. and should, in principle, provide common mechanisms for handling all of them. The framework will rely on existing or developing standards for representing multi-media.

Semantic adequacy. Representation structures must have a formal semantics, including definitions of logical operations. There must exist a centralized way of sharing descriptors and information categories

Incrementality. The framework must provide support for various stages of input interpretation and output generation, both during annotation (which may be accomplished at different times and with different software) and use. It must also provide for the representation of partial/under-specified results and ambiguities, alternatives, etc. and their merging and comparison.

Separability. As a complement to incrementality, it must be possible for NLP applications to easily separate or extract annotation types specific to the task at hand.

Uniformity. Representations must utilize same "building blocks" and the same methods for combining them.

Openness. The framework must not dictate representations dependent on a single linguistic theory.

Extensibility. The framework must provide ways to declare and interchange extensions to the centralized data category registry.

Human readability. Representations must be human readable, at least for creation and editing.

Processability (explicitness). Information in an annotation scheme must be explicit-that is, the burden of interpretation should not be left to the processing software.

Consistency. Different mechanisms should not be used to indicate the same type of information.

To fulfill these requirements, it is necessary to identify a consistent underlying data model for data and its annotations. A data model is a formalized description of the data objects (in terms of composition, attributes, class membership, applicable procedures, etc.) and relations among them, independent of their instantiation in any particular form. A data model capable of capturing the structure and relations in diverse types of data and annotations is a pre-requisite for developing a common corpus-handling environment: it impacts the design of annotation schema, encoding formats and data architectures, and tool architectures.

As a starting assumption, we can conceive of an annotation as a one-or two-way link between an annotation object and a point (or a list/set of points) or span (or a list/set of spans) within a base data set. Links may or may not have a semantics-i.e., a type--associated with them. Points and spans in the base data may themselves be objects, or sets or lists of objects. We make several observations concerning this assumption:

• the model assumes a fundamental linearity of objects in the base,3 e.g., as a time line (speech); a sequence of characters, words, sentences, etc.; or pixel data representing images;

• the granularity of the data representation and encoding is critical: it must be possible to uniquely point to the smallest possible component (e.g., character, phonetic component, pitch signal, morpheme, word, etc.);

• an annotation scheme must be mappable to the structures defined for annotation objects in the model;

• the encoding scheme must be able to capture the object structure and relations expressed in the model, including class membership and inheritance, therefore requiring a sophisticated means to specify linkage within and between documents; • it is necessary to consider the logistics of identifying spans by enclosing them in start and end tags (thus enabling hierarchical grouping of objects in the data itself), vs. explicit addressing of start and end points, which is required for read-only data;

• it must be possible to represent objects and relations in some (fairly straightforward) form that prevents information loss;

• it should be possible to represent the objects and relations in a variety of formats suitable to different tools and applications.

ISO TC37/SC 4's goal is to develop a framework for the design and implementation of linguistic resource formats and processes in order to facilitate the exchange of information between language processing modules. A well-defined representational framework for linguistic information will also provide for the specification and comparison of existing application-specific representations and the definition of new ones, while ensuring a level of interoperability between them. The framework should allow for variation in annotation schemes while at the same time enabling comparison and evaluation, merging of different annotations, and development of common tools for creating and using annotated data. For this purpose we envisage a common "pivot" format based on a data model capable of capturing all types of information in linguistic annotations, into and out of which sitespecific representation formats can be transduced. This strategy is similar to that adopted in the design of languages intended to be reusable across platforms, such as Java. The pivot format must support the communication among all modules in the system, and be adequate for representing not only the end result of interpretation, but also intermediate results.

Terms and definitions

The following terms and definitions are used in the discussion that follows:

Annotation: The process of adding linguistic information to language data ("annotation of a corpus") or the linguistic information itself ("an annotation"), independent of its representation. For example, one may annotate a document for syntax using a LISP-like representation, an XML representation, etc.

Representation:

The format in which the annotation is rendered, e.g. XML, LISP, etc. independent of its content. For example, a phrase structure syntactic annotation and a dependency-based annotation may both be represented using XML, even though the annotation information itself is very different.

Types of Annotation:

We distinguish two fundamental types of annotation activity:

1. segmentation : delimits linguistic elements that appear in the primary data. Including

• continuous segments (appear contiguously in the primary data)

• super-and sub-segments, where groups of segments will comprise the parts of a larger segment (e.g., a contiguous word segments typically comprise a sentence segment)

• discontinuous segments (linking continuous segments)

• landmarks (e.g time stamps) that note a point in the primary data In current practice, segmental information may or may not appear in the document containing the primary data itself. Documents considered to be read-only, for example, might be segmented by specifying byte offsets into the primary document where a given segment begins and ends.

linguistic annotation: provides linguistic

and/or communicative information about the segments in the primary data, e.g., a morphosyntactic annotation in which a part of speech and lemma are associated with each segment in the data. Note that the identification of a segment as a word, sentence, noun phrase, etc. also constitutes linguistic annotation.

In current practice, when it is possible to do so, segmentation and identification of the linguistic role or properties of that segment are often combined (e.g., syntactic bracketing, or delimiting each word in the document with an XML tag that identifies the segment as a word, sentence, etc.).

Stand-off annotation:

Annotations layered over a given primary document and instantiated in a document separate from that containing the primary data. Stand-off annotations refer to specific locations in the primary data, by addressing byte offsets, elements, etc. to which the annotation applies. Multiple stand-off annotation documents for a given type of annotation can refer to the same primary document (e.g., two different part of speech annotations for a given text). There is no requirement that a single XML-compliant document may be created by merging stand-off annotation documents with the primary data; that is, two annotation documents may specify trees over the primary data that contain overlapping hierarchies.

Design principles

The following general principles guide the LAF development:

• The data model and document form are distinct but mappable to one another

• The data model is parsimonious, general, and formally precise

• The data model is built around a clear separation of structure and content

• There is an inventory of logical operations supported by the data model, which define its abstract semantics

• The document form is largely under user control

• The mapping between the flexible document form and data model is via a rigid dump-format

• The mapping from document form to the dump format is documented in an XML Schema (or the functional equivalent thereof) associated with the document

• Mapping is operationalized either via schemabased data-binding process or via schemaderived stylesheet mapping between the user document and the dump-format document.

• It must be possible to isolate specific layers of annotation from other annotation layers or the primary (base) data; i.e., it must be possible to create a format using stand-off annotation

• The dump format must be designed to enable stream marshalling and unmarshalling

The overall architecture of LAF as dictated by these principles is given in Figure 1. The fundamental principle is that the user controls the representation format for linguistic annotations, which is mappable to the data model. This mapping is accomplished via a rigid "dump" format, isomorphic to the data model and intended primarily for machine rather than human use. The left side of the diagram represents the user-defined document form, and is labeled "human" to indicate that creation and editing, of the resource is accomplished via human interaction with this format. This format should, to the extent possible, be human readable. We will support XML for these formats (e.g., by providing style sheets, examples, etc.) but not disallow other formats. The right side represents the dump format, which is machine processable, and may not be human readable, as it is intended for use only in processing. The following set of practices will guide the implementation of the LAF:

• The data model is essentially a feature structure graph with a moderate admixture of algebra (e.g. disjunction, sets), grounded in ndimensional regions of primary data and literals.

• The dump format is isomorphic to the data model.

• Semantic coherence is provided by a registry of features in an XML-compatible format (e.g., RDF), which can be used directly in the userdefined formats and is always used with the dump format.

• Resources will be available to support the design and specification of document forms, for example:

o XML Schemas in several normal forms based on type definitions and abstract elements that can be exploited via type derivation and/or substitution group; o XPointer design-patterns with standoff semantics; o Schema annotations specifying mapping between document form and data model; o Meta-stylesheet for mapping from annotated XML Schema to mapping stylesheets; o Data-binding stylesheets with languagespecific bindings (e.g. Java).

• Users may define their own data categories or establish variants of categories in the registry. In such cases, the newly defined data categories will be formalized using the same format as definitions available in the registry, and will be associated with the dump format.

• The responsibility of converting to the dump format is on the producer of the resource.

• The producer is responsible for documenting the mapping from the user format to the data model.

Data Model

The data model is built around a clear separation of the structure of annotations and their content, that is, the linguistic information the annotation provides.

The model therefore combines a structural metamodel, that is, an abstract structure shared by all documents of a given type (e.g. syntactic annotation), and a set of data categories associated with the various components of the structural metamodel.

The structural component of the data model is a directed graph of feature structures capable of referencing n-dimensional regions of primary data as well as other annotations. The choice of this model is indicated by its almost universal use in defining general-purpose annotation formats, including the Generic Modeling Tool (GMT) (Ide andRomary, 2001, 2002) and Annotation Graphs [START_REF] Bird | A formal framework for linguistic annotation[END_REF]. A small inventory of logical operations over annotation structures is specified, which define the model's abstract semantics. These operations allow for expressing the following relations among annotation fragments:

• Parallelism: two or more annotations refer to the same data object;

• Alternatives: two or more annotations comprise a set of mutually exclusive alternatives (e.g., two possible part-of-speech assignments, before disambiguation);

• Aggregation: two or more annotations comprise a list (ordered) or set (unordered) that should be taken as a unit.

The feature structure graph contains elementary structural nodes to which one or more data category/value pairs are attached, providing the semantics of the annotation.

As specified by the LAF architecture, the dump format will implement a feature structure graph instantiated in XML. All annotations are stand-off-i.e., in documents separate from the primary data and other annotations-in order to support the requirements for incrementality and separability (see section 3.2).

The XML-based GMT will serve as a starting point for defining the dump format. The GMT implements a simple hierarchical structural model, with mechanisms to support long-distance dependencies, together with a basic set of features structure construction operators for conjunction, disjunction, etc. As such, it is sufficiently expressive to represent the information required in LAF. For examples of GMT application to different annotation types, see Ide, et al., 2000 (terminology, dictionaries and other lexical data); Ide and Romary, 2002 (morphological annotation); and Ide and2001b, 2003 (syntactic annotation).

The final implementation of the dump format may differ from the GMT, in particular by mapping its "plain" XML format to RDF/RDFS/OWL.

Data Categories and the Data Category

Registry

The central component of the LAF architecture is a Data Category Registry that will contain pre-defined data elements and schemas that can be used directly in annotations. Alternatively, users may define their own data categories or establish variants of categories in the registry; in such cases, the newly defined data categories will be formalized using the same format as definitions available in the registry.

We define a data category as an elementary descriptor used in a linguistic annotation scheme. In feature structure terminology, data categories include both attributes (hereafter called type descriptors) such as SYNTACTIC CATEGORY and GRAMMATICAL GENDER, as well as a set of associated atomic values taken by such attributes, such as NOUN and FEMININE. In both cases we distinguish between the abstraction (concept) behind an attribute or value, and its realization as some string of characters or other object. Figure 1 provides an overview of these relationships.

Whereas there is only one concept for a given attribute or value, there may be multiple instantiations. The DCR under development within ISO TC37 SC4 is built around this fundamental concept/instance distinction. In principle, the DCR provides a set of reference concepts, while the annotator provides a Data Category Specification (DCS) that comprises a mapping between his or her scheme-specific instantiations and the concepts in the DCR. As such, the DCS provides documentation for the linguistic annotation scheme in question. The DCS for a given annotation document/s is included or referenced in any data exchange to provide the receiver with the information required to interpret the annotation content or to map it to another instantiation. Semantic integrity is guaranteed by mutual reference to DCR concepts.

The DCR is intended to provide a set of formallydefined reference categories. "Formal definition" in this context includes natural language definitions for each category accompanied by specification of the possible values each category may take. At present, we envision instantiation of the DCR as a simple database in which each entry is either a type descriptor or value. Data categories will be referenced either by the DCR entry identifier, or, since the DCR will be publicly available on-line, via a URI.

Note that this simple instantiation of the DCR makes no distinction in terms of representation between type descriptors and values; each is considered as a data category and provided with an entry identifier for reference. Only minimal constraints on their use in an annotation are specified--i.e., constraints on descriptor/value combinations given in the descriptor entry. The broader structural integrity of an annotation is provided by placing constraints on nodes in the annotation graph (as defined in the LAF architecture) with which a given category can be associated. For example, the structural graph for a syntactic constituency analysis would consist of a hierarchy of typed nodes corresponding to the nonterminals in the grammar, with constraints on their embedding, and with which only appropriate descriptor/value pairs may be associated. Node types (e.g., NP, VP) as well as associated grammatical information (e.g., tense, number) may all be specified with data categories drawn from the DCR.

A more formal specification of data categories could be provided using mechanisms such as RDF Schema (RDFS) and the Ontology Web Language (OWL) to formalize the properties and relations associated with data categories. For example, consider the following RDF Schema fragment: This fragment defines a class of objects called " "Noun" that has a property "number". Note that the schema defines the classes but does not instantiate objects belonging to the class; instantiation may be accomplished directly in the annotation file, as follows (for brevity, the following examples assume appropriate namespace declarations specifying the URIs of schema and instance declarations):

<Noun rdf:about="Mydoc#W1"> <number rdf:value="Plural"/> </Noun>

where "Mydoc#W1" is the URI of the word being annotated as a noun. Alternatively, the DCR could contain instantiations of basic data elements, specifying values for properties, which can be referenced directly in the annotation:

<Noun rdf:ID="NMP"> <number rdf:value="plural"/> </Noun>

The annotation file could then reference the predefined instance as follows:

<rdf:Description rdf:about="myDoc#W1"> <POS rdf:resource="categories#NMS"/> </rdf:Description> 4

The class and sub-class mechanisms provided in RDFS and its extensions in OWL can also be used to model the structure of annotations-that is, to identify and constrain nodes in the annotation graph itself. For example, the hierarchical structure defined by ISLE/MILE for lexical entries [START_REF] Ide | RDF Instantiation of ISLE/MILE Lexical Entries[END_REF] has been modeled in an RDF Schema (Ide, et al., 2003). Another example is the time ontology5 developed by, the DAML6 effort, which reflects the internal structure of time and event annotations.

An RDFS/OWL specification of data categories would enable greater control over descriptor/value use and also allow for the possibility of inferencing over annotations. However, it would also demand definition of a precise hierarchy of linguistic categories and a distinction between classes (objects) and properties that could place unwanted constraints on annotation form and content. Therefore, any such specification of data categories is left to the annotator, at least for the time being.

It is anticipated that many annotators will use their own category names and values and provide a mapping to DCR categories. The DCR will include an XML template for specifying this mapping, as well as for defining variants and new descriptor/value pairs. The Data Category Registry will support multiple languages by providing the following:

• reference definitions for data categories in various languages;

• data element names for the data categories in various languages;

• description of usage in language-specific contexts, including definitions, usage notes, examples, and/or lists of values (e.g., GENDER takes the values masculine, feminine in French; masculine, feminine, neuter in German)

The goal of the Data Category Registry is not to impose a specific set of categories, but rather to ensure that the semantics of data categories included in annotations (whether they exist in the Registry or not) are well-defined and understood. It is possible that several different instantiations of the same category (e.g., noun) and/or different schemas describing the same phenomenon could exist in the registry, to be used as desired by annotators. The purpose of the registry is solely to gather together (and where necessary, harmonize) existing schemas and instances in use by the language technology community as a resource for the annotation of linguistic data. The formally defined set of categories will have several functions: (1) it will provide a precise semantics for annotation categories that can be either used "off the shelf" by annotators or modified to serve specific needs;

(2) it will provide a set of reference categories onto which scheme-specific names can be mapped; and (3) it will provide a point of departure for definition of variant or more precise categories.

Conclusion

In this paper we describe the Linguistic Annotation Framework under development by ISO TC37/SC 4 WG1. Its design is intended to allow for, on the one hand, maximum flexibility for annotators, and. on the other, processing efficiency and reusability. This is accomplished by separating user annotation formats from the exchange/processing format. This separation ensures that pre-existing annotations are compatible with LAF, and that users have the freedom to design specific schemes to meet their needs, while still conforming to LAF requirements.

LAF provides for the use of any annotation format consistent with the feature structure-based data model that will be used to define the dump format. This suggests a future scenario in which annotators may create and edit annotations in a proprietary format, transduce the annotations using available tools to the dump format for interchange and/or processing, and if desired, transduce the dump form of the annotations (and/or additional annotation introduced by processing) back into the proprietary format. We anticipate the future development of annotation tools that provide a user-oriented interface for specifying annotation information, and which then generate annotations in the pivot format directly. Thus the pivot format is intended to function in the same way as, for example, Java byte code functions for programmers, as a universal "machine language" that is interpreted by processing software into an internal representation suited to its particular requirements. As with Java byte code, users need never see or manipulate the pivot format; it is solely for machine consumption.

Part of the work of SC4 WG1 is to provide development resources, including schemas, design patterns, and stylesheets, which will enable annotators and software developers to immediately adapt to LAF. Example mappings, e.g., for XCES-encoded annotations, will also be provided. In this way, we hope to realize the goal of harmonized and reusable resources in the near future.

For general information on the work of the ISO committee on language resources, consult the ISO TC37/SC4 website (http://www.tc37sc4.org).

Figure

  Figure 1. Overall LAF architecture

  Figure 1. Data category overview

Note that this observation applies to the fundamental structure of stored data. Because the targets of a relation may be either individual objects, or sets or lists of objects, information with more than one dimension is accommodated.

In these examples, NUMBER is given literal values. However, with OWL it is possible to restrict the range of possible values by enumeration.

Accessible from http://www.cs.rochester.edu/~ferguson/daml/.

http://www.daml.org


Nancy Ide
email: ide@cs.vassar.edu

Adam Kilgarriff
email: kilgarriff@itri.brighton.ac.uk

Laurent Romary
email: romary@loria.fr

A Formal Model of Dictionary Structure and Content

We show that a general model of lexical information conforms to an abstract model that reflects the hierarchy of information found in a typical dictionary entry. We show that this model can be mapped into a well-formed XML document, and how the XSL transformation language can be used to implement a semantics defined over the abstract model to enable extraction and manipulation of the information in any format.

Introduction

The structure and content of dictionary entries and lexical information has been explored in considerable depth in the past (see, for instance [Ide et al. 1995]), primarily in order to determine a common model that can serve as a basis for encoding schemas and/or database formats. For the most part, descriptions of dictionary structure have been informed by the format of printed dictionaries, which varies considerably over dictionaries produced by different publishers and for different purposes, together with the requirements for instantiation in some encoding format (principally, SGML). However, the constraints imposed by these formats interfere with the development of a model that fully captures the underlying structure of lexical information. As a result, although schemas such as those provided in the TEI Guidelines exist, they do not provide a satisfactorily comprehensive and unique description of dictionary structure and content.

We believe that in order to develop a concrete and general model of dictionaries, it is essential to distinguish between the formal model itself and the encoding or database schema that may ultimately instantiate it. That is, it is necessary to consider, in the abstract, the form and content of lexical information independent of requirements and/or limitations imposed its ultimate representation as an encoded or printed object. This is especially important since these eventual representations will vary from one application to another; in particular, dictionaries may be encoded not only for the purposes of publishing in print or electronic form, but also for creating computational lexicons for use in natural language processing applications. It is therefore essential to develop a model that may be subsequently transformed into a variety of alternative formats.

In this paper, we outline a formal model for dictionaries that describes (a) the structure of dictionary information, (b) the information associated with this structure at various levels, and (c) a system of inheritance of information over this structure. We then show how the structure may be instantiated as a document encoded using the Extended Markup Language (XML). Using the transformation language provided by the Extensible Style Language (XSL), we then demonstrate how the original XML instantiation may be transformed into other XML documents according to any desired configuration (including omission) of the elements in the original. Because of its generality, we believe our model may serve as a basis for representing, combining, and extracting information from not only dictionaries, but also terminology banks, computational lexicons, and, more generally, a wide variety of structured and semi-structured document types. An XML document type definition designed to be used with the formal framework described below is discussed in [START_REF] Erjavec | The CONCEDE model for Lexical Databases[END_REF].

A formal representation of dictionaries

Basic notation

The underlying structure of a dictionary can be viewed as embedded partitions of a lexicon, in which no distinction is made among embedded levels (e.g., entry, homonym, sense, etc.).

A dictionary is thus a recursive structure comprised, at each level, of one or more nodes. This structure is most easily visualized as a tree, where each node may have zero or more children. That is, at any level n, a node is either a leaf (i.e., with no children) or can be decomposed as:

T=[T 1 , T 2 , …, T n ]
where each T i is a node at level n+1.

Properties may be attached to any node in the dictionary structure with the prop predicate:

PROP(T,P)

indicates that the property P is attached to node T.

Properties are associated with nodes either by explicit assignment, or they may be inherited from the parent node. The object of our model is to identify the ways in which properties are propagated through levels of dictionary structure. For this purpose, we consider properties to be Feature-Value 1 pairs expressed as terms of the form FEAT(F,V), where F and V are tokens designating a feature (e.g., POS) and a value. In the simplest case, values are atomic (e.g., NOUN) but may also consist of sets of feature-value pairs. This representation is consistent with the base notation associated with feature structures [START_REF] Shieber | An Introduction to Unification-based Approaches to Grammar[END_REF]], a common framework for representing linguistic information.

Propagating information across levels of dictionary structure

We define three types of features:

! Cumulative features that may take more than one value and may be thus inherited and combined along the dictionary structure. For example, for a cumulative feature ! Overwriting features that take only one value at a time. This implies that only one instance of an overwriting feature may appear at a given node and that the corresponding properties is propagated along the dictionary structure unless and until a new value is specified for that feature. In such a case, the new value "overwrites" the earlier one and is subsequently propagated to nodes in its subtrees.

! Local features, which apply only at the node with which they are associated; i.e., they are not propagated through the structure. Cross-references are an example of a local feature, since they apply only to the level of description with which they are directly associated.

Cumulative, overwriting, and local features are identified using predicates (CUM(F C ), OVER(F O ), and LOC(F O ), respectively.

Basic notation

To model the behavior associated with cumulative and overwriting features, we consider two types of inferences:

1. Classical implication: !"# which is monotonic 2. Defeasible implication: !># where # may be deduced from ! unless there exists some information incompatible with # 2

Propagating cumulative features

Since cumulative features may take several values at a time, their behavior can be described with a simple propagation rule:

PROP(T,FEAT(F C ,V)) $ CUM(F C )" %I PROP(T I , FEAT(F C ,V)),T=[T 1 , T 2 , …, T N ]
No specific constraints hold for cumulative features.

Propagating overwriting features

The propagation of an overwriting feature can be represented with a defeasible inference:

PROP(T,FEAT(F O ,V)) $ OVER(F O ) > %I PROP(T I , FEAT(F O ,V)), T=[T 1 , T 2 , …, T N ]
This inference is associated with a rule that expresses the incompatibility of two values of the overwriting feature at a given node:

PROP(T,FEAT(F O ,V)) $ OVER(F O ) " %V'!V, "PROP(T,FEAT(F O ,V'))

Expressing dependencies among features

In certain cases, overwriting the value of a given feature has implications for the values of other features, and even for the existence of other features themselves. This typically occurs for classes of features; for example, for the class of features representing grammatical information, overwriting POS=N with P O S = V implies that features such as GEN=F, PLURALFORM=-S no longer apply, and that features such as PASTPART are now applicable.

Here, we see that within the features supplying grammatical information, POS is a privileged feature because it dictates the features and values for other members of its class.

To deal with this situation it is necessary to introduce dependency rules that block the propagation of subordinate features when the privileged feature is overwritten. The following rule indicates that feature F is dependent on a given value V for feature G:

PROP(T,FEAT(F,_)) " PROP(T,FEAT(G,V)) dictionary
For instance, to express the dependency between GEN and CAT=NOUN, the rule is

PROP(T,FEAT(GEN,_)) " PROP(T,FEAT(CAT,"N"))
Rules of this kind apply only to overwriting features, since they block the activation of a defeasible rule.

Creating representations from dictionary entries

Lexical information in dictionary entries can be represented as a tree structure reflecting, in large part, the natural hierarchical organization of entries in printed dictionaries. This hierarchical organization (e.g., division into homographs, senses, sub-senses, etc.) enables information to be applied over all sub-levels in the hierarchy, thus eliminating the need to re-specify common information (for a detailed discussion, see [Ide, et al., 1995]; [Ide/Veronis, 1995).

For example, consider the following definition from the Collins English Dictionary (CED):

EX. 1: disproof disproof (dIs'pru:f) n. 1. facts that disprove something. 2. the act of disproving.

The information in this entry can be represented in tree form as follows:

Each node in the tree represents a partition of the information in the entry, and information is inherited over sub-trees as defined in section 2.2. The value of the brack feature is itself a set of feature-value pairs (see section 2.1); the use of this generic feature provides a means to group "Brigade de gendarmes" and the crossreference to "brigadier" together. Again, orthographic form and pronunciation appear at the highest level and apply to the entire entry, except at the lowest level, where the value of orth is overwritten for the subentry for "le gendarme".

The specification of alternatives, as for the two plural forms in Example 4, is common in dictionary entries when both alternatives apply to information lower in the hierarchy. However, this notation is effectively a shorthand indicating a new level of partitioning of the information, as shown in the corresponding tree structure:

EX. 4 : pinna (CED) pinna (pron1) n. pl. -nae (pron2) or -nas…

As it would in a dictionary entry, this results in the duplication of potentially large amounts of information. In section 5 we will show that the encoded version of the information avoids this duplication.

In general, the following rules apply to the creation of trees from dictionary entries:

! Headword information and other information appearing in the entry at the same level (typically, other form-related information such as pronunciation, hyphenation, etc.) is associated with the root.

! Sibling sub-trees for the current node (initially, the root) are introduced upon encountering the following:

! Partitions of the entry into different parts of speech;

! Partitions of the entry into senses at the same level (applied recursively to subsenses);

! Partitions between the main entry and a related entry.

Although expressed informally here, a precise algorithm for tree creation can, in principle, be implemented, thereby enabling the automatic or semi-automatic creation of trees from dictionary entries in any of a variety of electronic forms (typesetter tapes, word processor output, etc.). We define a tree traversal as any path starting at the root of the tree and following, at each node, a single child of that node. A full traversal is a path from the root to any leaf; a partial traversal extends from the root to any node in one of its subtrees.

As a tree created from a dictionary entry is traversed, each node is associated with a set of features including: (a) features associated with the node during tree creation, and (b) features determined by applying the rules for propagating overwriting, cumulative, and local features outlined in section 2.2. Thus, at any node, all applicable information is available for some unique partition of the lexical space. Nodes near the top of the tree represent very broad categories of partition; leaf nodes are associated with information for the most specific usage of the entry.

A traversal of the tree for Example 3 above (gendarme) proceeds as follows: The features orth (orthography) and etym (etymology) are overwriting features, and therefore values specified for these features at lower nodes of the tree replace those appearing above them. The feature def (definition) is cumulative, and so all definitions are retained during the root-to-leaf traversal. The features ex (example) and xr (cross-reference) are local and are therefore not propagated to lower nodes in the tree.

For "overdress", two traversals are possible:

(1) Root: orth : overdress Second level:

orth : overdress pos : verb pron : pron1 def:

To dress (oneself or another) too elaborately or finely (2) Root:

orth : overdress Second level:

orth : overdress pos : noun pron : pron2 def : A dress that may be worn over a jumper, blouse, etc.

Encoding the information in XML

We define an XML encoding format for the structures described above:

Elements

! <struc> represents a node in the tree. <struc> elements may be recursively nested at any level to reflect the structure of the corresponding tree. <struc> is the only element in the encoding scheme that corresponds to the tree structure; all other elements provide information associated with a specific node (i.e., the node corresponding to the immediately enclosing <struc> element). 4

! <alt> alternatives are bracketed in parallel <alt> elements, which may appear within any <struc>. The use of this element to encode alternatives corresponds to the shorthand often used in dictionary entries, where two equally applicable sets of information apply to the entire sub-tree.

! <brack> is a general-purpose bracketing element to group associated features.

! Base elements corresponding to features, including orth, pron, hyph, syll, stress, pos, gen, case, number, gram, tns, mood, usg, time, register, geo, domain, style, def, eg, etym, xr, trans, and itype, analogous to dictionary elements defined in the TEI Guidelines.

Attributes

Attributes are used to provide information specific to the element on which they appear and are not inherited in a tree traversal.

The following show the corresponding XML encoding for Examples 1, 2, and 3: The Extensible Style Language (XSL) is a part of the XML framework that enables transformation of XML documents into other XML documents. The best-known use of XSL is the formatting of documents for display on web browsers. However, XSL also provides a powerful transformation language that can be used to convert an XML document describing dictionary entries by selecting, rearranging, and adding information to it, using a powerful language enabling tree-traversal [clark, 1999]. Thus, a document encoded according to the specifications outlined in the previous section can be manipulated to serve any application that relies on part or all of its contents.

The XSL transformation language is relatively complex and will not be described in detail here. A short example can provide some idea of the possibilities. For example, the XSL script in Figure 5, applied to the XML-encoded entry for "overdress" given in Figure 2, produces an HTML document that will display the table in Figure 6 on a web browser: XSL's transformation language can also be used to implement the semantics of inheritance and overwriting outlined in section 2. The XSL code in Figure 7 shows the implementation for overwriting and cumulative features. The result is another XML document, but with the inherited information explicitly represented, as shown in Figure 8.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:template match="/"> <xsl:apply-templates/> </xsl:template> <!--the recursive copy is initiated at the dict level --> <xsl:template match="dict"> <dict> <xsl:apply-templates/> </dict> </xsl:template> <!--each struc is copied with supplementary features depending on 'over' or 'cum' rules --> <xsl:template match="struc"> <struc> <!--rule for an overwriting feature (here, "orth") --> <!--if there is not already an 'orth' take the first one you encounter --> <xsl:if test="not(orth)"> <xsl:copy-of select="ancestor::node()[orth][1]/orth"/> </xsl:if> <!--rule for a cumulative feature (here, "ex") --> <!--copy all the 'ex' you find above in the tree --> <xsl:copy-of select="ancestor::node()/ex"/> <xsl:apply-templates/> </struc> </xsl:template> <!--any other node is just copied --> <xsl:template match="*"> <xsl:copy-of select="."/> </xsl:template> </xsl:stylesheet>

Conclusion

This brief overview is intended to show that a general model of lexical information conforms to an abstract model that reflects the hierarchy of information found in a typical dictionary entry, and that this model can be mapped into a well-formed XML document.

The XSL transformation language can be used to implement a semantics defined over the abstract model, thus enabling sophisticated query and manipulation of the data to serve a variety of ends.

Figure

  Figure 5. XSL code to produce an HTML document

Figure 6 .

 6 Figure 6. Display of HTML output for XSL code in Figure 5

Figure 7 .

 7 Figure 7. XSLT code implementing overwriting and cumulative inheritance

Figure 8 .

 8 Figure 8. Results of XSLT code in Figure 7 for "overdress"

  DOMAIN, if the property FEAT(DOMAIN,NAVIGATION) is associated with a node at level n and FEAT(DOMAIN,LAW) is associated with its child at level n+1, by inheritance the node at level n+1will be assigned the property FEAT(DOMAIN,NAVIGATION + LAW).

dress (oneself or another) too elaborately or finely. ~n. (pron2) 2. A dress that may be worn over a jumper, blouse, etc.

  The orthographic form "overdress" appears at the top node and applies to the entire entry; the entry is then partitioned into two sub-trees, for verb and noun, each of which is associated with specific information about part of speech, pronunciation, and definition.

Thus in this example, orthographic form, pronunciation, and part-of-speech are inherited by both sub-nodes, each of which represents a partition of the information into two distinct (hence, sibling) senses. The following entry shows a different organization: EX. 2 : overdress (CED) orth : disproof pron : dIs'pru:f def : facts that disprove something. def : the act of disproving. overdress vb. (pron1) 1. To 3 EX. 3 : gendarme (Le Petit Robert) gendarme (...) n.m. (XV#; gendarmes; de g e n s , et arme)…II. (1790).

Mod. Militaire appartenant à un corps spécialement chargé de veiller au maintien de l'ordre et de la sureté de la publique…

	orth : overdress
	pos : verb	pos : noun
	pron : pron1	pron : pron2
	etym : 1790
	time : modern
	def : Militaire appartenant à un corps …
	xr : Gendarmerie
	xr : Marechaussée
	brack :
	ex : Etre arrêté par les gendarmes.

(V. Gendarmerie, Marechaussée). Brigade de gendarmes (V. Brigadier). Etre arrêté par les gendarmes. Jouer au gendarme et au voleur. & Le gendarme : symbole de la force publique, de l'ordre. La peur du gendarme… The tree representing the information in this entry is: orth : gendarme pron : ... … ex : Jouer au gendarme et au voleur. pos : noun gender : masculine etym : XV#; gendarmes; de gens, et arme ex : Brigade de gendarmes xr : brigadier orth : le gendarme def : symbole de la force publique, de l'ordre. ex : La peur du gendarme

XML encoding for first part of the entry "pinna"

	<struc>
	<orth>overdress</orth>
	<struc>
	<pos>verb</pos>
	<pron>...</pron>
	<def> To dress (oneself or another) too elaborately or finely </def>
	</struc>
	<struc>
	<pos>noun</pos>
	<pron>...</pron>
	<def> A dress that may be worn over a jumper, blouse, etc.</ def>
	</struc></struc>
	Figure 2. XML encoding for "overdress"
	<struc>
	<orth>gendarme</orth>
	< pron>...</pron>
	<struc>
	<pos>noun</pos>
	<gender>masc</gender>
	<etym> XV#; gendarmes; de gens, et arme</etym>
	…
	<struc>
	<etym>1790</etym>
	<time>modern</time>
	<def>Militaire appartenant à un corps …</def>
	<xr type="see">Gendarmerie</xr>
	<xr type="see">Marechaussée </xr>
	<brack>
	<ex>Brigade de gendarmes </ex>
	<xr type="see">brigadier</xr></brack>
	<ex>Etre arrêté par les gendarmes. </ex>
	<ex>Jouer au gendarme et au voleur. </ex>
	<struc>
	<orth>le gendarme</orth>
	<def>symbole de la force publique, de l'ordre.</def>
	<ex>La peur du gendarme</ex>
	</struc></struc></struc></struc>
	Figure 3. XML encoding for "gendarme"
	<struc>
	<orth>pinna</orth>
	< pron>pron1</pron>
	<pos>noun</pos>
	<struc>
	<alt>
	<plural>pinnae</plural>
	<pron>pron2</pron></alt>
	<alt>
	< plural>pinnas </plural></struc>…
	Figure 4.

XML documents are also described as trees, where the "parent" of a given element is the element in which it is immediately enclosed. To avoid confusion, we use the term "tree" and the associated terminology to refer only to the structures outlined in section 4.

Acknowledgements

The work reported in this paper was supported in part by National Science Foundation Grant INT-9815830, the Centre National de la Recherche Scientifique (France), and the European Commission INCO-COPERNICUS project CONCEDE (no. PL96-1142).

Notes

1 We deliberately avoid the term "attribute" to avoid confusion with the SGML/XML vocabulary.

2 At this stage, our notation is independent of any specific theory (default logic, autoepistemic logic, circumscription, conditional logics etc.).

3 Note that although the presentation of the verb and noun senses of the headword is effectively linear in the printed form of entry, the reader understands the implied parallelism apparent in the derived structure.