Florence Amardeilh Mondeca
email: florence.amardeilh@mondeca.com

Danica Damljanovic
email: d.damljanovic@dcs.shef.ac.uk

Kalina Bontcheva

CA Manager: a Framework for Creating Customised Workflows for Ontology Population and Semantic Annotation

Keywords: semantic annotation, semantic repositories, ontologies, content augmentation knowledge acquisition, ontology population, software artefacts, key concept identification

The aim of semantic annotation is to bridge the large gap between structured knowledge and the large volumes of unstructured text data that companies and people need to deal with daily. Alas, the process is very labourious and error-prone, even when performed semior fully automatically. Although widely researched, the two key steps in this process -semantic annotation and ontology population -still hold outstanding challenges. While there are numerous tools in existence, many lack compliance with recent standards, but more importantly, lack the flexibility to customise the annotation workflow. In this paper, we present the Content Augmentation Manager Framework, which bridges the gap between information extraction tools and semantic repositories by allowing easy plugin of various types of components. It is also capable of controlling the process of semantic annotation and ontology population by means of consolidation algorithms.

INTRODUCTION

Copyright is held by the author/owner(s). SAAKM '09 Berkeley, California USA ACM Gartner predicted in 2002 1 that for the next decade more than 95% of human-to-computer information input will involve textual language. They also predict that by 2012 taxonomic and hierarchical knowledge mapping and indexing will be prevalent in almost all informationrich applications. There is a tension here: between the increasingly rich ontology-based, semantic models on the one hand, and the continuing prevalence of human language materials on the other. This process may be characterised as the dynamic creation of interrelationships between ontologies and unstructured and semistructured textual content in a bidirectional manner. However, transforming huge amount of unstructured text into the semantically interlinked knowledge space is a big challenge. Two key parts of this process are: 1) semantic annotation, and 2) ontology population. We define semantic annotation as a formal representation of content, expressed using concepts, relations and instances as described in an ontology, and linked to the original resource. Ontology instances are usually stored in a knowledge base independently from the annotated resource, as in the case of KIM [START_REF] Popov | KIM -Semantic Annotation Platform[END_REF] or ITM [START_REF] Amardeilh | A semantic web portal with hlt capabilities[END_REF]. The process of adding new class instances or property values to a knowledge base is called knowledge acquisition or ontology population. As such, the ontology population looks at semantic annotation as means for data-driven enrichment of an existing knowledge base.

Although widely researched, both semantic annotation and ontology population tasks still remain a big challenge, as the role of human annotators remains paramount. Consequently, high-quality automation is one of the very important requirements in order to ease the knowledge acquisition bottleneck, particularly for annotating large collections of legacy documents [START_REF] Uren | Semantic annotation for knowledge management: Requirements and a survey of the state of the art[END_REF]. While there are numerous tools in existence, many lack compliance with recent standards, but more importantly, lack the flexibility to customise the annotation workflow. Moreover, the workflow needs to also accomodate human, as well as automatic annotators.

In this paper, we present the Content Augmentation Manager Framework (CA Manager) which is capable of performing and controlling the process of semantic annotation and ontology population by means of consolidation algorithms. This framework supports ontology population from text (semi)automatically, by allowing easy plugin of various types of components including information extraction tools, customised domain ontologies, and diverse semantic repositories. In other words, this framework helps to bridge the gap between information extraction tools and the semantic repositories which are used to store the collected knowledge. This paper is structured as follows. In Section 2 we present CA Manager. Evaluation results are presented in Section 3. Similar approaches are discussed in Section 4. Finally, we draw our conclusions and plan for future work in Section 5.

THE CONTENT AUGMENTATION MAN-AGER FRAMEWORK

The core philosophy of the CA Manager is to bridge the gap between the content augmentation tools, and the semantic repository tools. It is conceived as a middleware, meaning that it does not have the responsibility neither of the information extraction task by itself, nor of the knowledge storage, but it is flexible enough to adapt to any domain ontology, various content augmentation tools, semantic repositories and workflow requirements. It is, amongst other things, capable of controlling the quality and the validity of information extraction results against an ontology, matching them against existing resources (the application's knowledge base or repositories from the linking open data initiative for instance), and enriching them. To achieve that goal, the CA Manager relies on the recommendations formulated by the Semantic Web community (RDF/OWL languages, Service Oriented Architecture) combined with a UIMA-based infrastructure which has been enriched and customized to address the specific needs of semantic annotation and ontology population tasks.

The UIMA (Unstructured Information Management Architecture) framework aims at providing a development platform for systems that analyze large volumes of unstructured information in order to discover knowledge that is relevant to an end user2 . We adopted UIMA as the foundation of the internal architecture of the CA Manager, due to its ease of integration and composition of internal or external modules, and more impor-tantly its wide acceptance by the Information Extraction (IE) community. However, although UIMA provides the building blocks to develop knowledge acquisition applications based on text-mining components, it does not give any guidelines as to which steps should be arranged in which order. Moreover, none of the existing components in the pipeline addresses the issue of controlling the quality and validity of the generated annotations/instances. Moreover, its Common Analysis Structure (CAS) defines a high-level annotation schema but it needs to be redefined for each new application need. Lastly, it uses a proprietary way to expose webservices (the Vinci IBM protocol), that makes it even more complex, as it is not reusing open Semantic Web standards.

Therefore, we implemented the CA Manager in order to develop a flexible architecture based on a combination of several UIMA Analysis Engines. UIMA provides an Eclipse plugin Eclipse facilitating the definition and customisation of each Analysis Engines (stored in an XML file) needed in the target application and their ordering as a workflow. Therefore, each step of the annotation workflow is a component that can be plug in or out according to the objectives of the final application. At the same time, we aimed at improving the UIMA infrastructure with the systematic use of Semantic Web standards We defined an RDF-based annotation schema dedicated to ontology population and semantic annotation tasks, composed of entities, properties, annotations and offsets. This annotation schema is produced after applying the first Analysis Engine and then enriched and controled by the following ones during the whole duration of the workflow process. We also provide a distributed service-oriented architecture relying on languages and protocols defined for the Semantic Web, especially for easing the integration with external components through the use of opened web services.

Furthermore, the CA Manager proposes a default workflow composed of a list of logical steps dedicated to semantic annotation and ontology population, see Figure 1:

• Extracting the valuable knowledge;

• Consolidating knowledge;

• Storing

These three phases allow the connection with information extraction tools (being defined as UIMA analysis engines plugin or providing a web service), semantic repositories and domain ontologies or corpus.

Extracting knowledge from text

The information Extraction component is made of two steps, split (optional) and extract. Split divides the in-

KCIT

KCIT [START_REF] Damljanovic | A text-based query interface to owl ontologies[END_REF] automatically retrieves key concepts from legacy documents with regards to the domain ontology. These annotations are created based on the pre-condition that a specific part of a document is referring to a particular ontology resource if the lemmas of the two match.

By matching lemmas, we ensure that all morphological inflections of the relevant terms will be matched. The KCIT process can be broken down into several steps:

• Building a list of relevant terms. Given an ontology, lexicalisations of all ontological resources (classes, instances, properties, property values) are lemmatised and added to a gazetteer list.

• Annotating the legacy content. The legacy content is first lemmatised, with a morphological analyser.

It is then matched against the gazetteer list created in the previous step.

• Resolving conflicts. This step includes filtering annotations and solving ambiguity problems such as removing redundant annotations.

To build a list of relevant terms dynamically we first extract a list of the ontology resource names (i.e., fragment identifiers) and their assigned property values (e.g., label and datatype property values). Each item from the list is further processed, based on some heuristic rules derived from different ontology designs. Although there is no need to customise KCIT when using it with different ontologies, customisation might yield better results. By default, KCIT apply several rules such as replacing dashes and underline characters with spaces and also splitting camelCase words. In addition it applies some heuristic rules which are optional, as their usage depends on the lexicalisations available in the ontology.

KCIT remains domain-independent, as all domain-specific issues are implemented through parameters. Typically, the output of a content augmentation tool is in some kind of XML format and is not connected directly to the ontology. In such cases, it needs to be transitioned to the generic Annotation Schema (AS) of the CA Manager through the use of a mapping. However, as KCIT produces ontology-based annotations, such a mapping is not necessary in this case, because the annotations are directly grounded in the ontology model.

Consolidating annotations and knowledge with an ontology repository

We studied in [START_REF] Amardeilh | Semantic annotation and ontology population[END_REF] the various possible cases of instances and annotation creation and identified two axes of consolidation:

1. the first axis defines the ontological element concerned, i.e. an instance of a class, a property value or a semantic annotation 2. the second axis defines the constraints to be checked, i.e. non redundancy, the domain and range restrictions and the element's cardinality Each of the CA Manager consolidation algorithms takes into account these two axis. In the Information Consolidation component, they are performed through three steps, merge, control and optionally infer.

The first step, merge, eliminates the duplicates within the CAS (same entity or annotations occurring more than once in the CAS) and queries the semantic repository in order to retrieve the corresponding URI of the concerned entity or annotation if not present. These queries can be simple (class + string label) or multicriteria (class + set of required properties that identify unambiguously an entity in the repository). For instance, a person can be queried by its name. However, in cases of homonymy, looking at the person name is clearly not enough and one might want to query on particular properties such as the date of birth that can better discriminate several instances of persons sharing the same name. This multicriteraai search is built from the restricted properties where cardinality is minimum 1; we call these properties identifier properties as they are required to identify and define an instance of a concept. In case of law case reports, the identifer properties are the court, the location of the court, the date of the decision and the decision itself. If it is not possible to disambiguate between two instances of the semantic repository because for example no identifier properties have been extracted and annotated as such, then the new entity or annotation is tagged with a metadata "invalid".

The control step verifies that the extracted entity or annotation is valid against the ontology model. This implies controlling domains and ranges, cardinalities, date formats and the temporal information, the number formats and metric systems, etc. For instance, if in the preceding step the extracted entity was merged with an existing instance, the algorithms look at the properties of the extracted entity: are these properties types authorized for the entity's class? do these properties already exist on the merged instance? do they have the same values or different values? then how do we know which value is the right one, especially when dealing with thesaurus values such as geographical locations, or with time value such as dates? The algorithms try to automatically resolve these issues and when not possible, they also mark the new entity or annotation with the "invalid" metadata. All invalid statements are stored in the semantic repository on the server so that they can be retrieved and presented to the end-user for manual validation, if required by the target application.

The last step of this component, infer, is optional. It applies inference rules through a reasoning engine in order to discover new entities or new relations between them and also to control the overall coherence and quality of the semantic repository.

Storing annotations and knowledge in repositories

The Information Storage component has two steps, serialise and store. The serialization step parses the enriched and consolidated annotation schema in order to generate an output in the requested applicatoin format (XML, RDF, OWL, NewsML, CityGML, etc.). The second step, storage, is optional as it depends if the application directly digests the serialised format or stores the results in a knowledge store (such as ITM) and/or in a dedicated annotation server (such as Sesame).

The CA Manager framework is open-source and available on SourceForge.net 3 . The release includes a documentation describing how to install the CA Manager, how to configure workflows, how to develop new plugins as for connecting a new information extraction engine for example, and so on. Moreover, the above pipeline is also exposed as a web service, and a testing web client application is available from http://client2. mondeca.com/scan/.

EVALUATION

Within the TAO project 4 we implemented different workflows (a combination of different ontologies, semantic annotation tools, semantic repositories and corpora) for evaluating the flexibility and the scalability of the CA Manager framework. One of these was composed of loading the upper-level PROTON Ontology5 in the Sesame RDF repository and applying simple regular expressions on the corpus. The other two were based on the GATE Ontology developed within the TAO project, both using KCIT as the content augmentation system but one semantic repository being ITM 6 and the second one being Sesame RDF repository. Moreover, we have challenged flexibility of CA Manager beyond the TAO project by trialing it in different environments with various information extraction tools as well. For more details, see [START_REF] Damljanovic | CA Manager Framework: Creating Customised Workflows for Ontology Population and Semantic Annotation[END_REF].

The GATE domain ontology7 contains knowledge about GATE components, modules, plugins and language processing resources and was used by KCIT to annotate software artefacts about GATE software8 . In the rest of this section, we present evaluation of the two main steps described above: information extraction and consolidation, based on the described workflow configuration, using GATE ontology, KCIT annotation tool and ITM semantic repository.

We have first evaluated the KCIT information extraction tool, as consolidation is entirely dependent on the quality of the KCIT output. As KCIT is primarily a semantic annotation tool, we have measured its performance using standard information extraction measures, namely precision and recall. We collected 64 GATE software artefacts of various types, namely source code, source documentation, GATE user manual, publications and forum posts from the GATE mailing list. Then, we organised an annotation exercise with 12 human subjects to whom we gave a manual on how to proceed in validating annotations and creating a gold standard. Each document has been assigned to two participants so that we can calculate inter-annotator agreement.

Inter-annotator agreement

Inter-annotator agreement is used to establish the upper bounds on performance of information extraction tools such is KCIT. Table 1 shows precision, recall and F-Measure values, based on the results of this experiment.

For computing these measures between two annotation sets, one can use one annotation set as gold standard and another set as system output. One can switch the roles of the two annotation sets. The Precision and Recall in the former case become Recall and Precision in the latter, respectively. But the F1 remains the same in both cases.

Table 1: Inter-annotator agreement

Disagreements between annotators occurred for all document types excluding forum posts, with the source code being most problematic. In some cases, annotators did not agree on the annotation type. For instance, resources was annotated as a mention of gate:GATE-Resource 9 by one annotator, whereas another annotator labeled it as gate:ResourceParameter. In some cases, annotators decided to delete an annotation such as in the sentence Couldn't get resource data for gate.creole.morph.Morph, although the context indicates that resource refers to the GATE-Resource concept.

Another interesting example is annotating Annotation-Schema() with brackets included, or protected FeatureMap features annotated as a whole, not only features part as done by the other annotators.

We used the human-annotated corpus mentioned above to optimise the KCIT tool. In the first iteration we compared the automatic of KCIT against the gold standard. Then, the mistakes were examined manually and rules for improving KCIT performance were derived and implemented, mostly concerning the filtering and disambiguation phase.

After improving KCIT, 4956 annotations (out of 6175 in total) were correct, 49 partially correct 10 , 347 annotations were missing (manually added by participants who created the gold standard), and 823 annotations were wrong (manually deleted by the participants who created the gold standard). These were used in the calculation of the precision and recall figures reported in Table 2.

We conclude that with the proper configuration and 9 gate: is used instead of the full namespace which is http: //gate.ac.uk/ns/gate-ontology 10 Manually modified by the participants who created the gold standard, where modifications include for example extending the annotation to refer to the longer string; for instance, if ian roberts is not annotated as a whole, but only ian is, then this is considered partially correct.

tailoring the filtering phase, very good results can be achieved. The quality of the KCIT annotations approaches that of human annotators, while offering significant gains in terms of annotation effort required.

Evaluation of the information extraction and consolidation phase

We selected another 20 documents to serve as a representative corpus of GATE software artefacts, on which to evaluate KCIT's performance. This selection is made in order to cover not only the knowledge about GATE, but also different types of documents (structured, semistructured, and unstructured). As before, these were annotated manually and then used to calculate precision and recall values as presented next.

Information extraction

To evaluate the information extraction phase, the automatic annotations produced by KCIT were compared against the gold standard, using the GATE Benchmarking tool [START_REF] Cunningham | GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications[END_REF]. The results are shown in Table 3. For the 20 selected documents, 4523 created annotations were correct, 41 annotations were partially correct, 126 annotations were missing, and 255 were spurious. Further inspection of the annotated documents revealed that the majority of the spurious/missing annotations were due to errors by the GATE morphological analyser. For example, it could not extract correctly the roots of acronyms and camelCased words: the root of LanguageResources remained LanguageResources.

In addition, there were many wrong annotations of the word learn. The problem arises from KCIT not being able to disambiguate correctly based on the local context, resulting in each appearance of the word learn, e.g., learn GATE using movie tutorials, being annotated as referring to the GATE maching learning plugin. Similarly, many annotations were created for each mention of the word resources, even though there was no reference to GATE resources. For example, when reporting the problems on the mailing list, a user said I cannot waste too much resources on the server..., meaning computational time, not GATE components. Nevertheless, as demonstrated by the overall results, such problematic cases were quite infrequent.

Table 2: Average precision and recall values for all documents

The third class of mistakes arose from overlapping annotations not being filtered out properly by KCIT. For example, ANNIE NE Transducer was annotated as the whole string refering to the eponymous processing resource, but also Transducer was annotated as a mention of JAPE Transducer, which again should have been filtered out as redundant.

Overall, we can conclude that the performance of KCIT is of a good quality, especially on domain-specific documents and can be a good base for evaluating the consolidation algorithms. For example, forum posts and java classes were annotated with a very high precision and recall. As documents get more abstract and generalised, more ambiguities creep in (e.g., peer-reviewed publications) and KCIT's performance degrades.

Consolidation phase

To evaluate the consolidation algorithms, we applied recall and precision measures on the same corpus using the GATE ontology, KCIT exposed as a service, and ITM repository for storage.

Here the recall and precision measures are applied to the semantic annotations produced by KCIT exploited for evaluating the two following tasks:

• ontology population (knowledge instances newly created from annotations) and

• semantic annotation (semantic annotations controlled with regard to the instantiated concepts in the repository).

Hence, we obtain the two following adapted measures:

• Precision measures the number of annotations/instances correctly acquired divided by the total number of annotations/instances acquired.

• Recall measures the number of annotations/instances correctly acquired divided by the number of annotations/instances returned by KCIT.

Table 4 shows results according to the two different tasks. We want to emphasize the fact that we are not evaluating the KCIT results (done in previous section) or the quality of the ontology model but the performance of the consolidation algorithms themselves. First, it is important to notice that the consolidation algorithms are reducing the final number of knowledge instances or annotations based on the KCIT annotations. That means that the merging step is successful in resolving references to different annotations towards the same instance in the knowledge base. For example, in the movies.xml file, on 129 annotations generated by KCIT, CA Manager created only 46 knowledge instances and 27 semantic annotations. For instance, the KCIT annotated the two terms ontology tool and Ontology Tool that in fact correspond to the same instance of the class GATEPlugin, with both labels as names and the following URI, http://gate.ac.uk/ns/ gate-ontology\#Ontology\_Tools. This URI is then used as the reference link to create the semantic annotation between the document and this instance. So instead of having two annotations, only one is produced in this case and the labels have been consolidated on the knowledge instances. The same occurs with for example the term ANNIE which is annotated 9 times by KCIT whereas at the end of the CA Manager pipeline, there is only one semantic annotation referring to the same instance ANNIE of class "GATEPlugin"

It appears that sometimes the consolidation of the semantic annotation on one label has not been correctly achieved by the CA Manager, producing non pertinent annotations or instance references as some duplicates remained. For example, the terms "pipeline" and "Pipeline" produced four knowledge instances: two with the URI gate:Pipeline and two with the URI gate:GATEController. At most, we would like to have one of each URI if the CA Manager could not disambiguate to which instance the annotation refers to. But it should aggregate the two labels "pipeline" and "Pipeline" on each URI value.

This leads to only 76.5% of precision for creating knowledge instances first and as the semantic annotations are referring to these instances with their URI, the precision result is slightly better, reaching 93.3%. Indeed, if in the knowledge instances results, we obtain four instances with the same URI, each having their own label, in the semantic annotation we point to the URI, and not the individual labels. As such, the different knowledge instances possessing the same URI are merged into the one with now four labels and the semantic annotation can refer to this federated instance, improving the results. In fact, the more the CA Manager is used, the better it enriches the knowledge base and therefore the semantic annotation results.

The CA Manager obtains 100% recall on both tasks, hence there is no loss of information after processing the KCIT annotations to produce the final semantic annotations and knowledge instances. The CA Manager consolidation algorithms that deal with merging need to be improved in order to eliminate duplicated terms with different orthographic labels such as "datastore", "data store", "DATASTORE", and "Data Store" that must be merged together in the same knowledge instance and thus producing only one semantic annotation referring to the one particular instance. On the other hand, the consolidation algorithms that control the ontology model are performing nicely which is not very difficult in the GATE case study as we mostly refer to the class instances. There are no annotations which refer to the relation between knowledge instances for example. For doing this, we need to improve the linguistic analysis to support that feature and thus evaluate properly this functionality. This is the case in the research and industrial projects in which the CA Manager is now used.

RELATED WORK

KIM [START_REF] Popov | KIM -Semantic Annotation Platform[END_REF] performs semantic annotation and ontology population automatically in respect to their ontology, by identifying the Key Phrases and Named Entities (NE). As NE they consider people, organizations, locations, and others referred to by name. They use GATE [START_REF] Cunningham | GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications[END_REF] for NE Recognition. Our approach is more flexible than KIM, in the sense that the CA Manager is a generic framework and can thus support semantic annotation based on any information extraction tool, and also any ontology, or semantic repository.

Other similar work includes frameworks such as S-Cream [START_REF] Handschuh | S-CREAM -Semi-automatic CREAtion of Metadata[END_REF], MnM [START_REF] Motta | MnM: Ontology Driven Semi-Automatic and Automatic Support for Semantic Markup[END_REF], Artequakt [START_REF] Alani | Web-based Knowledge Extraction and Consolidation for Automatic Ontology Instantiation[END_REF] and OntoSophie [START_REF] Valarakos | Enhancing ontological knowledge through ontology population and enrichment[END_REF]. We can notice important differences between the CA Manager framework and similar approaches: some of them use machine learning techniques, some are based on high-level generic ontologies such as PROTON 11 rather than domain-oriented ontologies, or they perform either ontology population or semantic annotation. The main difference between those platforms and the CA Manager is the fact that the CA Manager preserves the independence between the content augmentation tool, such as KCIT, and the semantic repositories (Sesame 12 , ITM 13 ). It acts as a mediator, providing greater flexibility and adaptability capabilities as required by real world applications. For more details about the existing semantic annotation platforms we refer the reader to the survey in [START_REF] Uren | Semantic annotation for knowledge management: Requirements and a survey of the state of the art[END_REF].

Moreover, as the consolidation phase is a very important part of the CA Manager, we would like to emphasize that the tools for ontology population or semantic annotation which describe, or even mention, the consolidation phase in their workflows [START_REF] Alani | Web-based Knowledge Extraction and Consolidation for Automatic Ontology Instantiation[END_REF], are rare. However, this phase is extremely important to maintain the integrity and the quality of the application's referential.

In fact, most of them rely on manual validation only, to check the generated annotations or instances. In the knowledge acquisition field, the ArtEquAkt project [START_REF] Alani | Web-based Knowledge Extraction and Consolidation for Automatic Ontology Instantiation[END_REF] was concerned with the consolidation phase where Alani et al. defined four problems related to the integration of new instances in a knowledge base: duplicated information, geographical consolidation, temporal consolidation and inconsistent information. Their approach consists of instantiating the knowledge base based on the information extracted from the documents. They apply a consolidation algorithm driven by a set of heuristics and methods of terminological expansion based on the WordNet [START_REF] Fellbaum | WordNet -An Electronic Lexical Database[END_REF] lexical base.

Contrary to their approach, we are convinced that in order to preserve the integrity of the knowledge base, this consolidation phase must be carried out before the creation of the instances in the repository. Thus, only new and consistent information is created, yet preserving the integrity of the referential and thus improving the quality of the target application.

We presented the CA Manager framework which serves as a mediator between semantic annotation and ontology population, and is capable of consolidating and controlling this process while allowing human annotators to be involved, if required. We created various workflows to evaluate the flexibility and scalability of this framework, which offers adapted workflows for ontology population and semantic annotation based on Semantic Web standards and UIMA concepts. The main contribution of the CA Manager in comparison to other similar tools is that it allows easy plugin of information extraction tools, semantic repositories and ontologies.

We have created a gold standard corpus in the domain of software engineering, based on which we could calculate Inter Annotation Agreement, and also precision and recall values of automatically processed results. First, we have used this corpus to calculate the performance of KCIT -a GATE-based information extraction tool which has been exposed as a Web service and used by the CA Manager; then we calculated the performance of the consolidation algorithms based on the same corpus. The automatically produced annotations reach the level of human annotators and are therefore suitable for practical applications.

Figure 1 :

 1 Figure 1: Specialized UIMA processing pipeline

Table 3 :

 3 Precision and recall measures for the 20 GATE software artefacts

Table 4 :

 4 Performance results on the representative corpus of the GATE case study

	Element	Number	Number	Number	Recall	Precision	F1-measure
	type in	of correct	of missing	of	(A/A+B)	(A/A+C)	(R*P)/0.5
	the	elements	elements	spurious			(R+P)
	ontology	(A)	(B)	elements
				(C)
	Kb instances	208	0	64	1	0.765	0.867
	Annotations	168	0	12	1	0.933	0.965

http://www3.gartner.com/DisplayDocument?id=379859

UIMA website: http://www.alphaworks.ibm.com/tech/ uima

CA Manager release: http://sourceforge.net/ projects/scan-ca-manager

TAO website: http://www.tao-project.eu

PROTON website : http://proton.semanticweb.org

ITM website: http://mondeca.com/index.php/en/ intelligent_topic_manager

http://gate.ac.uk/ns/gate-kb

GATE website: http://gate.ac.uk

ACKNOWLEDGMENTS

This research was partially supported by the EU Sixth Framework Program project TAO (FP6-026460).


Danica Damljanovic
email: d.damljanovic@dcs.shef.ac.uk

Kalina Bontcheva

CA Manager Framework: Creating Customised Workflows for Ontology Population and Semantic Annotation

Keywords: I, 2, 4 [Knowledge Representation Formalisms and Methods]: representation languages, miscellaneous

We present the Content Augmentation Manager Framework for creating various adapted workflows for ontology population and semantic annotation based on Semantic Web recommendations and UIMA precepts. This framework supports ontology population from text (semi)automatically, by allowing easy plug-in of various types of components including information extraction tools, customised domain ontologies, and diverse semantic repositories. Our evaluation reveals that the framework offers flexibility, without compromising on precision and recall of the constituting components.

INTRODUCTION

Gartner reported in 2002 1 that for at least the next decade more than 95% of human-to-computer information input will involve textual language. They also report that by 2012 taxonomic and hierarchical knowledge mapping and indexing will be prevalent in almost all information-rich applications. There is a tension here: between the increasingly rich ontology-based, semantic models on the one hand, and the continuing prevalence of human language materials on the other. This process may be characterised as the dynamic creation of interrelationships between ontologies and unstructured textual content in a bidirectional manner.

However, transforming huge number of unstructured text into the semantically interlinked knowledge space is a big challenge. Two key parts of this process are semantic annotation and ontology population. While there are numerous tools to support these processes in isolation, many lack compliance with recent standards, but more importantly, lack the flexibility to customise and link them together.

In this paper, we present the Content Augmentation (CA) Manager Framework which is capable of performing and controlling the process of semantic annotation and ontology population by means of consolidation algorithms. CA Manager allows easy plug-in of various types of components including Information Extraction (IE) tools, customised domain ontologies, and diverse semantic repositories.

THE CA MANAGER FRAMEWORK

The core philosophy of the CA Manager is to bridge the gap between the content augmentation tools, and the semantic repository tools. It is conceived as a middleware, capable of controlling the quality and the validity of IE results against an ontology, matching them with existing resources (e.g. the application's knowledge base or repositories from the Linked Open Data initiative (linkeddata.org), and enriching them. To achieve that goal, the CA Manager relies on the recommendations formulated by the Semantic Web community (RDF/OWL languages, Service Oriented Architecture) combined with the UIMA-based infrastructure which have been enriched and customised to address the specific needs of semantic annotation and ontology population tasks.

The CA Manager proposes a list of logical steps arranged in a workflow, see Figure 1

EVALUATION

CA Manager was developed in the course of the TAO project (www.tao-project.eu), but is now used in several others. We implemented different workflows (a combination of different ontologies, semantic annotation tools, semantic repositories and corpora) for evaluating the flexibility and the scalability of the CA Manager framework, see Table 1. IE tool can either call a natural language processing tool or any kind of knowledge provider such as the semantic databases available within the Linked Open Data community. This particularly shows how the CA Manager is flexible to adapt its generic framework to all kinds of application needs. One of the workflows trialed during the TAO project was used to evaluate consolidation algorithms [START_REF] Amardeilh | Semantic annotation and ontology population[END_REF] of CA Manager. Namely, we used ontology-based IE tool KCIT [START_REF] Damljanovic | A text-based query interface to owl ontologies[END_REF], but one semantic repository being the ITM (see http://mondeca.com/) and the other one being the Sesame RDF repository. Using KCIT, we annotated 20 documents about GATE [START_REF] Cunningham | GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications[END_REF], with regards to the domain ontology (gate.ac.uk/ns/gate-kb). As consolidation is entirely dependent on the quality of the IE output, we have first measured precision and recall values for KCIT in isolation, by comparing automatically processed with human-annotated corpus. Then we calculated the performance of the consolidation algorithms based on the same settings.

The CA Manager obtains 100% recall hence there is no loss of information after processing the KCIT annotations to produce the final semantic annotations and knowledge instances. The CA Manager consolidation algorithms that deal with merging, need to be improved in order to eliminate duplicated terms with different orthographic labels such as "datastore", "data store", and "DATASTORE". On the other hand, the consolidation algorithms that control the ontology model are performing nicely which is not very difficult in the GATE case study as we mostly refer to the class instances. There are no annotations which refer to the relations between knowledge instances for example. In future, we plan to improve the linguistic analysis to support that feature.

CONCLUSIONS

We presented the CA Manager framework which serves as a mediator between semantic annotation and ontology population, and is capable of consolidating and controlling this process while allowing human annotators to be involved, if required. We created various workflows to evaluate the flexibility and scalability of this framework, which is based on Semantic Web standards and UIMA concepts. The main contribution of the CA Manager in comparison to other similar tools is that it allows easy plug-in of any IE tool, semantic repository, ontology or corpus, while also applying its own consolidation algorithm in order to link IE phase with ontology population.

  : a) Extracting the valuable knowledge and annotating the content; b) Con-solidating knowledge with regards to the ontology model and the semantic repository; c) Serialising the typesystem output in various formats and storing it in the semantic repository.

Figure 1 :

 1 Figure 1: Specialised UIMA processing pipeline

Table 1 :

 1 CA Manager workflows

	ontology	corpus	CA tool	Repository
	Architectural	3D objects DBPedia and	ITM
	ontology		Geonames
	(3D objects)		web services
	Adverse Drug	PubMed	Luxid
	Effect ontology	abstracts	(Temis)	ITM
	Tourism	Touristic	TimeFrame
	ontology	web sites	(Modyco,	ITM
			Univ Paris X)
			FunGen
	MiRNA	PubMed	Discovery	Sesame
	ontology	articles	(INSERM)

ACKNOWLEDGMENTS

This research was supported by the EU-funded TAO (FP6-026460), MUSING (FP6-027097), and ServiceFinder (FP7 215876) projects.


Sabine Marie Moulin
email: sabine.moulin@protonmail.com

The Implementation of Climate Change Provisions regarding Land Take in French Mountain Territories Land Use Planning

In 2019, the French government created a 'Citizens Convention for Climate' , with the task to make proposals to reduce greenhouse gas emissions in 2030 by 40% from their level in 1990, in accordance with social justice. These proposals were mainly converted into the Climate and Resilience Act of 22 August 2021. For land use planning, solutions mainly consist in protecting soils as carbon sinks with a purpose of dividing by two land take by 2031, in a move towards the aim of 'zero net land take' by 2050.

In this context, I here want to analyse if this first objective can be reached in French ski resorts with regard to the measures and timing enacted by Parliament.

As a follow up to IMC 2019, I chose the same three case studies in the Northern Alps I used two years ago with Anouk Bonnemains, a geographer, to analyse the implementation of climate change provisions in French mountain territories. These are urban plans at different spatial scales (municipalities' plans 'PLU' , intermunicipalities' plans 'PLUi' and territorial coherence programs 'SCOT'), concerning territories with different contexts (high and medium mountain areas) and development models.

It is unlikely that the target of dividing by two land take by 2031 will be reached. However, it will very much depend on how 'land take' is defined. Charts n° 2, 3 and 4: Comparison of cumulated land take planned in three urban plans for the period 2021-2030 according to the plans, their possible modification and the Climate and Resilience Act's target (point of reference since 2011) Chart 1 : Land take from 2011 to 2020 in Tarentaise-Vanoise, Coeur de Chartreuse and Chamrousse Tarentaise

Main results

Although land take has globally decreased from 2011 to 2020 (chart n° 1), the purpose of halving land take by 2031 will not be achieved unless the reality of land take is far below the figures planned (charts n°2 and 3).

There are two reasons for this :

-The Climate and Resilience Act has to cascade down from national level to local level through a 6-year timeline (see timeline below),

-Derogations based on unclear rules can be applied. In particular, urban plans that already aim at reducing land take by 33% do not have to change (art. 194). The methods for counting land take have nevertheless not been determined yet. Therefore, we do not know whether the studied plans meet this condition or not.

Therefore, to simplify the Act's rules, there are two possibilities : either the plan benefits from the derogation, or it will have to implement a different land take ratio (50% or less) no later than August 22, 2026 (Tarentaise-Vanoise and Coeur de Chartreuse) or August 22, 2027 (Chamrousse). The gap between the national and the local target will have grown by then. Furthermore, 'land take' has not been clearly defined yet. Chamrousse, that appears to be the most virtuous territory, plans to build an artificial pond, a waste reception center and various recreation facilities on top of its planned ratio considering them neutral for soils within the ski area.

The French government has been sued by some NGOs. It acknowledged that the measures provided for by the Climate and Resilience bill, which weakened citizens' proposals, were not sufficient to achieve its purpose of reducing greenhouse gas emissions by 40% (CS Commune de Grande Synthe v. France, July 1, 2021). This will probably also prove to be true for land take, especially if ski resorts persist in transforming mountains into amusement parks…. Unless citizens take legal action, once more.

  Implementation of Provisions regarding Land Take in the Climate and Resilience Act of 22 August 2021 land take area (ha), as planned Cumulated land take area (ha) modified according the Climate and Resilience Act Cumulated land take reduced by 50% compared to 2011-2020 Year Area (ha)


N Girard

E Baudrier
email: baudrier@unistra.fr

J-M Ogier

A perceptual image quality evaluation based on local spatial information

Keywords: Image comparison, Hausdorff distance, distance transform, local dissimilarity measure, quality measure

This paper presents a new comparative objective method for image quality evaluation. This method relies on two keys points: a local objective evaluation and a perceptual gathering. The local evaluation concerns the dissimilarities between the degraded image and the reference image ; it is based on a graylevel local Hausdorff distance. This new Hausdorff distance uses a generalized distance transform which is studied here. The evaluation result is a local dissimilarity map (LDMap). In order to include perceptual information, a perceptual map based on the image properties is then proposed. The coefficients of this map are used to weight and to gather the LDM measures into a single quality measure. The perceptual map is tunable but it gives encouraging quality measures even with naive parameters.

Introduction

Image quality evaluation is a key point in several domains including image compression algorithm assessment or graphical image quality evaluation. Even if the best method is the subjective method MOS (Mean opinion score), which is based on observers' evaluation, it is not always possible to seek it: it is subject to variations and it involves many people and a lot of time. An alternative is to use an automatic quality evaluation. In this frame, the measure can be estimated just on the transformed image itself or in comparison with a reference image. We focus on the latter kind of methods so-called comparative objective methods. There exists a lot of well known comparative objective methods like the Mean Square error (MSE) or the Peak Signal to Noise Ratio (PSNR), ... But none of the current methods take in account a perceptual evaluation because they are often based on a pixel to pixel difference and perceptual information include both local and global aspects of the image. In order to move closer to the evaluation of the final user, it is important to integrate a perceptual evaluation. We propose a new method which is based on two key points: 1) a local objective evaluation and 2) a perceptual gathering. 1) Unlike to human vision, the pixel to pixel difference is very sensitive to small translations. A generalization of it to a less sensitive local measure has been developed for binary images in [START_REF] Baudrier | Binary-image comparison with local-dissimilarity quantification[END_REF]: the so-called Local Dissimilarities Map (LDMap). It is based on the distance transform and several distance transform generalization to gray-level images are available. A comprehensive study of the LDMap generalization to graylevel images is presented here in quality measure aim. 2) Our method then exploits the spatial distribution of the dissimilarity measures gathered in the LDMap generalized to gray-level images thanks to a perceptual weighting that emphasizes image areas that are important for human vision. These weights, gathered in a socalled perceptual map, are based on brightness, shape and texture; they can be tuned in function of the final application. Our method results in a single measure. The proposed measure has been compared to subjective evaluations on the test image base IVC database and gives encouraging results. The two key points are detailed in the following sections.

The Local Dissimilarity Map 2.1 Definition

The LDMap is based on a local and adaptative evaluation of dissimilarities between images. It has been first defined on binary images via a local measure of dissimilarities, and is based on a distance between a pixel and a set of pixel. When Haussdorf distance is used for the local measure, the formula of the LDMap is simple (see Definition 1) and allows to generalize easily the LDMap to gray-level images. Fig. 1 illustrates the LDMAP between the image of letters CO and the image of letters ET.

Definition 1 LDM ap

Let I 1 , I 2 be two images, x ∈ R 2 a pixel, the LDmap between I 1 and I 2 is defined from R 2 to R by

LDM ap(x) = |I 2 (x) -I 1 (x)| max(|I 2 (x) -I 1 (x)|) max(d(x, I 1 ), d(x, I 2 )) ( 1
)
where d is a distance transform (DT), measuring the distance between a point and a set of points.

The distance transform and its continuous generalizations

The DT are based on an underlying distance between pixels. The different DT come from distinct underlying distances. We focus here on the underlying distance computation so as to choose the most adapted DT to image processing.

For binary images, pixels can be seen as points of the plan. The distance transforms are then based on 2D mathematical spatial distances (e.g. the Euclidean distance, the Mahalanobis distance and so on).

For gray-level images, the gray-level dimension is added to the two plan dimensions. There are different ways of including this dimension in the distance computation, and they influence the produced dissimilarity measure. Considering a gray-level image as a set of points of R 3 , the image is a surface in R 3 . Then the distance between two points of this surface is the shortest path between these two points accorded to a pathlength measure. There are two possibilities in the literature for this measure: the measure of the length of the path on the image surface or the measure of the area of the surface under the path on the image surface. A formal definition is given thereafter (Definition 2).

Definition 2 Let I be a continuous function defined on X the image support :

I : X -→ R (x, y) → I(x, y)
and let π : t ∈ [0, 1] → π(t) ∈ R 2 be a continuous path between the pixels p and q of I. The length of the path π (so-called L π ) is given by 1. the length of the path on the image surface

L 1 π = 1 0 |I (π(t))|dt (2)
2. the area of the surface under the path on the image surface

L 2 π = 1 0 |I(π(t))π (t)|dt (3)
Both of these possibilities haves been implemented in the discrete space. The discretization step implies also choices so the different versions of the DT underlying distances are briefly introduced in the next section.

Discrete implementations of the distance transforms

The presentation of the different DT is too long to be detailed here so we will only give a brief insight on them.

1. first type methods (introduced by P. Toivanen):

• the Distance Transform On Curved Space [START_REF] Toivanen | New geodesic distance transforms for gray scale images[END_REF][START_REF] Toivanen | Sequential local transform algorithms for gray-level distance transforms[END_REF] (DTOCS), the Weighted DTOCS (WDTOCS)

• the method improvements: 3-4-DTOCS and Optimal WDTOCS (Opt-WDTOCS).

second type methods

• the Gray-Weighted Medial Axis Transform [START_REF] Levi | A grey-weighted skeleton[END_REF] (GRAYMAT) introduced by G. Levi and U. Montanari. Its underlying distance is based on the pixel difference value and as a consequence, it promotes low gray-level pixel paths. It is aimed at skeletization. • the Gray-Weighted Distance Transform [START_REF] Rutovitz | Data structures for operations on digital images[END_REF] (GWDT) introduced by D. Rutovitz

Comments and choice

It is quite difficult to anticipate the qualities of the different DT from their definition, nevertheless, we have chosen the second family method because their underlying distance is closer to the notion of distance in the graylevel dimension. The second family indeed, takes into account differences of gray levels on a path and the spatial distance. It seems to us more appropriate than the first family which overwrites some spatial information for the benefit of gray level information (see Fig. 3). In the second family the differences between the versions are minor. Nevertheless the 3-4-DTOCS seems the most interesting to us because it does not underestimate the diagonal distance unlike to DTOCS. Our study on this point should be furthered to detail the choice. We want to take advantage of the spatial information contained in the LDMap to include perceptual information. The dissimilarities between two images are already quantified in the LDMap. Given an original image (reference image, original image) and a transformed image (compressed image, degraded image ...), then the sum of the coefficients in the LDMap (computed between these two images) can represent the transformation evaluation between the compared images. Nevertheless, it is known that the visual perception does not take the entire image into account [START_REF] Lorenzetto | Image comparison metrics: A review[END_REF]. So we construct a perceptual map (PMap) in order to use the local information of the LDMap and the observer's attention model, and we associate the two maps to obtain a single measure. The PMap combines the brightness, shape, texture attributes which are factors the eye is sensitive to. Each attribute is calculated on the reference image and for each one we define a map: for the brightness B, for the shape S and for the texture T . The Pmap is defined as weighted sum (Definition 3) of B, S and T . Then the PMap is normalized in order to be a weighting Map. Figure 4 presents an example of PMap.

Definition 3 P M ap

Let A be an image, the PMap of A is defined by :

P M ap A = p 1 • B + p 2 • S + p 3 • T (4)
Where p 1 , p 2 and p 3 are the weights respectively of the brightness, the shape and the texture, and where B, S and T are normalized.

By balancing the LDMap with the PMap as illustrated Fig. 5, we obtain the perceptual local dissimilarity map (PLDMap), containing local information on dissimilarities and visual perception, we can measure the quality of the transformed image by extracting the maximum value or the mean value of the PLDMap. We have study the two values and the maximum appears to be the best quality measure. The proposed measure has been compared to subjective evaluations on a test image base and gives encouraging results. The Fig. 6 shows our first results for one image compressed with different rates of compression. The smaller our measure, the better the quality, conversely for the Mean Opinion Score (MOS) the higher the score, the better the quality. Test results on a bigger database and in comparison with other quality are coming soon. In the future, the LDMap could be developed to color images and the PMap could take in account other attributes. We can refine weights of maps constituting the PMAP.

Figure 1 :

 1 Figure 1: Letters CO and ET and their LDMAP containing theirs local dissimilarities

Figure 2 :

 2 Figure 2: Distance transform of the binary image composed with a white pixel form and a black pixel background

Figure 3 :

 3 Figure 3: An example of some DT

Figure 4 : 2 Figure 5 :

 425 Figure 4: PMap with p 1 =2, p 2 =1 and p 3 =-2

Figure 6 :

 6 Figure 6: First result on two images compressed with different rates (r1 < r2 < r3 < r4 < r5) and with p 1 =4,p 2 =1,p 3 =-2