Jérôme Euzenat
email: cassia.trojahn@inrialpes.fr

Keywords: ontology matching, ontology alignment, evaluation, benchmarks, efficiency measure

The implementation of the automatic services for evaluating matching tools follows an iterative model. The aim is to provide a way for continuously analysing and improving these services. In this deliverable, we report the first iteration of this process, i.e., current implementation status of the services. In this first iteration, we have extended our previous implementation in order to migrate our own services to the SEALS components, which have been finished since the end of the first evaluation campaign.

Executive Summary

The goal of this deliverable is to report the current status of the service implementation for the automatic evaluation of matching tools.

For the first evaluation campaign, as reported in [START_REF] Meilicke | Services for the automatic evaluation of matching tools[END_REF], we have developed an evaluation service based on the use of a web service interface wrapping the functionality of a tool to be evaluated. This interface allowed to evaluate the tool without the need for a runtime environment. Contrary to this, the tools were executed on the machine of the tool developer. We have followed this approach mainly because some SEALS components were still under development in that time.

Since the end of the first campaign, several SEALS components have been finished. This allows to progressively migrate our previous implementation to the SEALS platform. Firstly, we have concentrated our efforts in using the Test Data and Results Repository by uploading and using test and results data (Chapter 2). Next, we have wrapped some tools in order to fulfill the requirements for deploying a tool into the SEALS platform (Chapter 3). Then, we have extended our BPEL workflow in order to use the new web service interfaces for accessing the repositories and invoking the tools (Chapter 4). In order to run integration tests, an express version of the SEALS Runtime Evaluation Service has been delivered. This express version allows to deploy locally some tools and to run a complete evaluation experiment (Chapter 5).

For the next development iteration (Chapter 6), we plan to improve the specification of our current raw results metadata. Further, we will modify the interpreter for generating interpretations to be integrated as custom services in the platform. Finally, we plan to extend the BPEL workflow in order to take into account fault handling.

Introduction

The main aim of SEALS is to provide a platform for supporting automatic evaluations. It plans to go beyond current evaluation software by providing a continuous evaluation platform that allows people to test their tools at any time and publish the results that they want to be recorded.

For the first evaluation campaign, due the complexity of the SEALS platform, required SEALS components (e.g., Runtime Evaluation Service) were under development. For that reason, we have implemented an evaluation service that bypasses one of the main sources of complexity, i.e., the deployment of a tool. Our approach was based on the idea of executing the tool on the machine of the tool vendor itself. In this scenario, the functionality of the tool is made available via a web service, which is accessed by the evaluation service during the execution of an evaluation workflow. We have developed a set of services for running a complete evaluation experiment, i.e., for iterating the test suites, invoking a tool, interpreting raw results and storing results into a relational database.

For the second stage of the project, where the focus is the second evaluation campaign, we aim at extending our previous implementation to integrate the SEALS components that have been released since the first evaluation campaign. This deliverable reports the first iteration of this migration process. The first step was to replace our own test suite web service iterator by the service accessing the SEALS Test Data Repository. It involved to require the development of a finegrained access to the repository. Once this functionality had been implemented, we specified the test suite metadata for each of our test suites and uploaded them into the repository. This repository can now also be accessed via a web service interface implementation provided by the platform. We have also worked with the interface and its implementation for uploading raw results into the SEALS Results Repository.

Regarding tools, in order to deploy them into the SEALS platform, developers must extend an interface and wrap its implementation in a ZIP file that includes a bridge as well all required libraries for running the tool. We have tested the required structure of the ZIP file by using some of the tools that have participated in the first evaluation campaign. Furthermore, a web service interface has been defined for invoking the deployed tools from evaluation descriptions. We have also tested this interface.

For running a complete evaluation experiment, an express version of the SEALS Runtime Evaluation Service has been delivered, which allows for invoking the web service implementations for accessing the SEALS Test Data and Results repositories and invoking the deployed tools. This whole environment must be deployed locally in a test machine and allows researchers to run integration tests.

The remainder of this deliverable is structured as follows. In Chapter 2, we discuss the usage of the SEALS repositories. Chapter 3 presents the required packaging format for deploying a tool into the platform. In Chapter 4, we present our extended evaluation workflow. Some notes about running an integration test are presented in Chapter 5. Finally, Chapter 6 concludes the deliverable and presents the perspectives for the next development iteration. For our evaluation workflows it was important to retrieve the DataItems in a finegrained way. We participated in the specification of a Suite metadata definition that supports this finegrained access. At the same time we took into account that the approach has to be generic enough to be usable by the other research workpackages. According to this approach, a test suite is then described using the following metadata:

< rdf:RDF xmlns:rdf = " http: // www . w3 . org /1999/02/22 -rdf -syntax -ns # " xmlns:seals = " http: // www . seals -project . eu / ontologies / SEALSMetadata . owl # " xmlns:owl = " http: // www . w3 . org /2002/07/ owl # " xmlns:dc = " http: // purl . org / dc / terms / " xmlns:xsd = " http: // www . w3 . org /2001/ XMLSchema # " xmlns:rdfs = " http: // www . w3 . org /2000/01/ rdf -schema # " > < seals:Suite rdf:about = " http: // www . seals -project . eu / alignment / benchmark # BenchmarkSuite " > < s ea ls :h as Su it eI te m rdf:resource = " http: // www . seals -project . eu / alignment / benchmark #101 " / > < s ea ls :h as Su it eI te m rdf:resource = " http: // www . seals -project . eu / alignment / benchmark #102 " / > < s ea ls :h as Su it eI te m rdf:resource = " http: // www . seals -project . eu / alignment / benchmark #103 " / > </ seals:Suite > < seals:SuiteItem rdf:about = " http: // www . seals -project . eu / alignment / benchmark #102 " > < se als:ha sDataI tem rdf:resource = " http: // www . seals -project . eu / alignment / benchmark #101 -source " / > < se als:ha sDataI tem rdf:resource = " http: // www . seals -project . eu / alignment / benchmark #102 -target " / > < se als:ha sDataI tem rdf:resource = " http: // www . seals -project . eu / alignment / benchmark # ref101to102 -ref " / > < dc:identifier > 102 </ dc:identifier > </ seals:SuiteItem > < seals:DataItem rdf:about = " http: // www . We have uploaded three test suites into the SEALS repository, which correspond to the data sets used in the first evaluation campaign: Anatomy1 , Benchmark2 and Conference3 .

Results Repository

The results repository stores raw results (i.e., alignments in Work Package 12) and interpretations of these results (i.e., precision and recall). In this deliverable, we focus on uploading raw results because custom interpreters, which are generating the interpretations, are not yet integrated into the express runtime version of the SEALS platform. For uploading raw results, metadata must be specified that describes the stored results.

In a similar way of what is done for a test suite, raw results are encapsulated in a ZIP file, together with the suite metadata file. In the current implementation, we have used the results composer service provided by the platform and a minimal version of the suite metadata. This service bundles the raw result of each test iteration (i.e., each resulting alignment) and encapsulates all the results in the ZIP file.

A minimal suite metadata4 has been specified for describing the raw results:

<? xml version ="1.0" encoding =" UTF -8"? > < rdf : RDF xmlns : rdfs =" http :// www . w3 . org /2000/01/ rdf -schema #" xmlns : seals =" http :// www . seals -project . eu / ontologies / SEALSMetadata . owl #" xmlns : rdf =" http :// www . w3 . org /1999/02/22 -rdf -syntax -ns #" xmlns : dcterms =" http :// purl . org / dc / terms /" > < seals : RawResult rdf : about =" http :// www . seals -project . eu / metadata / a n y U R I /" > < seals : hasName rdf : datatype =" http :/ www . w3 . org / XMLSchema # string " > Benchmark -Aroma -Raw -Results </ seals : hasName > </ seals : RawResult > </ rdf : RDF > For the next iteration, we have to extend the required metadata in order to fulfill the specifications in [START_REF] Marte | Iterative evaluation and implementation of the results repository service v1.1-beta[END_REF]. Furthermore, as we have reported below, at the time of writing this deliverable, the details about how to include interpreters in the current version of the Runtime Evaluation Service have not been provided. As discussed in the last Seals plenary meeting 5 , each WP will be responsible for defining their own interpreters (as WSDL interfaces) and integrating them into the platform. We have already defined an interface for measuring precision and recall and are waiting for guidelines for integrating it into the platform in a way that it will be available for access from the BPEL workflow (as a partner link).

Tools and Evaluation Description Repositories

Currently we do not use the other two repositories, because they are only partially integrated into the platform. In particular, the following reasons prevent us from using the services of these repositories.

• The SEALS Tools Repository is to our knowledge currently not connected to the upload functionality of the web portal. Even more important is the problem

FP7 -238975 Deliverable 12.5
that there is no connection between storing a tool in the repository and executing it on the platform. Currently tools have to be deployed manually in order to evaluate them. There is no automatism that retrieves a tool from the repository and deploys and executes it automatically.

• The situation of the SEALS Evaluation Description Repository is similar. Currently only the core element of the evaluation description -the BPEL itselfis required to run an evaluation. This BPEL workflow has to be deployed and executed manually. There exists no automatism that retrieves an evaluation description and executes it automatically.

To benefit from the functionality of the SEALS Tools Repository, it is first of all required to continue with the development of the automatic tool deployment. Currently it is not supported that a tool is deployed automatically. Thus, it is unclear why tool developers should upload a tool to the platform that cannot be deployed and executed. For that reason it is highly problematic that this functionality is currently set to the status unplanned according to the relevant Redmine issues. 6The use of the SEALS Evaluation Description Repository depends on the availability of the functionality to execute a SEDL document (or at least to execute the BPEL encapsulated in the SEDL document). To enable this, a tight collaboration between the developer of the SEDL specification and the people responsible for providing the execution environment of the BPEL workflow is required.

FP7 -238975

Deliverable 12.5

Tool Wrappers

In order to deploy a tool into the SEALS platform, developers must implement a tool interface (i.e., IOntologyMatchingToolBridge in the case of Work Package 12). This interface specifies the methods that will be called for invoking the tool from the platform. The interface implementation must be encapsulated together with all required libraries and additional files in a ZIP file, following the structure presented in the following example. The folder bin/lib/ contains all required libraries for executing the tool. In the specific example, demomatcher-bridge.jar contains the implementation of the tool interface. Besides the required libraries, some additional files and few empty folders can be found. The empty folders can contain additional resources that are not required with respect to our simple example. The four files deploy.bat, start.bat, stop.bat, and undeploy.bat have to be part of the package. They are currently not used, but will be used in the next version of the platform to automatically prepare the environment required by the tools.

One important component in this package is the descriptor.xml file, which provides the description of the tool, the list of required libraries and the specification of the class implementing the tool interface: We have contacted several tool developers to check if the wrapping works with real tools. For some of these tools we used the version that is available via the official webpage of the tool, while for other we contacted the developer to retrieve access to the tool. The tools we have tried to wrap so far are the tools AnchorFlood [START_REF] Seddiqui | Anchor-Flood: results for OAEI[END_REF], Aroma [START_REF] David | AROMA results for OAEI[END_REF], Eff2Match [START_REF] Wei | Eff2Match results for OAEI[END_REF], Falcon-AO [START_REF] Hu | Falcon-AO: A practical ontology matching system[END_REF], Lily [START_REF] Wang | Lily: ontology alignment results for OAEI 2009[END_REF], and Taxomap [START_REF] Hamdi | TaxoMap alignment and refinement modules: Results for OAEI[END_REF]. While doing this we noticed that many tools specify internally a relative path from the current working directory to a required configuration file (or a similar kind of resource). We have to clarify if this results in a problem or requires some additional agreements regarding tool deployment and execution. In parallel we are writing a tutorial that allows the tool developer to wrap the tool on his own.

Our first experiences indicate that some problems relevant in the deployment process can only be solved by the use of manual work-arounds. Even more, it is not clear whether such an approach will always be applicable given a high number of participants. This is also related to the fact that the automized deployment is not yet available. In particular, is is currently not the case that the start, stop, deploy, and undeploy-scripts are executed by the platform, nor it is planned to make this functionality available in near future. 1 FP7 -238975 Deliverable 12.5

Evaluation Workflow

We have modified the BPEL workflow described in [START_REF] Meilicke | Services for the automatic evaluation of matching tools[END_REF] in order to replace our previous partner link definitions (external web services) by the new SEALS services:

• test repository service: provides access to the Test Data Repository, in order to retrieve a test suite and its test items to iterate over the test suite; • results repository service: provides access to the Results Repository, in order to add a raw result and its interpretations; • results composer service: a utility service that provides operations for bundling the raw results generated during the evaluation execution in a ZIP file; • tool service: provides access to a deployed tool; • evaluation service: describes the common service interface for the workflow entry point and the callback service interface (asynchronous process).

In the current implementation, the workflow iterates over a test suite and invokes a tool that generates an alignment between the source and target ontologies for each test item of a test suite. As custom interpreters are not yet integrated in the express version of the Runtime Evaluation Service, we can not generate interpretations for the raw results.

Basically, the BPEL workflow (Figure 4.1) interacts with the services through partner links, which encapsulate the corresponding web service interfaces. The "evaluation" process starts by receiving from the client process two input parameters: the test suite name and version. Due to an internal platform requirement, some information (e.g., ExecutionRequestId) is passed to the BPEL process in the header part of each message exchanged with partner links. One assigns activity (setHeaderForInvocations) is used to initialize the header part of all messages.

As we reported in §2.2, for uploading results into the Results Repository one must specify their metadata. In our workflow, we firstly initialize the metadata for the results composer (activities assignMetadata and addMetadaResults). This metadata will be encapsulated later, by the results composer service, into the ZIP file containing all raw results. Next, the parameters for loading a test suite are initialized (setParamsToLoadTestSuite) and the suite is loaded (loadTestSuite). Figure 4.2 shows the iteration over a test suite. For each test case in the test suite, the ontology source (getOntoSource) and the ontology target (getOntoTarget) are retrieved from the Test Data Repository. These two elements are then used as input parameter (setParamsAlign) for invoking the tool (align). The output of this invoke is the URL of the file containing the generated alignment. This URL is then sent to the results composer (addDataItem), which uses this information later for generating the ZIP file containing all generated raw results.

After iterating over all test cases, the ZIP file, containing all raw results (alignment files) and the corresponding metadata, is generated by the results composer service (createBundle). Next, this bundle is submitted to the Results Repository by invoking the addRawResults method of the results repository service.

The workflow is asynchronous, i.e., the client does not wait for the complete execution of the evaluation. For that reason a notification must be sent when the process finishes (callbackClient). The full BPEL code can be found at https://svn.seals-project.eu/seals/ tags/deliverables/D12.5/M21/software/evals.

FP7 -238975

Deliverable 12.5

Integration test

For running an integration test, i.e., executing a BPEL workflow that accesses repository services and deployed tools, an express version of the SEALS Runtime Evaluation Service has been delivered for local deployment. This version is composed of two main sub-modules:

• res-core-sa: takes care of the complete orchestration of the activities involved in the evaluation of a tool. From this module, an evaluation description (BPEL workflow) is executed; • res-worker-sa: is responsible for the launching of tools. From this sub-module, tools are deployed, started and executed.

These two modules are deployed locally in different instances of a servicemix container. The BPEL workflow must be deployed in the same instance as res-core-sa, while the path for the tool package (ZIP file presented in §3) must be configured in the same instance as res-worker-sa. Both of these modules communicate with each other and share service endpoints. This allows to execute an integrated test.

We have successfully deployed res-core-sa, res-worker-sa, the BPEL workflow and one tool. Based on this we could successfully execute an integration test. In our integration test we have used the tool Anchor-Flood [START_REF] Seddiqui | Anchor-Flood: results for OAEI[END_REF], which is -wrapped in the appropriate way -available at https://svn.seals-project.eu/seals-dev/omt/tools/ aflood. For invoking the deployed BPEL we have used an EvaluationBinding test suite, created using the SOAP-UI tool. In Appendix A, the code for the test suite can be found.

FP7 -238975

Deliverable 12.5

Conclusion and Future Work

This deliverable has reported the current implementation status of Work Package 12. We have migrated our previous implementation in order to take into account the new delivered SEALS components. We have successfully wrapped some tools and started to write a tutorial to help tool developers. We successfully achieved to run an integration test which involves the execution of a BPEL workflow that accesses the SEALS repositories and invokes a deployed tool.

For the next iteration, we plan to extend the definitions of metadata for raw results, in order to fulfill the requirements specified in [START_REF] Marte | Iterative evaluation and implementation of the results repository service v1.1-beta[END_REF]. We already implemented the most important interpreters, however, we need to integrate them as custom interpreter services into the express version of the SEALS Runtime Evaluation Service in the next step. We plan as well to extend the BPEL workflow in order to take into account fault handling. Further, we have to gain more experiences in wrapping tools, to anticipate possible problems in running a fully automized evaluation on top of the SEALS platform.

Figure 4 . 1 :

 41 Figure 4.1: BPEL workflow.Figure 4.2: Iteration over a test suite.

Figure 4 . 2 :

 42 Figure 4.1: BPEL workflow.Figure 4.2: Iteration over a test suite.

Table of Contents

 of Test Data Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Results Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Tools and Evaluation Description Repositories . . . . . . . . . . . . . .

	List of figures
	1 Introduction
	2 SEALS Repositories
	2.1 3 Tool Wrappers
	4 Evaluation Workflow
	5 Integration test
	6 Conclusion and Future Work
	References
	A SOAP-UI Test Suite

http://seals.sti2.at/tdrs-web/testdata/persistent/Anatomy+Testsuite/2010/suite/

http://seals.sti2.at/tdrs-web/testdata/persistent/Benchmark+Testsuite/2010/ suite

http://seals.sti2.at/tdrs-web/testdata/persistent/Conference+Testsuite/2010/ suite/

One of our uploaded raw results can be found at http://seals.sti2.at/rrs-web/results/ Benchmark-Aroma+raw+results/

Sheffield, 31/01/2011-03/02/2011

See http://www.development.seals-project.eu/redmine/issues/344.

See for example http://www.development.seals-project.eu/redmine/issues/344.

//www.seals-project.eu/ Document URL EU Project Officer Carmela Asero

Project Consortium Information

Participant's name Partner Contact Universidad Politécnica de Madrid Asunción Gómez-Pérez Email: asun@fi.upm.es

List of Figures

A. SOAP-UI Test Suite

This section lists the xml file of the SOAP-UI project. It corresponds to a test suite for invoking the BPEL workflow.

<? xml version ="1.0" encoding =" UTF -8"? > < con : testSuite xmlns : con =" http :// eviware . com / soapui / config " name =" BPEL -TestSuite " > < con : settings / > < con : runType > SEQUENTIAL </ con : runType > < con : testCase failOnError =" true " f a i l T e s t C a s e O n E r r o r s =" true " keepSession =" false " maxResults ="0" name =" e v a l u a t i o n B i n d i n g TestSuite " s e a r c h P r o p e r t i e s =" true " id ="4 fd1db7e -8 ddd -4665 -913 a -604 e01e3c010 " > < con : settings / > < con : testStep type =" properties " name =" Initializing " > < con : settings / > < con : config xmlns : xsi =" http :// www . w3 . org /2001/ XMLSchema -instance " xsi : type =" con : Pr operties Step " saveFirst =" true " > < con : properties > < con : property > < con : name > header . destination . httpURI </ con : name > < con : value > http :// localhost </ con : value > </ con : property > < con : property > < con : name > header . destination . executionRequestId </ con : name > < con : value > urn : erq :6 ba7b810 -9 dad -11 d1 -80 b4 -00 c04fd430c8 </ con : value > </ con : property > </ con : properties > </ con : config > </ con : testStep > < con : testStep type =" transfer " name =" Transfer variables " > < con : settings / > < con : config xmlns : xsi =" http :// www . w3 . org /2001/ XMLSchema -instance " xsi : type =" con : P r o p e r t y T r a n s f e r s S t e p " > < con : transfers s e t N u l l O n M i s s i n g S o u r c e =" true " t r a n s f e r T e x t C o n t e n t =" true " failOnError =" true " ignoreEmpty =" false " transferToAll =" false " useXQuery =" false " entitize =" false " t r a n s f e r C h i l d N o d e s =" false " > < con : name > ToHttpURI </ con : name > < con : sourceType > header . destination . httpURI </ con : sourceType > < con : sourceStep > Initializing </ con : sourceStep > < con : targetType > Request </ con : targetType > < con : targetStep > evaluation </ con : targetStep > < con : targetPath >//*[ local -name () = ' HttpURI '] </ con : targetPath > </ con : transfers > < con : transfers s e t N u l l O n M i s s i n g S o u r c e =" true " t r a n s f e r T e x t C o n t e n t =" true " failOnError =" true " t r a n s f e r C h i l d N o d e s =" false " > < con : name > ToExecutionRequest </ con : name > < con : sourceType > header . destination . executionRequestId </ con : sourceType > < con : sourceStep > Initializing </ con : sourceStep > < con : targetType > Request </ con : targetType > < con : targetStep > evaluation </ con : targetStep > < con : targetPath >//*[ local -name () = ' ExecutionRequestId '] </ con : targetPath > </ con : transfers > </ con : config > </ con : testStep > < con : testStep type =" request " name =" evaluation " > < con : settings / > < con : config xmlns : xsi =" http :// www . w3 . org /2001/ XMLSchema -instance " xsi : type =" con : RequestStep " > < con : interface > evaluationBinding </ con : interface > < con : operation > initiate </ con : operation > < con : request name =" evaluation " > < con : settings > < con : setting id =" com . eviware . soapui . impl . wsdl . WsdlRequest@request -headers " >& lt ; xml -fragment /& gt ; </ con : setting > </ con : settings > < con : encoding > UTF -8 </ con : encoding > < con : endpoint > http :// localhost :8092/ evaluation </ con : endpoint > FP7 -238975 Deliverable 12.5 < con : request > <![ CDATA [ < soapenv : Envelope xmlns : soapenv =" http :// schemas .

xmlsoap . org / soap / envelope /" xmlns : v1 =" http :// www . seals -project . eu / resources / res / common / header / xsd / v1 " xmlns : v11 =" http :// www . seals -project . eu / resources / res / engine / evaluation / wsdl / v1 " xmlns : v12 =" http :// www . seals -project . eu / resources / res / common / types / xsd / v1 " > < soapenv : Header > < v1 : Header > <! --Optional : --> < v1 : Destination > < v1 : HttpURI > http :// localhost </ v1 : HttpURI > </ v1 : Destination > < v1 : ExecutionRequestId > urn : erq :6 ba7b810 -9 dad -11 d1 -80 b4 -00 c04fd430c8 </ v1 : ExecutionRequestId > </ v1 : Header > </ soapenv : Header > < soapenv : Body > < v11 : initiate > < v11 : argument > < v12 : name > Testsuite </ v12 : name > < v12 : value > Benchmark + Testsuite </ v12 : value > </ v11 : argument > < v11 : argument > < v12 : name > Version </ v12 : name > < v12 : value >2010 </ v12 : value > </ v11 : argument > </ v11 : initiate > </ soapenv : Body > </ soapenv : Envelope >]] > </ con : request > < con : jmsConfig J MS D el iv er y Mo de =" PERSISTENT "/ > < con : j m s P r o p e r t y C o n f i g / > < con : wsaConfig mustU nderstan d =" NONE " version ="200508"/ > < con : wsrmConfig version ="1.2"/ > </ con : request > </ con : config > </ con : testStep > < con : testStep type =" mockresponse " name =" e v a l u a t i o n C a l l b a c k " > < con : settings / > < con : config xmlns : xsi =" http :// www . w3 . org /2001/ XMLSchema -instance " xsi : type =" con : M o c k R e s p o n s e S t e p " > < con : interface > evaluationCallbackBinding </ con : interface > < con : operation > onResult </ con : operation > < con : path >/ evaluation / callback </ con : path > < con : port >8088 </ con : port > < con : response > < con : settings / > < con : re s po ns eC o nt en t xsi : nil =" true "/ > < con : wsaConfig mustU nderstan d =" NONE " version ="200508"/ > </ con : response > </ con : config > </ con : testStep > < con : properties / > </ con : testCase > < con : properties / > </ con : testSuite >


Jérôme Euzenat
email: jerome.euzenat@inrialpes.fr

Onyeari Mbanefo

Arun Sharma

Sharing resources through ontology alignments in a semantic peer-to-peer system

Keywords: Semantic peer-to-peer system, semantic annotation, ontology, heterogeneous annotation, resource sharing, ontology alignment, ontology matching, query, peer data management system, alignment composition, alignment inverse, PicSter, semantic web

In heterogeneous semantic peer-to-peer systems, peers describe their resources through ontologies that they can adapt to their particular needs. In order to interoperate, queries need to be transformed with respect to alignments between their ontologies before being evaluated. Alignments are thus critical for sharing resources in such systems. We report an experiment that explores how such alignments can be obtained in a natural way. In particular, asking users to provide alignments is a heavy constraint that must be relaxed as much as possible. This can be attempted through automatic matching. However, we suggest other possible solutions.

aims to illustrate the need for ontology alignments in such systems and the difficulties of providing such alignments.

After briefly presenting the field of semantic peer-to-peer systems, we describe extensively the PicSter picture sharing system. This description highlights the need for alignments in such systems. Then we elaborate on several possible solutions for obtaining these alignments and the difficulties facing these solutions. We do so through the description of an experiment.

Related work

In principle, peer-to-peer systems are distributed systems which do not rely, in their communication, on centralised resources. In practice, they often rely on some directory for joining a network. Once they have joined, all communication goes through adjacent peers. Peer-to-peer systems are well known as resource sharing devices in which peers offer resources and can query other peers for particular resources. We are concerned with such type of systems (other systems may be built on a publish and subscribe protocol, for instance).

Semantic peer-to-peer systems are peer-to-peer systems using semantic technologies for describing shared resources. The use of ontologies allows a richer description framework for resources than tags and simple categories. In particular, it allows querying the other peers with more precise queries ("Give me the Portraits depicting Paul Cezanne") instead of ("Portrait" and "Cezanne"). It also allows peers to draw inferences from the query and their ontology. For instance, something classified as a "Self-portrait" will be qualified as an answer for the above query if a "Self-portrait" is defined as a "Portrait". Similarly, a "Painting displaying Paul Cezanne" may be considered a "Portrait of Paul Cezanne" if "Portrait" is defined as a "Painting displaying some person". In summary, semantic peer-to-peer systems aim at providing a more complete and more precise answer to the queries that are exchanged between peers.

There have been several such systems. (Staab and Stuckenschmidt, 2006) provides a good overview of issues in such systems.

Edutella (Neijdl et al., 2002) was an early project for sharing educational resources. The system used the JXTA protocol to query over the network RDF annotation of these resources. One problem addressed by Edutella was to have more precise resource descriptions and queries while avoiding the fragmentation of niche P2P systems. The use of general purpose technologies such as those of the semantic web was the solution to this problem.

A first semantic peer-to-peer system was BibSter [START_REF] Haase | Bibster -a semanticsbased bibliographic peer-to-peer system[END_REF] developed in the framework of the SWAP European project. In BibSter, resources are bibliographic entries. They are expressed in RDF with regard to the same RDF schema (built from BibTeX). Hence queries are unambiguous. However, peers may have different ways to describe the same entry within BibTeX (using more or less information and more or less the same categories). Hence, the project has developed techniques for matching similar items [START_REF] Haase | Personalized information access in a bibliographic peer-to-peer system[END_REF].

In this chapter, we focus on open semantic peer-to-peer systems in which peers are free to use any ontologies they estimate suitable to their purposes.

Peer Data Management Systems (PDMS) are peer-to-peer systems whose shared resources are data. They have gone through some popularity in database. As data management systems, they are usually made of peers supporting a database that can be queried through an query language such as SQL. Peer databases are defined by a known schema which can be different from peer to peer. Although they do not provide a particularly expressive ontology, PDMSes address the problem of heterogeneity of the schema used by each peer through the use of mappings which are usually correspondences between queries.

Piazza [START_REF] Ives | Piazza: mediation and integration infrastructure for semantic web data[END_REF] is such a PDMS based on XML and Xquery. SomeWhere [START_REF] Rousset | Somewhere in the semantic web[END_REF]) is another such system initially based on Datalog with a well-defined query evaluation procedure. It has been extended towards RDF and OWL over the years. Most of the PDMSes are made of autonomous peers, non controlled by human beings. Other systems, such as BibSter and most of the well-known internet sharing systems, are directed by someone manipulating the system. This is also the context of the system we consider here.

Semantic peer-to-peer systems are confronted with important challenges such as routing queries in the network for efficiency or accuracy, replicating resources across peers, decomposing queries or finding the correct matching between two peers. These issues are well described in (Staab and Stuckenschmidt, 2006;[START_REF] Zaihrayeu | Towards peer-to-peer information management systems[END_REF]. This chapter considers exclusively the issue of establishing alignments between ontologies and offering solutions to end users who are not ontology nor matching specialists.

SETTING THE STAGE: The PicSter system

PicSter is an experimental semantic peer-to-peer picture annotation and sharing system developed in 2006 at INRIA [START_REF] Sharma | Lightweight synchronization of ontologies[END_REF]. In PicSter, both the semantic sharing aspect and the annotation aspects are important: this is because users have more freedom in modifying their ontologies that the system is heterogeneous and this makes answering queries difficult.

Semantic peer-to-peer framework

This case study considers a particular framework that we call semantic peer-to-peer. This framework involves the following elements:

 Peers: are the agents which describe and share a set of resources, e.g., pictures.

 Resources: are the resources that will shared among peers. These resources can be stored in peer directories or available on the web.  Ontologies: define the vocabulary used for annotating resources. They can be found on the web or be locally available at each peer.  Instances: are individuals described by the ontologies. They are stored separately in each peer. They can also be referred from the web through URIs.  Annotations: describe resources or resource fragments using the ontologies. Agents annotate resources by indicating that they belong to particular classes of the ontology, e.g., Picture, or they have relations with particular instances, e.g., depicts Grenoble.  Queries: are expressions used for selecting resources with regard to their annotations. They are usually expressed in a dedicated query language.  Alignments: express relationships between ontologies so that an annotation expressed with regard to one ontology can be understood with the vocabulary of another ontology. For instance, a "Lilium" under one ontology can be understood as a subclass of "Fleur" in another ontology. Each peer can work independently of the others as a fully functional system. As far as content is concerned, this framework covers the examples of peer-to-peer systems that we have considered previously.

This framework provides the basic functions of semantic desktops [START_REF] Schumacher | Combining Metadata and Document Search with Spreading Activation for Semantic Desktop Search[END_REF]:

 annotating a resource with elements from ontologies, and  searching for resources annotated by particular elements. Semantic desktops provide two additional and advanced features:

 extending ontologies, and  sharing annotations.

In this framework, resources are identified by URIs, they are annotated in RDF [START_REF] Carroll | RDF concepts and abstract syntax[END_REF] through vocabularies defined by ontologies in OWL [START_REF] Dean | OWL Web Ontology Language: Reference[END_REF] or RDF Schema [START_REF] Brickley | RDF vocabulary description language 1.0: RDF schema[END_REF]. Ontologies can be related by alignments and queries are expressed in SPARQL [START_REF] Prud'hommeaux | SPARQL query language for RDF[END_REF]. Hence, this framework takes the maximum advantage of standard semantic web technologies.

This framework is instantiated in PicSter that we describe in the remainder.

PicSter overview and architecture

Our working example is the use of semantic web technologies for annotating resources (and particularly pictures) on one's computer. This is already a very helpful application since it can help people querying their data with respect to very precise and personal vocabularies (this is more powerful than simple schemes such as those used in Web 2.0 picture sharing sites).

We have developed PicSter, a prototype for ontology-based peer-to-peer picture annotation that allows accommodating users' needs in the ontologies used for annotation. PicSter provides the ability to annotate pictures in RDF by selecting individuals and classes in chosen ontologies. The ontologies can be modified on the fly by users as they need to refine the classification, e.g., that a Fjording is a Horse, or to add individuals, e.g., that Tony is a particular Fjording. This requires very little overhead.

Since users can select and evolve ontologies as they need it, their ontologies are necessarily different from each others. Hence, when connecting to the peer-to-peer system, users have to provide (partial) alignments between the ontology they use and those of the peer they are connected to. These alignments are used for transforming queries from one peer to another so that the query can be interpreted with regard to the other peer's ontology. These alignments may be stored in an alignment server and can be retrieved on demand.

PicSter peers can be used off-line, for annotating local pictures and querying them, or online, when a peer accepts incoming queries from authorised peers and can issue queries to these peers. In order to properly work off-line, PicSter peers cache most of the information they deal with. Hence, peers store:  List of authorised peers,  Resources (here pictures),  Ontologies, as well as local extensions to these ontologies,  Instances,  Annotations. These resources are stored in separate are stored in several silos (actually file system directories) of individual PicSter repositories according to the following rules:

 pictures: contains all the resources as individual files, unchanged.

 ontologies: contains the original ontologies as unchanged files and the extension of these ontologies as separate prefixed files.  instances: contains all the individuals described with regard to the ontologies. They are stored as one file per concept containing all its instances.  annotations: contains picture annotations expressed in RDF. There is one file per picture containing all its annotations. An example of annotation is given in appendix. These directories are inspected when launching PicSter so that it can load ontologies and pictures. Annotations and individuals are loaded at query and annotation time.

PicSter offers two main functions (see Figure 1):

 Annotation editor allowing to (1) annotate artefacts with elements of the ontology, and (2) evolving the ontology to meets the annotator's need.  Query interface allowing to answer queries (1) against the local store, and (2) by submitting them to connected peers through their alignments. The goal of PicSter is to share the annotations and resources with other peers. For that purpose, peers are connected together and can send queries about their annotations to other peers and receive replies (under the form of lists of picture URIs satisfying the query). The queries are thus either evaluated locally or routed to other peers.

PicSter is based upon a peer-to-peer model without any centralised authority, but directories for finding peers. It is also an autonomous model: a PicSter peer without neighbours is a fully functional software. This authorises the off-line use of the tool: nothing pertaining to the peer is stored on the peer network.

The PicSter annotation tool is based on the PhotoStuff software [START_REF] Halaschek-Wiener | Photostuff -an image annotation tool for the semantic web[END_REF] that has been developed at MindLab (University of Maryland). PhotoStuff already had the ability to load images and ontologies, display images, select image areas, annotate pictures and zones with ontology concepts and properties, store the annotations in RDF and query them with SPARQL. We added the possibility to:  modify ontologies on the fly and to store and reload these ontologies;  load several ontologies;  route queries to other peers and declare related peers;

 transform queries through alignments. With regard to the description of Figure 1, PhotoStuff offered only the editor and the query evaluator. It had also a less rigourous way to store the data. This required many changes, in particular about how to name the new concepts or store the new ontologies.

PicSter step by step

We describe progressively how users can take advantage of PicSter in the way that was described at the beginning of the chapter: they first use PicSter for annotating their resources independently of any other peers. The first feature of PicSter is to help user organising their own resources without constraints. Then, users can connect to other peers and benefit from the work of others Users start with an empty PicSter instance in which they can load "pictures" and "ontologies". Both images and ontologies can come from their file system or the web. Under the hood, PicSter copies them in the images and ontologies directories, but it always maintain the initial URI identifying each of these objects.

Figure 2: Main PicSter edition interface. On the top panel, pictures can be selected; on the left panel, a selected picture is displayed and subareas can be selected; on the right panel, annotations, ontology and instances can be displayed and selected. The bottom panel is used for expressing queries.

In order to annotate pictures, users can either select several pictures, one picture or a particular area in a picture. Then from the ontology panel, they can select a class for annotating the selected item with this class. For instance, in Figure 3, the user as selected the "Rose" class in ontology "World". Instead of choosing classes, users can choose instances, i.e., individual objects. For instance, on Figure 2, the image has been annotated with the instance "GrenobleCoursliberation63Arriere". It may be that the ontology is not precise enough, in which case, the user can, on the fly, extend it. PicSter provides some basic ontology edition functions which are useful for quick personalisation of the picture annotation process. In particular, it allows users to extend the ontologies they use by:

 Creating new classes: The new class can be specified as a subclass of any class among the loaded ontologies. Labels and comments can be associated with it (see Figure 5).  Creating new properties: PicSter allows users to create new properties for any of the existing or newly created ontology classes. Users can specify the range and domain of the property and hence can create both OWL Datatype and OWL Object properties. The property can also be specified as a subproperty of any property among the loaded ontologies.  Creating new instances: of course, it is possible to create new individuals and to fill their properties, relating them to other individuals. This basic ontology editing functionality ensures that users of the system do not have to stick to a base ontology for annotating their pictures. As shown in Figure 4, users can create directly from the annotation interface new instances, new subclasses and new properties if these are necessary. When selecting these items, in Figure 4, the "Create Subclass", users are presented with a straightforward interface for providing minimal information about the new items. Here, the user knows that the depicted roses are from a particular variety, then they can create this new variety in the ontology as "PinkFlamingo12". This change will, of course, not affect the existing ontology on the web but it will be stored in the local ontology extension of the user directory. These simple changes enabled in the annotation interface free users from having to design elaborate and extensive ontologies before annotating. However, since the ontologies are stored in genuine OWL, they can be edited with any ontology editor before or after the annotation phase. At the end of the process, the user has set up the elements depicted in Figure 6. Once pictures are properly annotated, users can retrieve them through either full text search in annotations, concept search in annotations (directly from the bottom of the interface, see Figure 4 ) or through SPARQL querying (see Figure 7).

It is possible to evaluate simple queries like "what are the URIs of media elements representing something called Grenoble which contain a subarea which depicts a flower": PREFIX j.0: <http://www.mindswap.org/~glapizco/technical.owl#> PREFIX j.1: <http://exmo.inrialpes.fr/people/euzenat/JeromE_world.owl#> PREFIX j.2: <http://exmo.inrialpes.fr/people/euzenat/JeromE_space.owl#> SELECT ?uri WHERE { ?x j.0:depicts ?y. ?y rdf:type j.1:Flower. ?x j.0:subAreaOf ?uri. ?uri j.2:isIn ?z. ?z j.2:name "Grenoble". } This query returns the results provided in the right panel of Figure 7. The answer is provided in terms of instances and in terms of pictures. Thanks to the available ontologies it will also retrieve pictures which are not even annotated by "Flower" but by "Pink flamingo12".

Query evaluation is implemented by using the ARQ SPARQL processor for the Jena API.

Resource sharing

PicSter enables more than semantically annotating resources. It offers sharing resources and annotations among peers. For that purpose, peers are connected through the Java JXTA Protocol [START_REF] Oaks | JXTA in a nutshell[END_REF]. PicSter, through JXTA support, performs the tasks which are related to peer-topeer communication. The design principles for this component are the ones which are typical of peer-to-peer systems, namely:

 A peer announces its presence to other peers when joining the network;  Peer advertisements are sent and received in an asynchronous manner;  All peers maintain a local cache of known peers, so that they can send queries to known peers even if they have not received an advertisement for these peers. This peer-to-peer interface allows to declare some of the peers as friends. The support is visible in Figure 1 through the "Directory" component. It is dealt with, in each peer, through a "router" whose purpose is to route queries and answers from one peer to another.

Once a peer is connected to the network, it can evaluate its queries locally or against other peers. The interface provides the ability to send queries to all other peers or to select a subset of these (see Figure 7, left). A peer also enables other peers to send it queries to evaluate. However, more is needed in order to evaluate queries. Indeed, since users are allowed to choose the ontology they use and to adapt them to their needs, they will end up having different ontologies. Hence, simply evaluating queries in others peers' context is deemed to fail. For solving this problem, PicSter uses mediators (see Figure 1) which transform queries with regard to a source ontology into queries with regard to a target ontology. Mediators themselves take advantage of alignments between the ontologies. Alignments are expressed using the Alignment API [START_REF] Euzenat | An API for ontology alignment[END_REF]): an alignment is a set of correspondences between entities found in the ontologies. In our context, these entities could be classes, properties or individuals. For instance, the property "isIn" in one ontology may be more general than the property "city" in another one.

Query mediators translate queries. For this purpose, they take an ontology alignment and a query as input and generate the rewritten query with regard to the alignment. The current version of the mediator performs translations on the basis of equivalent correspondences given by the input alignment. The URI references within a query are replaced by their corresponding equivalent URI as mentioned in the input alignment. A further improvement would be to allow translations based on more elaborate correspondences. The mediator is implemented for queries written in the SPARQL (Prud'hommeaux and Seaborne, 2008) query language.

For instance, the following query returns the URI of all the resources which depicts some instance of "Fleur" in ontology Arun_sumo and which is situated in a "city" named "Grenoble".

CASE DESCRIPTION: Obtaining alignments in peer-to-peer networks

So far, we have described how annotating pictures by ontology elements and altering ontologies themselves is made easy for end-users. This feature is important for the comfort of users but it raises the important problem of heterogeneity. Hence, one important problem of a semantic peer-to-peer system such as PicSter is to obtain alignments so that queries can be transmitted from one peer to another. In order to solve this problem, users must obtain alignments between their ontologies. There are different ways to do this:

 manually: users can provide themselves the alignments by comparing the relevant ontologies;  by automatic matching: users, or the system itself, can rely on matching algorithms in order to establish alignments [START_REF] Euzenat | Ontology matching[END_REF];  by invert and composition: if enough alignments are available across ontologies, it is possible to combine them through algebraic operations (converse, composition, union, intersection) in order to obtain an alignment between the exact two ontologies that we have;  by retrieving them from an alignment store: if high-quality alignments are available between ontologies, it is then possible to retrieve alignments from these stores;  by local and aggregative interaction: it is possible to ask users for partial alignments which is sufficient for evaluating the query, then, as more queries are issued, the alignment between two ontologies will grow and eventually provide a full alignment.

It is possible to classify these techniques with regard to their expected quality and the effort required from the users, like in Table 1.

Expected quality Effort on the user

Manual high high

Automatic low low

Algebraic medium (depends in input) low

Store high (depends on input) low

Local medium (correct but incomplete) medium Table 1: Solutions for obtaining alignments classified according to their quality and difficulty. Since, they depend on existing alignments, the algebraic and storage methods need a critical mass of alignments to be usable.

In order to evaluate if PicSter is a viable system and if Table 1 is an accurate picture of reality, we ran an experiment that we report here.

Experimental setting

In PicSter, autonomous semantic peers share resources by relying on alignments. In 2006, we ran an experiment in order to investigate the practicality of this framework. There were several goals of this experiment in relation to the work of our team:

 Test and evaluate the PicSter application and infrastructure;  Provide experimental data for knowledge-based social network analysis;  Provide experimental data for alignment algorithm evaluation;  Provide experimental data on which to test alignment composition. We have experimented PicSter on a set of pictures and users free to choose and modify the ontologies they use. Users where asked to provide an alignment with one of the other users. These alignments were used to generate new alignments through inverse and composition, and to evaluate simple automatic matching systems against the given alignments.

The experiment was made among 7 researchers of the Exmo team (named here: P, E, L, A, Z, J, S) during January and February 2006. They had a set of 133 pictures taken by themselves on the topic of Grenoble and the INRIA centre to annotate. A common subset 47 randomly selected pictures was compulsory to annotate in order to ensure a common coverage of the data set and relations in annotations.

Participants were totally free to choose the way to annotate the pictures, they were only required to use the PicSter interface to do this work. In particular, they could use existing ontologies, design their own or extend existing ones.

In the end of the experiment, this provided us with a set of ontologies per participants, a set of annotated images per participants, and a set of annotation per participants. Participants were also asked to evaluate the time they spent in annotating and to name the other participants to whom they feel to be the closest ontology-wise. This is summarised in Table 2 2: Data from the participants in the experiment: #pic is the number of annotated pictures, #onto, the number of used ontologies, #inst, the number of instances that have been created, #class, the number of classes in the ontologies and their local extensions, #prop, the number of properties, and time, the approximate time spent for this annotation work. The closest column corresponds to the other participant, they felt their annotations would be the closest.

Participants took a relatively long time to provide the annotations. This was mostly due to the instability of the prototype.

What is surprising to us is that very few of the users (among the ontology-savvy) created their own ontologies. This is a good news for the semantic web showing that people are willing to reuse the existing resources if they are available. This can also be explained by the short time schedule or by the will of using the sharing philosophy of the web. Because this was possible in the interface, users used Swoogle [START_REF] Ding | Swoogle: a search and metadata engine for the semantic web[END_REF] for finding ontologies related to the annotation they had in mind and extended them.

The second remark is that users have used upper-level ontologies (SUMO, AKT) rather than more specialised ontologies (SPACE, FOAF). This resulted in very large ontologies to be used. After this step, each participant had been asked to evaluate and provide one alignment between his ontologies and those of another participant. In that step, we wanted to limit the burden put on users for providing alignments and yet preserve the possibility to compose alignments. The participants were organised in a circle (P → J → S → A → E → L → Z → P) in order to allow some manipulations related to reciprocity -convers -and transitivity -composition. No interface was provided to perform this task. The resulting alignments were used for two distinct purposes:

Alignment

 As reference alignments to compare with (simple) automatic matching techniques;  As seeds for obtaining other alignments through converse and composition. Additionally, we used the assessment by participants to evaluate the usability of the system and to compare their self-assessed affinity with distances on their ontologies.

Analysis

We analyse the results of this experiments with respect to several criteria. The usability of the PicSter system itself was considered fair by users. It was only a first prototype. Let consider the results concerning alignments and their computation.

Manual provision of alignments

The size of the resulting alignments is provided in Table 3. It shows a relatively low ratio of correspondences with regard to the size of the ontologies (especially if one compares with the number of potential one-to-one correspondences expressed in the third column). This easily confirms what was expected: users are not particularly willing to align ontologies by hand. This applies even if they have chosen themselves the ontologies. As a qualitative remark we had a lot of difficulties to obtain these alignments: while people have annotated pictures and altered ontologies within a few weeks, they spent several months providing the alignments.

As a result, there has been several workaround used by participants: some of them did not do the task, some of them did use very simple matchers (string equality), some of them did the task only partially. In particular, there were several people using SUMO which contains more than one thousand concepts. We could imagine that it is relatively easy to obtain an alignment from SUMO with itself and adding the few missing correspondences by hand. A simple program could be devised for doing most of this automatically. Unfortunately, these SUMO users found the task too overwhelming and returned very small alignments. This shows that, according to our expectations, hand-made alignments indeed require a lot of effort, but, contrary to our expectations, the results were not of a high quality.

Evaluating simple matching algorithms

We wanted to compare the results of simple matching algorithms on real world data in order to have an idea of which such algorithms were the most useful [START_REF] Mbanefo | Comparaison expérimentale d'algorithmes d'alignement d'ontologies[END_REF]. We designed and run a script that: (a) computed alignments between all pairs of ontologies, and (b) computed the alignments resulting from applying threshold to these alignments. This resulted in 588 alignments. Some of these alignments could be compared, through classical precision and recall to the reference alignments provided by users. We also performed a consensus analysis in which we took the best threshold for each method, we did the intersection of the resulting alignments and we compared them to the other results.

Unfortunately, the bad quality of our reference alignments prevented to obtain good evaluation of the quality of the alignment obtained automatically. When comparing the simple matchers with the result of the participants who had used the simplest automatic matchers, then the algorithms always had 100% recall and around a 10% precision. Worse, for those participants who had used SUMO, the precision of the automatic matchers was very bad (0%), while the results were indeed correct. This was a case where automatically computed alignments were better than those provided by human subjects, but our protocol was unable to find it.

In spite of these bad conditions, we can note that, for those users who did the matching work properly, we had significantly better results with our simple matchers that for others (yet around 30% precision and recall).

Evaluating composition and inverse

The third method that we had planned for obtaining alignments was to start from existing alignments and to obtain other alignments by algebraic operations. We wanted to check the quality of the results. Since participants were randomly organised in a circular manner, each alignment (x → y) can be computed by (a) inverting all alignments provided by participants, and (b) composing these inverses in order to go from x to y. For instance, the alignment from P to E can be obtained by (Z→P) -1 . (L→Z) -1 .(E→L) -1 or through (P→J).(J→S).(S→A).(A→E). Figure 8 shows how to start computing inverse and composition from the set of alignments provided by participants. Once this was achieved, it would have been possible to evaluate the results with simple measures:

 by comparing the result to the given alignment;  by having users evaluating the result;  by evaluating the amount of lost information.

We generated the alignments corresponding to the inversion method, unfortunately, our inverse operator was not working properly. This, conjugated with the poor alignments provided by participants, prevented us to exploit results. The algebraic operations are so highly dependent on the quality of initial alignments that we did not even perform consensus tests like comparing the results that could be obtained by the two paths above.

Conclusion

In conclusion, computer experiment is a very delicate task. It requires many people available and this is not easy for complex experiments like this one. The experiment was interesting but non conclusive due to a number of problems:  PicSter was not the most robust and easy going system in the world. This frustrated participants.  There was a bug in the invert function of the Alignment API which reduced our confidence in the results.  Providing alignments by hand is a very tedious task, thus some participants (with large ontologies) did not perform it with the required quality. Hence, results were not conclusive.  Some of the instructions were not clear enough. Most of these problems come from the data collection and not their exploitation, hence this kind of experiment could be run again with the above problems fixed.

A specific kind of evaluation that could have been performed is introduced in [START_REF] Isaac | Putting Ontology Alignment in Context: Usage Scenarios, Deployment and Evaluation in a Library Case[END_REF]: instead of evaluating alignments with regard to reference alignments, it consists of using the alignments for answering queries and asking users to evaluate the answers or having reference answers. More precisely, each participant would have to ask identified queries expressed in their ontologies and to identify the answers to these queries from the annotated pictures by anyone. Then, precision and recall could be given with regard to the answer returned by transformed queries with regard to the alignments. In the case of PicSter, such an evaluation would have been relatively natural since the only purpose of the alignments is querying. It would have had the advantage of showing to the users how bad their alignments were and enabled the evaluation of automatic alignments: automatic alignments would have provided better results than partial alignments. This would certainly have required more annotated pictures, but it would evaluate both the alignments, the ontologies and the annotations of all the peers. However, the main problem is the need for good quality alignments.

CURRENT CHALLENGES FACING THE ORGANISATION

PicSter is an experimental semantic peer-to-peer system. Beside its prototype status, it is functional but not really stable. Its characteristics are: Semantic by using ontologies (as opposed to tags or keywords) for annotating and querying; Serendipity by extending ontologies through annotating pictures, non obtrusively; Sharing by connecting peers in a natural and personal way.

The peer-to-peer aspect of PicSter is highly relying on alignments for reducing heterogeneity. We have attempted to assess the possible ways to obtain alignments in systems like PicSter.

The main conclusion of this experiment is that the task of providing good alignments manually is very difficult for users, even if the ontologies have been chosen by them. Hence, other ways should be investigated.

In particular, it is critical for this application and, we guess, for many others to have automatic matching systems delivering alignments of reasonable quality. These alignments could, of course, be reviewed before being used, either by users when they issue queries or by curators before entering alignments in a store.

Moreover, composing alignments in such networks may be useful when one does not have a direct alignment of his or her ontology to those of another peer. On the one hand this is supposed to reduce the effort; on the other hand, composition degrades the results. It would be worth finding the paths between peers and compose the alignments along those paths which minimise the loss. It would also be useful to really measure how much is lost and to investigate the trade-offs involved in composing alignments. In any case, algebraic composition can only be achieved if there are sufficient high quality alignments it cannot rely on those provided on the spot by users.

Since good quality alignments are difficult to obtain, efforts should be devoted to maintain them. In particular, when one of the aligned ontologies changes, the alignment should not be rebuilt from scratch but the alignments should be evolved with regard to the change. This can be achieved with the help of composition of the alignment with a unique alignment between the new and the old version of the ontology. Alignment evolution should be integrated in an alignment management perspective [START_REF] Euzenat | Ontology alignments: an ontology management perspective[END_REF].

We planned from the beginning to store alignments in an Alignment server [START_REF] Euzenat | Ontology alignments: an ontology management perspective[END_REF]. However, at the time PicSter was developed the Alignment server was not ready so we stored the alignments within the peers themselves. There was one alignment covering all ontologies used by a pair of peers which was used for transforming queries between these two peers. An Alignment server can either store alignments or compute them on the fly with embedded matching methods. The advantages of using an Alignment server is that alignments may be shared by several peers (at least concerning the non altered parts of ontologies) and even shared with other applications of the semantic web. They can be retrieved from the server and modified with regard to the changes a peer has made to these ontologies.

Last, but not least a new trend recently emerged for facing the difficulty of obtaining high quality alignments. It consists of obtaining partial alignments from users while they accomplish their tasks [START_REF] Conroy | Towards semantic mapping for casual web users[END_REF]. In our case, the task may be either modifying the ontology or, more likely, querying. Instead of asking users for an alignment beforehand, the idea is to ask users for their help. When a query is issued, the system can ask the user for a partial alignment regarding the query, i.e., the terms in the query or the query as a whole. This can be done relatively easily by displaying the query and the target ontologies. Users who want to see their queries answered should show more willingness to help the system in this specific way. This local alignment could be recorded as a special type of alignment that can be reused when these alignments are necessary. The peer-to-peer system may collect many of these alignment fragments and the aggregation of many such partial alignments provided by many users over time will eventually provide high quality alignments between ontologies. Even partial, such alignments may be sufficient for answering a large proportion of the queries. Hence, local query alignment is a cheap way to serendipitously obtain alignments from users: it seems to be a promising way to obtain the necessary alignments in semantic peer-to-peer systems.

The topic of finding alignments for heterogeneous semantic peer-to-peer systems is not closed yet. The problem of bootstrapping the semantic web with formal knowledge has resulted in that of bootstrapping with alignments. Once such alignments are available, we think that querying in semantic peer-to-peer systems will become easier. Like we promoted simple user interaction for stimulating ontology edition by users, the same could apply to alignments with local alignments: a simple interface for providing local and task-related clues to the system could go a long way.

The code of PicSter can be obtained from ftp://ftp.inrialpes.fr/pub/exmo/software/picster/PicSter-1.2.1m.zip However, PicSter is an unsupported research prototype and is not suitable for use. The test results can be obtained from ftp://ftp.inrialpes.fr/pub/exmo/tmp/picster-tests.zip.

Figure 1 :

 1 Figure 1: Architecture of a PicSter peer and relations to other elements. Simple arrows correspond to invocation and data flow. Double arrows correspond to query/answer interface. Aserv corresponds to the Alignment server, providing alignments to the mediator. It is in dashed lines because it is not available in PicSter's current implementation.

Figure 3 :

 3 Figure 3: Ontology representation and concept selection.

Figure 4 :

 4 Figure 4: Addition on the fly of an instance, a subclass or a property to an existing ontology.

Figure 5 :

 5 Figure 5: Required information for class creation.

Figure 6 :

 6 Figure 6: Resulting links between elements.

Figure 7 :

 7 Figure 7: On the left hand side, the query panel of PicSter 1.2.1. It allows to select if the query will be evaluated only locally or sent to related peers. On the right hand side, the results of query evaluation. Here these results can be either instances (I) or pictures (M).

Figure 8 :

 8 Figure 8: The set of participants and input alignments between them (plain arrows). In dotted arrows are examples of alignments obtained by composition (.) and inverse ( -1 ).

  .

	Participant #pic #onto	#inst #class	#prop	time closest
	S	47 3(SUMO,FOAF,SPACE)	66 158	242
	Z	47 5(AKT,office,SPACE+2)	87 141	45	18h E
	A	37 2(earthrealm,Travel)	81 738	38	11h E+S
	E	49 6(SPACE,FOAF+4)	78 390	30	35h Z+L
	J	47 1(SUMO)	49 1468	208	6h	S+L
	P	30 1(SUMO)	27 1449	208	5h	L+Z
	L	25 2(SUMO)	24 1395	416
	Table

Table 3 :

 3 Characteristics of alignment returned by participants.

		#Correspondences	#Potential correspondences
	P→J	14	2.127.132
	J→S	17	231.944
	S→A	50	116.604
	A→E	75	287.820
	E→L		544.050
	L→Z	19	196.695
	Z→P	138	204.309

Acknowledgements

We thank the PhotoStuff, Jena and JXTA teams for developing and making available the software that we have extended. We thank Faisal Alkhateeb, Jason Jung, Sébastien Laborie, Jérôme Pierson and Antoine Zimmermann for their participation in this project.

PREFIX j.0: <http://www.mindswap.org/~glapizco/technical.owl#> PREFIX j.1: <http://exmo.inrialpes.fr/people/sharma/Arun_sumo.owl#> PREFIX j.2: <http://daml.umbc.edu/ontologies/space-basic#> SELECT ?uri WHERE { ?x j.0:depicts ?y. ?y rdf:type j.1:Fleur. ?x j.0:subAreaOf ?uri. ?uri j.2:city ?z. ?z j.1:id "Grenoble". }

In this query, the ontology j.0 is the internal ontology of PicSter providing the notions of depiction and areas. The subAreaOf relation is reflexive (so that a picture is a subarea of itself). The two other ontologies are the local extensions of respectively the SUMO and DAML space ontologies.

If the target peer also uses two ontologies, JeromE_world.owl and JeromE_space.owl, then four ontologies are involved apart from the PicSter ontology itself. If the peer finds an alignment between the ontologies providing the correspondences stating:

Arun_sumo.owl#Fleur = JeromE_world.owl#Flower Arun_sumo.owl#id = JeromE_space.owl#name space-basic#city < JeromE_space.owl#isIn

The first correspondence relates two concepts and the latter relates properties. It is noteworthy that the ontologies do not coincide one by one.

The mediator rewrites the input query according to these correspondences. This will yield the query presented in the previous section:

PREFIX j.0: <http://www.mindswap.org/~glapizco/technical.owl#> PREFIX j.1: <http://exmo.inrialpes.fr/people/euzenat/JeromE_world.owl#> PREFIX j.2: <http://exmo.inrialpes.fr/people/euzenat/JeromE_space.owl#> SELECT ?uri WHERE { ?x j.0:depicts ?y. ?y rdf:type j.1:Flower. ?x j.0:subAreaOf ?uri. ?uri j.2:isIn ?z. ?z j.2:name "Grenoble". }

The rewritten query is able to find pictures which are not annotated by "Fleur" but by "Flower". This process works with SPARQL queries but it also works with more simple queries (such as those using only one concept name "Fleur").