Roland Airiau

Loïc Petit

Julien Rouland

Stéphane Séyvoz

Maxime Louvel

Radu Kopetz

Xavier Roubaud

Rémi Melisson

Rémi Drhuile

Jacques Pulou

Mathieu Anne

Anna André

Chris Ben

Simon Roula

Imad Joe Hassan

Bassam, Bernard, Charbel, Christelle, Cyntia Elie Assaf

Ubiquitous systems imagined by Mark Weiser are emerging thanks to the development of embedded systems and plug-n-play protocols like the Universal Plug aNd Play (UPnP), the Intelligent Grouping and Resource Sharing (IGRS), the Device Profile for Web Services (DPWS) and Apple Bonjour. Such protocols follow the service oriented architecture (SOA) paradigm and allow an automatic device and service discovery in a home network.

Once devices are connected to the local network, applications deployed for example on a smart phone, a PC or a home gateway, discover the plug-n-play devices and act as control points. The aim of such applications is to orchestrate the interactions between the devices such as lights, TVs and printers, and their corresponding hosted services to accomplish a specific human daily task like printing a document or dimming a light.

Devices supporting a plug-n-play protocol announce their hosted services each in its own description format and data content. Even similar devices supporting the same services represent their capabilities in a different representation format and content. Such heterogeneity along with the protocols layers diversity, prevent applications to use any available equivalent device on the network to accomplish a specific task. For instance, a UPnP printing application cannot interacts with an available DPWS printer on the network to print a document.

Designing applications to support multiple protocols is time consuming since developers must implement the interaction with each device profile and its own data description. Additionally, the deployed application must use multiple protocols stacks to interact with the device. More over, application vendors and telecoms operators need to orchestrate devices through a common application layer, independently from the protocol layers and the device description.

To accomplish interoperability between plug-n-play devices and applications, we propose a generic approach which consists in automatically generating proxies based on an ontology alignment. The alignment contains the correspondences between two equivalent devices descriptions. Such correspondences actually represent the proxy behavior which is used to provide interoperability between an application and a plug and play device. For instance, the generated proxy will announce itself on the network as a UPnP standard printer and will control the DPWS printer. Consequently, the UPnP printing application will interact transparently with the generated proxy which adapts and transfers the invocations to the real DPWS printer.

We implemented a prototype as a proof of concept that we evaluated on several real UPnP and DPWS equivalent devices.

First, I would like to thank the jury members, especially Stéphane Frénot and Michel Riveill for evaluating my dissertation and for their interesting feedbacks. I also thank Patrick Reignier for accepting to chair the defense session, and Lukasz Szòstek for examining my work.

Introduction

Smart homes are becoming a reality with the development of embedded systems, service oriented architecture and "Plug and Play" protocols. These new habitats, where computer systems are anywhere hidden in the environment, fit into the well known ubiquitous systems imagined by Mark Weiser in [ Weiser 91].

Devices supporting specific "Plug and Play" networking protocols are capable of announcing their type and capabilities in the home network. Consequently, smart applications and other devices are able to discover and cooperate with such devices to accomplish specific tasks. Such applications can be deployed on smart phones and PCs enabling the interaction between devices (lights, TV, printer) and their correspondent hosted services (Switch Light, Control TV, Print). For example, a Photo-Share application automatically detects an IP digital camera device and on the user command, photos are rendered on the TV and those selected can be printed out on the living room printer. The applications and devices configuration is completely transparent to the user who deploys the application on his home gateway, for example. Furthermore, the actual device plethora allows a proliferation in the applications development to provide a wide set of services like the multimedia user experience, energy saving, monitoring, maintenance or even surveillance and intrusion detection. Such applications create a new market perspectives which can be shared between three major actors: telecoms operators providing Internet and telephony access through already deployed settop-boxes and home gateways, manufacturers providing plug and play devices and third parties developing the applications and offering services based on the user home devices through the telecoms infrastructure.

However, the digital home is more than ever a heterogeneous and complex environment. Indeed, devices differ in terms of resources, communication mediums and networking protocols. Thus, such heterogeneity prevents the cooperation and interaction between the devices and applications to fulfill the user requirements and tasks. Thus, we focus in this work our effort to provide an interoperability between devices and applications supporting different protocols. We depict in the next section, the heterogeneity problem retaining the interaction between the smart applications and the "Plug and Play" devices.

Problem Statement

Devices supporting a "Plug and Play" protocol publish their capabilities and type on the network. More precisely, a device is constituted of three entities. The first is the protocols stacks used to communicate and interact with other devices and applications. The second entity consists in the provided services which represent the devices' capabilities. And finally, the third entity is the device and service description which are announced on the network. Based on the announcements, the applications deployed in the home network discover the provided services and interact with devices and applications to accomplish a specific task, such as dimming the CHAPTER 1. INTRODUCTION light intensity or printing a document.

Currently, the following four plug and play protocols are widespread: UPnP, DPWS, IGRS and Apple/Bonjour. Such protocols cohabit in home networks and share a lot of common features. They all announce their descriptions and can be remotely controlled by other devices and applications on the network. They also target similar device types. Multimedia devices are shared between UPnP, Bonjour and IGRS while the printing domain (printing, scanning) is dominated by UPnP, Bonjour and DPWS. More over, each protocol defines standard profiles with required and optional implementations that manufacturers need to support.

Even though those protocols have a lot in common, applications and devices can not cooperate due to the three following differences:

• Protocols Stacks: each protocol defines its own protocol stacks which is based on several protocols to announce the device description and to support remote interaction on the network. Thus, applications not supporting the same protocol stacks cannot discover the devices announcements and interact with.

• Description Format: each plug and play protocol proposes its own device annunciation and description format. Thus, the applications must be able to interpret the announced description on the network in order to discover the device type and capabilities.

• Description Content: each protocol provides a standard content description per device type. The content description includes the device type name, the required supported services names and versions. The standard content description also defines the names of the supported operations on the device along with their input/output parameters names, types and ranges. Thus, two standard equivalent devices such as the printers, supporting different plug and play protocols announce a syntactically different but semantically equivalent content. For instance, a UPnP light hosts a SwitchPower service with a Switch(true/false) method to control the light while a DPWS light[ SOA4D b] uses the semantically equivalent method SetTarget (On/OFF).

Thus the previously outlined three levels of heterogeneity: the description content, format and the protocols layers diversity, prevent applications to use any available equivalent device on the network to accomplish a specific task, such as printing a document or rendering a song. Designing applications to support multiple protocols is time consuming since developers must implement the interaction with each device profile and its own data description. Additionally, the deployed application must use multiple protocols stacks to interact with the equivalent devices. We believe that home applications should be set free from such heterogeneity and should be able to interact with any device capable of fulfilling a desired task. More over, application vendors and telecoms operators need to manage and orchestrate devices through a common application layer independently from the protocols layers and the devices descriptions heterogeneity.

Contributions

To tackle the devices heterogeneity and to accomplish interoperability between plug-n-play devices and applications, we propose to represent non-UPnP devices as standard UPnP devices by automatically generating proxies. Each proxy announces itself as a UPnP standard device on the network and controls an equivalent non-UPnP device. Thus, UPnP applications can interact with non-UPnP devices transparently through the UPnP generated proxy. The use of the UPnP profile description and stack as a common pivot is due to its wide acceptance among device manufacturers and vendors. However, another protocol pivot and profile can be chosen instead, since our approach is generic and is independent from a specific protocol.

The contributions of this thesis focus on providing to the applications targeting specific devices, with specific protocols, the ability to interact with any equivalent device on the network. In other words, our approach provides semantic and behavior interoperability between two equivalent devices based on the intersection of two major domains. The ontology matching crossed with the model driven engineering domain. The main contributions of this thesis are three fold. The first two contributions propose a scientific solution to the heterogeneity problem. The last fold is a technical contribution which proves the realization of the first and second contributions.

• An End To End Adaptation Solution: To allow a cross protocol interaction between UPnP applications and non-UPnP devices. We propose in this thesis an end to end novel solution to provide Plug and Play interoperability. The solution, consists of six major steps [El Kaed 10,El Kaed 11a], as shown in Figure 1.1. The first step, the "Ontology Generation", resolves the description format heterogeneity by automatically generating ontologies representing the devices descriptions in a uniform ontology format.

The ontology expresses the devices capabilities independently from the technical details and the specific protocol description format. The second step, the "Ontology Alignment", attempts to resolve the content description heterogeneity.

Ontology Generation

It applies on the previously generated ontologies semi-automatically matching techniques. The matching aims to detect correspondences between two equivalent ontologies which represent two devices having the same type. The ontology alignment detects the translation rules to go from one device capabilities to another.

Since the matching techniques are heuristics based, the third step, the "Alignment Validation", requires a human intervention to validate and edit the detected correspondences. Based on the validated alignments, the fourth step, the "Pattern Detection", applies rules to detect services and actions compositions based on their parameters. The rules automatically annotate the ontology when a composition pattern is detected. This step also detects correct compositions and returns to the expert non valid ones. The expert, based on the devices specifications and behavior, tries to adapt the non valid compositions by adding an adaptation behavior.

The fifth step, the "Alignment Adaptation", is an optional step. It depends on the previously detected correspondences and compositions. This step allows the expert to add an adaptation behavior using a high level language. The final step, the "Proxy Generation" consists in exploiting the validated correspondences between the two devices descriptions to automatically generate proxies [El Kaed 11b]. The generated proxy acts according to the validated correspondences which contains the adaptation behavior to allow a transparent interoperability between the two devices capabilities. For instance, a UPnP application can now interact with an available DPWS Printer as a UPnP standard Printer. In fact, the generated proxy exposes itself as a UPnP printer and adapts the received invocation from the UPnP application to invoke the DPWS printer.

The protocols stacks heterogeneity is resolved in this final step by the generated proxy which relies on a technical solution. The generated proxy uses a high level API to abstract the technical details of the protocols stacks.

• Matching Technique: The second contribution of this thesis occurs in the second step, the "Ontology Alignment", of the solution. We propose an ontology matching technique inspired from the SMOA [ Stoilos 05] matching approach. Our technique relies on a semantic dictionary which allows to detect synonyms and antonyms relations between two entities from the two ontologies.

• Prototype Validation On Real Devices: In order to prove the feasibility of our proposed solution, we implemented a prototype of three modules. The first module handles the automatic ontology generation from devices annunciations. The second module allows to trigger the ontology matching and to validate the alignments. And finally, the third module automatically generates proxies based on the previously validated ontology alignments. The automated generation, provides an adaptation without any human intervention in the proxy implementation process, which is time consuming and error prone.

We tested our approach on three different devices: lights, clocks and printers. The evaluation proved that our approach is applicable on simple devices such as the lights and the clocks. More importantly, the evaluation shows that our approach is scalable and can be carried out on complex devices like the UPnP and DPWS standard printers. In fact, the two printers are the two most complex standardized devices so far proposed in the UPnP and DPWS forums. Furthermore, today, it seems that we are the first to provide the interoperability between the two standard printers.

Thesis Outline

We expose in this section the organization of this document which details our proposed approach to resolve the devices heterogeneity. This document is organized in three parts.

• The first part outlines the context of our work as follows:

Chapter 2 depicts the ubiquitous computing environment through a scenario, then extracts the ubiquitous computing characteristics and challenges. Based on such challenges, we propose the characteristics of an ideal ubiquitous system which are mainly the following: control, eventing, discovery and interoperability. Then, we overview the actual state of the digital home environment which tends towards a ubiquitous one. In chapter 3, we overview the service layer of a device in a service oriented architecture (SOA) vision. Then, we insist on the SOA characteristics and show how they meet the ubiquitous system characteristics. And finally, the chapter outlines an SOA based framework, "OSGi", along with its architecture.

The SOA supports the characteristics of a ubiquitous system, however, the interoperability between the devices remains unsolved. Therefore, in chapter 4, we first detail the following play protocols: UPnP, DPWS, IGRS, Bonjour along with their underlying stacks. Then, we draw a comparison between such protocols and detail their common and divergence points. Finally, we highlight the three layers retaining a complete interoperability between the plug and play devices: protocols stacks, description format and content heterogeneity. Since, the protocols stacks heterogeneity can be tackled by an OSGi based technical solution, the description format and content heterogeneity remain. Thus, in chapter 5, we point out the knowledge representation in general then, we go through the service and device description languages and formats. We overview three knowledge representation techniques which can be useful to resolve such heterogeneity. Chapter 6 concludes the part and details the problem statement.

• The second part overviews the related work.

THESIS OUTLINE

Chapter 7 first outlines the currently existing solutions providing interoperability between heterogeneous services and devices. Then, the chapter classifies the proposed approaches into three categories. A detailed comparison is presented to expose the advantages and drawbacks of each category. Based on the comparison between the proposed approach, Chapter 8 gives an insight on the advantages of the ontology matching techniques to resolve the description content layer.

• The third part presents our contribution. Chapter 9 details the contribution of this thesis. A first section presents the motivation and the overview of the approach. Then, the automatically ontology generation is exposed. Another section introduces the device matching, the ontology alignment techniques and strategies. Then, we detail the ontology validation performed by an expert. The rule-based pattern detection to automatically annotate the ontology and detect services' methods compositions. Then, we detail the global overview of the architecture with regard to the digital home's and the operator's sites.

And finally, we overview the proxy generation based on the ontology alignments.

Chapter 10 outlines the implementation of our approach and exposes the carried out experimentations on several devices. Chapter 11 evaluates each module of our proposed approach. The first section evaluates the ontology generation while the second section details the device matching and ontology alignment evaluation on three equivalent devices types. The last section reports the evaluation results of the proxy generation module. Chapter 12 concludes the thesis and presents the major perspectives of this work.

Part I

Context

Chapter 2

Ubiquitous Computing

"Ubiquitous computing names the third wave in computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives."

-Mark Weiser In this chapter, we first introduce the context of ubiquitous environments through a well detailed scenario covering various aspects of such an environment. Then, in section 2.2, we extract from the previously detailed scenario, the ubiquitous environment characteristics. From such environment characteristics and features, we point out in section 2.3 the characteristics and challenges that a ubiquitous system needs to handle. In section 2.4, we detail the current state of the digital home and the challenges to be faced. We also provide an overall layer architecture of the applications and devices of the digital home. Then, we discuss the scope of each characteristic with respect to the actual device and application architecture. Finally, we present a conclusion of this chapter and outline the challenges to resolve in our work.

Ubiquitous Scenario

Mark Weiser defines in [Weiser 91] the ubiquitous computing as an environment with inter-connected computers, used by humans unconsciously to accomplish everyday tasks. In such environments, a computer is a machine capable of one or more operations like analyzing, processing, storing and transmitting information. Nowadays and thanks to the development of embedded technologies in the twentieth century, computers are embedded in thin devices having different memory and CPU resources ranging from scare resource devices like sensors measuring ambient temperatures to more abundant resource devices like laptops, set-top-boxes capable of processing high definition videos.

CHAPTER 2. UBIQUITOUS COMPUTING Three essential elements constitutes the ubiquitous computing:

• Devices ranging from tiny sensors measuring temperature, to home appliances machines such as heaters or consumer electronics like digital cameras and televisions. Such devices provide one or more services along with their control commands. For example, a light device hosts a SwitchPower service and the action Switch to turn on or off a light, a printer offers a Printing service along with actions like Print to print a document or to cancel it with CancelPrint action.

• Ubiquitous applications acting as orchestrators by controlling devices to accomplish a specific task. The aim of each application is to hide the devices interaction complexity and permit users to manipulate a large set of devices unconsciously to accomplish their needs. For example, on a ring-door button notification, an application activates the camera of the front door and redirects the video stream to an adequate display interface near the user so he can identify the visitor.

• A network inter-connecting the different devices and transporting the events and notifications to the correspondent application or device. The network is an assembly of different network types ranging from short range communication like Bluetooth [Bluetooth ] and infrared to medium range communication like the Wifi or the Ethernet or even a wider area like the cellular network and the Internet. In the left part of figure 2.1, "Bob" installed a ubiquitous application in his home which orchestrates home devices upon human activities such as the lights, the blinds and the coffee machine. As soon as "Bob" lies on his bed at night, sensors disseminated in his bed notify the home application.Based on his current movements and his previous sleeping behavior [ Helaoui 11], the application is capable of inferring if "Bob" is currently sleeping.

Then, the application ensures that no lights are involuntary forgotten on and turns them off. It also activates the security monitoring system and alarm.

Eventing i s an e s s e n t i a l f e a t u r e i n t h e u b i q u i t o u s environment , d e v i c e s d i s s e m i n a t e d i n t h e e n v i ro n m en t l i k e s e n s o r s n o t i f y a p p l i c a t i o n s i n s t a n t l y when an e v e n t o c c u r s When "Bob" picks up his cup of coffee and heads to take his breakfast, the application fades out the music and turns on the kitchen Television to display the daily news from his preferred channel along with the current weather forecast of Grenoble city where he lives. Once "Bob" finishes his breakfast, the home application requests from the fridge and the kitchen cabinets an update of the food missing. Once received, the application notifies him that there no milk and cereals left for the next breakfast. Thus a grocery list is displayed on the kitchen Television so he can add or/and remove items. "Bob" validates the list and chooses to pick up the grocery on his way back from work, the home application sends the command to the grocery shop.

I n t e r o p e r a b i l i t y between u b i q u i t o u s a p p l i c a t i o n s i s primary t o communicate with a n o t h e r e n v ir o n m en t t o p r o p o s e more s e r v i c e s . The "Home A p p l i c a t i o n " i n t e r a c t s with a w e a t h e r f o r e c a s t a p p l i c a t i o n and r e n d e r s i t on t h e TV. I t a l s o i n t e r a c t s with a " Grocery A p p l i c a t i o n " t o command m i s s i n g goods .

As soon as "Bob" leaves the home with his smart phone and laptop, the application closes the blinds, turns off the lights and re-activates the security monitoring system. When "Bob" is identified by his car application, it opens the door and requests from the home application the last channel visited by "Bob" recently, then activates the car radio and sets the channel. The content of the smart phone and the laptop become available in the car environment. On the way to work, the car applications connects to a global positioning system (GPS)

and guides him through the streets where there is less traffic. The car application is also notified that "Bob" has to pick up the grocery on his way back home.

Once arrived at work, the car application requests from the "work application" available parking spots and guides "Bob" to the nearest spot. Then, the car application notifies the work application that "Bob" has arrived and he is entering the office. Thus, the work application renders on his office display screen (laptop or PC wide screen) his important meeting hours of the day along with the high priority received emails.

Mobility , p e r s o n a l d e v i c e s f o l l o w t h e u s e r from one u b i q u i t o u s e nv i r on m e n t t o a n o t h e r . Bob takes his tablet and starts the meeting application, a list of available display devices in his office is shown. He chooses the wide screen television in his office as the main display screen while Edouard 's chooses the digital wall of the meeting room. Bob starts the meeting by overviewing the headlines of the cooperation terms with the Asian firm. Charles takes the presentation role and proceeds with the meeting, he approaches a white board and would like to draw a schema to represent the product they will be working on. Charles prefers to draw on real papers instead of digital ones. With a gesture, Charles notifies the disseminated cameras to focus on the white board to show Edouard what he is currently drawing. The meeting application is notified, it activates the video projector device. The work applications connect the digital wall and the wide screen television in both sites. They are now capable of rendering the white board drawing. Edouard updates the schema by drawing on his digital wall, the work applications synchronize the content and update it on the display screens, the video projector also displays a projected image update of Edouard 's drawing on the white board. Bob would like to clarify an aspect of the drawing, therefore, he draws directly on his tablet. The work applications update Bob clarification on the display screens. Two hours later, the meeting comes to its end, the work applications display the work calender of each participant along with the common available time for another meeting. The applications then update each participant work calender with the schedule of the next meeting and notifies Bob that he will be in Asia for the next meeting.

Gestures Interpretation i s a new human computer i n t e r f a c e , an a p p l i c a t i o n r e c o r d i n g t h e m e e t i n g d e t e c t s a human s p e c i a l g e s t u r e t o command a d e v i c e o r i n t e g r a t e a p h y s i c a l o b j e c t i n t o t h e d i g i t a l e n vi r o n me n t .

Integrating physical objects i n t o u b i q u i t o u s e n v i r o n m e n t s u s i n g cameras and v i d e o p r o j e c t o r s f o r example a l l o w s t o t r a n s f o r m s t a t i c o b j e c t s l i k e w h i t e b o a r d s i n t o a d i g i t a l s u p p o r t t o s h a r e i n f o r m a t i o n with o t h e r u s e r s .

On his way back home, Bob would like to have a Pizza for dinner, therefore using his smart phone, he sends his order to his preferred Pizzeria. The smart phone notifies the car application that a Pizza order has been issued and the home itinerary need to be adjusted to pick up the Pizza and the grocery on the way back. The car application recalculates the fastest itinerary with the minimum traffic.

Once at home, as shown in the central part of figure 2.1, Bob enters the living room and sits on his chair in front of his wide screen television. Bob browses an online movie repository and chooses one. Upon the movie selection, the home application is notified and since the living room chair already notified Bob presence. the application is aware that Bob is in the living room, therefore the living room TV will be used to display the movie. Thus, the home application sends commands for multiple devices in the living room, the blinds are closed and the lights are dimmed, the Hi-Fi system is activated and is ready to receive the TV audio output.

After a while, the home application receives a notification from the smart phone placed on the entrance hall battery charger that Alice is visiting Bob in an hour and he did not yet acknowledge the notification. The application redirects this important notification to the wide screen television in order to attract his attention along with the remaining time of the movie. Bob acknowledges the notification and continues to watch his movie. An hour later, Alice arrives with her digital camera, she just came back from her safari trip and wants to share her photos of animals and wildlife with Bob. In the right part of figure 2.1, the digital camera identifies itself and announce its supported services and capabilities, thus, the home application notifies Bob that a new device has been detected and asks if it should be integrated to the home network. After its integration, Bob uses his smart tablet to display Alice's photos on the wide screen television and to flick her safari album. While seeing the photos, Bob stores his preferred ones on his living room Network Attached Storage (NAS) device.

He also prints out some of her photos on his living room printer.

Device Discovery and Interaction a r e n e c e s s a r y t o i n t e g r a t e new d e v i c e s i n t o t h e u b i q u i t o u s e n v i ro n m en t . The d i s c o v e r y i n c l u d e s i d e n t i f y i n g t h e d e v i c e t y p e and i t s s u p p o r t e d s e r v i c e s .

Alice also videotaped great scenes of animals that she stored on her living room NAS. To watch the videos, Bob's home application interacts remotely with Alice's home application, and after security checks, the movies can be remotely displayed on Bob's television. After a while, Alice's smart phone notifies her that she has a wine tasting activity in an hour in a near by restaurant. Alice asks Bob to join her. Once arrived, the restaurant application redirects the wine menu to Alice and Bob smart phones. The menu also details information about each wine type such as the year of harvest, the region, the climate and the wine characteristics. Alice and Bob can order the wine directly or ask for a waiter assistant to help them choose.

The next day Bob has to travel to Asia early in the morning for a week to complete the cooperation agreement with the Asian firm in Japan. Therefore, the home application adapts his home alarm to his work schedule. The home application checks his destination and downloads on his smart phone the Japanese to French audio/visual translator along with Tokyo's city map, weather forecast and his hotel reservation. The application noticed a six hours free time between his meetings and his way back flight. Therefore, the application proposes a touristic city tour with the major monuments to visit in Tokyo.

Inference a l l o w s t o p r e d i c t c e r t a i n n e e d s f o r t h e u s e r , such a s p r e d i c t i n g a d i c t i o n a r y o r a map download based on h i s d e s t i n a t i o n t r i p .

Once arrived at the airport, his smart phone notifies the check-in application that Bob has arrived and requests the check-in gate for his flight along with the gate and seat information. The smart phone displays these information and guides Bob in the airport. After his landing in Tokyo's airport, his smart phone traces the itinerary to follow from the airport to the hotel where Bob is staying. He takes the subways and on his way, Bob approaches his smart phone from the advertisement bill board and activates the visual French-Japanese translator which redisplays the advertisement in French. Bob arrives at the hotel, his smart phone connects to the "Hotel" application and notifies that Bob has arrived and checks in. Then the smart phone displays Bob's room number and floor.

During his stay, Bob used his smart phone for multiple purposes, for example, the smart phone displays the Japanese restaurant menus in French and allows to order food without any knowledge in the Japanese language, Bob also used the audio Japanese to French translator application which takes an audio as input and translates it to French to understand the Taxi driver and able to communicate with. During the city site seeing tour, Bob used his smart phone to move around the city and to obtain more information about a monument.

For example, he uses a monument recognition application, Bob only places his smart phone in front of the monument, then the application identifies it and connects to the Japanese ministry of tourism or wikipedia for example to download more information about it. The smart phone also notifies Bob that it is time to head to the airport to take his flight back to Grenoble. Offices, homes, airports, restaurants and even more are an excellent example of an ubiquitous environment where devices are interconnected through a network and orchestrated by applications to accomplish a task. In the following section, we extract from the previously detailed scenario some characteristics of the ubiquitous environment and then detail the arising characteristics of a ubiquitous computing system.

Ubiquitous Environment Characteristics

In the light of the previous detailed scenario, we extract and analyze in the following section the major characteristics of the ubiquitous environments.

Dynamicity

A ubiquitous environment is a highly dynamic one, where devices join and leave the network due to several reasons, we summarize in the following some of such reasons:

User Mobility Personal devices dynamicity is correlated with the end-user mobility. Smart phones, digital cameras and laptops usually follow end-users in space and time from one ubiquitous environment to another.

Applications are expected to handle the service departure proposed by those devices. For example, when a digital camera leaves a ubiquitous environment, the application marks its functionality as unavailable. Such personal devices are also expected to be integrated transparently into the new environment along with a smooth interaction with existing devices in the new environment.

Device Energy

The development of embedded devices as predicted by Moore's Law [Moore 65] where processing capacity doubles approximately every two years allows such devices to handle more tasks and therefore consume more energy. The display screens and wireless cards are also greedy in energy despite advances in battery manufacturing and energy saving techniques. An average energy autonomy of a laptop holds between 3 to 9 hours, therefore, the device availability is dependent on its energy. Meaning the dynamicity is dependent on the device energy.

Device Non Usability Devices are also disconnected when there is no need for its functionalities at some given time, for example, the heating system is turned off during the summer season.

Replacing Devices

The end-user is also expected to renew his devices, i.e. replacing a device with another or adding a new device to his home.

Device Proteanism Devices have become recently multi-functions supporting different roles and functionalities. For instance, a smart phone is used to make phone calls, take pictures and videos, listen to music and even as a remote controller. Therefore, the user might switch off a functionality and activate another one on the same device. The environment will perceive a new capability arriving to the network and another leaving.

The high dynamic nature of ubiquitous environments requires highly dynamic ubiquitous applications able to detect the device arrival and departure. Upon a device arrival, the application need to detect or to be notified of its capabilities to enable and apply the orchestration among different devices and applications.

Heterogeneity

One of the most challenging features in ubiquitous environments is heterogeneity which is present on different levels, we outline some of them in the following: [IGRS ] or the Apple Bonjour [Bonjour ] protocol.

Resources A
However, ubiquitous applications are expected to communicate transparently with devices regardless of the protocols they use.

Software and Hardware platforms

The heterogeneity of resources, communication mediums and protocols, constitutes major factors influencing the hardware device components selection and constitution. The device resources and hardware, influence the software platforms selection. Such software platforms varies from tiny and embedded operating systems deployed on sensors to operating systems and virtual machines deployed on servers and devices with abundant resources. Thus, the hardware and software platforms are also heterogeneous and vary from a device to another. Moreover, ubiquitous applications are defined as software components running on software platforms to accomplish users tasks. Such components cannot be installed on any platform since they depend and rely on the functionalities offered by the specific underlying software and hardware platform.

The diversity of the software platforms makes installing ubiquitous applications on any available device with enough resources and capabilities a difficult task.

Device and Service Description Devices arriving to a ubiquitous environment are expected to identify themselves along with their supported services and capabilities. However, a lot of XML-based description formats are being used to expose the device description and capabilities. For instance, devices supporting the UPnP protocol uses an XML based format while other devices supporting the DPWS protocol uses the WSDL (Web Service Description Language) to expose the service descriptions.

Other than the description format, the syntax content is not the same even for the same device type and functionalities, for instance a UPnP Intel Light uses the "BinaryLight" to identify the device type while a DPWS Light declares its type as "SimpleLight". The services and the actions have also different syntax, the UPnP Light offers a "Switch" service with a "SetTarget" action to turn on or off the light, while the DPWS light uses the action "Switch" to offer a similar functionality.

Ubiquitous Applications deployed in ubiquitous environments are expected to identify devices along with their services and integrate them in the ubiquitous environment so they can be used transparently by users, other devices or applications.

User Interfaces

The classic mouse and keyboard are not the only input interfaces, touch screens and gestures among others have emerged in ubiquitous environments due to their simple usage. Some users also tend to use statical objects like the old white board to interact with other users or with the ubiquitous applications.

Therefore special gestures are used as a human interface to interact with the applications has become very promising [ Sharma 98].

The output display interfaces also became diversified with advances in display screens and video projectors technologies. The display interfaces ranges from flexible paper-thin screens to smart phones and wide TVs and walls. Ubiquitous applications are expected to use any display interface near to the user to deliver information.

For instance, an unacknowledged meeting on the smart phone will be redirected to the main display screen in the room to attract the user attention so he can validate the meeting.

Ubiquitous System Characteristics and Challenges

Different challenges arise from the previously enumerated ubiquitous environment characteristics. A ubiquitous system must handle such challenges. Such system is defined as a centralized or distributed middleware deployed on different devices in one or multiple ubiquitous environments. The middleware executes software components which represent among other service, the ubiquitous applications. An application offers or consumes services provided by other applications or devices. For instance, the grocery shop application offers the ordering items service to other applications. The grocery application also consumes a secure on-line payment service provided by another application. Devices also offer or consume services. For instance, the "Bob"'s Home application interacts with services provided by home devices, like the "Switch" service provided by a light to allow turning on or off the light.

We detail in the following the characteristics and challenges of an ideal ubiquitous computing system.

The Ubiquitous computing vision consists in helping users to accomplish their everyday tasks by easily using devices and technology. To achieve such target, ubiquitous applications must be aware of the environment surrounding the user in order to propose services that fit at best his needs. For instance, an application aware that the user is sleeping will mute his smart phone and turn off all forgotten lights in his home.

The user mobility and the device dynamicity are the main factors leading ubiquitous applications toward context awareness in order to fulfill the users need. To achieve such awareness, applications receive information from different resources: the user, other ubiquitous applications or devices. The user interacts with the environment, he actually activates the ubiquitous applications, replaces devices, triggers notifications and informs applications about his preferences. For example the user sets his favorite news or music channel. Devices like smart phones, lights or motion detectors disseminated in the environment communicate their notifications to the applications for multiple reasons, such as informing about an accomplished task like a printed document or to raise an alarm upon fire or smoke detection or simply to announce its services description and capabilities.

Receiving information from the context and the surrounding environment implies that an event took place, it is then up to the ubiquitous application to decide whether to ignore it or to react. A representation of an ideal ubiquitous system is exposed in Figure 2.3 where different layers are shown. The first layer (L1) contains devices in the environment capable of sending notifications and information to the application. The static objects can also be integrated in the ubiquitous system using devices such as cameras and video projectors. In the previous scenario, Charles's white board is integrated in the system through disseminated digital cameras capable of detecting human gestures. The (L2) context aware layer in Figure 2.3 gathers notifications from various devices and applications then notifies the applications that an event occurred. Then it is up to the application to ignore then event or to treat it. The reaction takes different aspects: an adaptation of the environment, an auto adaptation of the application or both. The adaptation can be handled by the system, the application or both.

Adapting the surrounding context occurs by interacting with devices and other applications. For instance, an application opens the blinds in a room to adapt the environment and restore a predefined level of light intensity.

Another form of adaptation is the auto adaptation which is the ability of an application to extend or remove features depending on the context change. For example on a device discovery, the system handles the device arrival and departure then informs the applications which adapt and propose an extended feature allowing users to accomplish more tasks such as the possibility to print a document on the newly discovered device. The More often, adaptation includes the environment and the self adaptation. The content adaptation is an excellent example of both adaptations. For instance, a displayed text on a smart phone screen will not be the same on a TV or on a giant billboard. A redirected notification or information firstly received on the smart phone will be reshaped and enlarged when redirected on a wide screen display.

Moreover, ubiquitous applications should be able to use and access some sort of a "Contextual Memory" containing previously entered users or applications preferences and actions. Therefore, applications request the user preferences and habits in order to adapt the context. For example, by setting Bob's favorite channel or even infer and predict assumptions based on previously collected information. For example, since no movement action has been detected in the house and since Bob slept at midnight all this month, the application can infer that he is currently sleeping, thus, the application can turn off his house lights.

Additionally, ubiquitous applications must be capable upon user request of integrating static objects such as a paper, white boards or pens into the digital environment. For example, a ubiquitous application recognizes a user gesture in order to render the white boards content on the television? Thus, the static object becomes a part of the digital ubiquitous environment. User gestures represents the peripheral between the digital and the physical world.

We outline next the characteristics and challenges.

Discovery

The device discovery constitutes a serious challenge in the ubiquitous computing environment since devices are highly dynamic. Applications must be aware of the device arrival and departure in order to apply adaptation on the context or on the application itself. Therefore, a device must first join a network and requests an address identifying it on this network. Then, the device must announce its arrival along with its description which includes, for example, its device type, name, manufacturer, serial number, software and hardware version along with its supported services and capabilities. The description annunciation allows applications to uniquely identify a device along with its capabilities and check if it responds to the requirements in order to interact with it to accomplish a given task. The device should also announce its departure on the network.

Moreover, the device physical location in the environment is an essential information. For example, a user would like to view photos stored on his friend's NAS (Network Attached Storage) the one located in the living room. Applications also need to access the device location, a motion detector notifies that a person has just entered the room, based on this information an application activates devices around the user's new location to ensure a continuity of a certain service or to accomplish a given task. For example the home application activates Bob's TV located in kitchen as soon as he enters to display the daily news, or dims the living room lights while watching a movie.

Therefore, we differentiate between the device discovery and the service discovery. The device discovery is the ability to identify a device presence on the network along with its physical location. The service discovery is identifying a device provided services and capabilities. Thus, to be discovered a device should perform the following operations:

• Addressing: A device joining the network is capable of acquiring a valid networking address.

• Annunciation: The device announces its description, including its general information (type, manufacturer, etc) along with its supported services and capabilities.

Control

The control consists in interacting with the device for multiple reasons, such as retrieving its information, requesting tasks.

Discovering a device and its capabilities is not enough for an application to control it. The device must also announce, how ubiquitous applications should interact with its services and supported capabilities. Therefore, a device must also describe how to use a service and its functionalities and the input/output parameters.

For instance, a light device describes its supported service "SwitchPower" along with the supported actions "Switch" to turn on or off the light and the "GetStatus" action to retrieve the light status. The description details also that the action "Switch" takes a boolean variable "true" or "false" as an input and also details that the "GetStatus" returns an output boolean value. Such information will allow ubiquitous applications to discover the device capabilities and functionalities. Thus, it will allow applications to perform device control and interaction.

Eventing

Eventing has a high impact on the ubiquitous application reaction and adaptation. It consists in informing other applications and devices that an event has occurred. Such event can be a device arrival or departure, an accomplished task or parameter change. The eventing allows applications to be context aware.

There is different sorts of eventing, the pulling is when a device sends information regularly, upon change or upon task accomplishment. For instance a temperature sensor sends the ambient temperature value every hour, while an intrusion detection sensor sends an event upon an intrusion detection. Eventing can also be upon request, an application subscribe to receive notifications regularly, upon a value change or upon task accomplishment. In this case the device only notifies the subscribed applications, instead of broadcasting non-interesting information on the network to multiple applications and devices.

Interoperability

One of the most challenging characteristics of the ubiquitous environment is the heterogeneity. Devices communicate with a multitude of communication mediums and protocols. Additionally, devices uses multiple description formats to detail their capabilities, their supported services along with their functionalities.

The interoperability is the ability of an application or a device to interact with different devices or applications supporting different protocols and representation description. The ubiquitous system must be able to integrate any device in the environment and interact with it independently from its underlying supported protocol and description format.

Inference

The inference is the capability of the applications to take a decision of adaptability by correlating previous and present information. Such information can be collected by the system through a period of time, like the sleeping hours or set by the user like the preferred radio station.

Interpretation

The system should also be able capable to interpret human gestures which constitutes a new peripheral. The gestures allows users to easily interact with the system or the ubiquitous applications to accomplish a given task.

For instance, a gesture can be used to raise up the volume, or close the blinds or even integrating a physical object into the ubiquitous digital environment. In the scenario, we pointed out the gesture interface, however, speech, vision, and touch also represent another computer human interfaces which need to be considered. Thus, the information returned from the peripherals supporting such new communication interfaces must be interpreted to interact and adapt to the humans' needs.

Security

Ubiquitous applications interacts with other applications and devices in local and remote ubiquitous environments. During the interaction, different information are exchanged ranging from the user location, to his media contents and private data. Therefore, a secure system and communication channels are a necessity to preserve the user's data and private information.

Additionally, applications can access devices in the environment to retrieve and share information like photos, videos and documents. Even though, devices are shared among multiple users, the access to private data should be restricted. Roles should be attributed to users and applications, for instance a group or a category of users cannot access contents, another category can have only a read access while a third category can benefits from all the privileges.

We detail in the next section the main actors of the digital home, then we identify the characteristics of such a complex realm and the challenges that arise from each characteristic to be treated in our work.

The Digital Home Towards a Ubiquitous Environment

The actual digital home is an excellent ubiquitous environment. Multiple characteristics and challenges from the ubiquitous environments are found in today's digital home. In this section, we detail the actual state of the digital home along with its characteristics and challenges.

The actual home device plethora allows a proliferation in the development of ubiquitous applications providing a wide set of services like multimedia user experience, energy saving, monitoring, maintenance or even surveillance and intrusion detection. Such applications create a new market perspectives which can be shared between three major actors: telecoms operators providing Internet and telephony access through already deployed set-top-boxes and home gateways, manufacturers providing certified plug and play devices and third parties developing ubiquitous applications and offering services based on the user's home devices through the telecoms infrastructure1 .

However, the digital home is an heterogeneous and complex environment for those three actors and the enduser. Devices differ in terms of resources and networking protocols. A wide variety of devices exists in the home and ranges from scare resource devices like sensors measuring ambient temperatures to more abundant resource devices like laptops, set-top-boxes capable of processing high definition videos. Ubiquitous applications are installed on some home devices like STBs and home gateways which share their physical resources. However, telecoms operators and third party application vendors must guarantee a minimum level of QoS (quality of service) when providing their applications. Therefore, the resource sharing between applications has to be taken into account to provide an excellent user experience.

As for the networking heterogeneity, multiple "Plug and Play" protocols have emerged like the Universal Plug and Play (UPnP) [UPnP ], the Intelligent Grouping and Resource Sharing (IGRS) [IGRS ] and the Device Profile for Web Services (DPWS) [OASIS 09a]. Each protocol defines or uses a specific networking layer and a set of standard profiles per device type (printer, light, clock) with required and optional implementation that manufacturers could support. This protocol diversity, prevent applications to use any available equivalent device in the home to accomplish a specific task. For example, a ubiquitous application searching to interact with a UPnP printer cannot use an available DPWS printer. Designing applications to support multiple protocols is time consuming from the developers perspective since they must implement the interaction with each device. For the telecoms operators, such protocol diversity makes the diagnostic and the troubleshooting a more complex operation and adds a burden costs on their infrastructure and tools to target additional protocols. Additionally, the lack of cross-device interoperability, frustrates the users since they will be restrained to a certain protocol and technology if they want a complete interoperation in their home network [ Dixon 10].

Moreover, telecoms operators, device manufactures and third parties application vendors provide a maintenance service along with their proposed applications. Thus, devices and applications of the home network must be monitored to provide sufficient information during diagnostic and troubleshooting operations.

Additionally, with thousands of proposed applications, end-users will find them selves confused when it comes to decide if their devices are compatible with the application they are ready to pay for. Therefore, the proposed applications to each end-user should be based on his devices type and software version.

In a such sophisticated environment, we believe that a middleware is essential for the digital home to simplify the complexity for the actors and the end user. The main characteristics and challenges of the digital home are the following:

Discovery and Adaptation for Dynamicity

The digital home is a dynamic environment where devices join and leave the network due to several reasons: personal devices dynamicity is correlated with the end-user mobility. Smart phones, digital cameras and laptops usually follow end-users in space and time from one ubiquitous environment to another. The dynamicity is also dependent on the device energy (battery down) and the non usability of its functionalities at some given time, for example, the heating system during the summer season. The end-user might also replace a device with another or add a new device to his home.

Multiple challenges arise from this characteristic, the middleware must be able to discover the device appearance then identify it along with its hosted functionalities. The arrival of new devices allows to propose new ubiquitous applications to the end-user or to activate features on some already deployed applications. For example, upon the discovery of a UPnP printer, an application can propose to print contents. The middleware should also detect the device disappearance, ubiquitous applications must be notified of such change and should be able to re-adapt.

Interoperability for Heterogeneous Plug and Play Protocols

UPnP [UPnP ], IGRS [IGRS ] and DPWS [OASIS 09a], cohabit in the home local area network (LAN) and share a lot of common features. Their protocol layers support: discovery, description, control and eventing. Plug and Play protocols follows the service oriented architecture paradigm, devices can be discovered on the LAN since they announce their description (device id, friendly name, manufacturers etc) along with their supported services and capabilities. For example, a printer hosts a service which allows to carry out printing operations.

The description allows applications to discover and identify the device in order to control it. Plug and Play devices also support eventing upon a parameter change or upon accomplished tasks.

Those protocols also target similar device types. Multimedia devices are shared between UPnP and IGRS while the printing domain (printing, scanning) is dominated by UPnP and DPWS. Even though protocols have a lot in common, devices can not cooperate due to two main differences: the protocol layers and the service description. Each Plug-n-Play protocol use its own protocol layers: they all use the SOAP protocol for interaction, UPnP and IGRS use SSDP for discovery and GENA for eventing, while DPWS uses a set of standard web services protocols (WS-*).

Moreover, each protocol defines its own description format, UPnP uses an XML proprietary format while IGRS and DPWS use the Web Service Description Language (WSDL). Plug-n-Play protocols also define the syntax description using standard profiles per device type. For example, a standard UPnP light [UPnP 03] profile hosts a SwitchPower service with a Switch(true/false) action to control the light while a DPWS light [ SOA4D b] profile uses the equivalent action SetTarget (On/OFF). The syntactic heterogeneity along with the protocols layers diversity, prevent applications to use any available equivalent device on the network to accomplish a specific task.

Management for Devices and Applications

The diversity of devices and applications interacting and sharing common resources in the digital home increases the need of remote management operations for the three actors and the end-user. Device manufacturers frequently perform firmware and software updates to improve a functionality or fix a security hole. Telecoms operators deploy at the end-user site customized residential gateways (RGW) and set-top-boxes (STB) to deliver a "Triple Play" service. For example, such service includes multimedia content, Internet and telephony along with a guaranteed QoS (quality of service).

End-users can experience a low quality service at home, for example the user receives a bad quality video when watching a movie. Such bad quality is more often due to an improper configuration of the STB or the TV. Third party application providers need also to carry out management operations to remotely install their proposed applications and to allow a successful orchestration and interaction between home devices. For example a device firmware update is needed to enable the new application. Some of these operations cannot be totally automated since they depend on the LAN configuration (DHCP, DNS-Servers, IPv4/IPv6), the device type and its execution environment (OS or virtual machines), therefore a remote diagnostic and intervention are often needed.

To support remote management operations and minimize the cost of a technician intervention on the enduser site, multiple telecoms operators, device manufacturers and service providers adopted a standard wide area network (WAN) management protocol, see However, it seems that the CWMP is the most adopted protocol so far [Broadband ].

The CWMP scope is clearly the WAN management and targets only equipments supporting the CPE profile and capabilities, while "Plug and Play" protocols support the discovery, control and eventing in LANs. Management protocols adopted by device manufacturers and telecoms operators proved their efficiency to administrate and remotely configure home devices at the end-user side [ UPnP 11], [Broadband 10a]. The management operations include the device and applications diagnostics, and troubleshooting. Those two operations require a historical trace of events taking place in the digital home like the appearance and disappearance of home devices, firmware and new applications deployment and management. Additionally, the network topology and its connectivity are also required to detect unplugged or malfunctioning devices, bandwidth use and other resource consumption to guarantee a QoS for the end-user experience. An historical trace of the digital home

gives the call center valuable information to correlate previous events and the currently occurred failures and simplifies the diagnostic and troubleshooting operations. For example, the new installed application can be conflicted with previously installed ones or even provoke failures and crashes of other applications, or the STB is unable to connect to the TV, the trace can reveal the STB and TV device status, their link connectivity and the last time they were connected.

The diversity of the management and the plug and play protocols forces the three actors: telecoms operators, device manufacturers and application providers to support several tools in order to provide the end-users with the required management and administration operations. Indeed, the three actors must maintain and update their multi-protocol diagnostics platforms and tools in order to provide the end-users with the necessary diagnostic operations.

Other characteristics [ Kurkovsky 07], [ Aipperspach 08], [ Dixon 10] and challenges [ Edwards 01], [ Lyytinen 02],

[ Kindberg 02] has been pointed out in the literature, however, in this work we only focus on the dynamicity, the heterogeneity and the management aspects. We believe that an efficient middleware for the digital home must handle challenges arising from such a complex environment and provides a simple platform for the three actors and the end-users. Third party application providers will be able to offer efficient and well adapted ubiquitous applications to suit end-users needs with an easy installation. Manufacturers remotely managing devices and telecoms operators offering a guaranteed QoS along with diagnostic and management operations for devices and installed applications. And finally, for the end-user a great ubiquitous experience and a simple technology to accomplish everyday tasks.

The Overall Actual Device and Application Architecture

The digital home tends towards a ubiquitous environment, therefore it inherits a lot of characteristics presented in section 2.2. Consequently, a ubiquitous system handling the digital home also inherits the challenges presented in section 2.3. We provide in this section an overall layers architecture of the applications and devices of the digital home. We also rediscuss the characteristics and challenges of a ubiquitous system with regard to the actual devices and applications architecture. Therefore, the interaction between entities goes through a communication medium. The communication layer (L0) is used as a medium to transport information between entities like devices and applications.

L1: Protocols Protocols at the layer (L1) are used on top of the communication medium to exchange information between two or more entities. Such protocols are embedded by applications and devices in order to be used by the provided services or the control points at the L2 layer.

L2: Interaction An entity can either be acting as a control point, as a service provider or both, situated at the L2 layer in Figure 2.5. Control points only use and coordinate capabilities provided by other applications and devices. For instance, the "Home Share Application" acts as a control point and orchestrates Bob's devices in order to allow a media sharing across the home. Devices also act as a control point, for example a TV would only control a media device like a Network Area Storage to retrieve media contents. Devices and applications can also provide one or more services. In section 2.1, Bob's home application interacts with a weather forecast application which provides the current temperature and the weather condition.

Devices also offer services, for example a light device provides a Switch service with two different actions, the SetTarget to remotely control the device (turn on/off) and the GetStatus to retrieve its actual state.

L3: Description

The description layer is where the entities providing services announce a general description about the device information and its supported capabilities. The description involves the actions invocation behavior along with the means of interaction with the provided capabilities. It also exposes the actual device state. Thus, control points interpret the description provided by the applications or devices in order to invoke their capabilities.

The service and device description are essential for discovery and interaction as mentioned before in section 2.3. It allows other devices and applications to identify the device along with its supported services and provided functionalities. Moreover, the description details the input, output parameters and formats for the action invocation and event notifications.

L4: Data Analysis The final layer is on top of all these previous layer. The data collected from the applications and devices are analyzed in the final layer to efficiently predict the user's behavior and preferences.

Discussion Around the Ubiquitous System Characteristics

Figure 2.5 overviews the scope of the ubiquitous system characteristics and challenges as presented in section 2.3.

This section sets the scope of each characteristic with respect to actual device and application architecture.

Discovery The device discovery, as defined previously, is the ability to identify a device along with its capabilities and provided services. The discovery phase is involved in almost all the layers of the device and application architecture, L1, L2, L3. Some might argue that the device discovery should be limited to the protocol layer while others insist that the protocol layer choice is highly dependent on the communication medium L0. Indeed, the protocol is dependent on the communication medium since some of the IP based protocols are not suitable in terms of efficiency on a low range and data transmission communication mediums. Therefore, the discovery characteristic in Figure 2.5 is placed partially in the L0 communication medium layer. The device and application discovery is obviously dependent on the protocol stack used by the entity to announce its arrival or departure on the network. Moreover, the entity identification is dependent on its announced description containing the provided services and supported capabilities. The provided functionalities constitutes a major criteria whether an entity is selected to interact with or not.

Control and Eventing

The control and eventing depend on the discovery. The interaction and the notification between applications and devices take place once the entity is identified along with the supported services and functionalities. These two characteristics depend also on the service description since it details the behavior and interaction mean when an action is invoked or when a notification has been triggered. The description includes the format of input and output parameters along with the notification delay and format.

The scope of these two characteristics ranges from L1 to L3. Obviously, the control and eventing also depend on the underlying protocols which allow to exchange control and eventing messages.

Interoperability The interoperability however is scattered over the first four layers, from L0 to L3. It is involved in the communication mediums. A lot of effort has been carried out in the "Internet of Things" domain in order to connect RFID communicating objects to the Internet through RFID-Internet [ Jain 10] proxies. Such type of proxies bridges the communication between low power communication mediums and the wide web. There is a multitude of use cases pointed out such as monitoring of manufacturered objects or tracing a mail package on the globe through the web. The interoperability is also needed at the protocols layer in order to discover a device and interact with it. For instance an application embedding a UPnP protocol stack cannot be aware of the presence of a DPWS device. The interoperability at the service level is desired to allow applications and devices to interoperate independently from the protocol and communication mediums. The main limitation of the interoperability at the service level is the description of devices, applications and their capabilities. Many representation format and languages has been proposed to describe a service along with its supported capabilities. This multitude of representation makes it difficult for services to interoperate. Moreover, even if the description is expressed in the same description language and format, the syntax is different even if the semantics are the same. For instance a light service offers an action SetTarget to turn on or off the light while another similar light device represents the same action with the name Switch. The two actions are semantically equivalent, however they are syntactically different. An application or a device searching to interact with a device interprets the description syntactically and not semantically therefore the service identification and the interaction will not take place.

Interpretation and Inference Obviously, those two characteristics stand in layer L4 on top of the other layers. The interpretation analyzes (gesture, speech, movement) data provided by devices and applications in order to trigger actions and execute tasks. The inference, however is placed on top of all these layers including for example, the gesture or the speech interpretation, it takes as an input all sort of data, provided by devices, applications and users. The inference makes decisions based on some logic and statistical analyses from previously collected data. For instance, an application will be able to decide if Bob is sleeping based on the actual sensors detector information and an average of his sleeping hour time during the month.

Security the security is clearly transversal to all these layers starting from the communication mediums to the inference and decision making. In all these layers, devices and applications handle private and sensible data transmitted through communication mediums. Therefore the security is a must on all these layers.

In our work we will focus on the following characteristics: discovery, interoperability, control and eventing in the L1, L2 and L3 layers.

Conclusion

In this chapter, we presented the context of our work which is the ubiquitous environment and more specifically the digital home. We extracted in section 2.2 the ubiquitous environment characteristics and outlined in section 2.3 an ideal ubiquitous system characteristics. In section 2.4, we detailed the actual digital home's state and characteristics. Then we rediscussed the characteristics of a digital home system.

In our work we will focus mainly on the following characteristics and challenges already pointed out in section 2.4.

1. The device and service discovery.

2. The interoperability between Plug and Play Protocols.

3. The device and application management.

Obviously, these three challenges pointed out in the digital home environment must be tackled by a ubiquitous system architecture capable of supporting the following features: the discovery, the control, the eventing and the interoperability.

Thus, we present in chapter 3, the L2 service layer in a service oriented architecture (SOA) vision along with the its three entities: the service providers, consumers and the service registry. Then, we mainly show how the SOA characteristics meet the ubiquitous system requirements and features. And finally, we describe a Service Oriented Framework, "OSGi" with its architecture and show how it can be used to fulfill the following ubiquitous system characteristics: discovery, control and eventing. However, only the interoperability remains unsolved by the SOA characteristics.

Therefore, in order to understand the layers retaining a complete interoperability between the plug and play devices, we detail in chapter 4, the following plug and play protocols: UPnP, DPWS, IGRS, Bonjour.

Then, we draw a comparison between such protocols and detail their common and divergence points. Moreover, we enumerate the three layers retaining a complete interoperability between the plug and play devices: the protocols stacks, the description format and content heterogeneity.

Thus, in chapter 5 we point out the knowledge representation in general then we go through the service and device description (L3) languages and formats. Then, we provide three knowledge representation techniques which might be useful to resolve the plug and play description format and content heterogeneity.

Chapter 3

Service Oriented Architecture In this chapter, we detail the Service Oriented Architecture along with its entities and characteristics. Then, we report how the SOA characteristics meet those of the ubiquitous system. We also provide an overview of OSGi, an SOA framework used later in our work.

SOA Principles

The Service Oriented Architecture (SOA) according to the OASIS The visibility allows those who need to consume, and those who offer capabilities to see each others. Each entity expresses the capability or the need through a description containing different aspects, such as the input, output, technical requirements or policies. Such description is specified by domain experts or service designers and programmers for example. The interaction involves information exchange, through messages for example, and an action invocation to accomplish a need through a capability. The effect is the result of the interaction which may result a return of information or a change of state.

Three entities constitute the service oriented architecture as shown in Figure 3.1: the service provider, the service consumer and the service registry. Those entities either provide or consume services. A service can be defined as a container of one or more capabilities. For instance a light device offers a Switch service supporting different actions to turn on/off a light or retrieve the actual light intensity and state. Service providers publish their service description in the service registry. Service consumers search in the registry for a defined service to accomplish a given task. A service consumer can also subscribe to a service, it will be notified when a service provider meeting its needs appears.

Two modes are possible to discover a service, the active discovery is when the provider is available and the request is satisfied instantly. The passive discovery is when the provider is unavailable, however, the request is saved in the registry. As soon as a provider appears satisfying the request, the consumer is notified.

The service registry matches the service description with the service consumer request and returns a reference to the consumer about the found services. The service consumer receiving the service provider reference, binds to it and interacts with its provided actions. The service registry can be represented by real physical centralized or distributed entities. The OASIS proposes the UDDI [ OASIS 04] protocol and registry platform for service registration and search. A service registry can also be distributed in the Local Area Network.

In a distributed registry service in the local area network scope, devices announce their services on the network while other devices and applications search or listen for the device annunciation. Once the service is found, the consumer binds with the service provider and interacts with it. Once the service provider becomes unreachable or disappears, then, the service registry removes the service description and the service provider reference. The service consumers are also notified when a service disappears.

The service description provides information about the service and its functionalities independently from the underlying implementation. It represents an abstraction of the existing invocation mechanisms supported by the service. Each service description is defined by an interface and a signature for each provided action. It details the means of interaction with the provided service along with its actions. It includes the information to be provided to the action along with the returned values and the effect of the interaction. Moreover, the interface and the description information are represented syntactically in a format that can be interpreted by the service consumer [MacKenzie 06].

As mentioned in [MacKenzie 06], there are theoretic limits on the effectiveness of the syntactic description.

There will always be an ambiguity or an unstated assumption used by the service provider when defining the service interface and description. The service interface and other specific properties are used to register and request services in the service registry. The service client must know in advance the interface name of the requested service. Since the request is based on the interface name, a client requesting an interface with a semantically equivalent but syntactically different name will not receive the reference of a similar service. The

SOA CHARACTERISTICS MEETING UBIQUITOUS SYSTEM ONES

limitation of the SOA approach resides in the matching of the requested services and provided ones. Such limitation is due to the syntactic service description which can be ambiguous.

SOA Characteristics meeting Ubiquitous System Ones

The SOA characteristics has been largely discussed in the literature, like in [MacKenzie 06, Bottaro 07b, Tigli 09b]. We overview some of these characteristics and describe how they meet the ubiquitous system ones.

SOA services are loose-coupled since the dependency between services is minimized and only holds to the service interface. Indeed, the SOA services follow the separation of concerns concept between different entities, and the use of functionalities through interfaces. The loose coupling increases the services re-usability. The description provides information about the service behavior and how to invoke its actions. Such abstraction makes the interaction implementation independent since the action can be invoked according to the service interface. Discoverability is one of the main characteristics of SOA. Each service announces its description and the means of interaction through its service interface description. Such description is published in a service registry to be discovered dynamically at run-time by the service consumers. The loose coupling and the reusability allow service composition. Service provider can offer services relying on many other services as shown in Figure 3.2 where the service C depends on the two services A and B. The SOA characteristics contains three of the ubiquitous environment characteristics. The discoverability feature of SOA allows to handle the device dynamicity in ubiquitous environments. SOA services announce their description and their supported capabilities to be discovered by other services. The service registry handles the service registration, it allows service providers to publish their description. The service registry also handles the service departure, it removes the service description the services disappear.

Consequently, a ubiquitous system following the SOA paradigm can actually handle the dynamicity of devices and applications if such devices also follow the SOA paradigm. In other words, if devices announce their departure/arrival on the network along with their description. A ubiquitous system can discover such devices and handle their registration to allow ubiquitous applications to interact with.

Moreover, the SOA has proved its worth to deal with applications interacting with real physical devices.

Such interaction is made possible thanks to the eventing mechanism suggested by the Event-Driven Architecture (EDA) [ Michelson 06] which is a part of the advanced Service Oriented Architecture. The EDA introduces a new kind of interaction between services, real devices and software applications. Such interaction is needed in the ubiquitous environment to allow an interaction between the real physical world and the ubiquitous system and applications.

At last, the SOA provides some kind of interoperability between services. Actually, the SOA allows an interaction between services independently from the implementation. The interface represents the service facade which describes the service behavior and effect. Thus, the service capabilities are invoked based on the method signature expressed in the interface description. Therefore, the interaction is dependent on the service description.

Discovery, control and eventing are the main features of the service oriented systems that suit at best so far ubiquitous systems characteristics. However, one of the main constraints still resides in the service composition and substitution since such operations highly depend on the interpretation of the service description and interface.

We outline next, OSGi an service oriented framework.

OSGi a Service Oriented Framework

The OSGi framework is a service oriented architecture framework. It has been proposed by the Open Service

Gateway initiative [OSGi ] alliance in 1999. The OSGi alliance is an industrial consortium joining effort to specify and promote a residential gateway framework. Such framework hosts multiple services on the client residential gateways such as TV decoders and other TV services. The success of the first version attracted many firms to join the alliance. Thus, the OSGi framework enlarged its scope to become an SOA framework supporting generic service deployment and execution. the digital home and intercommunicate to accomplish a given task for the user. However, such devices usually communicate in the local scope of a network. The residential gateway allows to connect such devices with other remote applications in order to provide an added value services to the end-user. For instance, surveillance camera devices and sensors located in the digital home can be connected to a remote monitoring system for intrusion detection and security surveillance. Additional services can also be proposed to the end-user based on to his already existing devices and applications in his home. For instance, the ubiquitous "Home Share Application" will enable the user to share the media content across his devices. Such applications can therefore be remotely installed on the residential gateways already connected to the Internet. Telecoms operators providing such residential gateways to the end-user will manage and administrate such devices. Moreover, third parties can offer applications to the end-user such as medical assistance or security surveillance services. The third parties service providers propose such services to the end-users by deploying services on their residential gateways.

Thus, the Telecoms operator share the gateway with the third parties applications to offer the end-user various services ranging from multimedia to the surveillance and health care applications.

Modularity and loose-coupling constitute the main advantages promoted by the OSGi framework. In OSGi, each service is independently packaged and executed on the framework. Moreover, the framework offers a management life cycle for each service. This feature allows a remote service management and deployment by both third parties and Telecoms operators.

An Architectural Overview of OSGi

OSGi is a dynamic module system based on the Service Oriented Architecture. An OSGi implementation has three main elements: the framework, the bundles and the services. As shown in Figure 3. From the conceptual point of view, the OSGi framework has three layers [ Hall 10] defined in the OSGi specification:

• The Module Layer defines the packaging and code sharing.

• The LifeCycle Layer defines the execution model management in the framework.

• The Service Layer defines the publish, search and interaction model based on the service oriented architecture.

The Module Layer

This layer defines the bundle concept which is a basic deployment unit. As shown in The bundle is deployed on an OSGi framework, it can implement either a service client, a service provider or simply an API providing packages to other bundles. The bundles interact with each others at the service layer.

The LifeCycle Layer

This layer defines the bundle management in the OSGi framework. It provides a well defined life cycle for the bundles at the download, install and execution time. Such life cycle allows to dynamically manage and evolve the bundles on the fly without the need to restart the OSGi framework.

The OSGi lifecycle state diagram is shown in Figure 3.6. The mechanism offers different operations to manage dynamic installation, start, stop, update, removal and resolution of dependencies between bundles. We overview in the following those commands. Uninstall A bundle can be uninstalled, then the JAR will be removed from the framework's file system.

However, some classes will still be present in the local cache of the framework if other bundles are dependent.

An update or a refresh operation suppresses the remained charged classes.

The uninstall operation transits an INSTALLED bundle into an UNINSTALLED state. An active bundle can also be uninstalled. The framework automatically stops the bundle first, which transits to the RESOLVED state, then the framework transits the bundle state into an INSTALLED state before uninstalling it.

The Service Layer

The service layer details the interaction model between services following the service oriented architecture principles. Service providers publish their services into the service registry while service consumers search or subscribe to consume specific services. Once the request is satisfied, the services can bind and interact with each others.

// P r i n t e r S e r v i c e p r o v i d i n g two c a p a b i l i t i e s : public i n t e r f a c e P r i n t i n g { public void c o l o r P r i n t i n g ( f i l e f ) ; public void b w P r i n t i n g ( f i l e f ) ; } Listing 3.1: An OSGi Printer Service Example An OSGi service provider is described by a Java interface representing an abstraction facade of its specific implementation. The service interface is used by other service consumers to interact with the service capabilities.

The service interface details the provided methods that can be invoked by a service consumer. The interaction goes through a standard method invocation provided by standard Java objects.

In the Listing 3.1, an OSGi printer service example is shown. The interface printing provides two capabilities through two methods. The colorPrinting and the black & white print.

As mentioned before, a bundle implements either a service provider, a consumer or a simply provides an API for other bundles. Once started, a service provider bundle registers its service specifying the service interface along with associated specific properties. Such properties are actually a set of a key-value objects allowing to better identify a service. The Listing 3.2 shows an example of such properties. The bundle registers its services using the standard OSGi framework API method:

registerService("Printing", ServiceJavaObj, props). This method publishes the service in the register. It takes the interface service, the properties and the Java Object reference, see Listing 3.3. The BundleContext is a Java object inherited from the OSGi framework to allow the bundle implementation to interact with the framework and its entities such as the service registry and service listener.

public c l a s s A c t i v a t o r implements B u n d l e A c t i v a t o r { private S e r v i c e R e g i s t r a t i o n r e g = n u l l ;

private P r i n t i n g p r i n t i n g = n u l l ; public void s t a r t ( BundleContext c t x t ) throws B u n d l e E x c e p t i o n { p r i n t i n g= new P r i n t i n g I m p l ( ) ; // I n s t a n t i a t i n g t h e s e r v i c e D i c t i o n a r y p r o p s=new P r o p e r t i e s ( ) ; p r o p s . put ( " documentType " , "PDF" ) ; p r o p s . put ( " p r i n t e r L o c a t i o n " , " F i r s t F l o o r , m e e t i n g room" ) ; // R e g i s t e r i n g t h e s e r v i c e r e g = c t x t . r e g i s t e r S e r v i c e ( " P r i n t i n g " , p r i n t i n g , p r o p s ) ; } public void s t o p ( BundleContext c t x t ) throws B u n d l e E x c e p t i o n { i f ( r e g != n u l l ) r e g . u n r e g i s t e r ( ) ; }

Listing 3.3: Printer Service Implementation Example

A service consumer searching for a Printing service uses the method getServiceReference("Printing") specifying the service interface name. The search can be refined using an LDAP 2 [RFC 4517] syntax-like filter with a key-value properties. For example, getServiceReferences("Printing","(documentType=PDF)").

Once the service is found by the service registry, it returns the Service Java object reference to the consumer which binds to it. Then, the service consumer interacts with it according the methods signature specified in the interface. The service consumer can also subscribe to a service using a Service Listener. The listener will notifies the service consumer as soon as the required service provider appears on the framework.

Service Hooks

The OSGi framework provides service primitives: publish, find and bind. However, such primitives do not allow services to be informed about what other services are searching for. For instance, a bundle service be aware that another service is searching for a "Printing" service or is listening to "Printer" services. This information can be useful to provide an on demand service. For example, a service can proxify another service. To allow such proxification, the service must first intercept the demand then it can publish the proxy service. We applied the service hooks in our work to generate proxies on demand.

Base Drivers

Base Drivers are defined as a set of bundles enabling to bridge devices with specific network protocols such as UPnP, DPWS, Bonjour and IGRS. According to the OSGi Service Compendium [OSGi 09b], a UPnP base driver must provide the following functions:

• "Discover UPnP devices and map each discovered device into an OSGi registered UPnP Device service".

The base driver listens to devices in the home network then creates and registers on the OSGi framework a proxy object reifying the founded device. Meaning that services offered by real devices on the home network are represented as local OSGi services with the same description. Moreover, services provided by the real device can be invoked by local service consumers on the OSGi framework. Additionally, the local invocation on the OSGi reified service is forwarded to the real device. The device reification is dynamic, it reflects the actual state of the device on the platform. Figure 3.8 shows a Printer supporting the DPWS protocol imported by the DPWS base driver and reified as an OSGi DPWS Printer. A Print invocation from a local service application is forwarded to the real DPWS printer by the DPWS Base Driver.

• "Present UPnP marked services that are registered with the OSGi Framework on one or more networks to be used by other computers". Service providers registered as UPnP devices on OSGi are exposed by the UPnP Base driver as real UPnP devices on the network and can be used by other services, applications or computers. Figure 3.8 shows a UPnP OSGi Light service exported as a UPnP Light on the home network.

We used the reification in our work to expose the generated UPnP proxies as real devices on the network.

Thus, the UPnP applications will interact transparently with the generated UPnP proxies as real devices.

Conclusion

In this chapter we detailed the service oriented architecture along with its three entities: service providers, service clients and the service registry. Service providers publish their service information in a service registry.

The published information contains description about the service such as the service provider, service behavior and capabilities. The description contains also information on how to interact with the service and invoke its functions through input and output parameters types and values. A consumer searches for services in the registry, if the requested capabilities are found, then the consumer can bind and interact with the service provider. We also showed how the SOA characteristics: the discovery, eventing and control suit at best so far ubiquitous systems characteristics.

We outlined the service oriented framework, OSGi, along with its architecture and modules. We explained how the base drivers can represent real devices on the network as local OSGi services. Thus, with the required base driver API, a reified device can be invoked locally on the OSGi framework, the base driver will handle the invocation transfer to the real device on the network. Consequently, base drivers refine the protocol stacks heterogeneity into an API and service invocations heterogeneity, since each base driver requires the use of its specific API.

Even though base drivers resolve the protocol heterogeneity, the service representation and content heterogeneity are still to be solved to allow a complete interoperability. Base drivers reify the Plug and Play devices as local OSGi services along with their supported capabilities and their description. Thus, since each protocol uses its own description format and syntax, the service description and interpretation still constitute one of the main constraints to provide a complete interoperability between services.

Chapter 4

Plug and Play Protocols In this chapter we detail four of the plug and play protocols existing in the actual digital home. We focus only on the following IP-based protocols: UPnP, DPWS, IGRS and Bonjour. Other protocols also exist in the digital home such as the X10 [Smarthome 04] or KNX [ Konnex 04] used for the home automation but are out of scope of our work.

We introduce in section 4.1 the plug and play concept, then in section 4.2, we outline the common features relation the four plug and play protocols. We detail the four protocols in sections 4.3, 4.4, 4.5 and 4.6. In section 4.7, we draw an comparison between the four protocols and show the divergence points retaining a complete interoperability between the four protocols. Finally, we conclude in section 4.8

Plug-and-Play

The "Plug-and-Play" (PnP) concept is defined in its early days as the automatic installation of new hardware devices by a computer system [Combs 02] such as modems, printers, network and video cards. The aim of the PnP is to simplify adding new hardware devices for novice computer users [ Bigelow 99]. Therefore, the PnP allows the operating system or the driver to automatically allocate input/output addresses and memory regions for the new added device. The Plug-and-Play concept follows the three rules [Combs 02]:

• A PnP device is completely configurable by a software.

• A PnP device is capable of uniquely identifying itself to the software when required.

• A PnP device exposes to the system the required resources to operate.

The Plug and Play concept simplifying devices installation for the users encouraged device manufacturers to push this vision further. Advances in embedded systems and wireless communications allowed to establish IP based plug and play protocols following the same features as the early PnP devices. Thus, different IP based protocols has been proposed, we focus on the most wide spread:

• The Universal Plug and Play (UPnP) [UPnP ] was proposed by a UPnP Forum industrial consortium in 1999 [UPnP 06b]. UPnP is the most adopted protocol so far due to the Digital Living Network Alliance (DLNA), an organism certifying that a UPnP device is conform to a standard profile and specifications.

Common features

These protocols share a lot in common, they follow the service oriented architecture (see Chapter 3) and cohabit in home networks. The term service in the plug and play domain is referred to as a container holding one or multiple capabilities represented as methods to invoke.

The multimedia domain is dominated by UPnP, IGRS and Bonjour, while the printing domain is shared between UPnP, DPWS and Bonjour. UPnP is widely adopted among routers and Wifi access points manufactures.

Moreover, Plug and Play protocols support the same basic following features:

• Addressing is the first step when a device connects to a network, it gets an IP networking address either by searching for a Dynamic Host Configuration Protocol (DHCP) [RFC 2131] and using an allocated IP address or by assigning an Auto-IP according to the [RFC 3927].

• Discovery and Description occur when a PnP device joins a network, first it gets a networking address then it advertises its description on the network. The device description advertisement includes general device information such as its unique identifier, device model, type, friendly name, its hosted services and pointers to more detailed information. Ubiquitous applications acting as control points listen to the advertisement message, identify and discover the device. Then it can interact with its hosted services based on its description. A Plug and Play device also notifies its departure on the network.

• Control is used by control points or ubiquitous applications searching to orchestrate plug and play devices.

First the control point identifies the device and browses its supported capabilities in order to interact with it. Then the service capabilities are invoked to accomplish a specific task.

• Eventing is the notification sent by a plug and play device to a subscribed control point. The subscription can be made on an accomplished task, a parameter update or device status change. The device notifies each subscribed control point.

• Security is discussed by the three following protocols: UPnP, DPWS and IGRS. However, the presence of real devices supporting security aspects remains shy. Although, the increasing demand to secure the user's data will probably promote this feature further into a real implementation and support on real devices.

Universal Plug and Play Protocol

"The significance of worldwide adoption of UPnP technology comes down to one simple idea: an easy to-use network becomes a ubiquitously used network."

-UPnP Forum [UPnP 06b]
The Universal Play and Play (UPnP) protocol is proposed in 1999 by the UPnP Forum which gathers different industrial companies: device manufactures, telecoms operators, companies in computing, printing, networking and home appliances. The UPnP Forum goal is to create a standard universal plug and play protocol to simplify the interaction and control between devices and applications. Currently, UPnP is the most widely adopted protocol with millions of device types and models certified by the Digital Living Network The UPnP Forum proposes the following specifications and standards to allow an efficient interoperability between UPnP devices.

• The UPnP Device Architecture (UDA) [ UPnP 08] (see Table 4.1), defines the protocol stack to use for the communication between control points and devices. Table 4.1 shows the protocols over IP that a UPnP device needs to implement. The Simple Service Discovery protocol (SSDP) [Goland 99] is used during the discovery phase, the device sends its description on the network to be discovered by control points. The General Event Notification Architecture (GENA) [ Cohen 98] protocol is used for the notification. The The UPnP forum defines the StateVariable as the argument manipulated by an action. There is an ambiguity in using the name StateVariable since its scope is wider than just the device or the service state. Thus, even though we will use the same terminology as the UPnP forum, however, the scope related to the StateVariable is wider than the just the state. The UPnP Device Architecture allows a universal interaction between UPnP devices from the networking perspective, since each device uses the underlying protocols (SSDP, GENA and SOAP) as specified.

Additionally, the UPnP device behaves as expected during the addressing, discovery, control and eventing steps. As for the description, the UDA defines through the UPnP Device Template only a format and structure to represent a UPnP device, its hosted services and the supported actions. However, the device description content is not unified, i.e. any UPnP device manufacturer is totally free to choose the syntax and semantics of the service operations and behavior. For example one manufacturer could assign the type "BinaryLight" (see listing 4.1, line 2) to its device while another light manufacturer can use the type "SimpleLight". The UPnP control point or ubiquitous application scanning the local network for device announcement identifies a device through its description, therefore the difference in the device type "SimpleLight" vs "BinaryLight" syntax prevents the control points to use the required device. To allow a universal plug and play interaction, the UPnP forum proposes the Device Control Protocols (DCP) on top of the UPnP Device Architecture (see table 4.1). Thus, the DCP offers a standard device profile descriptions.

1 < actionList > 2 < action > 3 < name > S e t T a r g e t </ name > 4 < argumentList > 5 < argument > 6 < name > newTargetValue </ name > 7
< r e l a t e d S t a t e V a r i a b l e > Ta rge t </ r e l a t e d S t a t e V a r i a b l e > We summarize in the following how the UPnP Device Architecture allows to a UPnP device to support the Service Oriented Architecture main characteristics: discovery, control and eventing. We detail next the UPnP protocol stacks exposed in Table 4.1 which enables such features.

8 < direction > i n </ direction > 9 </ argument > 10 </ argumentList > 11 </ action > . . .
Discovery is the first step in the plug and play interaction. A UPnP device or a UPnP application (control point) use the Simple Service Discovery protocol (SSDP) [Goland 99] during the discovery phase. A UPnP device relies on the SSDP to announce through multicast messages its presence on the network along with its supported capabilities. A UPnP Control Point relies on the SSDP and listens to the devices' advertisements. When a device type or a capability matches, the control point requests additional information from the device. The UPnP Control Point can also uses a multicast SSDP search request.

The UPnP devices listen to the search requests on the network, if a search matches the device capabilities, then the UPnP device responds with a unicast SSDP. The unicast and the advertisement messages both contains a pointer (url) to retrieve additional information on the capabilities. UPnP devices also rely on the SSDP to announce their departure messages.

Description is announced by a UPnP device on the network. It is exposed in the UPnP XML format and content detailed previously. A UPnP Control Point interprets the advertised description and if a match is found then the interaction and control can take place.

Control is supported by the Simple Object Access Protocol (SOAP) [W3C 00]. The action invocations are exchanged through SOAP messages which contain the action to invoke and the parameter(s) value(s).

Eventing is supported by the Generic Event Notification Architecture (GENA) [ Cohen 98]. It allows for a control point to subscribe in order to receive notifications, for example, on a parameter change or upon an ending task. The control points interested in receiving notifications, subscribe to an evented state variable, and provide the destination location where to send the event. The control point also specifies a subscription time which can be renewed or canceled.

The DPWS protocol is detailed in the next section.

Device Profile for Web Services

The Device profile for Web Services also known as Web Services on Devices (WSD) was initially proposed by Microsoft in 2004 as an enhanced version of the UPnP protocol. Actually, the UPnP protocol is based on non standard protocols like GENA, SSDP and uses non standard description language to announce its capabilities.

Therefore, the DPWS protocol insisted on the fact that it is built on common Internet standards promoted by the W3C. The DPWS targets similar devices and objectives as the UPnP protocol. However, it is fully dependent on standard Web Services technology and protocols. The DPWS is seen as an extended Web Services provided by devices. Therefore, the interoperability with the existing Web Services is supposed to take place with a minimum integration effort. The internal device description architecture is described is Figure 4.2. The device or the hosting service hosts one or multiple services (referred to as hosted services). Each hosted service (represented as "portType" in the WSDL) offers one or more capabilities which can be accessed through operations. The operation is invoked through input messages and can return output messages. The message content contains an element which could be simple or complex and contains itself several simple or complex elements. The DPWS protocol also follows the service oriented architecture which allows discovery, control and eventing. We detail in the following paragraph, the DPWS protocol stacks exposed in Table 4.2 which enables those features. We also overview their scope in an exchange example between a client and a device as exposed in Description Once the device discovery takes place, the client can retrieve additional information such as the supported services and capabilities. The WS-Transfer [W3C 06c] and the WS-MetaDataExchange [ W3C 11] protocols, see Table 4.2, are used to retrieve specific meta data along with the service description WSDL.

The WS-Policy can also be used to express the capabilities and constraints of the Web Service. In Fig- Eventing the notification is supported by WS-Eventing [W3C 06b]. It allows applications to subscribe to a specified event, such as when the printing operation is finished. The WSDL also contains information about the subscription and the information delivery structure and content data type. In Figure 4.4, messages [11][12], the client subscribes to the printer status during the printing job.

Messaging The DPWS uses the Message Transmission Optimization Mechanism [ W3C 05] with the SOAP [W3C 00] protocol in order to exchange large amount of data such as the audio/video streams and large documents. In Figure 4.4, message 13, the client sends the document to print using MTOM over SOAP.

The printer informs the client about the printer and the job status, messages [14-15].

Security The DPWS protocol offers through WS-Security a solution to exchange secured data between a device and a control point.

Intelligent Grouping and Resource Sharing

The IGRS [IGRS ] The IGRS protocol stack as described in [IGRS 06], adopted the same protocol stacks used by UPnP. The IGRS standard uses GENA for eventing, SSDP for discovery and SOAP for control. However, IGRS changed the header syntax of protocol messages on the eventing and control levels. Additionally, IGRS supports a peer-to-peer discovery mode and a centralized mode (master-slave) similar to the proxy discovery mode in the DPWS protocol.

Another difference between UPnP and IGRS resides mainly in the device and service description level. Interaction between the groups is enabled and goes through the security resource group which is present at the control and transport levels.

The IGRS labs publish standard IGRS profile devices highly inspired from the UPnP standard specifications.

For instance, there is an IGRS standard profile for the set-top-box, mobile phone and even an IGRS AV [ IGRS 07] (Audio, Video) specification similar to the UPnP AV. IGRS devices seem to be commercialized in Asia but not in Europe.

IGRS relies on the same UPnP protocol stacks, it also supports the control, eventing and control.

Bonjour

The Additionally, the DNS-SD allows to provide additional information during the annunciation. Such information are stored in a DNS TXT records in a key-value pairs. However, the information provided by the TXT records is considered as inferior and less reliable than those advertised initially. For each device profile, there is a standard description key record. For example, the Bonjour printer service [ Apple 05] defines a set a standard keys such as product, name, pdl. The Listing 4.4 shows an example of an LPR TXT record of PostScript printer.

1 . t x t v e r s =1. rp=auto . q t o t a l =1. p r i o r i t y =25. t y=Apple L a s e r W r i t e r 8500 The next section draws a comparison between the four plug and play protocols.

Plug and Play Protocols Divergence

"Increased homogeneity in the domestic environment plainly offers attractions such as convenience. ... However, this is a double edged sword, resonating with concerns of McDonaldization, the process by which modern society takes on the characteristics of a fast-food restaurant."

-Aipperspach et al. [Aipperspach 08]
UPnP [UPnP ], IGRS [IGRS ], Apple Bonjour [Bonjour ] and DPWS [OASIS 09a], cohabit in home networks and share a lot of common points. They are all service-oriented with the same generic IP based layers: addressing, discovery, description, control and eventing. They also target similar device types: multimedia is shared between UPnP, IGRS and Bonjour while the printing domain is dominated by UPnP Even though those protocols have a lot in common, devices can not cooperate due to following three main differences:

Protocol Stack : As shown in Table 4. GENA handles the eventing in UPnP and IGRS, while WS-Eventing is used in DPWS. In Bonjour devices, the eventing depends on the protocol layer employed on top of Bonjour.

The security in DPWS is handled by WS-Security and WS-Policy to provide secure channels during sessions. IGRS uses IGRS-Security present in the control and eventing protocols. UPnP launched initiatives to support security, however, it was not adopted by the industry because it was too complex to implement.

Another simplified security layer is expected to appear in the next version of the standard profiles. In UPnP and DPWS, the security is supported separately from the specifications while in IGRS it is directly involved in the control and eventing layers. Thus, the structural difference in the representation format adds another layer of difficulty when its comes to provide interoperability between description devices and services. This heterogeneity prevents smart applications to use any available device, regardless of their protocol, to accomplish a certain task such as printing or dimming a light.

Concepts

Conclusion

In this chapter, we detailed the following Plug and Play protocols: UPnP, DPWS, IGRS and Bonjour.

The UPnP protocol is the most widely spread so far. The DLNA organism which certifies UPnP devices and ensures their conformity to the standard profiles allowed the wide adoption of UPnP by a large number of manufacturers and Telecoms operators. Furthermore, to this date, UPnP is the protocol that offers the highest standard profiles among other protocols. It proposes various standard device profiles ranging from simple devices like binary lights to more complex devices like printers and scanners.

The DPWS protocol proposed by Microsoft and standardized in 2009 is already deployed in Windows Vista and Seven. Moreover, printer manufacturers adopted the printer standard profile and already propose various DPWS-capable printers. A DPWS device uses a set of (WS-*) standardized protocols and expresses its information and services with the Web Service Description Language. Such conformance with the web services protocols and description allows DPWS devices to communicate with Web services clients. To this date, DPWS published only a printer and a scanner standard profiles.

The Bonjour protocol proposed by Apple, is already deployed in i Tunes and the i P* products. It uses the multicast-DNS and DNS-SD protocols, therefore, it benefits from the large deployment of the DNS protocol and servers. However, Bonjour is only used for device addressing and discovery. Another set of protocols are used on top of Bonjour, depending on the device domain to allow eventing and control.

The IGRS protocol has a lot of similarities with UPnP and it is already emerging in Asian markets.

We also presented in this chapter, the common features and divergence points between the plug and play protocols. We pointed out in detail the main challenges restricting the interoperability between those protocols: the protocol stacks, the device and service description, and finally the content description.

Those plug and play protocols are either proposed or carried out by giant corporations. Thus, devices supporting UPnP, DPWS and Bonjour are already spread at the globe scale. Moreover, IGRS is emerging in Asian markets. Therefore, the interoperability between the four protocols is essential and constitutes a major challenge for different actors.

Applications Designers specify and implement applications to interact with various devices present in the digital home to accomplish specific tasks. Supporting multiple protocols is time consuming from the developers perspective since they must implement the interaction with each device.

Telecoms Operators deploy Internet gateways for the end-user and share the platform with third parties applications providers. They also provide tools to troubleshoot and diagnostic devices and applications to avoid an expensive human intervention at the end users side. Thus, such protocol diversity makes the diagnostic and the troubleshooting a more complex operation and adds a burden costs on their infrastructure and tools to target additional protocols.

End Users can be frustrated since they will be restrained to a certain protocol and technology if they want a complete interoperation in their home network.

Indeed, the OSGi framework along with the base drivers allow to resolve the protocol stacks heterogeneity by representing each device and its services as an OSGi service on the network. However, device and service descriptions and content remain heterogeneous. As detailed in chapters 3 and 4, plug and play devices and services announce their description in a service registry or on the network. Such description contains information on their supported capabilities and how to interact with their provided actions. The information provided by such services is a knowledge shared between service providers and service clients searching for specific services to use. Service consumers request and interact with the service providers according to their capabilities expressed in their description. Therefore, the description representation by the service providers and the correct interpretation by the service clients are essential for the interaction between services.

In the comparison of chapter 4, we detailed the main elements preventing plug and play device interoperability. It is due to the heterogeneity of the three following elements: the protocol stacks, the format description and the description content. Chapter 3 presents a technical solution to wipe off the protocols heterogeneity by combining the OSGi framework along with specific network base drivers. The presented solution reifies plug and play devices on the network as services on the OSGi framework. Thus, it allows applications and other services to interact with the reified OSGi services independently from the networking protocol. The solution resolves the protocol stacks heterogeneity, however, the service description and content heterogeneity remain unsolved.

In order to resolve such heterogeneity between representations, we must first have a look on the knowledge representation techniques. Therefore, in this chapter, we first define the knowledge representation. Then, we provide an insight of some representation techniques such as the ontologies, the rules and models. The languages used in the plug and play protocols, allow to mainly capture the description syntactically. In the description structure, some semantics can be retrieved such as the nature of the used entities, such as device or service. However, there is no direct semantic properties pointing out the relation between the entities. Thus, the expressivity of the languages used in the plug and play protocols lacks of expressivity since it relies only on the syntactic information representation. • Models enable capturing the knowledge on different abstraction layers form the real world.

These three techniques are described in the following sections.

Ontologies

The Ontology" comes from the Greek, "Ontos" for "being" and "Logos" for "study". It refers in philosophy to the nature of existence, the categories of being and their relations. More precisely, according to [Kalfoglou 01],

"An ontology is an explicit representation of a shared understanding of the important concepts in some domain of interest".

Gašević et al [ Gašević 06] define the ontology as a language and a sort of catalog used to manipulate entities of a domain, "To someone who wants to discuss topics in a domain D using a language L, an ontology provides a catalog of the types of things assumed to exist in D; the types in the ontology are represented in terms of the concepts, relations, and predicates of L." Moreover, the ontology provides a classification in a some sort of a taxonomy along with the relations between the concepts (hierarchies and constraints). The aim of an ontology is to capture the semantics of a domain through the concepts and the relation between them independently from the syntax.

Ontologies are expressed in triplets (Object -Attribute -Value).

Ontology Entities

Even though a lot of ontology languages have been proposed in the literature, they all deal with the following entities:

Concepts are the main entities of an ontology, they are the substitute of the things from the real world.

Individuals are considered as instantiated entities of concepts. In Figure 5.3, a Hotel Room:#1345 is an instance if the HotelRoom concept.

Relations constitute the glue associating concepts between them and individuals with their concepts. In Data Types are specific entities to specify a data type, such as a URI, a String, an integer or a boolean.

Data Values are values that a concept or an individual can have.

Ontology Development Methodologies

According to [Noy 01], the ontology development process starts by determining the domain and the scope of the ontology. Once the domain and scope are identified, the authors suggest to look up for existing ontologies.

It is easier to reuse ontologies instead of building a new ones. The building process starts by enumerating important terms of the ontology in order to define the concepts and the hierarchy between such concepts. Then, the relations between the concepts are established. Defining the data types and values then follows. Finally, only the first iteration ends by creating instances.

Multiple iterations need to be carried out to refine the ontology and adapt conflicting issues between entities in order to achieve a minimum acceptable version.

Several methodologies has been reported in the literature like in • "There is no one correct way to model a domain", the best solution depends on the anticipation of the ontology application and use.

• "Ontology development is necessarily an iterative process", it requires to consider conflicting concepts and relations. However, after a time-consuming iterations, a minimum consensus about the ontology should be found [ Gašević 06].

• "Concepts in the ontology should be close to objects (physical or logical) and relationships in your domain of interest". The ontology should represent the real world, therefore the concepts and the relations selection should be close to the real world objects as much as possible.

Semantic Web Services

As mentioned in chapters 3 and 4, the service description exposes information about the interface and the means of interaction between services. However, the interaction between services is dependent on the correct interpretation of the representation and the associated data content as well. The RDF-Schema is an extension of RDF, it offers more expressivity, for example it allows to specify properties and associate them with classes. This enables to create taxonomies through reserved terms in the RDF(S) language such as rdfs:Class, rdfs:SubClassOf, rdfs:Property and rdfs:Subproperty.

RDF(S) has a limited expressivity, for instance, one cannot specify in RDFS that a class is equivalent to another class.

OWL The Web Ontology Language [ Bechhofer 04] is a standard promoted by the W3C. OWL is the successor of two previous ontology languages DAML and OIL. OWL is based on the RDFS language and adds more expressivity by adding additional relations between entities. For instance, OWL allows to specify a symmetric, inverse or transitive properties between relations. It also allows to express that two entities are equivalent owl:equivalentProperty or different from each others through a well defined vocabulary. The (minimum, maximum) cardinality (owl:cardinality) of a property can also be specified. An entity can be also specified as an intersection or union of two or more other entities.

Semantic Service Description Languages

The ontologies are also used to capture semantics in service descriptions. We overview in the following semantic service description languages and frameworks built on top of the ontology languages.

OWL-S OWL-S [Martin 04

] is a Semantic Markup for Web Services. It relies on the OWL language and is used to describe web services. OWL-S specifies a service with three different parts: the service profile, the process model and the service grounding.

The service profile provides a description of the service capabilities along with its input, output, preconditions and effects. It also allows to specify the service type and category. The process model describes how the service works and provides information about the service whether it is an Atomic Process executing alone or a Composite Process combining other processes. The service grounding holds the technical details on how to interact with the service, such as the communication protocol or the serialization techniques.

OWL-S allows composition and cooperation between web services using well defined terms to express the service entities and relations through the different profiles. However, service providers may use different ontologies to represent the input and output types, the service types and categories. Therefore, the interoperability between different ontologies is still needed.

WSDL-S The Web Service Semantics Language [Akkiraju 05] aims to extend the Web Service Description

Language with semantic annotations added to the WSDL tags. It uses similar aspects from OWL-S such as precondition and effects. The WSDL-S benefits from the adoption of the WSDL as an industrial standard used to describe web services. The annotation can refer to concepts from different existing ontologies. Thus, this aspect makes the interpretation of the annotations difficult since no common semantics or relations are defined between the different ontologies and concepts. Therefore, it is almost impossible to match service requests and descriptions annotated from different ontologies using different concepts with a sort of mapping between the several used ontologies.

WSMO The Web Service Modeling Ontology[ Roman 05] is proposed to describe various aspects related to Semantic Web Services. WSMO has four main concepts to specify service descriptions: Ontologies, Services, Goals and Mediators. The ontologies provide defined terminologies to capture the semantics of a domain.

Services request or announce their description using the same terminology from a specific ontology. The Goals Other similar languages and frameworks were also proposed in the literature to add semantics to the service description. For example, the Semantic Web Services Framework (SWSF) [Battle 05] has also been proposed along with its two parts: the Semantic Web Services Language (SWSL) and the Semantic Web Services Ontology (SWSO).

The ontologies allow to capture information of a domain and to semantically express the relations between its entities. Such captured information is exploited to extract new information from what is already represented in order for example, to take a decision or interpret information. To extract information from what is already represented in the ontologies, rules combined with logics can be employed. We outline in the next section how rules can be used to infer new information from what is already represented.

Rules

Rules are another form of knowledge representation [Davis 93], [ Gašević 06], [ Studer 07]. Actually, human knowledge is also described using rules to make actions or think about situations. The rule structure is a combination of two parts, the preconditions and the conclusion. It is usually expressed in the form of an IF.. THEN ... For instance, IF "the service description matches the request" THEN "Bind to the service and interact". Rules are usually formalized using logics to provide defined semantics in order to remove ambiguity.

Combined with logic, the rules are used to exploit the information from what is already available [Horrocks 03].

Even more, rules are used to infer and deduce new information from what is already represented. A typical example of the deductive inference is the famous quote of Aristotle: "All men are mortal. Socrates is a man. Therefore, Socrates is mortal". Thus, the rules are not limited to representing knowledge but are also used to manage and extract information from what is already represented. Additionally, extracting information from existing data allows to trigger various operations which perform data classification or annotation on the existing information. Thus, the rules are used to represent and manage knowledge.

Logic-Based Representation

Logic based representation paradigms provide defined semantics to avoid ambiguous expressions. Such different representations allow a different level of expressivity. We overview in the following three logic based representations along with the limitations of their expressivity.

Propositional Logic

Propositional logic is about symbolic reasoning through propositions. A proposition is a logical statement assigned to a symbolic variable. A statement is written in a natural language and it cannot be interpreted by a machine. For example, "A= Device D provides Service X", "A" is a symbolic variable, it has a true or false value representing the truth of the logical statement.

Additionally, a proposition can be linked with another using logical operators such as AND(∧), OR(∨), NOT(¬), IMPLIES(⇒) or EQUIVALENCE(⇔) to form more complex rules. For example, let "B= Application P requests service X" and "C= An application interacts with a device". The relations between the three propositions can be expressed as follows: A ∧ B ⇒ C. Meaning, if A and B are true then C is true.

The propositional logic allows reasoning with rules through logical operators and variables. It mainly depends on the truth tables of logical operators. The semantics of symbols is unimportant since the reasoning depend on the values of the propositions (true, false). Moreover, the truth value of a statement can be subjective and not a common truth.

First Order Logic

The first order logic extends the propositional logic and offers more expressivity by introducing two quantifiers:

the Universal ∀ and the Existential ∃. It also uses constants, variables and predicates. A constant begin with a lowercase letter while a variable with an uppercase. A predicate links two constants or variables together.

For example "Device D provides Service X" would be written provides(D, X). For example, the rule "Device D provides a service X" and "Application A requests a service X" then "Application A interacts with the device D" would be written as:

∀ D provides(D, serviceX) ∧ ∀ A requests(A, serviceX) ⇒ interacts(A, D).
The first order logic has enough expressivity and allows other representations to be based on it. However, the First Order Logic cannot represent inclusions, union or intersection relation between elements. We present next the description logic which offers more expressivity.

Description Logic

The description logic is based on the First Order logic and offers more expressivity. It allows to define a vocabulary of a domain in a knowledge base called TBox. The vocabulary include an atomic terminology which consists of concepts and roles. Concepts represents a set of individuals while the roles are binary relations between the concepts. From the atomic terminology, a composite terminology can be composed using various symbols such as intersection ∩, union ∪, or inclusion ⊆ between elements. The TBOX expresses the semantic relation between entities. For example, "Hardware" and "Software" are atomic concepts. From these atomic concepts, we can propose composite concepts. For instance, "Device ≡ Software ∩ Hardware". The Assertion Box ABox represents new instances using the TBox terminology. For example: Device(Printer) allows to specify that a Printer is a Device concept.

OWL allows to express the concepts of an ontology in description logic using its sub-language, the OWL-Description Logic language. The description logic encounters limitations when defining predicates of arbitrary arity (more than two) and in the results decidability.

Rule Languages

Various rule languages have been proposed in the literature such as Lisp [Steele 90] and Prolog [Clocksin 03].

We outline in the following only rule languages specified to reason and extract information from ontologies.

SPARQL This language [Prud'hommeaux 04] is specified to query ontologies based on the RDF language. A SPARQL query targets a pattern in an RDF ontology. The pattern can be a simple RDF triplet (O-A-V)

or a complex one combined of multiple simple and complex RDF triplets. The SPARQL syntax is SQLlike. The SELECT clause holds variables starting with "?", the clause WHERE contains the pattern to be matched. A query of the Listing 5.1 can be specified as shown in Listing 5.2. Another interesting knowledge representation vision is tackled by the "Models" which offer different layers of representation. The high level layers manipulate abstract concepts while the lower level layers contain concepts more close to the real objects with implementation and technical related details. Thus, the high level layers offer to the designers the ability to manipulate abstract concepts with a minimum specific and technical information.

Moreover, the "Models" domain offers the ability to generate low level concepts from the high abstract ones.

Thus, compared with the ontologies, the "Models" offer a knowledge representation with abstract concepts away from detailed information dependent on the technical details of the real world objects. We outline next the knowledge representation using "Models".

Models

Hagget et al [Hagget 67] define a model as a simplified abstraction of reality. An abstraction is also considered as another form of knowledge representation. Figure 5.4 shows different levels of abstraction. The M0 layer represents objects from the real world, such as a real printer. At the M1 layer, a drawing of a printer abstracts the printer from the real world. Thus, the printer's drawing is a model abstracting an object from the real world. The M2 layer, the meta-model layer specifies the basic concepts to be used by the underneath layer.

Thus, each model (i.e. drawing) at the M1 layer is constituted only of concepts from the meta-model such as a circle or a triangle. Therefore, models (i.e. drawings) at the M1 layer are conformed to the M2 layer since they are constituted only from the specified concepts at the M2 layer. The M3 layer also defines concepts to be used by the M2 layer models and so on. However, the last layer Mn also includes self-defined concepts to be used by the layers underneath and by the Mn layer itself. In this example, the dot concept is the self defined concept which allows to define all the concepts at the lower layers. In the pervasive applications development, a lot of works have been proposed in the literature which provide a high level abstraction model to specify applications. Such specification is carried out by designers through high level languages detailing the concepts to be used, their behavior along with the interaction between. Then, using MDE techniques, the high level language is automatically transformed into an executable code. Thus, high level specification allows designers to focus only on the high level aspects in the application development which are mainly the design of modules, their behavior and the interactions between. For instance, designers will only manipulate high level modules such as a light device concept, a clock concept and a movement detector concept. The designer should only be aware of the concepts functionalities and their behavior without knowing the technical details allowing applications to communicate. Additionally, the designer will specify the behavior of each module and how it should react upon receiving events from other modules. Once the specification is achieved, code generators transform the high level specifications into lower levels.

Identically, the development process of applications goes through almost the same levels: the specification, then the implementation which holds a lot of technical details such as how to interact with real devices or what data structure should be sent. Thus, automating the transformation from high level specifications into low level code generation can actually save a lot of time for designers and can reduce the human errors occurring during the programming phase. The automation is carried out by transformation rules previously written by designers.

Moreover, the high level concepts help designers and developers to share and to agree on a common representation of the system to be built.

Conclusion

In this chapter, we presented different techniques to represent information. Ontologies are used to capture the semantics of a domain. Each important element of a domain is represented as a concept then various properties relate concepts together. Individuals can also be represented and are considered as an instance of a concept.

Data types and values are also represented in ontologies. Ontologies can be used in services description to provide a common semantics which can be shared between service providers and consumers. Different languages has been proposed to increase the expressivity of the service description. Such expressivity is supported through semantic annotations of the service description.

We also presented rules as a mean to exploit information from ontologies. Combined with logics, rules offer In our contribution, we present how these three representation techniques: "Ontology", "Rules" and "Modles" combined together can be used to resolve the device and service heterogeneity and achieve interoperability.

Chapter 6

Conclusion & Problem Statement: Device Interoperability

"A problem is a chance for you to do your best".

-Duke Ellington

In the part I of this thesis, we outlined the context of ubiquitous computing. We drew in chapter 2, the global vision of a ubiquitous environment than we extracted its characteristics. Based on such characteristics, we summarized the important features a ubiquitous system should support.

From an ideal vision of a ubiquitous environment and system, we presented the digital home, a current environment moving towards a ubiquitous one. We detailed the actual state of the digital home along with the current available technologies. Then, we pointed out the following three challenges in such an environment.

First, the dynamic discovery of existing devices along with their supported services and capabilities. Second, the interoperability between various plug and play devices and services. In an ideal ubiquitous environment, applications interact with devices and other applications transparently to accomplish a specific task such as printing or rendering a video. Thus, applications should be set free from a device or protocol technology. And finally, management operations in the digital home are needed mainly to control, deploy, troubleshoot and diagnostic different devices and applications. However, the heterogeneity of the plug and play protocols adds a burden cost on the telecoms operators, device manufacturers and applications providers. Therefore, those three actors must maintain a wide variety of administration and diagnostic tools in order to support troubleshooting and maintenance on different devices communicating through heterogeneous protocols.

Obviously, these three challenges pointed out in the digital home environment must be tackled by a ubiquitous system architecture capable of supporting the following features: the discovery, the control, the eventing and the interoperability.

Therefore, in chapter 3, we exposed the Service Oriented Architecture and showed how the SOA characteristics meet at best so far the ubiquitous system characteristics. In fact, the SOA architecture consists of three elements: service providers, consumers and the service registry. Service providers publish their service description which allows service consumers to discover the provided capabilities in order to control the service providers. Additionally, the SOA architecture supports the eventing between services. In fact, Plug and Play devices in the ubiquitous environment announce their description while ubiquitous applications listen to such advertisements in order to interact with specific devices to accomplish a given task.

Thus, the SOA can provide to a ubiquitous system the following characteristics: discovery, eventing and control. Moreover, the SOA refines the plug and play device heterogeneity into a SOA service description heterogeneity. In fact, the main limitation of the SOA resides in the service description which is expressed only syntactically, thus making the service matching dependent on the syntax description instead of the semantics.

In order to identify more deeply the plug and play device heterogeneity, we detailed in chapter 4 the following plug and play protocols: UPnP, DPWS, Bonjour and IGRS. Devices supporting such protocols follow the service oriented architecture.

We also drew a detailed plug and play protocols comparison showing their commonalities and divergence areas. OSGi and the base drivers allow to represent devices and their services as local OSGi services which can be invoked independently from the protocol stack. Base drivers will transfer local invocations received by the OSGi reified services to real devices. The OSGi and its base drivers move then the protocol stack heterogeneity into the base drivers API calls. However, the service description and the content heterogeneity remain to be solved.

Device and Service Description Each protocol also defines a specific description format to announce the device information and the services description. UPnP uses an XML template based description, while IGRS and DPWS uses the Web Service Description Language. Bonjour also defines its description format.

Thus, the heterogeneity in the description representation makes it impossible to applications or devices to interpret the announced description or the received events and interact with the provided service.

Description Content Another layer of heterogeneity is present on top of the description content. Each protocol defines a set of standard profiles per device type which imposes standard devices to use the same content syntax to describe the supported services, actions and parameter types and names.

Since each protocol defines its own standard content, the interoperability is impossible. The discovery will not occur since an application searching for a UPnP Printer with a type "PrinterDevice" only binds to device holding the same specific name. Thus, a DPWS printer holding the name "PrintingDevice" will not match even though the application is searching for any device capable to print.

The interaction and control cannot occur since each action is described using various content. For instance, a standard UPnP printer supports the action CreateURIJob to print a document while a DPWS standard printer supports an equivalent action expressed syntactically differently, the CreatePrintJob action to print a document. Moreover, since applications and devices interpret the description syntactically, then an application cannot interact with a UPnP printer since the content description is different, even though the printer supports semantically equivalent actions.

The three levels heterogeneity enclose smart applications into specific and preselected devices and services orchestration. Smart applications need to be set free from protocol and service syntax heterogeneity. The user must not be restrained to one type of protocol and devices, but should be able to integrate easily and transparently equivalent devices to his home environment.

The description and the content heterogeneity along with the protocols layers diversity, prevent applications to discover and use any available equivalent device on the network to accomplish a specific task. Designing ubiquitous applications to support multiple protocols is time consuming since developers must implement the interaction with each device profile and its own data description. Additionally, the deployed application must use multiple protocols stacks to interact with the devices. An interoperability between plug and play devices requires an interoperability on the three different layers.

Moreover, application vendors and telecoms operators need to orchestrate devices through a common application layer [ Spets 10], independently from the protocol layers and the device description. Figure 6.2 overviews the application vendors and telecoms operators vision to interact with plug and play devices. They would like to develop applications interacting with plug and play devices transparently through a common application layer [ Spets 10]. Such common layer actually abstracts the three layers of heterogeneity and allows ubiquitous applications to interact with any device through common instructions specific to a device type. For instance, the UPnP and DPWS Light devices would be controlled using the same actions provided by the common application layer. It is up to the common application layer to adapt and translate the commands according to the specific target device using its own protocol stack, description and content. However, proposing a common application layer is very complex since the common layer will need to capture common concept and find common semantics between the actions from the same type device. Additionally, common parameters (types and content) need to be proposed which is not simple since some actions require a set of obligatory elements and other optional ones. Therefore, the main focus of our work will concentrate on the service adaptation to allow a plug and play interoperability between devices and their provided services.

To address the description representation format and content heterogeneity, we need to clarify through actual available techniques, how knowledge and information can be represented. Therefore, we provided in chapter 5 We survey in this chapter, research efforts proposed in the literature to allow service adaptation. We refer to the term adaptation as a transformation of a service in order to adjust to precise conditions. Such conditions as mentioned in chapter 2, are due to various reasons such as a device appearance, user demand, scarce resources, and many others.

The service adaptation can occur at the compile time or at run-time, on demand or dynamically triggered.

A service adaptation is involved in one or multiple aspects. For example, it can include a transformation in one of the following: the input/output data of a service, the service interface, the communication medium, the service behavior, the resource allocation and use, platform migration and execution and many others.

We focus in this chapter only on the service interface adaptation, input/output data and behavior. Different approaches have been developed to solve the interoperation problem through service adaptation, it can be put in three major categories. The first category uses a common ontology capturing all the semantics of a domain. The adaptation process queries the common ontology which holds all the concepts from a domain and the relations between them. The adaptation is possible only if the service description uses the same concepts of the ontology. The adaptation is usually performed at run-time.

The second category is based on a abstract model capturing abstract and high level concepts of a domain. Then, using transformation rules and code generation techniques, an adaptation at the lower level is achieved. The adaptation can be performed at run-time or at compile-time.

The third category handles the adaptation through a rewriting of the service description using a universal language or a unified interface. The adaptation is usually not performed at run-time since the description is re-written offline.

Common Ontology

This section overviews research work proposing service adaptation for interoperability based on a common ontology. The global ontology is manually built by an expert and represents the main concepts of the domain and the relations between such concepts. A common ontology is used mainly to classify concepts in a hierarchical organization which is referred to as taxonomy. For conciseness, we choose to detail only three service composition approaches. Paolucci's semantic matching algorithm, the PERSE and the MySIM middlewares. if the output R is a subclassOf output A, i.e. if the service provider advertises an output parameter having as a type Device and the service consumer requests an output parameter having as a type Printer, then there is an exact match between the two outputs.

PlugIn occurs if the advertised output subsumes the requested output. For example, if the Device is advertised and LaserJet is requested. Since Device is a more general concept than LaserJet, the provided output can be a LaserJet. There is a weak relation between the two concepts.

Subsume if the requested output subsumes the advertised output. For instance if the requested output is a

Device and the advertised output is a LaserJet. Modifications on the service consumer are probably needed.

Fail if their no relation of subsumption between the the advertised and requested outputs. For example, there is no relation in the ontology of Figure 7.1 between Scanner and LaserJet or Wifi.

The algorithm returns to the service consumer, a reference to bind with the service provider. The exact matches are returned first, then the plugIn and finally the Subsume.

This approach is relatively close to the inheritance principle in the object oriented programming where the adaptation is resolved by casting objects. However, Paolucci defined rules to limit the substitution by restricting the inheritance degree. The more the gap between two concepts increases, the less the substitution between concepts is possible.

Paloucci implemented his algorithm and applied tests using a UDDI [OASIS 04] service registry and used the DAML-S language to annotate the service description. An evaluation of the semantic service matching showed that most of the time is spent during reasoning and concepts classification. Therefore, they proposed an encoding mechanism to represent the service description (input, output and category) with regard to the ontology concept. Their encoding approach revealed to be efficient since the classification and matching time were highly reduced.

They also proposed an Interoperable Service Description Language (ISDL) [ Mokhtar 07] quite similar to OWL-S, previously described in chapter 5. A translation mechanism is introduced to represent other service description in ISDL. ISDL allows to represent service description with semantic annotation and is based on the WS-BPEL [ Alves 07] which allows to specify a service behavior.

The PERSE middleware also supports service composition, i.e. assemble or orchestrate many available services to satisfy users' task. The general idea is to transform each service behavior specified in the service description to a finite state automate then to select only sub-automates verifying the user's task constraints. Then, the selected sub-automate is used to satisfy the user's task by composing various services in the environment.

The PERSE [ Mokhtar 07] middleware has been implemented in Java and validated using the Salutation (ex-Bonjour) and UPnP protocols, they also used a UDDI service registry.

MySIM

Ibrahim [Ibrahim 08] presents the MySIM middleware which allows a service matching, transformation and composition. The service description is annotated with concepts from a common ontology. MySIM takes into account during the service matching and composition, non-functional properties such as the QoS which can be specified in the service description as shown in Listing 7.2. Each service is annotated with a predefined concept from the common ontology, such as the "printer" concept. The input and output parameters are described in triplets, <parameterName, type, semantic concept>. Additionally, each description contains non functional properties which can be qualitative Npql (line [4] in Listing 7.2) or quantitative Npqn (line [5]). The adaptation carried out by MySIM is transparent to the user and the application. The service matching is syntax and semantic based and takes non-functional properties into account. also focus on satisfying users requests through service composition. A user's request can be expressed with an "intuitive form" like a natural language, than it is transformed into a semantic request. Then, the request is satisfied through a service composition from already available services. Fujii 's approach also uses annotations from what they refer to as a "Semantic Graph".

Abstract Model

Abstract models are also used in middlewares to provide interoperability between services and protocols. As shown in chapter 5, models can be used to capture information about a domain. Assisted with model driven engineering (MDE) [Schmidt 06] techniques, an abstract model can be transformed into an executable code.

Thus, the methodology consists in proposing an abstract model independently from a language or a technology then to transform it into an executable code through transformation rules and code generation techniques. We present in this section, three middlewares relying on MDE techniques and abstract models.

DOG: Domotic OSGi Gateway

The ) are applied to connect some properties to the instance. For example, if the instance is a light device then a control functionality is automatically added. Moreover, it is up to the designer to add the device description to the ontology which contains the device name as well as all the low level details like the device address and its format (see the simpleHome1 ontology which an instance of the DogOnt).

DOG uses specific Network Drivers similar to the base drivers described in chapter 3 to communicate with the real devices. On DOG start up, the Platform manager [Bonino 08a] queries the DogOnt and retrieves the list of device instance to be manage along with their descriptions and their low level details such as the address and the communication protocol type.

When DOG receives a command such as a turn off light command for a specific light device. Based on the device light type and address, DOG queries the ontology using SPARQL [Bonino 08b] and SWRL [ Bonino 10] to retrieve the allowed parameters and constraints. Once the command passes the syntactic and semantic validation, MVEL2 rules are then used to generate specific messages which are then sent to the real device.

EnTiMid

PervML: Pervasive Modeling Language

In [ Serral 08], the Pervasive Modeling Language is proposed to allow pervasive applications developers to deal with service composition and heterogeneity. The PervML is designed to easily specify and describe functionalities of pervasive devices and services in conceptual models which are technology independent. A structural model which provides information about the available instances of each service and the relations between services and their interfaces. And the interaction model which describes through sequence diagrams the interaction between services.

A system architect uses three models to bind the supported services to available physical devices. The with its supported capabilities. The high level description is then used to generate OSGi bundles to interact with sensors and actuators through network converters similar to the OSGi base drivers.

Binding

Uniform Language/Interface

The last category uses a uniform language or interface in order to represent devices and their supported capabilities in a universal representation format. Additionally, some of the proposed works in the literature define a universal content representations either per device type (printer, TV) or device domain (multimedia, printing).

We depict in the following three approaches using a universal language or interface. The refined drivers re-expose devices based on their supported operations type. For instance, since blinds also can vary their aperture and closure values, then they can expose dimming operations. Thus, the refined drivers represent the blinds using the Dimmable Switch API which is common for blinds and light devices.

HomeSOA

The same applies for the other device types, a UPnP refined driver mediasource searches for a UPnP Media Server and re-exposes it as mediasource using a media device interface and exposing its description and capabilities in a WSDL file.

The smart devices reification allows ubiquitous applications to search for devices by category of operations and interact with their services and actions using the Smart Device API. However, the smart API only reexposes the device description format using the WSDL. The description content is not resolved by the smart API. In other words, applications must know in advance the description content of the device to interact with it. For instance, an application must use the action SetTarget to interact with a UPnP light and the Switch action with a DPWS light, however, it will rely on the dimmable switch API to search for the lights and retrieve their descriptions.

UMB: Universal Middleware Bridge

The Universal Middleware Bridge [ A specific router in the UMB architecture is used to route message and events to the proxies. The proxy messages are forwarded by the UMB adapters to the real devices. The TechManager also reifies the virtual device as a real device. For instance a LightWS which represents an X10 light is also represented as a UPnP Light in the UPnP network.

Other similar approaches have been proposed in the literature such as the uMiddle [ Nakazawa 06,[START_REF] Nakazawa | uMiddle[END_REF] and the Web Services on Universal Networks [Yim 07] Frameworks.

Comparison & Discussion

In this section, we outline a comparison (see Table 7.1) between the previously overviewed approaches already classified in the three following categories:

Common Ontology used to capture all the semantics of a domain. The common ontology is built by an ontology expert. The interoperability between devices or services is possible only if the description is annotated by the concepts of the common ontology.

Abstract Model allows to manipulate services' interactions and behavior through high level concepts. Then using MDE techniques and transformation rules specified by designers, these high level concepts can be automatically transformed into an executable code which provides interoperability.

Unified Language/ Interface proposes a common description format and content to represent devices and their services through a unified description format and semantics. This category requires maintaining a mapping between various heterogeneous description and the unified proposed representation. The table between the local descriptions and the unified description is maintained by a domain expert which manually proposes the correspondences.

We present in Table 7.1 a comparison of the previously detailed interoperability approaches based on the following criteria. The various protocols used in each approach in order to interconnect the devices and applications, see Table 7.1. Then, for each approach we expose how the heterogeneity problem is tackled at the three layers detailed in chapter 6: the protocol stacks mediation, the presentation format and the content mediation. Additionally, for each layer, we outline the module used in each approach in order to resolve the heterogeneity problem. Furthermore, we depict the overall realization techniques used in each approach. We also take into consideration during the comparison the description annotation, whether (or not) the services' descriptions need to be manually annotated in order to resolve the heterogeneity problem at the presentation and content layers. Finally, we outline the human intervention in the adaptation and interoperability process.

Paolucci's approach along with the MySIM and PERSE middlewares use a common ontology to capture the domain semantics. This category shares the four following common points.

• A common ontology needs be specified by an ontology expert in order to capture the domain. Additionally, common semantics need to be found between relatively common concepts. In section 5.1.2, we outlined that the ontology development methodology is a difficult task, where multiple iterations are necessary until a correct version with coherent semantics can be found. Besides, building a common ontology for relatively complex devices can be more difficult and error prone. In the appendix B. • The common ontology is used to resolve the content heterogeneity of descriptions by representing concepts using a common syntax in a common taxonomy along with the relations between.

• The realization of the interoperability involves an ontology reasoning which classifies the concepts in a sort of a taxonomy along with the shared elements and properties. After the reasoning, comes the matching between the service categories, input and output parameters. Then, the adaptation process can be carried out through different techniques such as bundle generation.

• The service description must be annotated with elements used in a common ontology (annotation can also refers to other ontologies). Thus, a sort of a manual mapping is first applied by a human expert on the service description by employing the same concepts from the common ontology. Additionally, the common ontology might need to be extended and updated if the concepts of the service description are not available in the current version of the common ontology. The update process can also be difficult and requires resolving conflicts where two concepts might have crossed semantics.

Paolucci's [ Paolucci 02] approach deals with the service matching between web services having WSDL descriptions annotated with concepts from a common ontology. Thus, obviously there is no protocol or rep- theoretical and well defined methods to achieve adaptation and service composition. However, the realization of such adaptation is not available in their approach.

Additionally, both MySIM and PERSE tackle the QoS during service matching and adaptation. In our context, we discard this feature, since the actual device and service description do not provide any QoS information.

The second category uses an abstract model which allows experts to manipulate high level concepts to provide interoperability. This category shares the two following common points:

• In this category, the three approaches propose high level models to conceptualize the main entities of a domain. This high level of modeling allows designers to manipulate concepts without facing technical requirements and constraints. However, this high level conceptualization requires from experts instantiating each device to be used. The instantiation includes providing information such as the device address, the device type, its supported services and actions using a high level language. Thus, each new device appearing in the network cannot be handled unless its instantiation is present.

• Two kinds of transformation rules must also be specified by experts. The transformation rules from one model to another which correspond in our context to the behavior of the proxy. The proxy's behavior is usually specified using a high level behavior language. For instance, a proxy resolving the heterogeneity between two or more devices holds behavioral information on how to react to events and messages by applying redirection techniques or applying modifications on the messages content. Thus, the first type of transformation rules specifies the translation from one model to another mainly detailing the behavior of adaptation entities such as proxies. Such rules are usually written manually or generated using graphical tools using specific languages like ATL [ Jouault 08].

The other type of transformation rules is the code transformation rules which also needs to be specified by as Java classes and executable code. The transformation rules to automatically generate high level models into executable code are written once and used in each code generation. However, the transformation rules of behavioral rules to adapt from one model or service to another must be specified for each device or service type.

DOG [Bonino 08b] tackles the heterogeneity of home automation protocols such as the KNX, myOpen and X10 protocols. DOG uses specific OSGi network drivers similar to the base drivers detailed in section 3.3 which handle the protocol and presentation mediations. DOG uses the DogOnt ontology as a taxonomy like in the common ontology category and also uses it as an abstract model. DogOnt models functionalities such as dimming or switch apart and then it is associated to a device in the ontology. The human intervention occurs at three levels the ontology specification, the device instantiation ad transformation rules specification.

The realization is carried out by reasoning and specific message generation to provide interoperability between devices without specific bundle generation.

EnTiMid [Nain 08] represents devices like myOpen and Knx as plug and play devices (UPnP and DPWS) through models. They use base drivers like network adapters to resolve protocol heterogeneity. The service profiles generation handles the representation format. However, it is not clear if the content mediation is resolved, i.e. if a KNX light is exposed as a standard UPnP light with standard content and semantics. The EnTiMid realizes the adaptation by applying transformation rules from a model to model EnTiMid to UPnP and from high level to low level code using the bundle generation.

PervML [Serral 08] follows a different approach, where analysts and architectures specify through 6 steps, as shown in Figure 7.5, all the components to be used in an application. They specify using high level language, the devices, the services and actions used along with the behavior of each component involved in the adaptation. Then, using code generation techniques, the high level modeling is transformed into executable bundles.

However, the PervML approach does not take into account mobility and dynamicity of devices. It supposes that the devices are always present in a room such as a meeting room.

The final category uses a common language to expose the representation format and content in a unified semantics. This category shares the three following common points:

• Universal representation format and content are re-proposed by an expert in order to achieve a unified format and content. The proposed universal semantics tries as much as possible to capture semantics of the various device and service heterogeneous content.

• A mapping is maintained by an expert between the universal and the local content representations.

The mapping includes the device, service types and versions along with their supported actions names, input/output parameters names and types.

• The realization of the adaptation is based on the service discovery where specific modules discover the service or device appearance, then query the mapping table for its universal equivalence. Then a virtual device or a proxy exposes its description in the universal representation format and content.

HomeSOA [Bottaro 08a] handles heterogeneity between UPnP, DPWS, Bonjour and X10 protocols. Base drivers resolve stacks heterogeneity while another layer of refined drivers expose unified interface per device type and domain through smart devices. A smart device resolves the format representation by exposing the device and its capabilities using a WSDL description. However, services, actions and parameters names along with their types remain the same from the original description, thus the content heterogeneity is not resolved.

Once a smart device is discovered by an application, a test of the action names and parameters is needed in order to invoke the desired action. A UPnP light and a DPWS light will advertise a dimmable-Light interface, however, the actions names remain different, i.e. Switch and SetTarget. In [Bottaro 07b], the HomeSOA outlines a solution where a repository is used to store service matching. However, no technical and performance details are provided.

The UMB [Moon 05] approach also holds a mapping table between the universal and the local representation and content. On a local device discovery, an adapter discovers the device and registers a proxy using the universal representation and content. DomoNet [Miori 06] adopts a similar approach to resolve the heterogeneity between various protocols. Tech-Managers handle the protocol and presentation heterogeneity while the DomoML tackles the content universal representation. They proposed a set of standard profiles to represent devices and their supported capabilities.

The realization is carried out by the registration of a virtual device upon the real device discovery. A manual mapping table is maintained between the DomoML profiles and the profiles of other protocol devices. A virtual device is re-exposed by TechManagers in each sub-network protocol, i.e. a UPnP virtualized device appears as a DPWS, X10 device. In [ Miori 10], they outlined a limitation of this approach due to data synchronization problem between the various virtualized devices appearing in multiple networks.

Each of the three categories offers advantages and drawbacks. The first approach is based on a powerful semantic representation which captures the main concepts of the domain and the relations between. However, the common ontology need to be manually built by an expert. In section 5.1.2, we outlined that the ontology construction is an iterative and hard task. Moreover, plug and play devices can have complex representations, for The second category allows a manipulation of high level concepts by abstracting technical and implementation details. Thus, it allows to tackle the interoperability easily. A model to model transformation rules allow to specify the adaptation behavior between two services or actions. The main strength of this approach is the transformation of high level concepts into lower level executable code. The main draw back resides in the modeling process which can be composed of up to six modeling steps as shown in Figure 7.5. The modeling process includes an adaptation behavior specified manually by an expert to allow a correct adaptation between devices and services.

The final category proposes a unified representation format and content. It consists of maintaining a manually established mapping table between local and the unified proposed representations. The main drawback of this approach resides in the manual established mapping between descriptions. In this category, the main advantage consists in using a unified representation format and usually (in some works) a unified content representation.

We believe that an adaptation approach can be inspired from the three categories and benefits from their advantages. For instance, the ontology allows to semantically represent information of a domain and captures the relations between the expressed entities. Furthermore, the models offer the ability to manipulate abstract concepts independently from the technical details. Additionally, a set of transformation rules can be applied on the models. Such rules allow to go from one model to another on the same level of abstraction, or even from a high level to a low level concepts containing more technical details.

However, as mentioned before, building an ontology is a hard and an iterative task which can be error prone.

Therefore, reusing ontologies is recommended instead of building new ones from scratch [Noy 01, Wache 02].

Moreover, we also outlined the effort needed to propose the models and the transformation rules through a time consuming process and iterations, see Figure 7.5. However, another alternative, the ontology matching [ Euzenat 07] seems to enable detecting transformation rules and mappings semi-automatically through heuristic based algorithms. Thus, we provide in the next chapter, the ontology alignment techniques and algorithms that can be used to achieve a semantic interoperability.

Chapter 8 Ontology Matching

"It is of course unrealistic to hope that there will be an agreement on one or even a small set of ontologies ... . We will still need to map between ontologies" Ontologies have proved their advantages in the data mediation and integration from various heterogeneous sources such as databases and web pages. The main strength in using an ontology resides in its ability to capture semantics of a domain through a representation of the domain's main concepts and the relations between.

Moreover, the ontology reasoning allows to infer new information from what is already present. Thus, as long as the various information sources refer to the same semantic concepts in a common ontology, the integration problem can be resolved by reasoning and extracting information to achieve interoperability.

UPnP

DPWS IGRS Other

Global Ontology Moreover, real world experiences demonstrate that a central common information model capturing all the semantics is not realistic [Euzenat 07, Noy 04]. The integration problem exists in different domains where each representation specified by different experts is specific to a certain application or context. Furthermore, as mentioned in chapter 5, the knowledge representation is imperfect and captures only a part of the reality. Thus, each representation will be oriented and built by domain experts to capture a part of the reality. Additionally, a common ontology is usually expressed in specific concepts used in a predefined jargon from a specific domain.

As a result, different ontologies emerge and the need of re-usability and extension of ontologies lead to the ontology mapping which is proposed by the alternative known as the federated approach. ] agree on the fact that a representation will never be large enough to capture all the concepts and relations of a domain. Thus, a mapping between representations is needed at some point, which can be achieved through matching techniques. According to Euzenat [Euzenat 07], a "Matching is the process of finding relationships or correspondences between entities of different ontologies", the result of the ontology matching is the ontology alignment representing the correspondences between various ontologies. Such correspondences allow then heterogeneous ontologies represented by different semantics and jargons to be interoperable. The detected correspondences between ontologies represent the transformation rules to go from one ontology to another which leads to information integration from various resources. The challenge in the federated approach consists in finding correspondences between the pivot and each domain. Such correspondences can be detected using the alignment techniques which are semi-automatic and based on the syntax, the semantics and the structure [ Euzenat 07]. However, the alignment techniques are heuristic based, thus, a human intervention is still required to validate the detected correspondences. Since the mappings are independent, an update in a domain requires an update of the concerned mappings. If the pivot was updated then all the mappings have to be updated.

UPnP DPWS

The federated approach is already employed in the semantic web and medicals domains such as in [Elbyed 09], to integrate data from different heterogeneous information sources like data base schemes and web pages. It is also used in some works for peer-to-peer interactions and web services composition where an interaction model is proposed to define the constraints that services have to satisfy such as in [ Giunchiglia 06] and [ Robertson 06].

However, to our knowledge the federated approach is not yet explored to resolve interoperability between Plug and Play devices. We overview in this chapter, ontological matching techniques and tools proposed in the literature. In section 8.1, we outline different matching techniques. We provide in section 8.2, ontology alignment tools and frameworks, then, we conclude in section 8.3.

Matching Techniques

Various ontology matching techniques have been proposed in the literature [ Euzenat 07] to detect correspondences between ontologies. Such matching techniques are based on various elements like the syntax, the language, the structure, and the semantics. For conciseness, we overview in this section only three categories.

Syntactical techniques are usually based on the comparison between strings. Before the string comparison, the string-based techniques apply normalization procedures to improve the matching results. Such normalization includes for example converting upper case letters to lower case (or lower to upper), or accents removal. Once the normalization is applied then one of the following string-based techniques can be used.

• The Hamming distance [Hamming 50] counts the number of positions in which two strings differ.

• The SubString similarity measures if a string can be derived from another string based on the prefix or suffix. For example, LightDevice is more specific than Device, therefore the LightDevice concept might be considered as a subconcept of Device. However, this technique can also give good results with non related strings such as the article string which is a substring of particle.

• The Levenstein distance [Levenshtein 65] calculates the minimum number of characters to transform one string into another using character insertion, deletions or substitutions.

• The String Metric for Ontology Alignment (SMOA) [ Stoilos 05] technique is interesting in our context. SMOA is based on how computer researches and programmers choose descriptive names for the variables they use. Usually, they employ a mix of concatenating and trimming operations on strings such as numPages and numberOfPages. The general idea is to search for substrings then remove the common substring until none can be identified. For example, (numberOfPages, numPages) → (numberOf, num) → (berOf, ""). Then, the similarity value is based on the commonality between the two strings and the maximum common substring length. The commonality is calculated as follows:

Comm(S 1 , S 2 ) = 2 * i length(maxCommonSubString i ) length(s1) + length(s2) (8.1)
Thus, the SMOA similarity is the commonality value omitted from the difference value which represents the difference between the two strings. For example, the SMOA similarity value between numberOfPages and numPages reaches 0.91.

Other measures also exist such as the Jaro and the Jaro-Winkler based on characters' positions in the string. The Path distance can also be used to compare paths such as Device/Service/Switch and Device/Service/Button. This measure compares the strings' position and apply penalties when two strings are not at the same position in the path.

The string-based techniques can be used to rapidly detect relatively similar strings and mainly string extensions like a prefix or a suffix. However, if synonyms are used, for example, article and paper then it is obvious that such techniques will not detect any similarity. The linguistic techniques can be used to tackle synonyms and antonyms (down = up).

Linguistic The language based techniques are based on the natural language processing (NLP) techniques.

In [ Euzenat 07], they are classified in two categories: intrinsic and extrinsic.

• The Intrinsic methods are based on linguistic processing techniques such as the tokenisation which allows to extract tokens by eliminating punctuations, digits and blank characters.

• The Extrinsic methods use external resources such as the WordNet [Fellbaum 98] dictionary. Word-Net is considered as a large lexical database of English The linguistic techniques used with external tools such as WordNet allow to enhance the mapping between various entities based on the semantics instead of the syntax. However, the dictionaries like WordNet provide multiple senses for the same concept, thus proposing multiple potential matches. This can take place for example if the concept printer is present in an ontology O1 and the concepts device and equipment are both present in an ontology O2. Then, in general and according to WordNet both matchings are valid.

In order to provide more accurate results, the structural methods can be used.

Structural based techniques relies on the properties between concepts. Two types of structural techniques

exist in the literature. The internal structure based and the relational structure based.

• The internal structure based techniques rely on specific properties between the concepts. The data type property (string, int, float) can be used to determine which alignment to keep when two or more have relatively close similarity values. For instance, in Figure 8.4 since id and the isbn are both "uri", then a potential match is detected. It is obvious, that this technique need to be coupled with other techniques since the price and the year can also be considered as potential match.

The cardinality of a concept can also be used to chose and decide about close alignments, for example some concepts can have at most 3 relations with another specific concept while others do not have any restrictions. Other properties can also be taken into account such as the transitivity and symmetry of a relation in an ontology.

The internal structure methods can be used to determine or decide between two relatively close similarities. Used alone, such techniques are not efficient since two ontologies can have relatively similar internal structure but capturing different domains. Thus, the internal techniques must be coupled with the previous overviewed techniques. Matching Strategies Each of the previous categories has its own advantages and drawbacks. The syntactical techniques allow to detect prefixes, suffixes and common substrings. Thus, they can be applied to obtain a quick evaluation on the similarity between two strings. However, their precision is not accurate since they are based only on the syntax. For example, the two strings article and particle which are not related will score a high similarity value using the syntactic matching techniques since they share the same suffix.

The syntactic matching techniques can be reinforced with the linguistic based techniques. The intrinsic methods can be used to normalize the strings to be matched. For instance, the intrinsic methods will remove punctuations or accents. Thus, the normalization can help to achieve better results since the punctuations and other non essential characters are removed.

Another advantage of the linguistic techniques relies on the use of external dictionaries which allows to detect synonyms or antonyms. Thus, the linguistic techniques are more accurate than the syntactic techniques since their metrics are based on a linguistic dictionary. Even though, the linguistic techniques provide more accurate results than the syntactical ones, however, the time to access and query a dictionary must be taken into account when dealing with large ontologies.

The structural matching techniques exploit the structure of the ontologies and take into account the number of childes and other criteria such as the parameter types and properties relating the entities together.

Since each category has its own advantages and drawbacks, these matching techniques can be combined in a global matching strategy to improve the ontology matching. For example, the techniques can be used together in a parallel composition. Each technique (k) can be assigned a weight α k , which represents the accuracy to its calculated similarity value. Then, the similarity values of each techniques can be summed and normalized as follows.

Let Sim k (i, j) be the similarity result between the concept i from O 1 and the concept j from O 2 using the technique k. The average weighed similarity between the concepts is calculated as follows:

W eightedSum Similarity i,j = n k=1 (α k * Sim k (i, j)) with n k=1 α k = 1. ( 8

.2)

A sequential composition can also be used to combine the matching techniques.

Several systems apply a combination of different techniques, such as in the ROMIE system [Elbyed 09] where the string-based techniques are used with the WordNet dictionary and a structural mapping technique which enforces the similarity values between the concepts.

Filters can also be applied on the alignment to keep only those with similarity values above or equal to a predefined threshold.

We overview in the next section some ontology alignment tools and frameworks. A Leaf matcher is also supported to match instances.

Ontology Alignment Tools & Frameworks

The COMA++ [ Aumueller 05] is an enhanced version of COMA, ontologies written in OWL are supported.

Additionally, COMA++ provides a GUI to help users validate and edit ontologies. A taxonomy matcher can also be used where a specific domain taxonomy is used as an intermediary ontology instead of WordNet which holds a more general taxonomy. The ROMIE and the OMIE framework are not available publicly.

Alignment API [Euzenat 04, David 10] is an alignment tool which provides high level abstractions to manipulate ontologies. The Alignment API supports some basic string-based matchers such as SMOA, Levenshtein. It also provides an alignment format which encourages sharing and reusing alignments results.

The alignment format contains references to the two compared entities (belonging to the two ontologies)

and the calculated similarity value, see Listing B.3 in the appendix B.5. Additionally, the API provides alignment transformation from the alignment format into ontology OWL axioms. The transformation allows to add properties such as the equivalentTo property between two matched entities. The transformation result can be merged with the two ontologies used as an input of the alignment to produce a third ontology holding the matching between entities in a standard ontology language. Such merge allows other algorithms, techniques or reasoners to exploit the alignment result for example to retrieve information or apply some context or service adaptation.

The Alignment API is available publicly as an open source tool.

Conclusion

We provided in this chapter, the motivation behind the ontology matching to integrate information from various heterogeneous sources. Additionally, the ontology matching is considered more realistic than having an agreement on a common ontology.

We overviewed in section 8.1 three categories of the existing matching techniques available in the literature.

The syntactical techniques are string-based and can be used to detect a matching between a string and its extension such as the prefix, suffix and some concatenation between two or multiple strings. The SMOA is an interesting syntactical technique in our context since it is based on how computer scientists choose entity and variable names. Such names are usually a concatenation of two or more strings. The linguistic techniques have two categories, the intrinsic methods can be used for a pre-processing step since they can eliminate non usable information such as the punctuations and the digits and outputs tokens to be used by other techniques.

The second category of the linguistic techniques uses an external resource such as a dictionary, WordNet for example. The linguistic techniques allows to detect mappings between strings based on synonyms, antonyms and hypernyms. The structural based techniques represent the third category where the matching is based on the internal structure of the ontologies such as the datatype and other properties such as the cardinality. The structural techniques are based on the taxonomic nature of the ontologies (subconcept, superconcept) in order to detect alignments between entities. The alignment result can also depend on the alignment result of the sub-entities and the super-entities. Such techniques can be composed in order to produce a matching strategy.

We summarized in section 8.2, two alignment frameworks using various simple techniques and matching strategies. We also presented an alignment API which offers an alignment format and transformation techniques to exploit the ontology alignments.

In the next part, we detail our contribution and how these basic matching techniques and strategies assisted with rules can be used to provide interoperability between plug and play services. We detail in this chapter our approach to tackle the plug and play device and service interoperability. In the next two sections, we provide the motivation and the overview of our proposed approach. Our contribution is based on the automatic generation of ontologies capturing the device description which is presented in section 9.3.1. Then, we explain in section 9.4, our ontology matching strategy which consists of four steps to semi automatically find and validate correspondences between ontologies representing equivalent devices. In section 9.3.4, we present the global view of the architecture between the operator's platform and the end-user's digital home with regard to our proposed adaptation modules. And finally, we describe in section 9.3.3 the architecture of the automatic proxy generator and how we exploit the ontology alignment to generate specific proxies to provide a transparent plug and play interoperability.

Motivation

In this section, we overview the main motivation behind our approach with respect to the advantages and drawbacks of the various approaches proposed in the literature tackling the device and the service heterogeneity.

Thus, we briefly revisit the three types of the proposed solutions, detailed in chapter 7. Furthermore, we depict their limitations in the light of chapter 8 which proposes a semi automatic alternative to find correspondences between descriptions in order to provide interoperability.

The common ontology offers a powerful representation language to capture the semantics of a domain. It allows to represent the main concepts and the relations between. The inference capability and the taxonomic structure represent the main strength points in this category. Those two points allow to automatically extract information from what is already represented in order to provide semantic interoperability.

However, in chapter 5, we overviewed the ontology development methodology and showed that such task is tedious, error prone and requires multiple iterations until unified concepts can be found and agreed upon.

Furthermore, the proposed solution in this category requires a manual annotation of the service description exclusively with services from the common ontology. Thus, the description annotation is a manual mapping task, which relies on a first step where an expert must understand the semantics of the service and device description. Then, the second step consists in identifying the service, actions and parameters categories presented in the description and associate them with the concepts from the common ontology. The final step is the manual description annotation with the identified concepts from the common ontology.

Additionally, in chapter 8, we pointed out that a common ontology will never be large enough to capture a domain, therefore the common ontology must be extended to represent additional information included in some service or device description. Moreover, the new integrated concepts can hold crossed semantics with the already represented concepts transforming the ontology into an incoherent structure. Furthermore, at some point the integration effort becomes too tedious to integrate it manually since additional common semantics must be found. Though, the integration can include a reuse of an already developed ontology by other experts to add additional concepts.

Furthermore, information integration experts agree that at some point, the ontology mapping is needed.

In fact, various ontologies will be proposed and developed by experts choosing concepts from their domain specific jargon. In chapter 8, we presented the ontology matching techniques which can be applied to semi-automatically find alignments between ontologies.

Thus, from this category, we retain the following: ontologies represent a powerful tool to represent information. However, developing ontologies manually must be avoided, we should privilege instead reusing already proposed ontologies and explore semi-automatic alignment techniques.

Abstract model allows to manipulate high level concepts independently from the technology used underneath.

Thus, the interoperability can be tackled using high level concepts using a high level language which specifies the behavior of a proxy device concept in order to achieve an interoperability between services.

This specified behavior represents the transformation and the adaptation rules which allows two services to interact.

The main strength of this category resides in the automatic generation techniques offered by the model driven engineering domain which allows to go from high level abstract concepts to lower level executable code.

However, this category requires an expert intervention to instantiate the devices, services and actions from the abstract model and add additional information in order to be used once the code is executed.

Additionally, two types of rules must be specified by an expert. The high-to-low level transformation rules are specified once and used to automatically generate code. The other transformation rules are those which allow to go from one model to another, i.e. the translation and behavior rules which provide interoperability. The behavior and translation rules must be specified by an expert for each interaction between services and devices.

Thus, from this category, we retain its two major strengths: manipulation of high level concepts and the automatic generation of an executable code to provide interoperability. The manual specification of the interoperability behavior rules can be probably reduced through semi-automatic ontology alignment techniques.

Unified interface/language exposes the representation in a unified interface or language to resolve the heterogeneity. A mapping table is maintained between the local descriptions and the unified one. Thus, applications will transparently interact with the adapted devices by using the same interface. The main strength of this category is its unified language to resolve the representation format and content. However, the mapping table is manually established by an expert.

Thus, from this category, we retain the unified language representation to resolve heterogeneity. The manual effort for the table mapping between the local and the unified might be reduced probably through semi-automatic ontology alignment techniques.

Based on the previous analysis detailing the advantages and drawbacks of each category, we select the major strengths in order to establish our contribution which is overviewed in the next section.

Overview

In our proposal, we took the best of the three previously detailed categories to tackle the plug and play device interoperability. The Ontologies allow to represent information of a domain through defined relations and concepts. The second category uses models and transformation rules to automatically go from high level to low level. The third category offers interoperability using unified semantic concepts. The approaches proposed in the related work requires an expert intervention and a manual effort to resolve heterogeneity and achieve interoperability. Thus, we would like to minimize the expert intervention in the adaptation process.

Applications running locally or remotely that communicate only via UPnP Additionally, the number of UPnP device types standardized by the UPnP Forum is the highest among the other protocols. There is 26 1 UPnP standard device type while DPWS has only 3 standard types. IGRS relies on the UPnP standards while Bonjour has a limited standard devices like printers and the audio sharing devices i products. Additionally, as mentioned in section 2.4.3, the BroadBand Forum promotes the CWMP standard protocol [Broadband 10a] which can be coupled with a CWMP-UPnP bridge [Broadband 10b, Lupton 07] to offer a standard remote management solution adopted by many telecoms operators and device manufacturers [Broadband ]. Such solution allows to remotely administrate from the operators' platforms the end-users devices located in the digital home. Bridging a remote administration protocol with a local plug and play protocol offers the telecoms operators, the device manufacturers and the end-users a lot of advantages [Lupton 07, Broadband 10a, UPnP 11] such as the remote diagnostic, troubleshooting and update operations. As for the other protocols, DPWS can be coupled with WS-Management protocol promoted by the DMTF [DTMF ] , however it is so far mainly adopted by Microsoft and Schneider Electric. Bonjour and IGRS use proprietary administration protocols.

UPnP as a Common Application Layer

The UPnP is chosen as a common pivot in this thesis, however, our proposed solution is generic and another pivot can also be chosen instead. The ontologies resolve the description heterogeneity format in the M1 layer, therefore, in order to resolve the content heterogeneity, we apply ontology alignment techniques in order to find correspondences between two equivalent devices and their supported capabilities, as shown in Figure 9.3, the (c) arrows. The alignment techniques, as shown in chapter 8, are heuristics-based techniques, thus an expert intervention is required to validate the semi-automatically found alignments. Rules are also applied on the alignment to detect patterns in order to assist the expert during the validation. Additionally, the rules are used to automatically annotate the ontology with new properties expressing the detected patterns between the actions such as the composition relations.

Finally, once the ontology matching is validated between equivalent devices, the automatic code generation techniques allow to go from a high level independent technology representation in the M1 layer to an executable proxy code in the M0 layer, as shown in Figure 9.3, the (d) arrows. The proxy exposed as a standard UPnP device transfers the received invocations to non-UPnP devices.

The remainder of this chapter exposes our contributions which are two folds. An end to end architecture which provides the interoperability between plug and play devices and applications. The device matching which enables to semi automatically detect correspondences between equivalent devices.

Therefore, first, we present in section 9.3 our end to end architecture and detail each of its modules in four subsections. We outline in subsection 9.3.1 our automatic ontology generation. Then, we overview in subsection 9.3.2 the device matching which allows to detect correspondences between two equivalent devices.

In subsection 9.3.3, we describe the proxy generation from the previously detected correspondences. Then, subsection 9.3.4 shows the global architecture and how the proposed modules can fit in the digital home.

Then, we detail the device matching which relies on the ontology alignment and the different steps to achieve a valid matching between two equivalent devices. And finally, we end this chapter with some concluding remarks.

An End To End Architecture

We overview in this section our end to end architecture along with its three independent modules used in order provide a transparent plug and play interoperability between applications and devices. Figure 9.4 shows the modular architecture of our approach. The first module, "Ontology Generation" see Figure 9.4, handles the automatic ontology generation from the device and services descriptions. Then, the generated ontologies are used by the "Device Matching" module. This module takes two generated ontologies describing two equivalent devices type and applies heuristics based algorithms in order to find correspondences between the equivalent devices.

Since the device matching module is heuristics based, an expert intervention is needed in order to validate the detected correspondences. Finally, the validated correspondences are used by the "Proxy Generation" module in order to automatically generate an adaptation code.

OWL Writers

Plug and play devices announce their device description and capabilities on the network, see chapter 4. In section 3.3, we outlined how base drivers can represent real devices as local OSGi services on the platform. Such OSGi representation allows local services on the platform to interact with the real devices on the network. Thus, we designed software entities, the OWL Writers [El Kaed 10] as shown in Figure 9. We define a general device model [El Kaed 10] as shown in Figure 9.6, inspired from the UPnP Meta Model [UPnP ]. We use the concepts as follows: every device has one or multiple services, every service has one or multiple actions and each action has one or multiple input/output variables. A variable can also be directly related to one or several services. Additionally, each state variable has a type, a range and can have multiple default values. Thus, each OWL Writer automatically generates an ontology which holds two types of information, the taxonomic structure of concepts and the relations between them expressed in unified semantics. In each generated ontology, the general concepts Device, Service, Action, Parameter and the relations between the concepts found in the device description are represented in unified properties such as the hasService, hasAction, hasInput, hasOutput, etc. The ontology represents the information in a high level (M1 layer) using high level concepts independently from the devices format description. An OWL writer also generates ontologies from a device file description statically, i.e. without the need to detect the device on the network and retrieve its description. The OWL Writers can be placed on the clients' side, in digital homes, for example on a Set-Top-Box. Upon the detection of a new device, an OWL Writer checks if an ontology representing the detected device exists locally or on the operator's site. If it is not the case, then the OWL Writer generates an ontology and sends it to the operator. An OWL Writer can also be placed at the operator's site to generate ontologies statically based on the devices' descriptions files.

We overview next the device matching module.

Overview of the Device Matching

The device matching module takes as input two generated ontologies which represent two equivalent devices descriptions, for example, a UPnP and a DPWS lights. Then, the device matching applies the ontology alignment techniques presented in chapter 8 along with other proposed methods detailed in section 9.4. Such techniques are heuristics based and allow to automatically detect correspondences between the two ontologies based on several criteria such as the ontology structure, the semantic names and number of elements. Furthermore, since the alignment is heuristic based, a human intervention is needed to validate the detected correspondences. Figure 9.9 shows a simplified part of the device matching result applied on two lights (UPnP and DPWS) ontologies. Equivalent entities from both ontologies are now related through the equivalence property. Figure 9.9 shows a simple mapping relations where an entity is equivalent to only one another entity. However, other union mappings exist which are detailed in the device matching module in section 9.4. The equivalentTo property connecting the entities in Figure 9.9 represent the semi-automatically detected translation rules to go from one ontology to another. Now that the alignment is validated between the ontologies, the adaptation can start in order to provide an interoperability between the plug and play devices and services. The next section gives a detailed explanation about DOXEN and the proxy generation.

DOXEN

In this section, we detail our proposed DOXEN [El Kaed 11b] module, a Dynamic Ontology-based proXy gENerator which can be deployed on a SetTopBox for example. DOXEN automatically generates, a proxy for each valid equivalent non-UPnP device based on an ontology alignment. DOXEN takes as input an ontology alignment containing the equivalent devices' descriptions and the mappings between equivalent entities (devices, services, actions and variables). Based on the information in the ontology, DOXEN can generate using predefined templates, a proxy exposing a UPnP standard profile and able to interact with real devices having the same description as in the ontology alignment. The generated proxy is exposed as a UPnP standard device, it translates the action invocations to the real non-UPnP device.

The necessary steps carried out by DOXEN to generate and start a proxy are overviewed next. Then, section 9.3.3.2, points out the template filling principles to go from a high level representation contained in an ontology into a low level execution code.

Proxy Generation Overview

We describe in this section, the steps performed by DOXEN to automatically generate a proxy, as shown in Once, the ontology is explored, DOXEN starts the code generation by filling (5) the pre-written code templates based on the extracted information from the ontology alignment. Once the code files are filled and generated, DOXEN compiles the files (6) and builds a package (7) at runtime. Then it installs (8) and starts (9) the new generated package which corresponds to a UPnP proxy for the non-UPnP device. The started proxy requests the general device information (manufacturer, model, friendly name, ID, etc) from the non-UPnP device and publishes the same retrieved information during its UPnP format annunciation on the network. The end-user or the application identifies the proxy using the same type and the friendly name of the non-UPnP device. As soon as the non-UPnP device leaves the network, the proxy itself announces its departure from the network. When the non-UPnP device re-appears, DOXEN re-starts the proxy again which re-binds to the real device and re-announces its UPnP description.

The next step overviews the template filling steps ( 4) and ( 5) of the proxy generation.

Template Filling

DOXEN takes an alignment as an input, as shown in Figure 9.11, and uses predefined templates along with a compiler to generate executable code which corresponds to the proxy containing the behavior adaptation. The templates consist of several files containing the implementation structure of a proxy. They hold the general behavior of a proxy when it comes to registering its services and announcing its description. Moreover, once started, each proxy searches for a specific non-UPnP device type and binds with it in order to translate the received UPnP actions' invocations to this real device. Thus, the search and the binding mechanisms performed by each proxy are supported in the templates. Additionally, extracting the parameters values from the received invocations along with the translation between values and parameters constitute common adaptation functions for all the proxies. Thus, such functions are also already supported in the templates. Furthermore, the actions adaptation and composition represent a general shared attitude between the proxies. Therefore, their implementation is embedded in the templates. The high level API code also exists in a file template and is injected in the code files when an expert adds a behavior adaptation to the ontology.

Thus, since the templates contain the general behavior of a proxy, they only need to be filled with specific information, such as the name of the device to search for, the equivalent services, actions and parameters. The template also needs to be filled with the composition relations between actions (simple, union, sequential union) and an adaptation code if necessary.

Figure 9.9 shows a part of an alignment between two lights and their equivalent relations. Moreover, the ontology contains unified relations which allow to go through all the concepts such as the hasService and the hasAction properties. The relations between semantically compatible entities are either the equivalence or the patterns properties detailed in the previous section. Based on such information, DOXEN walks through the ontology starting from the UPnP device concept and extracts the valuable information such as the entity names, the equivalent mappings and valid composition, then fills the templates. Each filled template will generate a file containing code ready to be compiled, packaged and executed.

The template filling allows to go from a high level representation model (M1 Layer) specified with an abstract representation language into a lower level (M0 Layer) language which is the code ready to be executed. The realization of such transformation relies on the content of the template, the ontology visiting and the template filling. The high-to-low level transformation is accomplished once and used for every proxy generation. In fact, the high-to-low transformation constitutes the main strength of the Model Driven Engineering domain promoting the "write once and use anywhere" slogan.

The next section overviews the global architecture between the operator's platform and the end-user's digital home with regard to our proposed adaptation modules.

Global Architecture

The previously detailed modules are used in the global architecture as shown in Figure 9.12. The OWL Writers detailed in section 9.3.1, automatically generate ontologies from devices' descriptions. Moreover, OWL Writers operate in two modes, either they are deployed in a network and when a new device appears, the OWL Writer discovers the Plug and Play device and then generates an ontology reflecting its description. The other mode is more static, using a shell command, an expert gives the device file description as an entry to the OWL Writer.

Thus, an OWL Writer can be deployed at the end-user's digital home. The OWL Writer will detect new devices and generate their correspondent ontologies when a non identified device is discovered. Then, the generated ontologies are uploaded to the operator's site. The OWL Writer can also be placed at the operator's site and will generate ontologies on the expert demand.

The Device matching explained in section 9.4 is performed on the operator's site through the ATOPAI framework. The expert retrieves from a database two OWL Writer generated ontologies representing two equivalent plug and play devices. The expert triggers the automatic alignment by selecting a matching technique.

Then, the expert validates the alignment result though the ATOPAI's GUI. The valid alignment can then be deployed automatically or on demand on the home network in order to be used for the proxy generation.

DOXEN can be deployed on a set top box for example, it generates a proxy from each valid alignment which contains two devices profiles along with the mapping between. The generated proxy is then installed and started in order to resolve the heterogeneity between the two profiles. The proxy announces itself as a UPnP device and implement a client to interact with the non-UPnP equivalent Device. The proxy generation is triggered when an application is searching for a UPnP device, a UPnP printer for example and there is an available DPWS printer instead. DOXEN can also trigger the proxy generation when the DPWS printer appears on the network. DOXEN can also be installed at the operator's site instead of deploying it on the client's site. Then, the proxy generation will take place on the operator's site, the generated proxies will be remotely installed in home networks when a new device appears or on applications demands. When the DPWS printer leaves the network, the proxy receives the departure announcement and then sends its equivalent UPnP departure announcement on the network and then goes into an "OSGi" installed bundle state, see section 3.3. The generated proxy contains the adaptation behavior to provide interoperability between two equivalent devices profiles. Thus, the proxy can be exposed as a UPnP printer and will interoperate transparently with any UPnP application searching to invoke a printer with standard UPnP actions.

The following section details the device matching which constitutes the second fold of our contribution.

Device Matching

The device matching as shown in Figure 9.14 is dependent on the ontology generation step. It takes as an input two automatically generated ontologies by the OWL Writers. The device matching allows through four major steps to semi-automatically detect translation rules between two devices' descriptions. We rely on the semiautomatic ontology alignment to detect correspondences, instead of specifying the adaptation rules manually.

The detected adaptation resolve the content heterogeneity and allows to go from one description device to another.

Figure 9.14: The overall major steps of the process

The device matching detects compatibility between two equivalent descriptions like two printer devices. Such compatibility is in fact a valid matching between the device services, actions and parameters. In other words, if the device matching is valid, then using a particular proxy, the device can be substituted with its equivalent announcing a different description and using a different protocol. The proxy will handle the redirection and the invocation translation with the required parameters. However, since the matching is heuristics based, an expert intervention is needed to validate and edit the semi-automatically found correspondences. In this second step, "Align Validation (2)", detailed in section 9.4.2, the expert refers to devices specifications to update and validate the matching. After the expert validation, the alignment file containing tuples and similarity values is transformed into an OWL ontology. In other words, the left and right entities are currently related with the owl:equivalentTo property. Then, the left, right and the alignment ontologies are merged into a single ontology containing the equivalent properties. The merged ontology contains the entities from both ontologies and the relations between.

The third step applies "Rules (3)" to automatically detect patterns between the equivalent correspondences.

The rules detect compositions between actions. For example, an action on a device can be equivalent to the union of two other actions on another device. For each detected pattern between entities, the ontology is automatically annotated using composition relations relating the concerned entities.

Moreover, a matching between two actions is not enough to presume that they are compatible, their input/output parameters also need to be considered. Therefore, the patterns explained in section 9.4.3, detect compatible actions based on their input/output matched parameters.

The fourth step, "Alignment Adaptation (4)", is the final and optional step of the device matching. Actually, not all the correspondences between actions are simple and can be resolved only by linking the entities. The adaptation might needs data conversions and loops, for example, a temperature conversion, • C = (5/9)( • F-32).

Thus, we overview in section 9.4.4, how the expert specifies an adaptation behavior using a high level API.

The valid ontology is then used as an input for the proxy generation and the device and service adaptation.

We detail next the four major steps of the device matching.

Ontology Alignment

The ontology alignment step is applied on two ontologies O 1 and O 2 which abstract two devices description, for example the UPnP and DPWS Light ontologies shown in Figure 9.8. The ontology alignment uses the following matching strategy, shown in Figure 9.16, to automatically detect correspondences between equivalent services, actions and parameters. The matching strategy consists of four steps, it takes two generated ontologies conformed to the meta model and detects correspondences between. We detail next each step of the matching.

Basic Methods

The first step of the alignment relies on the basic matching methods described in chapter 8. Thus, for each method k used, such as SMOA, Hamming, Leveinstein, we obtain an output similarity matrix Sim M atrix K shown in (9.1) between n (resp. m) entities of the ontology O 1 (resp. O 2 ). However, as mentioned in chapter 8, the string-based techniques can be used to detect relatively similar strings and mainly string extensions like a prefix or a suffix. Table 9.1 shows the similarity results returned by five string-based matching techniques. It is obvious that such techniques cannot detect antonyms (Up = Down)

Sim M atrix K =      SimK 1,1 • • • SimK 1,n . . . . . . . . . SimK m,1 • • • SimK m,n      (9.1)
or synonyms (Clock ≡ Timer) and detect false matches between two similar strings (Particle, Article).

Method δ 1 ("SetVolumeUp", "SetVolumeDown") δ("Clock", "Time") δ("Particle", "Article")

1-Ngrams 0.7 0.0 0.9 The SMOA matching technique relies on some patterns used by programmers when it comes to choose a descriptive names for their variables. The chosen names are usually a set of words trimmed and concatenated together. For example, the SetVolumeUp or the SetVolumeDown actions names. In the SMOA technique, the biggest common substring is searched between the two strings. Once found it is removed. Then, the search continues for the next biggest common substring between the two strings until none is found.

Definitely, in our context the device and service description contains trimmed and concatenated names.

However, the SMOA technique is unable to detect synonyms, antonyms and cannot differentiate between two words semantically different having similar strings.

Thus, we propose the SMOA++ [Mora 10] also based on the substring search. The SMOA++ [Mora 10] consists of two major steps:

• Tokenization: The first step consists of separating the strings into tokens based on the WordNet dictionary.

The SMOA++ searches for the biggest substring that can be identified in WordNet in each of the strings.

Once the substring is identified, it is removed. Then, the search continues until no substrings can be identified. For example, the tokenization applied on the action name CreateJob returns the following three substrings based on WordNet: {Create, eat, Job}. However, since eat is a substring of Create then the token eat is not considered.

The Tokenization is given more formally in the following definition:

Definition 1 (T okenization SM OA++ ). Let a string S composed of a set of substrings {s 1 , s 2 , . . . , s n }.

The Common Sub function between two strings returns the string having the biggest common substring. If no common substrings is found, the Common Sub returns both strings. The Common Sub can be defined as follow:

3 SMOA++ is proposed by Felipe MORA during his final year project under my supervision

Common Sub (s1, s2) =        {s2} if s1 ⊂ s2 {s1} if s2 ⊂ s1
{s1, s2} else.

The List W ordN et (S) function keeps only substrings of S which can be found in WordNet. As shown in Figure 9.18, the equivalent substrings will be assigned a similarity value of 1.

We chose the string equality technique, however other basic techniques can also be applied.

Then, we continue the matching for the non-equivalent strings. We query WordNet for their synonyms and antonyms. If the two strings are synonyms (Clock ≡ Time) then we assign the similarity value δ Syn = 1 and if the they are antonyms (up = down), then the whole antonym similarity δ Ant is set to zero. The search for the antonyms is first applied on the two substrings, for example up and down. If no antonyms are found, we extend the search by retrieving the synonyms of each substring, i.e. the synonym list of up and the synonym list of down. Then we find antonyms between the two synonyms list. This allows to extend the antonyms search.

The matching is given more formally in the following definition:

Definition 2 (SMOA++ Matching).

δ StrEqu (S 1 , S 2 ) =    1 if S1 = S2 0 else δ Syn (S 1 , S 2 ) =    1 if S1 and S2 are Synonyms 0 else δ Ant (S 1 , S 2 ) =    0 if S1 and S2 are Antonyms 1 else
Thus, with two strings composed of a set of substring S 1 = {s 1 1 , s 1 2 , . . . , s 1 n } and S 2 = {s 2 1 , s 2 2 , . . . , s 2 m } from the tokenization step. The SMOA++ similarity function is given by the following equation:

δ SM OA++ (S 1 , S 2 ) = 2 * n,m i,j=1 δ StrEqu (s 1 i , s 2 j ) + δ Syn (s 1 i , s 2 j ) * n,m i,j=1 δ Ant (s 1 i , s 2 j ) length(S 1 ) + length(S 2 ) (9.2) 9.4.1.

Filters

The result of the previous step, as shown in the equation 9.1, is a matrix holding n * m similarity values between n concepts from the ontology O 1 and m concepts from the ontology O 2 . The aim of the alignment is to find translation rules between equivalent types in order to replace for example an action invocation with another equivalent one.

Therefore, we apply two types of filters. The first keeps the best alignments between entities. For example, the filter only keeps the best similarity value between the four pairs of Figure 9.17. In other words, we apply a one-to-one cardinality mapping, this allows to pick up only the best pair of the matched entities between a concept of the ontology O 1 and the m concepts from O 2 . Thus, the first filter is a maximum function which is applied on the matrix as follows:

Sim M atrix = max 1≤k≤m (Sim k,1 , Sim k,2 , . . . , Sim k,n ) (9.3)
The similarity values of the Sim M atrix , obtained after applying the first filter will be reused in the step 4 to improve the similarities.

The second filter is a "structural type" filter, we go through the alignment and keep only the correspondences between entities belonging to the same concepts types, (device-device), (service-service), (action-action) and

(parameter-parameter).

The two following steps consist in enhancing similarity values based on the two ontologies structures.

Similarity Propagation

In this step, we apply a "down-top" similarity propagation on the ontology structure as shown in Figure 9.19.

For example the similarity of the two services depends on the similarity of their matched actions if it is higher than a predefined threshold propagation value t P . Furthermore, the actions' similarities is dependent on their input/output parameters' similarities. The function to propagate the similarity between two services S 1 and S 2 , having n t P matched actions with a similarity value greater than a threshold t P , is given by the following equation. We normalize the similarities which explains the division by n t P and 2:

if δ(S 1 .Action i , S 2 .Action i ) ≥ t P / t P ∈ R + [0, 1], δ P ropagated (S 1 , S 2 ) = δ(S 1 , S 2 ) + ( n i=1 δ(S 1 .Action i , S 2 .Action i ) /n t P ) 2 . (9.4)
More generally, each similarity between two entities of a certain level (device, service, action or parameter) depends on its sub-levels similarities. In other words, the (device-device) similarity depends on the similarity of their matched services similarities. A service similarity depends on their actions' similarities and finally the actions similarity is dependent on their parameters' similarities.

Enhancer

This step is optional, it uses the similarity results of the first step before applying the "structural type" filter presented in section 9.4. Figure 9.19: An illustration of the similarity propagation For example, in Figure 9.20, a service S 2 has a high similarity with the action A 1 of the service S 1 , and there is also a similarity between the two actions A 1 and A 2 . However there is no alignment detected between the two services, S 1 and S 2 . Since there is a match between the two actions (same-level) and between the actions and the service (crosslevel), we can suppose that there is a possible relevance between the two services. Thus, we propose to enhance the similarity between the two services based on the similarities of the same and the cross levels if their values are equal or higher than a threshold t E , i.e. between the actions pair and the service-action pair. We assign a confidence value α to the same-level similarity and the confidence value β to the cross-level similarity with, α, β ∈ R + [0, 1] and α + β = 1. Thus, the enhancer function is provided by the following equation:

if δ same-level (A 1 , A 2 ) ≥ t and δ cross-level (A 1 , S 2 ) ≥ t E / t E ∈ R + [0, 1], δ Services Enhanced (S 1 , S 2 ) = δ Services (S 1 , S 2 ) + α * δ same-level (A 1 , A 2 ) + β * δ cross-level (A 1 , S 2 ) 2 (9.5)
The equation 9.5 can be re-applied several times depending if S 1 and S 2 have multiple (action-action) and The alignments are based on heuristics, thus, an expert intervention is required to validate the found correspondences. We present next the alignment validation tool.

Expert Alignment Validation

Unfortunately, the ontology matching is still imperfect and heuristic based, even when adding a semantic dictionary such as WordNet, false matches can still be proposed as potential matches. Therefore, an expert intervention is needed to validate the mappings. The expert relies on the standard device specifications and manuals to validate the correspondences. Thus, the alignment validation is the second step which follows the ontology alignment step, as shown in Figure 9.21. The ATOPAI framework is hosted on the operator platform, the ontologies can be statically generated by the OWL Writers then loaded in ATOPAI by the expert. The ontologies can also be uploaded by the OWL Writers deployed on the end users' sites upon a new device type detection. ATOPAI allows to load any ontology conformed to the predefined meta model. Each entity is annotated with its type (D:Device, S:Service, A:Action, etc) as shown in Figure 9.22. To calculate a mapping, the alignment method is chosen along with the trimming threshold t value (alignments above t are kept) and a semantic dictionary if needed.

Figure 9.22, shows an automatically calculated alignment between a UPnP and a DPWS light using SMOA [ Stoilos 05]. The similarity value is shown on the line between the entities. The alignment result is expressed with an alignment format [ Euzenat 04] with cells containing tuples (leftEntity, rightEntity, similarity). The result can be saved to an RDF file which can be loaded later. The expert can also saves the found tuples in a data base which can be queried to enhance a similarity value of a tuple if the saved value is higher then the current detected similarity.

The expert fixes the alignment by dragging lines between same type entities (device-device, service-service, etc). He can also add an alignment by only selecting two entities and using the Add Alignment button. ATOPAI assigns to the alignment added by the expert a similarity value set to 1 to avoid removing a valid alignment when trimming. To remove an alignment, the expert selects the line between the two entities and removes the alignment. All the alignments edited by the expert are shown in blue, the others are detected automatically.

Pattern Detection

The matching between entities detects only one-to-one cardinality mapping as stated in section 9.4.1.2. In simple cases, we have one-to-one mapping actions, such as in Figure 9.23 where the SetTarget (boolean: true/false) is equivalent to the action Switch (String: ON/OFF). However, the adaptation between devices can require a composition adaptation. For example, on a device, an action can be equivalent to the composition of two or more other actions. Thus, the pattern detection step follows the alignment validation as shown in Figure 9.24. In order to detect such compositions, we apply patterns on the previously expert-validated alignments. The pattern recognition as defined by Gonzalez [ Gonzalez 78], is "a classification of input data via extraction of important features from a lot of noisy data". We detailed in chapter 5, how rules combined with basic logics can be used to extract information from what is already represented. In our approach, we use patterns to classify the matching between the actions: equivalent actions and complex compositions. The applied rules will detect patterns based on the alignment between entities. The rules will automatically annotate the ontology using specific properties between the actions to describe their composition type. Such composition information is exploited during the code generation in order to generate specific composition adaptation code between the affected actions.

However, annotating a composition type between actions is not enough to decide if the composition is valid, their input/output parameters need to be considered. For simple devices, with actions having small number of input/output parameters, a manual checking of the compatibility can be carried out manually by an expert. Therefore, another set of decision rules are applied in order to detect valid matchings between actions compositions based on their parameters. This allows the expert to focus only on non valid ones in order to check if a potential adaptation or conversion can resolve the incompatibility. For example, the expert can make two actions valid by adding default values or writing high level adaptation code by referring to the specifications.

Thus, the patterns are used for three purposes. First to automatically annotate the ontology with union and sequential compositions of actions based on the expert validation. Second, to automatically propose to the expert non valid mappings in order to explore possible adaptation mechanisms such as setting default values and adding high level adaptation code. And the annotation will be exploited during the proxy generation in order to include in the proxy the adaptation behavior trough composition and redirection.

The pattern detection is an automatic operation triggered initially by the expert through ATOPAI once the expert's validation is accomplished.

We detail next, the three major categories of patterns used in our approach. The patterns are detected by applying rules on the ontology [El Kaed 11a].

The Patterns

In this section, we present the patterns used to automatically annotate the ontology with union and sequential compositions of actions based on the expert validation. We also provide the formal definition of each pattern in the first order logics along with the some rules expressed in the Ontology Pre-Processing Language [ Šváb Zamazal 10] overviewed in chapter 5.

The following paragraph presents some general definitions and notations used in the definitions and the equations employed later in this section.

Definition 3 (General symbol definition).

Consider O an ontology, A a set of actions of the ontology O and P a set of input/output parameters.

• ∀f, g ∈ O, f ≡ g ⇐⇒ (f, owl:equivalentTo, g)

• ∀f ∈ A, ∀x, y ∈ P f , y = f (x) ⇐⇒ f hasInput (x) and hasOutput (y) y = f (φ) is used if the action f has no input or the input is not considered. For simplicity of illustration, two equivalent input/output parameters will be represented with the same name but with the index of their action names. For example, x f is a parameter of the action f.

1. Simple Mapping is a pattern between two actions, having a one-to-one simple mapping. This property has the following three sub-properties:

(a) Simple Mapping Input: two actions are related with a Simple mapping input, if there is at least one matching between their input parameters. For example, Switch(Power) ≡ SetTarget(Target).

More formally, the Simple Mapping Input definition can be specified using the first order logics as follows:

Definition 4 (Simple Mapping Input). ∀f, g ∈ A, ∃x f ∈ P f , x g ∈ P g / f (x f ) ∧ g(x g ) ∧ (f ≡ g) ⇐⇒ f Simple M apping Input g
To detect the Simple Mapping Input pattern, we apply the following rule shown in Listing 9.1 written in the Ontology Pre-Processing Language [ Šváb Zamazal 10] already overviewed in chapter 5. Once the rule is applied on the ontology, it will add a relation between the concepts (classes) of the ontology. For example, the rule will link the detected pair of entities Switch and SetTarget with the object property Simple Mapping Input. For conciseness, we detail one example written in the OPPL language per pattern category. 4 ? f e q u i v a l e n t T o ? g , ? x f e q u i v a l e n t T o ? xg ,

BEGIN

6 ADD ? f s u b C l a s s O f S i m p l e M a p p i n g I n p u t some ? g 7 END;

Listing 9.1: The OPPL rule applied to detect the Simple Mapping Input Pattern

The first line of the Listing 9.1 declares four variables of type class. Line 2, selects any class related with another class in the ontology through the property has UPnP Input. Line 4, specifies that the selected classes f and g must also be related with an equivalentTo property. The same applies to the selected classes x f and x g . Line 6, annotates the ontology with the property Simple Mapping Input between the selected classes f and g verifying the previous criteria specified in the select part of the OPPL rule.

In other words, the rule specified in the Listing 9.1, will select a class f (respectively a class g) having as UPnP Input (respectively DPWS Input) a class x f (respectively a class x g ). Moreover, the selected classes f and g must be equivalent in the ontology. The classes x f and x g must also be equivalent. The found classes verifying the previous criteria will be annotated with the property Simple Mapping Input.

(b) Simple Mapping Output: two actions having at least one matching of output parameters. For example, (Power=GetStatus()) ≡ (Status=GetStatus()). More formally, the Simple Mapping Output definition can be specified as follows:

Definition 5 (Simple Mapping Output). ∀f, g ∈ A, ∃y f ∈ P f , y g ∈ P g / (y f = f (φ)) ∧ (y g = g(φ)) ∧ (f ≡ g) ⇐⇒ f Simple M apping Output g (c)
Simple Mapping Input Output: is the association of the two previous patterns. It is defined as follows:

Definition 6 (Simple Mapping Input Output). ∀f, g ∈ A,

(f Simple M apping Input g)∧(f Simple M apping Output g) ⇔ f Simple M apping Input Output g 2. Union Mapping : this pattern occurs when an action is equivalent to the composition of two or more actions with no predefined order. There is three categories of the union mapping. We rely on the previously simple detected patterns in order to identify the Union Mapping pattern. For conciseness, we only define the Union Mapping Input, where the input parameters are considered. However, we also detect patterns for the Union Mapping Output based on the output parameters.

(a) One-to-N: At least three actions are involved. An example of this pattern is shown in Figure 9.25 where the SetClock is equivalent to the union of the two actions SetDate and SetTime.

SetClock(T ime, Date) Union 1 to n    SetDate(Date)
SetT ime(T ime)

We provide in Definition 7, One-to-2 Union Mapping, however the definition can be generalized to n actions.

Definition 7 (Union, One-to-N). ∀f, g, h ∈ A / (f Simple M apping Input g) ∧ (f Simple M apping Input h) ⇐⇒ f U nion Input 1 → n {g, h}
The Union 1 to n Input pattern can be detected through the following OPPL rule shown in Listing 9.2. Applied on the ontology of Figure 9.25, the rule will add the property Union 1 to n pattern between the three actions SetClock, SetDate and SetTime. Listing 9.2: The OPPL rule applied to detect the Union 1 to n Input Pattern

The Union 1 to n Output pattern can also be detected based on its output parameters as shown in the following example. The action f is equivalent to the actions g and h along with their parameters y and z. The Union 1 to n Output rule is based on the Simple Mapping Output.

{y f , z f } = f (φ) Union 1 to n Output    y g = g(φ) z h = h(φ)
The Union 1 to n Input Output pattern is based on the two previously detected patterns.

(b) N-to-M: The Union n to m patterns can be found on devices where two or more actions are equivalent to the union composition of multiple actions. Such union is detected based on their equivalent parameters as shown in Figure 9.26. The action f is a union of the actions h and k. However, to invoke the action h, the input parameter b h is needed which can be retrieved only if the action g is invoked. Thus the N-to-One union mapping and the N-to-M represents a sort of a deadlock union.

f (a f , c f ) g(b g ) h(a h , b h ) k(c k )
Figure 9.26: N-to-M Union Mapping

The two actions f and g must be both invoked to proceed with the adaptation and the invocation of the equivalent actions.

These two patterns make the service adaptation indeterministic. When the generated proxy receives an f invocation, it has two options. The first, is to wait for a lapse of time and adapt other received actions in the meanwhile until the proxy receives the invocation of the action g. Then, the proxy can invoke the actions h and k on the equivalent device.

The second option consists in the following, when the action f is received, the next received action invocation must be g in order to apply the adaptation and invoke h and k. If another action is received instead, the invocation redirection of h and k are ignored.

The N-to-M pattern in general cannot be adapted, since the invocation is unpredictable. The two overviewed options provoke undesirable behavior since the application might be expecting an immediate return value after an action invocation in order to proceed with a predefined ubiquitous task. In the first option, the application can wait indefinitely before receiving a return value or detecting the execution of the desired action in the ubiquitous environment. The second option, ignores the adaptation if the two actions are not invoked one after another. Thus, the proxy might not execute the desired task at all. Additionally, there is no guarantee that the application will invoke both actions f and g since the application might just need to execute the action f or g separately.

Despite of these drawbacks, when the N-to-M union patterns occur, the ontology is still annotated with the composition relation Union n to m. However, its is up to the expert to decide whether to consider the detected union as valid or to ignore it. His decision is based on the device type and the dependency degree between the actions f and g. A high dependency between the two actions makes this pattern valid for composition and adaptation. Such high dependency between two actions is found for example on the DPWS standard printer where the action CreatePrintJob must be followed up by the invocation of the action AddDocument or SendDocument in order to accomplish the printing task 4 . Now consider the following, we chose the DPWS devices as a pivot instead of UPnP and we generate a DPWS proxy which exposes any UPnP printer as a DPWS printer and is able to interact with the printer. Thus, in this case the proxy can carry out the adaptation since the DPWS application interacting with the proxy will invoke the two DPWS actions on the proxy, the CreatePrintJob and the AddDocument. Thus, the proxy receiving both actions can extract the required values and invoke the equivalent UPnP action CreateURIJob on the UPnP Printer. Thus, this pattern can be valid, depending on the device and service specifications' and behavior.

Therefore, the detected patterns are annotated in the ontology and it is up to the expert to chose according to the device specifications and manuals if an adaptation can apply by setting default values or special configurations. We detail how the expert can carry out such adaptation operations using a high level language in section 9.4.4.

(c) N-to-One is a specification of the N-to-M union, therefore, the same drawbacks and constraints apply.

3. Sequential Union: is a union mapping with a predefined order between actions. There is also three categories for a sequential union. In order to detect a sequential union, the rules rely on the previously union detected patterns. The only difference between a sequential and the previously detailed union pattern resides in the parameters. In a sequential union, an output parameter of an action is actually an input parameter of another action. Therefore, an invocation order is established between the two actions.

(a) One-to-N: At least three actions are involved in the One-to-N sequential union as shown in the following:

y f = f (x f ) Sequential 1 to n    (1) r g = g(x g ) (2) y h = h(r h )
The action f has an output parameter y f and an input parameter x f . The action f shares an equivalent input parameter with the action g. The action f also shares an output parameter with the action h. Thus, the action f is related with a Union 1 to n property with the actions g and h. However, the action h has an input parameter r h which is the output parameter of the action g.

Therefore, the action adaptation requires first the invocation of the action g, then, waiting for its return value r g . Furthermore, invoke the action h having as input the parameter r h . And finally, return the y h value. 9.2 shows a part of the input parameters used by the three actions. We detail in chapter 11 the mapping parameters between the three actions.

In order to detect the sequential union mappings, we first detect the union mapping then the sequential dependency (has Next) between actions of the same device. has Next is a binary relation defined as follows:

Definition 8 (has Next). ∀f, h ∈ A, f has Next h ⇐⇒                (1) f = h and Output(f ) ∩ Input(h) = ∅ and Output(h) ∩ Input(f ) = ∅ (or)
(2) ∃ g ∈ A /f has Next g and g has Next h

We detect the hasNext pattern by applying the three following OPPL rules:

The rule shown in Listing 9.3 allows to detect the dependency between the actions based on the equivalence between the actions and the equivalence between their input and output parameters.

Lines 2 and 3 allows to detect the unions and the lines 5, 6 and 7 selects the actions having inputs and outputs. And finally, lines 7 selects the actions with equivalent parameters.

The second rule is used to detect a cycle between two actions, (g has Next h ∧ h has Next g). In such a case, the expert validating the ontology is notified and the cycle is broken by removing the has Next properties, as shown in Listing 9.4. The third category of rules is used to detect the transitive clause. (3) Recursively compute a topological ordering of (List-1) (3a) Append this after f in the Ordered List

Definition 9 (Sequential Union 1 to n). ∀f, g, h ∈ A, / f Sequential 1 to n (g, h) ⇐⇒ (f U nion 1 to n g) ∧ (f U nion 1 to n h) ∧ (g hasN ext h).
The following rule shown in Listing 9.5 is applied to detect the sequential union.

? f : CLASS, ? g : CLASS, ? h : CLASS, ? x : CLASS, ? y : CLASS, ? r : CLASS

SELECT ? f s u b C l a s s O f U n i o n 1 t o n some ? g , ? f s u b C l a s s O f U n i o n 1 t o n some ?h , ? g s u b C l a s s O f hasNext some ?h
WHERE ? f != ? g , ? g != ?h BEGIN ADD ? f S e q u e n t i a l 1 t o n some ?h , ADD ? f S e q u e n t i a l 1 t o n some ? g END;

Listing 9.5: An OPPL rule applied to detect a Sequential Union Pattern (b) The N-to-One and the N-to-M sequential unions categories share the same drawbacks as the

N-to-M unions.   (1) a g = g(x g ) (2) y h = h(a h )   Sequential to One y h = f (x h )
The sequential union N-to-One in general is hard to adapt. Consider an application invoking a g = g(x g ), it expects a return value a g . This value is then used as an input when invoking y h = h(a h ).

Thus, the proxy carrying out the adaptation cannot generate a value of a unless it is well defined in a certain context or it is a random number.

However, if the parameter a is the JobId parameter returned by the DPWS standard action Cre-atePrintJob, then, in this case, the proxy can be configured to return any number and wait for the application to invoke the action h so it can return the y value.

Thus, the detected patterns are annotated in the ontology and it is up to the expert to choose according to the device specifications and manuals if an adaptation can apply by setting default values or special configurations. We detail how the expert can carry out such adaptation operations using a high level language in section 9.4.4.

In this section, we detailed how the patterns can be applied on the ontology in order to detect union and sequential compositions. However, a detected composition between two actions is not enough to presume that they are compatible, their input/output parameters equivalence and cardinality also need to be considered.

Therefore, we detail in the next section how we detect compatible actions and point out the non valid ones for a potential adaptation.

The Matching Concepts

We provide in this section, the matching concepts which allow to automatically decide if a simple or a union mapping is valid or might be adapted by an expert. The matchings concepts are based on the equivalent input/output parameters of the actions' pattern mapping type previously detected using rules. The detected pattern will provide information when specifying the matching concept and counting the number of equivalent parameters. For instance, a union mapping refers to more than one action, thus more parameters need to be taken into account.

In order to detect a valid mapping between two sets of actions, we take several criteria into account such as the number of equivalent input/output parameters between the actions. Moreover, an important criterion relies on the number of the satisfied entry parameters values required by an action. For example, consider the following SequentialUnion 1 to n composition between the actions f, g and h:

c f = f (a f , b f ) SequentialUnion 1 to n    c g = g(a g ) d h = h(c h , b h )
The mapping between the two sets of actions {f } and {g, h} is valid even if the number of parameters is not the same. The action f is not expecting a return value d therefore, the return value of the action h can be ignored by the proxy during the adaptation. Additionally, the parameter c g is an output parameter of the action g and an input parameter of the action h, thus invoking the two actions in the correct sequential order will resolve the dependency. The input parameters a and b are equivalent in the both sets of actions, therefore the values can be transfered when the invocation of f is received.

The mapping validity on two sets of actions can be carried out manually on a small number of actions and parameters. However, on more complex compositions and devices where an action has several parameters, the manual check of the validity becomes a tedious task and error prone. Therefore, we provide the matching concepts based on Paolucci 's matching [Paolucci 02] detailed in section 7.1.1. Paolucci proposes four matching degrees which can be applied on a common ontology by exploiting the taxonomic structure between the concepts' semantics. In our work, the ontology contains only equivalent relations and the composition patterns.

We provide in the following some definitions used later to calculate the decidability of the actions validity.

Definition 10 (Matching Definitions).

• ∀a ∈ A, np Input (a) is the number of parameters the action a has as input.

• ∀ a, b ∈ A, nbEqual Input (a, b) is the number of equivalent input parameters between actions a and b.

• ∀ a, b ∈ A, np Common (a ∩ b) = number of common parameters between actions a and b.

• ∀ n ∈ N + , ∀a 1 , a 2 , ...a n ∈ A, , S an : {a 1 , a 2 , . . . a n } is a set of actions.
In order to automatically detect valid compositions and mappings, we propose the MConcept equation. The

M Concept

Input takes two sets of actions S an , S bm , for example S an = {f } and S bm = {g, h} from the previous example. The M Concept Input returns an MConcept similarity value in R + [0,1] which is calculated based on the equivalent input parameters divided by the number of the input parameters. There is also the M Concept Output which is based on the equivalence of the actions and their output parameters. For conciseness, we only detail the M Concept Input since the M Concept Output can be calculated using equivalent output equations. Thus, the matching concept is defined as follows:

Definition 11 (The Matching Concepts). ∀ S an , S bm / a i , b j ∈ A, n, m ∈ N + * , M Concept Input (S an , S bm ) = nbEqual Input (S an , S bm ) * 2 P arameters Input (S an , S bm )

• nbEqual Input (S an , S bm ) is the number of equivalent input parameters between the two sets of actions.

It is calculated as follows: ∀S an : {a 1 , a 2 , . . . a n }, S bm :

{b 1 , b 2 , . . . b m }, nbEqual Input (S an , S bm ) = n i=1 m j=1 (nbEqual Input (a i , b j )).
• P arameters Input (S an , S bm ) is the number of input parameters between the two sets of actions. It is based on the number of input parameters of each action. However, we count only once the common parameters shared between sequential actions. The P arameters Input (S an , S bm ) is calculated as follows:

P arameters Input (S an , S bm ) = np Input (S an ) + np Input (S bm ) -np Common (S an ) -np Common (S bm ).

• np Input (S an ) = n i=1 np Input (a i ).

• np Common (S an ) = np Common (∩ n i=1 a i ).

If S an and S bm have no input parameters and only outputs, then the M Concept Input (S an , S bm ) = 1.

The M Concept Input between the two sets of actions {f } and {g, h} returns the following similarity value

M Concept Input = 2 * 2 2+3-1 = 1.
There is two equivalent input parameters between the two sets of actions. {f } has two input parameters while {g, h} have three input parameters and one common parameter c between the two actions g and h.

We extended Paolucci's[ Paolucci 02] four matching degrees between matched services: Exact, PlugIn, Subsumes and Fail. Paolucci applied his proposed matching degrees on concepts belonging to the same ontology to detect compatibilities between services. Paolucci uses a reasoner to determine the compatibility between concepts based on a hierarchical classification in the common ontology.

In our approach, we only have the equivalent and compositions relations provided by the alignment and the patterns, therefore we redefine the following matching degrees between two sets of actions S an , S bm , as follows:

• ExactM atch Input (S an , S bm ): for each input parameter of S an there is an equivalentTo relation with each input parameter of S bm , unless if the sequential union parameters resolve the non existing equivalence relations. For example, f(x,y) and g(x,y) have an ExactM atch Input . The sequential union example between {f } and {g, h} is another ExactM atch Input example where the sequential union parameters resolve non-existing equivalence relations 5 .

The ExactMatch applies if M Concept Input (S an , S bm ) = 1.

• P lugIn Input (S an , S bm ): for each input parameter of S bm there is an equivalentTo relation with some input parameters of S an . For example f(x,y,z) ≡ g(x,y). The parameter z can be ignored during the invocation since it has no equivalence on the action g. The PlugIn applies iff the following three conditions are valid:

-M Concept Input (S an , S bm ) = 1 -np Input (S bm ) = nbEqual Input (S an , S bm ) -np Input (S an ) > np Input (S bm )
• Subsume Input (S an , S bm ): for each input parameter of S an there is an equivalentTo relation with some input parameters of S bm . For example, f(x,y) ≡ g(x,y,z). The Subsume matching degree do not guarantee a successful translation between actions since some values of S bm are missing. The parameter z cannot be ignored, g is expecting a value. Therefore, it is up to the expert validating the alignment to verify the specifications and check if the parameter z can have a default value or other adaptation operations using ATOPAI. The Subsume applies iff the following three conditions are valid:

-M Concept Input (S an , S bm ) = 1 -np Input (S an ) = nbEqual Input (S an , S bm ) -np Input (S an ) < np Input (S bm )
• U nknown Input (S an , S bm ): for some input parameters of S an there is an equivalentTo relation with some parameters of S bm and does not verify any previously defined matching concept. For example, f(x,y,z) ≡ g(x,b,c).

(S an , S bm )

Exact In P lugIn In Subsume In U nknown In Exact Out ? ?

P lugIn Out x or ?

x or ?

x or ?

x or ?

Subsume Out ? ?

U nknown Out x or ?

x or ?

x or ?

x or ? Table 9.3: Decision table, ( :success, x:fail, ?:undefined)

Once the M Concept Input and the M Concept Output are calculated, we define the Table 9.3 to decide if the mapping between actions is a success (the translation between actions is satisfied), a failure or an undefined.

We detail next, some examples to show how the table decisions are found.

Consider an Exact Input /Exact Output matching, such matching is clearly valid and don't need an adaptation, thus the result is always valid. Consider the following mapping, (y=f(x,y,z)) ≡ (y=g(x,y)) which clearly represents a P lugIn In and an Exact Out mappings. The parameter z is not mapped and is not needed to invoke the action g, thus it can be ignored by the proxy during the adaptation, therefore the mapping decision is always valid. Now consider another Exact I n/P lugIn O ut mapping, (y=f(x,y)) ≡ (g(x,y)), such mapping cannot be automatically classified as valid. The application invoking the action f on the proxy, is expecting a return value y, thus it is up to the expert to check if an adaptation is possible based on the returned value.

Table 9.4, shows the decision returned to the expert by ATOPAI between the DPWS and UPnP Printers.

The first two actions are successful, the expert then focuses only on the undefined decisions to check if it can be turned to success. The CreateURIJob mapping, see Table 9.4, is turned to success by setting the LastDocument to true as a default value of the action AddDocument. The GetPrinterAttributes in Table 9.4 is turned to success by setting the PrinterStatus as default input of the action GetPrinterElements. All the successful mappings are used for the code generation while the failures are ignored.

The successful mappings are directly used in the code generation without any expert intervention. The undefined decisions require an intervention from the expert which checks if an adaptation is possible by providing default values set using ATOPAI. The adaptation is applied using a high level API to specify the behavior as detailed in the next section 9.4.4. The failure/undefined decision (x/?) is harder to adapt since the adaptation depends on the output or input parameters not having an equivalent relation. Thus as mentioned in section 9.4.3.1, with Sequential Union n to m, such adaptation is in general impossible and depends on the actions specifications and descriptions.

The adaption of the undefined decisions can be accomplished either by setting default values or by using a high level API to specify the adaptation behavior as detailed in the next section.

Expert Code Annotation

Rules automatically detects patterns, however, not all the correspondences between actions are simple and can be resolved only by linking the entities or setting default values. The adaptation might need data conversions GetPrinterAttributes ∪ 2 (GetPrinterElements, GetActiveJobs) Susume Input , Subsume Output ?

1 Sequential 1 to n

2 Union 1 to n Table 9.4: Equivalent actions for UPnP-DPWS Standard Printers and loops, for example a temperature conversion between C degree and F using the following equation:

• C = (5/9)( • F -32
). This step is optional and depends on the previous matching concepts results, as shown in To add the adaptation code, the expert selects an action or state variable entity on the left ontology, the uses Add Code Button which allows to write the conversion behavior.

• Use Case #1, Consider two TV devices. T V 1 has a service VolumeManager with one action a 1 :SetVolume (int newVol) used to set the new volume value. The T V 2 has a VolumeManager service with three following actions:

(a 1 ) SetV olume(newV ol) ?

        
(a 2 ) currentV olume = GetV olume() : retrieves the current value.

(a 3 ) V olumeU p() : to increase the volume value by 1.

(a 4 ) V olumeDown() : to decrease the volume value by 1.

The action a 1 has no direct equivalence with the actions a 2 , a 3 and a 4 , however the following adaptation is possible. In our approach, the generated proxy is exposed as a device T V 1 and interacts with a device T V 2 as shown in Figure 9.31.

Listing 9.6 shows the adaptation behavior involving the three actions a 2 , a 3 and a 4 . When the proxy receives the SetVolume invocation it retrieves the newVol value, (line 5) then it invokes the GetVolume and retrieves the current volume level currentVolume on the T V 2 , (lines [6,7]). The proxy calculates the difference between the two volumes' values, the newVol and the currentVolume on the T V 2 . If the the devices and the relations between to generate a proxy which transparently reconciliate the heterogeneity between the two devices. Thus, the generated proxy will expose itself as a specific UPnP device on the network.

The UPnP applications will interact with the proxy transparently as a UPnP device. The received invocations from the UPnP applications are redirected to the real non-UPnP equivalent device on the network.

The next chapter details the implementation of the three modules OWL Writers, ATOPAI and DOXEN, then presents the experiments carried out on devices of the digital home.

Chapter 10

Implementation "A good idea is about ten percent and implementation and hard work, and luck is 90 percent." We present in this chapter the implementation of our proposed approach. The implemented prototype performs a device to device adaptation based on three modules: the OWL Writers depicted in section 9. The same applies on the rest of the entities which are also connected through object and data properties. with double frames concepts, such as the UPnP Device, UPnP Service. Then, the OWL Writers goes through the device description and services and fills the ontology by placing the elements according to their type. For example, the BinaryLight is a device type, thus it will be represented in the ontology as a sub-concept of the ontology class UPnP Device. The same applies for the rest of the entities, service, action and variable.

-Guy Kawasaki
Additionally, for each sub-concept added, the OWL Writer, connects it to a specific property. For example, The DPWS OWL Writer applies the same mechanism to generate ontologies, except for the operations and the messages' content. Such information cannot be retrieved using the DPWS Base Driver API, thus the DPWS OWL Writer retrieves the WSDL file from the real device. Then, the OWL Writer parses it and generates the operations names, variables and instances then attaches them to the ontology. The OWL Writers also support a shell command to generate ontologies by passing a description file (UPnP-XML or a WSDL).

The next subsection, gives more information about the WSDL structure construction. DPWS devices receiving invocations, expect to find the same structure when an action is received. Therefore, the same structure must be used when the proxy has to interact with the real device. In order to keep the structure and gain time during the code generation phase, we chose to flatten the parameters during the automatic generation of ontologies. Thus, the JobName element, in The drawbacks of this solution are in the ontology construction and the aligning process since time is spent on the structure flattening by browsing the hierarchy. Additionally, instead of aligning once the structure, ./../JobDescription/JobName, the JobName will be matched several times since the root of the structure is not the same.

However, One of the advantages of our choice is the clarity during the expert validation, parameters are now flattened and directly visible in the GUI/ATOPAI. Thus, the expert don't need to explore the hierarchy 4 or 5 level to validate an alignment. Second, it allows to gain time during the code generation since the hierarchy is represented as a path in the name, therefore, no need to explore the structure.

Consequently, the generation of structures is much more faster during proxy generation (less than 3 seconds for a printer proxy). Since the construction is relatively fast and the alignment is performed off line, then, loosing time during these two steps is acceptable in favor of gaining time during the dynamic code generation.

In chapter 4, we mentioned that the IGRS protocol relies on similar UPnP stacks and exposes its description using a WSDL. Thus, an IGRS OWL Writer will have the same implementation in order to go through the WSDL and build the ontologies automatically. Currently, there is no IGRS Base Driver supported on OSGi.

Therefore, we did not develop an IGRS OWL Writer.

We detail next a possible solution for the Bonjour devices. Once the ontologies are automatically generated, they are used by the ATOPAI framework to perform the matching between two equivalent device types.

ATOPAI

In this section, we detail the Alignment and annotation Tool framewOrk for Plug and plAy Interoperability (ATOPAI) which allows to perform the device matching between two generated ontologies by the OWL Writers. We detail next the implementation of the main steps: ontology alignment, expert validation, patterns and adaptation.

Compute Alignment

Ontology Alignment

The Align API supports string-based matching techniques, however, such techniques clearly cannot detect synonyms and antonyms. Thus, we propose the SMOA++ [Mora 10] technique based on the external dictionary WordNet [ Fellbaum 98]. We used the Rita 2 WordNet Java API, which allows to retrieve the synonyms, antonyms needed in our context for the ontology alignment.

ATOPAI supports the basic matching techniques along with SMOA++, however, it does not yet supports combining strategies by assigning confidence or weight values for each basic matching technique. We describe next, the implementation of SMOA++.

SMOA++

The SMOA++ technique relies on the substring search in the wordnet dictionary. It is inspired from the SMOA technique which searches for the biggest common substring between two strings to match. In SMOA++, first we apply the tokenization, to search for the biggest substring in WordNet and then we remove it from the string and search for another substring until none can be found. For example, between the Create and eat, the kept substring is Create. The start index of each found substring in kept in the subString index table. The search continues until no substring is identified in WordNet. Then, based on the saved indexes, we extract the substrings and add them to the list of valid WordNet substrings.

The previous tokenization step returns two lists of substrings found in WordNet. The matching step consists in finding synonyms or antonyms in order to decide if the two string are equivalent or not. Using the Rita API, we can retrieve the synonyms and antonyms of each substring. Additionally, the Rita API returns the best position of a string in WordNet and whether it is used as a verb, noun, adjective or an adverb. We exploit the four possibilities when searching for synonyms. If a synonym is found between two substrings then the similarity value is set to 1 between the two. The Rita API also allows to retrieve if two strings are related with a coordinate relation, i.e. they share the same hypernym or superconcept. For example, "fork" and "spoon" are coordinate terms, they are both eating utensils. We also retrieve the list of coordinates and handle it as the synonyms list.

During the antonyms detection, we apply a first round of search between the two substrings lists. Using the Rita API, we check if there is at least one pair of antonym between two substrings from each list. If the first round did not detect any antonyms, we apply a second round, where we query the WordNet dictionary for the synonyms of each substring in the two lists. Then, for each synonyms, we retrieve its antonyms. If a pair of antonyms matched then the global similarity value is set to 0. Listing 10.4 shows a part of the implementation to retrieve the second round antonyms using the Rita WordNet API. the position pos and the integer both refer to the best position whether it is used as a verb, noun, adjective or an adverb.

1 public Set<S t r i n g > getAntonyms2Round ( S t r i n g s t r 1 , i n t i , S t r i n g pos ) {

2 Set<S t r i n g > synonyms = getSynonyms ( s t r 1 , i ) ; To detect the equality between two substrings, we rely on the equalsIgnoreCase method provided by the Java String API, however, other basic matching techniques can also be used instead.

Filters

As mentioned in section 9.4.1, the basic matching between an ontology O 1 having n elements and an ontology O 2 having m elements results in a matrix[n][m] holding similarity values between the entities. We filter these results by applying two types of filters, as detailed in section 9.4.1.2. The first keeps the best alignment similarity value in each line of the matrix. The implementation is simply applying a max function on each line of the matrix. The second filter applies a test on the resultant tuple (leftEntity, rightEntity, similarity) to check if the leftEntity and the rightEntity belongs to the same type, device-device, service-service, etc. If it the case then the similarity value is kept, else it is set to zero. A tuple holding a similarity value 0 will be removed from the alignment file when a trimming threshold is applied at the final result.

Similarity Propagation

In order to enhance similarity values between entities, we apply a similarity propagation based on the structure.

After applying the filters, we revisit the alignment and enhance the similarity values by applying the Equation 9.4. We apply a down-top propagation, we enhance the similarity of the actions if the similarity values of the parameters are higher than a defined threshold set by the expert using ATOPAI. The propagation proceeds by enhancing the similarities of the services based on the actions. And finally, the similarity of the device which is based on the similarity of the services.

The WSDL Structure

As mentioned in section 10.1.1, the WSDL is flattened, therefore the actions parameters in the DPWS, IGRS and Bonjour ontologies will hold a path in their names as in CreatePrintJobRequest/PrintTicket/Copies. Therefore, when it is matched with another parameter from the UPnP ontology like the parameter Copies, we only consider for now the last element of the path (Copies).

Another strategy can also be used. For instance, we can keep the maximum found similarity between all the path elements and the UPnP matched parameter. For example, the matching between the two parameters will perform a SMOA++ matching on the following pairs of elements: (CreatePrintJobRequest, Copies), (PrintTicket, Copies), (Copies, Copies) and then keep only the maximum similarity. Applying the matching on each of the elements and not only the last element might allow to detect more mappings since the elements in the path are somehow semantically related with the last element of the path.

We detailed in section 9.4.1.4, the enhancer step, which consists in enhancing the similarity values between two services, for example, based on the similarity values of their actions and if there is a high similarity value between one of the services and the actions. The enhancer is not supported yet in the current version of ATOPAI.

We detail next the implementation of the pattern detection which annotates the ontology with composition relations between the actions.

Pattern Detection

The ATOPAI framework supports the pattern detection by triggering a set of pre-written rules specified in the Ontology Pre-Processing Language [ Šváb Zamazal 10]. In order to detect the patterns detailed in section 9. The pattern detection implementation takes a left ontology, a right ontology and an alignment ontology, then merges the three ontologies into a single one. The rules are then applied on the merged ontology to detect the patterns. This step also adds the new patterns properties such as Simple Mapping Input to the ontology which are used later by the the OPPL2 API 3 rules to connect the classes. The pattern detection implementation loads a file containing the 12 OPPL rules and then applies them on the ontology in a specific order since a default value or an adaptation behavior. However, the non valid compositions of mappings are removed manually one by one by the expert which selects the entities in ATOAPI and removes the lines between.

We present next, the adaptation which can be carried out by the expert to add an adaptation behavior or a default value. ATOPAI also supports adding a target name space to the parameter structure which is the case of the GetPrinterElements action, see Table 9.4. We set the input parameter to tns:PrinterElements to validate the mapping with the UPnP.GetPrinterAttributes action.

ATOAPI also supports fetching data from a URL and transforms it into a file to be used by another action. This feature is applied on the sequential mapping between the CreateURIJob and the CreatePrintJob, SendDocument. The SendDocument expects a file while the CreateURIJob contains the url location pointing at the file. Thus, the fetch feature allows to add an adaptation between the two actions, from url to document by retrieving the document from the url at runtime. Since, the fetch from url feature is a generic method and can be used for other cases, we decided to implement it in ATOPAI. However, it can also be used as an external service call instead. There is another possible sequential mapping between the printers, the CreateURIJob is equivalent to CreatePrintJob and AddDocument which takes a url. Thus, in this case the fetch from url is not used.

The meta data annotation allows to apply basic adaptation on the actions input and output parameters.

However, not all the adaption can be resolved by setting default values. The adaptation can also require adding a behavior using loops or treatment on the values using high level operations on the ontology entities as shown next.

Adaptation API

The UPnP and DPWS API require an advanced knowledge in the plug and play technology in oder to apply an adaptation behavior and translation between the actions and their parameters. Applying a low-level adaptation by manipulating the Plug and Play protocols API can be a tedious and an error prone task. Thus, we specified the high level adaptation API to offer the expert the ability to specify a behavior relating two or more actions along with their related parameters independently from the UPnP and DPWS API. The expert manipulates the entities (actions or parameters) requiring adaptation through high level Java syntax-based operations such as set, get and invoke.

classes while the Behavior Code part will hold the adaptation code added by the expert. Figure 10.5 shows the adaptation code between the two TVs overviewed in section 9.4.4, the added adaptation code is shown in Listing 9.6. The added code is saved in a file and the selected action from the ontology is annotated with the following data property has Adaptation Code which relates the specific action to the adaptation code file location. Such information is exploited during the code generation, the added code will be injected in specific positions in the templates. So far, we only support an adaptation code written in Java based syntax.

The interface of the Adaptation API is shown in Listing 10.6. It offers methods to select an action from the right ontology. It also allows to invoke any selected action. The input/output values of a selected action can be set or retrieved. To apply the methods on the selected action of the left ontology, the keyword this is used, see Listing 9.6. The Adaptation API offers a high level entity manipulation to adapt action invocations. However, some adaptation might require an external conversion or transformation from a generic service such as the ps to pdf conversion. We detail next, the external service API which allows to invoke external services.

External Service API

The external service API is used by the expert to invoke an external service such as an OSGi service present on the framework. The use case #2 detailed in section 9.4.4 shows how the adaptation can be performed by invoking a general purpose service which converts a ps document into a pdf.

To use an external service, the expert relies on the API when defining its adaptation code exactly as in the adaptation API. The action in the ontology is also annotated with the reference to the code file. The added code is also injected in the templates during the code generation. The implementation of the External Service API, shown in Listing 10.7, relies on the reflection API (java.lang.reflect 4 ) to search for the method to invoke along with its input/output parameters and values. moreover, the proxy subscribes and searches for the required service. Thus, the expert must first specify the service name or the class to search for on the OSGi framework.

Then, the method to invoke along with the input parameters names and values (since the same method can

Ontology Visiting

DOXEN visits the ontology to extract necessary information for the proxy generation such as the name of the devices, services, actions and parameters. It also retrieves the properties between the entities such as equivalent parameters, union and sequential union compositions between actions. Additionally, the annotated meta data and behavior code added by the expert is taken into account.

DOXEN loads the alignment file and points to the entry point, the "UPnP Device" OWL class. Each generated entity (by OWL Writers) in the ontology is a subclass of a well defined concept from the metamodel. In the lights ontology for example, (see Figure 8.3), BinaryLight (resp. SimpleLight) is a subclass of UPnP Device (resp. DPWS Device). The same applies on the rest of the entities in the ontology.

Using an OWL Entity Visitor which implements an OWLClassExpressionVisitor, OWLDataRangeVisitor and OWLIndividualVisitor (see OWL API 3.0 [OWL ]), DOXEN explores the ontology then instantiates Java classes. Entities in the ontology are linked through object and data properties such as equivalentTo, Union 1 to n or has Adaptation Code, therefore when the visitor explores an entity, it instantiate a Java Class depending on its OWL Super Class. For example, when the visitor encounters the BinaryLight entity, since it is a subclass of the UPnP Device entity, then the visitor instantiates a UPnP Device class. Then, DOXEN continues visiting the ontology by relying on the properties between the entities. For example, the BinaryLight entity is related to the SimpleLight with the equivalentTo property. The BinaryLight entity is also related to the SwitchPower service entity with the has UPnP Service. Thus, based on such information, DOXEN visits the entities and instantiates the required Java classes needed for the code generation. Figure 10.6 shows the Java structure instantiated by the visitor.

All the classes in Figure 10.6 extends the abstract class Entity. Each class contains fields specific to the entity, for example, the UPnP Service contains the service id, type and version extracted from the ontology.

The UPnP Action also contains the action names and the list of their parameters.

The addProperty method allows to extract the properties between entities such as the equivalentTo, Union 1 to n or has Adaptation Code and according to the property an operation is executed. For example, when the has Adaptation Code is encountered, DOXEN prepares to inject the code behavior in the template code, along with the Adaptation API dependencies. Moreover, the UPnP Action class contains the two methods orderList and findAction. These two methods are used to order a sequential union composition. It applies the Algorithm 1 described in section 9.4.3. Additionally, the generate Impl() allows to trigger the code generation detailed next.

Code Generation

Once the necessary information is extracted from the ontology, the code generation can be triggered. This step fills predefined Java templates with the information carried by the instantiated objects. We used the FreeMarker5 template engine to generate the proxy Java code. For clarity of presentation, we refer to the instantiated classes holding information from the ontology as objects and by Java files, the generated files obtained by filling (.tpl) templates.

1 Template a c t i v a t o r T p l = c f g . g e t F r e e M a r k e r C o n f i g ( ) . getTemplate ( " t e m p l a t e s / A c t i v a t o r . f t l " ) ; 2 Outs = new F i l e O u t p u t S t r e a m ( c f g . NewJavaFile ( " A c t i v a t o r " ) ) ; Once the ontology is visited and the classes are instantiated, DOXEN starts preparing the templates to be filled. Listing 10.9 contains a part of the implementation. The first line in the Listing 10.9 loads the Activator template to be filled. The lines 2,3 creates an empty Java File "Activator.java" on the disk in a specific location contained in the configurator (cfg). The activatorInformationMap in line 4 will hold the necessary information to be filled in the template such as the package name, and the upnp device name to publish and the dpws device type to search for. Lines 6 and 7 fill the map with the information from the instantiated Java structure, more specifically from the UPnP Device and DPWS Device classes fields, see Figure 10.6. And finally, the line 8 invokes the FreeMarker template engine in order to replace in the template the $upnp device name and $dpws device name with the values contained in the map as shown in Listing 10.10.

1 System . out . p r i n t l n ( "1-A c t i v a t i n g Bundle : Proxy UPnP-DPWS $ { u p n p d e v i c e n a m e } D e v i c e " ) ; 2 System . out . p r i n t l n ( "2-S e a r c h i n g f o r an e q u i v a l e n t DPWS $ { d p w s d e v i c e n a m e } D e v i c e " ) ; . . . The expert added code is actually saved in template files and then injected in the actions templates using the <#include "$code"> feature of the FreeMarker template engine. Finally, DOXEN generates additional files such as the DPWSElementFactory which implements methods used by the actions to handle the conversion between the flat UPnP parameters and the structural elements of the WSDL.

Compiling and Bundle Generation

Once all the code source is generated, DOXEN compiles it on the fly using the Janino6 compiler. We chose to use Janino since it is a light compiler (569 KB) compared to other compilers such as the Sun JDK compiler.

The Janino compiler is used mainly to compile and execute the compiled code on the fly. Therefore, we added an extra method generateBytecodes extended to return the byte code of each compiled file.

We rely on two class loaders to compile as shown in Listing 10.12 [lines 1-2], the DOXEN Class loader and an additional urlsOfJar array which contains urls pointing to several Jar packages needed to compile. The line 3 starts the compilation by passing the Activator.java file, then Janino loads and compiles the dependent classes in the other generated files. Once the source is compiled, we write the byteCode in .class files and then we package all the classes in a Jar. In section 2.4.3, we gave an insight on the importance of the device and application management for an operator, a device manufacturer and an application vendor. Therefore, we also provide DOXEN and its generated proxies with management services in order to allow remote administration and action invocation. We detail next, DOXEN's supported services and capabilities. A use case in the experimentation section, shows how such management services can be used to carry out diagnostic operation. Such operations allow to detect basic malfunctions of applications interacting with the generated proxies in the digital home. need to be added in order to support a transparent interaction with the new protocol's devices.

We detail next, the capabilities and services provided by DOXEN.

Two Proxy Generation Modes

DOXEN supports two proxy generation modes: brutal and on demand. The brutal mode consists in generating the adequate proxies on the appearance of non-UPnP devices on the network. The implementation of the brutal method relies on registering a service listener for the DPWS devices.

As for the on demand mode, DOXEN listens to the applications requests (search or subscribe) for a UPnP device using the service hooks feature detailed in section 3.3.1.3. Once a UPnP device request is intercepted, a UPnP Printer request for example. Based on the request information (device type, meta-data information), DOXEN queries its configuration to check its non-UPnP equivalent device types and versions. Then, DOXEN scans the network searching for the specific non-UPnP device, a DPWS equivalent printer for example. Once found, the proxy generation is carried out as described previously. The on demand mode can be used for example when DOXEN is deployed on low energy and scarce resource devices such as smart phones. DOXEN will trigger the proxy generation only upon applications requests.

Uniqueness In Interaction

Consider two DPWS lights are detected in the network, DOXEN will generate two UPnP-DPWS light proxies since each proxy reflects the same state of the real DPWS device. Therefore, the invocations received by the proxy must be sent to one specific real device. Therefore, each generated proxy must be handling a unique DPWS light device. Therefore, DOXEN must guarantee the uniqueness in interaction.

To achieve such uniqueness, each started proxy interacts with DOXEN to check if other UPnP-DPWS lights proxies are already generated. If it is the case, the newly generated proxy receives a list of the real devices unique identifiers already handled by actual UPnP-DPWS lights proxies. Based on these communicated identifiers, the new generated proxy will search for a DPWS light having an identifier not included in the list. Once found, the proxy notifies DOXEN about the identifier which updates its list of identifies. When the DPWS Light leaves the network, the proxy handling the device sends a notification to DOXEN about the device departure, then the proxy stops. DOXEN re-updates its unique identifier list by removing the received one. DOXEN supports the three following UPnP services:

• DMS: The DOXEN management service offers several management operations such as start, stop and update DOXEN, retrieve and set the proxy generation mode. Other operations also allow to add, remove or update an ontology alignment, a template code file or a list of supported equivalent devices and services.

Furthermore, DOXEN can also be invoked to return a list of its generated proxies.

• BMS: The basic management service [UPnP 10a] as overviewed in section 2. • CMS: The configuration management service [UPnP 10b] is also a standard UPnP service, it returns several information such as the device type and manufacturer, the network configuration of the machine where it is deployed. For example, CMS reveals the number of active network interface along with their IP and MAC addresses, the DNS and DHCP hosts. CMS also exposes general information about the physical device and its manufacturer. Information about the operating system can also be provided.

The current version of DOXEN relies on the exec process and the methods of the java.net package.

DOXEN retrieves the following information from the underlying device and the network: the number of sent/received bytes using the ("netstat -e") exec process, the IP and MAC address, the subNetMask, the default gateway IP, the number of existing interfaces and also detects which interfaces are up and the type network's card type, WIFI or LAN.

Development Effort Table 10.1 shows the development effort of each module expressed in lines of code.

The UPnP OWL Writer is achieved in 582 LoC while the DPWS OWL Writer has 1119 LoC. The difference is due to the fact that in the DPWS OWL Writer, the WSDL service description file need to be retrieved and parsed along with the XSD file. Moreover, as mentioned in section 10.1.1, the WSDL and the XSD allow to have complex and structured elements which makes the ontology generation even more complex.

The ATOPAI framework and the GUI with the four steps mentioned in section 9. The next section draws the experimentations we carried out to validate our approach. It also exposes in a use case, how the combination of the BMS and the CMS management services can be used to perform basic diagnostic operations.

Experimentations

We validated our approach on 6 existing devices which we took on the shelf, we also implemented fake devices in order to carry out additional tests. We detail next a basic diagnostic use case showing how the BMS and CMS can be used to perform basic troubleshooting operations.

Basic Connectivity Diagnostic Use Case

The basic and configuration management services are also supported by each generated proxy. The UPnP standard CMS service [UPnP 10b] exposes only information about its hosted device. On the generated proxies, we extended the CMS to additionally expose networking configuration information of the real device such as the IP and MAC address, the DNS and DHCP servers along with the general device information (manufacturer, unique ID, etc). Thus, each generated proxy exposes the following services types:

• A set of specific services related to the device type and domain. Such specific services are generated based on the ontology alignment.

• A Basic Management Service [UPnP 10a] which provides basic operations such as the reboot, ping and traceroute.

• A Configuration Management Service [UPnP 10b] which provides two sets of information. The first refers to the hosted device while the second set provides information from the real DPWS device it is proxifying.

To retrieve networking information from the real DPWS device, the proxy relies on the real device description containing general information such as the manufacturer, model etc. The proxy also retrieves the IP address of the DPWS device from the DPWS base driver. The MAC address is retrieved by applying two operations, first the proxy pings the DPWS device and then performs an ARP request. The proxy also detects the interface name of its underlying device used to communicate with the real device. and a UPnP print application. The device hosting the printer proxy will be referred to as the "proxy-device" while the device hosting the application will be referred to as the "application-device". The application and the proxy can be hosted on several devices in the digital home such as a Set-Top-Box, a GuruPlug12 or a PC. Alice remotely performs the following operations through the CWMP-UPnP Bridge and relies her diagnostic on their response:

1. Gateway.ping ("Application-Device"). The gateway pings the "Application-Device".

2. Gateway.ping ("Printer").

3. Gateway.ping ("Proxy-Device").

4. Application-Device.ping ("Proxy-Device").

Proxy.ping ("Printer").

The operations #1, #2 and #3 are used to test if all devices are connected to the Gateway. The operation #4 tests if the UPnP application reaches the proxy. And the operation #5 tests if the Proxy can reach the Printer.

When Edouard reports the problem to Alice, she asks him specific questions such as "Does your UPnP application shows the UPnP Printer?" and then according to Edouard's answers, Alice performs diagnostic operations to detect the potential malfunctions:

1. Edouard: "My UPnP Printer Application is not showing the UPnP Printer.".

For such error and based on the end-user feedback, there is the following potential malfunctions:

• Connectivity Failure between the Application and the Proxy. It is verified if the operation #4 returned an error. Clearly, if there is no connectivity between the application and the proxy, the application will not receive the proxy annunciation and will be unable to interact with.

• The DPWS Printer is not connected. Thus, the proxy did not detect the printer. It is verified if the operation #5 returns an error.

2. Edouard: "I am able to select the UPnP Printer using my UPnP Printer Application. But it is still not printing".

• Printer was suddenly unplugged from power. As mentioned in section 9.3.3, the proxy's state depends on the real device's state. Thus, when the DPWS printer device announces its departure and leaves the network, the proxy receiving the DPWS announcement also announces it departure. Therefore, if the DPWS printer leaves the network, the proxy announces also its departure. However, if the printer was suddenly unplugged from the power switch, then it did not send the departure announcement.

Since it is not received by the proxy, then the proxy remains present on the network and can detected by the UPnP Printer Application. To detect this problem, the operations #2 and #5 fails.

• No Connectivity between the proxy device and the printer. Thus, the proxy cannot invoke the action on the printer, the physical link is down. The results of the operations #2 and #3 allow to detect whether the problem lies in the Gateway-Printer or in the Gateway-Proxy connectivity.

Other specific printer errors and specific DPWS devices messages are intercepted by the proxy and forwarded to the UPnP applications as UPnP Exceptions, for example, the "Tray Paper Empty", "Paper Jam" or "0%

Ink".

We provided in this paragraph only a part of the experimentation regarding the interoperability, We show in [El Kaed 11c] how our approach can be mapped with other diagnostic and monitoring tools in the digital home.

We detail in the next chapter, the performance evaluation of our proposed modules, the We evaluate in this chapter our proposed approach to resolve the plug and play devices heterogeneity. In section 11.1, we evaluate the ontology generation on a PC and a Set-Top-Box. We outline in section 11.2, a comparison between our proposed method SMOA++ and the other syntax based methods. Then, we present the ontology alignment results of equivalent device types based on the SMOA and the SMOA++ techniques. In section 11.3, the proxy generation evaluation is provided. And finally, we discuss in section 11.4 the evaluation of our approach with respect to the proposed approaches in the literature.

OWL Writer

We tested the OWL Writers on an Intel x86 Centrino Duo Core PC, with a 2 GHz clock frequency and 1 GB RAM capacity. We were also capable of deploying and testing the OWL Writers on a SodaVille Set-Top-Box with an intel Atom 1.2 GHz frequency and (256+128) MB of RAM. The evaluation on the PC provides the reader with an insight on the OWL Writers' performance, while the evaluation on the SodaVille aims to prove that our modules can be deployed on real embedded devices in the digital home.

The OWL Writers generated ontologies for the UPnP and DPWS Lights, Clocks and Printers. We used a UPnP standard light and printer profile proposed by the UPnP Forum. The UPnP Clock profile is proposed by Apache Felix as a fake device. The DPWS Light profile is found at the SOA4D [SOA4D b] forge, while the DPWS clock profile can be retrieved from the WS4D1 forge. The DPWS standard printer is proposed by Microsoft. Thus, the six devices are on the "shelf devices" which can be purchased or are prototypes of development projects. In other terms, we used existing devices to validate our approach without modifying and arranging their descriptions to meet our needs. 11.1 with the five devices2 . From the first observation, one might concludes that the ontology generation time is exponential with regard to the description (LoC). However, in order to have a clear insight on the OWL Writers evaluation and the impact of the description size on the generation time, we used fake devices with various fake descriptions, as shown in Figure 11.2. We assigned a label to each fake device to indicate the description size in LoC 3 . Figure 11.2 reveals that the ontology generation time tends to be exponential with regard to the device description (LoC). The next section details the evaluation of the device matching.

Device Matching

We provide in this section an evaluation of the SMOA++ method on simple examples to reveal its advantages and drawbacks. Then we provide the ontology alignment results applied on five equivalent devices.

SMOA++

We provide in 2 "Set" and "Get" not found in WordNet as antonyms.

3 Found on the UPnP and DPWS Clocks devices. Thus, even though the SMOA++ detects antonyms and synonyms, a specific device domain dictionary can be used to enhance the matching. The tokenization step can also be improved using other metrics to reduce the tokenization errors.

The next section details the matching between the automatically generated ontologies representing the devices descriptions.

Alignment

In order to test the alignment, we took existing devices descriptions and applied our matching strategies. We

Discussion

We propose in this work to resolve the heterogeneity between plug and play devices by applying ontology matching techniques to find the correspondences between two equivalent devices. As mentioned in chapter 4, there is three levels of heterogeneity between the plug and play devices. The first level of heterogeneity resides in the protocol stacks, the second level exists in the description format and the third and final level is in the description content. The protocols stacks heterogeneity is handled by the OSGi base drivers which represents the devices as local OSGi services. Thus, the applications can interact directly with the devices through the API base drivers methods.

In order to resolve the second and the third levels, we propose three modules. The first module consist in automatically generating ontologies based on the devices' descriptions and capabilities. The generated ontologies resolve the second level of heterogeneity since the description is now represented in a common format, the ontology format which is conform to the meta model device. Additionally, the ontology allows to express the relation between the entities, such as the sub-concept or the equivalent relation. Thus, the ontology represents each device, its services along with the actions and variables in a common representation using the ontology domain semantics. The automatic ontology generation as shown in section 11.1, varies from less than 0.05 seconds to 4 minutes when dealing with complex devices such as the printers.

To resolve the third level, we rely on the semi-automatic ontology matching techniques to resolve the content description heterogeneity. The ontology matching assisted with a semantic dictionary allows to detect similar entities independently from their syntactic name description. The semantic dictionary is used to detect the synonyms and antonyms between the entities. Section 11.2 reveals the ontology alignment evaluation, the matching can detect up to 78 % of the valid correspondances. Then, it is up to the epxert to valid, update and edit the alignments to achieve a valid ontology mapping. We also propose the ATOPAI framework along with a GUI to make the ontology validation easier. Based on the valid alignment, we apply several rules on the ontology to detect actions compositions (union or sequential) based on their input and output variables.

The rules automatically annotate the ontology with the detect pattern category and composition. Then, we automatically classify the compositions and return to the expert the non-valid compositions. The expert then checks based on the devices specifications if an adapation is possible. There are three possible adaptations, the first consists in setting default input/output parameters values. The second category requires an adaptation behavior on the actions and their parameters. The third and final category allows to delegate the adaptation to external services such as OSGi services on the framework.

Once the ontology alignment is validated and contains the transformation to go from one description to another. DOXEN takes the high level representation then generates and installs a specific proxy which handles the invocations adaptations. As shown in section 11.3, the proxy generation is feasible, the DOXEN module can be deployed in the digital home to generate proxies based on the ontologies. The proxy generation ranges from less than a second and up to 16 seconds. Furthermore, the invocation overhead through the generated proxies ranges from 17 to 297 ms which can be considered inconsiderable.

We complete in Table 11.16, the comparison by adding our approach characteristics. The proposed approach can be applied to the following plug and play protocols: UPnP, DPWS, IGRS and Bonjour. Our proof of concept prototype is applied for now on the UPnP and DPWS devices. However, since the IGRS protocol relies on the same protocols stacks and uses the WSDL to announce its capabilities. Then, our approach should be applicable on the IGRS devices. We already treat the WSDL descriptions provided by the DPWS devices. Furthermore, in [Xie 09] a technical solution has been proposed to resolve the heterogeneity between the UPnP and IGRS protocol to allow the device discovery and interaction.

As for the Bonjour devices, we propose a solution in section 10.1.2, based on the HomeSOA framework which exposes the Bonjour devices in a WSDL.

Thus, in Table 11.16, we specify that the base drivers in our approach allow to resolve the protocol heterogeneity by representing the device and their hosted services as local OSGi services. The OWL Writers modules resolve the representation heterogeneity by automatically generating ontologies using common representation concepts. The content mediation reconciliation is provided by the semi-automatic device matching assisted by the expert. And finally, the realization of the interoperability is handled by DOXEN and the generated proxies.

In our proposed approach, the manual device and service description annotation is not required since the rules automatically detect the patterns and annotate the ontology. The human intervention is needed however to validate the device matching since it is heuristics based. The main advantages of this approach can be resumed as follows: First, already written applications which targets only standard UPnP devices can now interact with other non-UPnP devices thanks to the dynamically generated proxy. Applications can now interact transparently with any equivalent non-UPnP device using standard UPnP service and action names since the proxy is exposed as a UPnP standard device. More over, all the code generation, compilation and installation is automatic and transparent to the user and the UPnP applications.

Protocols

Second, there is no need to add additional networking stacks to support other protocols on devices hosting applications. The same UPnP stack already deployed is used to interact with other devices supporting a different protocol via the proxy. For instance, we only deployed on the Android Smart Phone the UPnP Home Controller and a UPnP protocol stack. The application which can only interacts with the UPnP devices is currently able to interact with a standard DPWS printer through the generated proxy.

And finally, the construction process of ontologies is relatively simpler and faster than building a global common ontology specially when dealing with complex devices like the printers. The ontologies can also be reused if another protocol is chosen as a pivot. The expert validating the alignment using ATOPAI needs only to remove or add lines between the equivalent entities, the expert may also add an adaptation behavior using a high level API. The specifications are protocol and technology independent therefore the expert performing the validation off line on the operator site using ATOPAI can be a technician or a domain expert. For the printers validation, we validated the mappings by referring to the standard printers profiles [Microsoft 07, UPnP 06a].

Furthermore, to our knowledge, we are the first to resolve heterogeneity between two standard profile printers.

Conclusion

We evaluated in this chapter our proposed solution with its three modules: the OWL Writers, the device matching and DOXEN. The evaluation of the OWL Writers proves that such modules can be deployed in the digital home, on a set top box for example. The OWL Writers are capable of generating ontologies in a reasonable time when new devices appear in the network. The ontology generation vary from few seconds up to 4 minutes when dealing with complex devices.

The device matching evaluation on complex devices can actually detect up to 78% of the valid correspondences between two ontologies. Then, it is up to the expert to validate and edit the remaining correspondences in order to achieve a valid ontology alignment. We also pointed out the strengths and weaknesses of the SMOA++. The evaluation reveales that the SMOA++ techniques can achieve better success rates and lower false matchings than the SMOA technique. However, the SMOA++ spends more time to align than SMOA due to the access to the WordNet dictionary. The device matching can aslo rely on the similarity propagation in order to enhance the similarity values based on the structure and therefore increase the success rates of the alignment.

The evaluation of DOXEN shows that it can be deployed in the digital home on a Set-Top-Box for example to generate proxies. The proxy generation time on the Set-Top-Box ranges from 4 to 16 seconds which remains acceptable. The end-user and the application will wait at most for 16 seconds for the proxy generation before discovering the new proxy-device. Furthermore, the proxy invocation overhead can achieve at most 0.2 seconds which in inconsiderable. The proxy or the end-user will wait up to 0.2 sec before the execution of their tasks.

Chapter 12

Conclusion

"Reasoning draws a conclusion, but does not make the conclusion certain, unless the mind discovers it by the path of experience."

-Roger Bacon

The digital home is heading towards a ubiquitous computing environment where devices and applications cooperate and interact to accomplish the user's daily tasks. Heterogeneous plug and play protocols cohabit in the same home network, however, the cross protocol interaction between devices and applications is hard to achieve. Such heterogeneity between protocols, prevents the ubiquitous applications to use any available device, regardless of their protocol, to accomplish a certain task such as printing or dimming lights.

The plug and play protocols share a lot in common, they all announce their presence and departure on the network, they provide a list of their supported services. Each service provides a set of operations which can be remotely invoked on the network to accomplish a specific task.

Even though, the plug and play protocols have a lot in common, three levels of heterogeneity retain the plug and play interoperability. First, each plug and play protocol relies on specific protocols stacks to announce its presence/departure, and its supported capabilities. More over, each protocol uses a different set of other protocols to support the interaction and control. Thus, this heterogeneity in the protocol stacks does not allow to UPnP applications for example, searching for a UPnP specific device to detect other equivalent plug and play devices.

The second heterogeneity level resides in the description format. Each plug and play protocol defines or uses a specific description format to announce the device description and its supported capabilities. Thus, applications targeting a specific plug and play protocol are unable to interpret other devices descriptions since they are expressed in a different description format.

The content is the final heterogeneity level. Each plug and play protocol defines a set of standard profiles which contains the capabilities to be supported by a standard device. The profile also specifies the exact names to be used in the description annunciation for the device type, services, actions and variables. Additionally, the profile defines the behavior of the device when an operation is invoked along with the input/output required parameters for a successfully invocation. Thus, two devices having the same type will express their description content in a syntactically different representation. However, the description is more or less semantically equivalent. In fact, similar devices share common functionalities, for instance a light is always expected to turn on or off a light, a clock will always return the current time and a printer is always expected to print.

Thus, this three levels of heterogeneity between the plug and play devices retain the applications to use any equivalent available device in the network to accomplish a specific task. The ubiquitous computing vision

consists in an open world, where devices are aware of each others and put in common their capabilities to assist And finally, the DOXEN module realizes the interoperability by generating specific proxies behaving according to the ontology alignment. The proxy generation relies on the model driven engineering techniques which allow to go from a high level description into an executable code. The high to low level transformation is specified once and used each time an adaptation is needed. The high-low level transformation relies on the template filling and code generation techniques.

• SMOA++: The second contribution of this thesis resides in the device matching technique. We proposed an enhanced matching technique SMOA++ inspired from the SMOA [Stoilos 05] technique. SMOA++ relies on an external dictionary WordNet [ Fellbaum 98] to provide a similarity value between two entities.

The SMOA++ consists of two steps. The first is the tokenization step, where a string representing an entity name such as the service name is split into several substrings. The splitting process relies on the dictionary, each substring must exist in WordNet. The, we retain from the returned list of substrings, only the biggest common substring from those having common parts.

The tokenization is applied on the two entities from two ontologies, thus it returns two lists of substrings.

The second step is the matching between the two lists of substrings. SMAO++ computes the similarity value between two strings based on their relation in WordNet which can be queried for the synonyms and the antonyms.

• Prototype Validation On Real Devices: In order to validate our approach, we implemented three independent modules. The first module contains the OWL Writers which automatically generate ontologies from the devices annunciations on the network. The evaluation of the OWL Writers show that such modules can be deployed in a home network, on a Set-Top-Box or a PC for example, to automatically generate ontologies when new devices appear.

ATOPAI, the second module, it handles the device matching to detect correspondences between equivalent devices' ontologies. ATOPAI allows an expert to easily update or edit detected correspondences using a GUI. More over, ATOPAI allows to apply the rules on the ontology to detect the pattern compositions.

Additionally, ATOPAI offers to the expert a high level adaptation API in order to add adaptation behavior between actions.

SMOA++ is used in this device matching step, the evaluation of SMOA++ shows higher or equivalent successful correspondences than those detected by SMOA. More over, the number of false matched in SMOA++ is equal or lower that SMOA results.

The device matching evaluation on the three devices: clocks, lights and printers shows encouraging matching results, 78% of the successful correspondences are semi-automatically detected. Then, it is up to the expert to correct and complete the matching using A GUI as the one we propose with ATOPAI.

The third and final module is DOXEN which takes an ontology alignment and generates a specific proxy.

The generated proxy behaves according to the adaptation and compositions annotated in the ontology by the previous module.

We successfully deployed DOXEN on a Set-Top-Box in a home network. The proxy generation time carried out by DOXEN ranges between 3 to 10 seconds. In other words, when a DPWS printer appears in the home network, the user or the application waits for 10 seconds, the required time for DOXEN to generate a proxy, install it and start it. Moreover, the generation has to be done only once: when the new device is firstly discovered. Furthermore, the generation is quite fast, compared to the time spent on the implementation process carried out by a human. Additionally, to our knowledge, we are the first to print, cancel a job, retrieve the printer's and the job's attributes on a DPWS printer from a UPnP control point.

In fact, we are even the first to generate ontology-based proxies to resolve the heterogeneity between two equivalent devices.

More over, we showed in a basic diagnostic use case the benefits of supporting two management services (BMS and CMS) on each generated proxy and how such services can be used to detect basic connectivity failures.

The next section overviews the perspectives of this thesis.

Perspectives

This thesis proposes two kinds of perspectives, the first category containing the first three perspectives is related to the adaptation process, while the second category holding the last two aims to technically improve our prototype.

Machine Learning Based Alignment

ATOPAI allows to store a validated alignment. It actually stores the tuples (leftEntity, similarity, rightEntity) in a database to be reused later to enhance current computed alignments. In fact, to enhance the current alignment, ATOPAI queries the database searching for a tuple having the same left and right entities with a higher stored similarity value. If an equivalent stored tuple is found with a higher similarity value, then, the current similarity value between the tuples is replaced with the similarity from the database.

The current enhancement method is basic and cannot differentiate between absolute correct correspondences and device specific correspondences which are relative correspondences. For instance, the two strings timer and clock are absolute correct correspondences while SetTarget and Switch are relative correspondences which are only valid on the lights devices.

The absolute correspondences are usually found in WordNet while the relative correspondences are set by the expert validating the ontology. Thus, it would be relevant to take into account the relative and absolute correspondences when enhancing and even when applying the matching techniques.

The absolute and relative correspondences constitute only a small step to enhance the ontology alignment.

More generally, the ontology alignment can rely on the machine learning to detect correspondences. Machine learning algorithms can be used to observe and learn from the expert when he intervenes for validation. Such algorithms can rely on the previous used metrics and methods to compare and analyze the correction made by the expert. Consider for example, that the expert removed a matching between SetTarget and GetTarget. Then, the machine learning can apply the tokenization step used in SMAO++ which returns the following substrings:

{Set, Target} and {Get, Target}. And then, the machine learning algorithm discovers that Set and Get can be probably antonyms.

Furthermore, the machine learning can rely on statistics to take decisions during the alignment. Consider for example, that the same two strings were considered only twice as correct matchings and 10 times as false detected matching. Then, the decision would take such metrics when applying the ontology alignment.

Absolute and relative correspondences along with statistics and probability constitute only few of the other metrics the machine learning can rely on to enhance the device matching.

Device Composition

Our approach tackles the device heterogeneity and provides interoperability between two equivalent devices.

Currently, ATOPAI allows to load and detect mappings between two ontologies. The actual standard devices are mono-functions, however, new multi-function devices are currently arriving on the market, such as the multi-function printer scanner device. Thus, a multi-function device has to be probably matched with other several mono-function devices.

To support the multi-functions devices, an investigation must be carried out to check how such devices are specified. Whether they announce their selfs on the network as two logical devices supporting independent services or as one logical device providing several independent services. If the services are independent then the matching is still supported by ATOPAI. However, if the services are highly dependent one on another, then ATOPAI must be extended to support multiple ontology matching.

More over, DOXEN will also need to be extended to visit and generate code from multiple ontologies.

Additionally, the code generation templates will be impacted if the several equivalent mono-function devices support different plug and play protocol. Thus, the templates need to be extended to support several Base drivers API instead of currently only two base drivers API.

Security & Privacy

Plug and Play devices can be controlled on the network by any protocol compatible control point. Even worse, any control point can connect to an end user's media library and retrieve content saved on its Network Area Storage (NAS) device without going through any authentication or identity verification. Therefore, new discussions are initiated in order to define privacy roles on some devices. Meaning, that according to the enduser's identity distinct access authorizations are allowed. For instance, the end-user's owner will have a full access on its device content and use while a visitor will only use a part of the provided services of the device.

Thus, according to the authentication method, an adaptation must be carried out on the generated proxies.

Indeed, the proxies should be able to provide to the applications and end user's authentication mechanisms similar to those used on the real device. More over, if necessary, the generated proxy must establish secured channels between the application and the real device.

Adaptation Code

The following perspective aims to improve the actual version of ATOPAI. In fact, currently an expert can add through ATOPAI an adaptation code using the high level adaptation and the external service APIs. We made the assumption that the code is correct, thus, it is directly injected in the templates without any verification.

Therefore, a parser like the JavaCC can be added to parse and verify the code before attaching it to the ontology.

Additionally, the expert specifies using ATOPAI the imported packages needed in his adaptation code.

So far, ATOPAI does not allow to add external dependencies from specific Jars and other native libraries.

Extending ATOPAI to support adding external packages dependencies would be handy for some adaptation use cases.

DOXEN

In the current global architecture, DOXEN is placed in the home network on a Set-Top-Box. It generates on the fly specific proxies based on the ontology alignments. Another alternative is still to be considered. It consists in placing DOXEN on the operator's site and download the generated proxies on demand to the digital homes.

Placing DOXEN on the operator's site could accelerate the proxy generation time since more powerful machines can be used. A comparison of the two alternatives need to be considered. x m l n s : s w r l=" h t t p : //www. w3 . o r g /2003/11/ s w r l#"

x m l n s : p r o t e g e=" h t t p : // p r o t e g e . s t a n f o r d . edu / p l u g i n s / owl / p r o t e g e#"

x m l n s : s w r l b=" h t t p : //www. x m l n s : n s 0= ' h t t p : // knowledgeweb . semanticweb . o r g / h e t e r o g e n e i t y / a l i g n m e n t '

x m l n s : r d f= ' h t t p : //www. w3 . o r g /1999/02/22 -r d f -syntax-ns# '

x m l n s : x s d= ' h t t p : //www. w3 . o r g /2001/XMLSchema# '

x m l n s : a l i g n= ' h t t p : // knowledgeweb . semanticweb . o r g / h e t e r o g e n e i t y / a l i g n m e n t# '>

  Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Ubiquitous Environment Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Dynamicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Heterogeneity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Ubiquitous System Characteristics and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.3 Eventing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.4 Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.5 Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.6 Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.7 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 The Digital Home Towards a Ubiquitous Environment . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Discovery and Adaptation for Dynamicity . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Interoperability for Heterogeneous Plug and Play Protocols . . . . . . . . . . . . . . . . . 2.4.3 Management for Devices and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.4 The Overall Actual Device and Application Architecture . . . . . . . . . . . . . . . . . . . 2.4.5 Discussion Around the Ubiquitous System Characteristics . . . . . . . . . . . . . . . . . . 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Service Oriented Architecture 3.1 SOA Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 SOA Characteristics meeting Ubiquitous System Ones . . . . . . . . . . . . . . . . . . . . . . . . 3.3 OSGi a Service Oriented Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 An Architectural Overview of OSGi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Base Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Machine Learning Based Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.2 Device Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LISTINGS Terminology ACS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Auto Configuration Server ATOPAI . . . . . . . . Alignment and annotation Tool framewOrk for Plug and plAy Interoperability BD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Base Driver BMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Basic Management Service CMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuration Management Service CP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Control Point CPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Customer Premises Equipment CWMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CPE WAN Management Protocol DCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Device Control Protocols DLNA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Digital Living Network Alliance DOXEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dynamic Ontology-based proXy gENerator DPWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Device Profile for Web Services GENA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . General Event Notification Architecture IGRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Intelligent Grouping and Resource Sharing KR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Knowledge Representation NAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Network Area Storage (Device) OASIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Advancing Open Standards for the Information Society OSGi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Open Service Gateway initiative OWL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ontology Web Language OWL-S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Semantic Markup for Web Services (OWL) PnP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Plug and Play SMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Software Management Service SOA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Service Oriented Architecture SOAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple Object Access Protocol SSDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple Service Discovery protocol STB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Set Top Box UDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UPnP Device Architecture UPnP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Universal Plug and Play UPnP-DM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UPnP-Device Management XSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XML Schema Definition Language Chapter 1

Figure 1 . 1 :

 11 Figure 1.1: Steps of the approach

Contents 2. 1

 1 Ubiquitous Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.2 Ubiquitous Environment Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.3 Ubiquitous System Characteristics and Challenges . . . . . . . . . . . . . . . . . . . . 35 2.4 The Digital Home Towards a Ubiquitous Environment . . . . . . . . . . . . . . . . . . 38 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Figure 2 . 1 :

 21 Figure 2.1: Ubiquitous Environments

  Contextual Memory t o p r e d i c t a c e r t a i n b e h a v i o r . The c o n t e x t u a l memory i s a s o r t o f c o l l e c t e d data from d i f f e r e n t s o u r c e s i n t h e e n v ir o n m en t . I t can be s e t by t h e u s e r o r c a l c u l a t e d o v e r a p e r i o d o f time . A p p l i c a t i o n s a c c e s s such i n f o r m a t i o n t o i n f e r b e h a v i o r s l i k e Bob ' s u s u a l s l e e p i n g h o u r s .

  Device and User l oc a t io n e n a b l e u b i q u i t o u s a p p l i c a t i o n s t o m i g r a t e s e r v i c e s t o t h e new l o c a t i o n and t h e r e f o r e o f f e r i n g a s e r v i c e c o n t i n u i t y . The c a r a p p l i c a t i o n tuned t h e c a r r a d i o t o l a t e s t c h a n n e l p r e v i o u s l y s e t by "Bob" a t home . Communication mediums between d e v i c e s and a p p l i c a t i o n s a r e h e t e r o g e n e o u s , t h e c a r u s e s a GPS, a l a p t o p u s e s a W i f i network w h i l e a smart phone u s e s a 3G network .

Figure 2

 2 Figure 2.2: Ubiquitous Work Environments At 3 PM, Bob starts a scheduled meeting with his coworkers, Charles present in Bob's office and Edouard in Japan. The work applications on both sites, mute all the four smart phones and other notifications since Bob and his colleagues agreed that they should not be bothered during the meeting. The work applications on both

  Device I n t e r o p e r a b i l i t y a l l o w s an e f f i c i e n t d e v i c e and s e r v i c e o r c h e s t r a t i o n t o a c c o m p l i s h a s p e c i f i c u s e r t a s k , TV, m i c r o p h o n e s and s p e a k e r s a r e o r c h e s t r a t e d t o e n a b l e a m e e t i n g s e s s i o n o r a home cinema e x p e r i e n c e .

  Security and Privacy a r e n a t u r a l l y e s s e n t i a l t o p r e v e n t m a l i c i o u s a c c e s s t o p e r s o n a l d e v i c e s c o n t a i n i n g s e n s i b l e and p r i v a t e i n f o r m a t i o n . Roles must a l s o be d e f i n e d t o s p e c i f y t h e a c c e s s and a l l o w e d o p e r a t i o n s o f each u s e r and a p p l i c a t i o n . For example , A l i c e ' s roommate has a c c e s s t o t h e NAS and can o n l y view some o f A l i c e ' s p h o t o s . No d e l e t e o p e r a t i o n s a r e a l l o w e d .

  wide variety of devices co-habit in ubiquitous environments and ranges from scare resource devices like sensors disseminated in the furniture detecting movement to a more abundant resource devices like laptops, set-top-boxes capable of processing high definition videos. This resource heterogeneity translates into a difference capabilities of storage, computing and communication. A temperature sensor has less computation power and transmission range then a Set-Top-Box capable of treating data streams and connecting to other devices in a wireless local area network.Ubiquitous applications are installed on devices in ubiquitous environments and share the common resources of their underlying physical devices. Therefore, the resource heterogeneity must be taken into account to allow a fluent ubiquitous computing experience for the end-user.Communication MediumsThe difference in resources imposes the use of different communication mediums, an embedded sensor uses low power radio communications such as infrared or low power radio waves mediums to exchange low amounts of data with other sensors or devices. Lights and Switches can use PLC (Power Line Communications) to exchange data over electrical power wires [Yousuf 07]. A smart phone uses a mobile telecommunication medium like the 3G or UMTS and can also switch to a wireless local area network to communicate with other devices like printers or TVs. Ethernet is also used to connect devices with relatively abundant resources like Set-Top-Boxes and TVs to exchange video/audio streams. A car uses the GPS to communicate with satellites using special radio waves needing more computational power and energy. A network is then generally constituted of devices communicating through the same medium. Gateways are used to connect different networks and devices. The communication medium diversity forces ubiquitous applications to support different medium communications or use a multitude of gateways in order to interact with multiple devices supporting heterogeneous communication means. Communication Protocols General purpose protocols are being proposed on top of the communication mediums, which can be classified into two categories, the IP based protocols and non-IP based protocols. The IP-based offer a lot of benefits such as packet routing, communication reliability, packets ordering, etc. Therefore, the IP-based protocols require devices with enough computational and transmission capacities. However, the non-IP based protocols like ZigBee[ZigBee 09] or Bluetooth[Bluetooth ] are deployed on low power and scare resource devices to exchange small amounts of data without the need to handle packet routing or reliable communications. The data packets size is smaller than the IP based protocols packets size. Therefore, no extra communication and computational capabilities are needed. For example, the IETF proposed recently the 6LowPAN[IETF 07] (IPv6 over Low power WPAN) for low power communication. On top of these protocols layers, specific application protocols layers are added for a specific purpose like the DHCP [Droms 97] (Dynamic Host Configuration Protocol) allowing devices to acquire an IP address. Other protocols have been used by devices to announce their services descriptions along with their capabilities like the SSDP (Simple Service Discovery protocol) [Goland 99] or the Web Services Discovery [OASIS 09b]. The GENA (General Event Notification Architecture) protocol or the Web Services Eventing [W3C 06b] are also employed by devices for notification and event broadcasting. Other protocols built on top of these previous protocols layers are proposed and implemented by devices to communicate in ubiquitous environments, like UPnP the Universal Plug and Play Protocol [UPnP ], DPWS The Device Profile for Web Services protocol, IGRS the Intelligent Grouping and Resource Sharing (IGRS)

Figure 2 . 3 :

 23 Figure 2.3: Ideal Ubiquitous System Architecture

Figure 2. 4 .Figure 2 . 4 :

 424 Figure 2.4: Remote Administration through CWMP and UPnP-DM

  Nikolaidis et al.[Nikolaidis 07] proposed a bridge between the CWMP and the UPnP protocols to extend the operator control to non-CWMP devices in the LAN. For each UPnP device, an entry in the CPE data model is added along with its supported services, actions and state variables. The main drawback of the solution is that the operator can invoke via CWMP supported actions on the UPnP device, i.e. if the UPnP device supports only interaction commands (print, turn on light, getTime), the management operations remains limited.To extend the management operations to the LAN, Bottaro et al[Bottaro 08a] proposed the UPnP Device Management (UPnP-DM) [UPnP 11] which was adopted by the UPnP Forum. The UPnP DM is a standard profile which consists of two required and one optional services. The Basic Management Service (BMS)[UPnP 10a] allows to perform basic operations like rebooting or resetting the device, performing a "Ping", a "Traceroute" and self-tests diagnostics. It also allows to retrieve the device status and logs. The Configuration Management Service (CMS)[UPnP 10b] is designed to retrieve configuration information such as IP and MAC address, the DNS and DHCP servers IP addresses, the device CPU and memory current usage. And finally, the optional Software Management Service (SMS) [UPnP 10c] allows to manage software entities, (un)install, update, stop/start service. A proxy bridge between UPnP-DM and CWMP is proposed in[Broadband 10b] to extend the operator's management operations from the WAN to the LAN.

Figure 2. 5 :

 5 Figure 2.5: Overall Layers System Architecture

Figure 3 . 1 :

 31 Figure 3.1: Service Oriented Architecture paradigm

Figure 3 .

 3 Figure 3.2: Service Composition

Figure 3 . 3 :

 33 Figure 3.3: OSGi on residential gateways

  4, the OSGi framework is an execution framework placed on top of a Java Virtual Machine (JVM). Deployment units referred to as Bundles are installed on the OSGi framework. Once started, a bundle implementation either offers a service, consumes a service or simply provide an API for other bundles. The services cooperate at the OSGi framework and can also interact with external services or native OS APIs.

Figure 3 . 4 :

 34 Figure 3.4: Multi Layer Architecture of an OSGi Framework [Richard S. Hall]

Figure 3 . 5 ,

 35 a bundle contains Java binary classes and other resources such as files, images or native APIs packaged in a Java ARchive (.jar) file. Each JAR file also packages a manifest file containing metadata about the bundle. The OSGi manifest holds different information about the bundle such as its name, version, author and vendor. The manifest also expresses the bundle package dependency imported from other bundles. It also exposes the exported packages shared on the framework and can be used by other bundles. Specifying the bundle dependency allows the OSGi framework to manage the bundle resolution automatically. The manifest also contains the Activator class name which is equivalent to the main class in a standard JAR file. The Activator class allows to the OSGi framework to handle the module life cycle by invoking its specific methods start and stop as shown in section 3.3.1.2.

Figure 3 . 5 :

 35 Figure 3.5: OSGi Bundle

Figure 3 . 7 :

 37 Figure 3.7: The Service Hooks

  Currently, there is a UPnP base driver implementation published by Apache [Apache b]. Multiple DPWS Base Drivers implementations has been proposed. Orange Labs, proposed through The HomeSOA [Bottaro 08b]

Figure 3 . 8 :

 38 Figure 3.8: OSGi Base Drivers

  Alliance (DLNA) [DLNA 03]. DLNA is a certification organism ensuring that a device description, its hosted services and behavior are conformed to the standard specifications and profiles proposed by the UPnP Forum. A DLNA certified UPnP device supports the required standard behavior and description published by the UPnP Forum. The DLNA certification guarantees that a UPnP device description and behavior are conformed to the standard specifications. Thus, DLNA pushed the adoption of the UPnP protocol among various manufacturers and telecoms operators. Additionally, device manufacturers or vendors can also extend the device standards and propose their own services on top of the standard specifications, as shown in

  interaction goes through the standard Simple Object Access Protocol (SOAP)[W3C 00]. The UDA also specifies a UPnP Device Template which details the description format and structure to be announced by each device on the network. The template model is shown in Figure4.1, each device hosts one or more services, a service implements one or more actions and state variables. The actions have input and output arguments related to a state variable.

Figure 4 . 1 :

 41 Figure 4.1: UPnP Device Description Structure

  12 < s e r v i c e S t a t e T a b l e >13< stateVariable sendEvents =" no " > 14 < name > Ta rge t </ name > 15 < dataType > b o o l e a n </ dataType > 16 < defaultValue >0 </ defaultValue > 17 </ stateVariable > 18 </ s e r v i c e S t a t e T a b l e > Listing 4.2: UPnP SwitchPower Service Description (Simplified)• The UPnP Device Control Protocol defines a standard profile description with mandatory and optional services, actions and state variables that manufacturers need to implement. Its details for each device type the required and optional services along with the actions, their input/output arguments, data types and their range. A UPnP device conform to the UPnP Device architecture and a UPnP Device Control Protocol allows any UPnP Control Point (conform to the UDA and DCP) to interact with the device and reach a universal plug and play experience. The UPnP Forum proposed multiple standard profiles covering different domains. For example, the Home automation domain includes standard profiles for solar protection blinds, lighting controls and HVAC (Heating, Ventilating, and Air Conditioning) systems. An Internet gateway and WLAN Access point profiles are proposed in the networking field. The multimedia and printing domains also include standard profiles for a printer, camera and scanner devices. For instance, the UPnP standard printer profile promoted by the UPnP Forum under the Device Control Protocol details the printer behavior and interaction along with the supported service and actions names.The UPnP Forum also proposes a UPnP Audio/Video Architecture [UPnP 02] to enable media content sharing on the network. The UPnP AV architecture is implemented on the same level as the DCP on top of the UDA. It defines the three following profiles: Media Server (MS), Media Renderer and an AV Control Point. The MS profile is used to describe a device capable of offering media content for example, MP3 players, DVD Players, radio. A Media Renderer profile is used to describe a device capable of playing or rendering a media, like TVs and stereos speakers. And finally, the AV Control point allows the discovery and the browsing in the media server content. The AV control point also establishes the interaction between the UPnP Media server and the Media Renderer.Moreover, the UPnP Forum proposes other standard profiles such as the telephony, QoS, remote Access and even UPnP Low Power device to save energy devices. the UPnP Forum is currently working on the security and privacy support in the UPnP v2.

Figure 4 .

 4 Figure 4.2: DPWS Device Description Structure

1 3 4

 3 < in SOAP:ENV WS -Discovery > 2 < wsd:Types > w s d p : D e v i c e w p r t : S i m p l e L i g h t </ wsd:Types > < w s d l : d e f i n i t i o n s name =" Lighting " ... 5 < xsd:simp leType name =" PowerState " > 6 < x sd :r es t ri ct io n base =" xsd:token " > 7 < x sd :e nu m er at io n value =" ON " / > 8 < x sd :e nu m er at io n value =" OFF " / > 9 </ x sd :r e st ri ct i on > </ xsd: simpleT ype > 10 < xsd:element name =" Power " type =" tns:Po werState " / > 11 12 < wsdl:message name =" SwitchMsg " > 13 < wsdl:part name =" PowerOut " element =" tns:Power "/ > 14 </ wsdl:message > 15 16 < wsdl:portType name =" SwitchPower " ws e: Ev e nt So u rc e =" true " > 17 < wsdl:ope ration name =" Switch " > 18 < wsdl:input message =" tns:SwitchMsg "/ > 19 </ wsdl:o peration > . . . 20 </ w sd l : d e f i n i t i o n s > Listing 4.3: DPWS SwitchPower Service Description (Simplified) The service description file is expressed using the Web Service Description Language [Christensen 01] and XML Schema Definition Language [W3C 01] to represent the content and element data type. The Listing 4.3 shows a part of a simple light [SOA4D a] DPWS device description. In the Listing 4.3, lines [16-19] defines the operation Switch and its input message provided by the service SwitchPower. Lines [12-14] defines the message content and structure by referring to the element Power. The element structure and content is described in lines[4-10], the type is a token having values ON or OFF. In the simple light example, the element is flat, meaning there is no other elements embedded in the Power element. The WSDL specification allows a hierarchical structure of elements, an element can contain more than one element.

Figure 4 .Figure 4 . 3 :

 443 Figure 4.3: Hierarchical Parameters in the WSDL Printer description

Figure 4 . 4 .

 44 Figure 4.4.Discovery is the first step where consumers discover devices providing services. The WS-Discovery[OASIS 09b]

Figure 4 . 4 :

 44 Figure 4.4: DPWS Exchange Example [Microsoft 06]

Figure 4 .

 4 Figure 4.4 shows an example of interaction. The client first sends a Probe message searching for a printer device. The printer device listening to Probe messages replies with a probe match message containing a WS-Addressing reference, a UUID (Universal Unique IDentifier) along with the device capabilities. The client can use the WS-Addressing [W3C 06a] reference, shown in messages [3-4] of Figure 4.4, and ask for a resolve match to retrieve more information to identify the device such as the IP address.

ure 4 . 4 ,

 44 messages[5][6][7][START_REF]1 A Fragment of WordNet 2.1 Results for the word[END_REF] are initiated by the client to retrieve additional information about the device and its supported services and capabilities. The device responds with the meta data and the WSDL. Those messages are optional if the client knows that the device is conformed to a standard profile.Control The interaction between devices and consumers is message based over the SOAP protocol. DPWS defines also how the message and its content are serialized during the exchange. The message format, structure and content are defined in the WSDL file, see Listing 4.3. The WS-Addressing[W3C 06a] is used on top of SOAP and involved in different features. It is used to exchange addressing information to identify a device and a service through a SOAP level address. In Figure4.4, messages[9][10], the client starts a printing job on the device which responds that the job is started. The interaction content follows the description exposed in the WSDL file.

  alliance was established in 2003 to provide interoperability between three worlds called the "3C": Computing (laptops, PC), Consumer Electronics (TV, Set-Top-Box) and Communication (mobiles, PDAs). In 2005, the IGRS standard was formally approved by the Ministry of Information Industry of China which became the governing body of the IGRS working group. International corporations like Philips, LG, Cisco and STMicroelectronics joined the alliance transforming IGRS into a high potential standard protocol at least in the Asian markets.

  UPnP uses an XML template description format, whileIGRS uses the Web Service Description Language to express the device and service description. Interoperability guidelines between UPnP and IGRS are provided in chapter 11 of [IGRS 06]. UPnP devices can be discovered by IGRS control points and IGRS devices can be discovered by UPnP control points. Xie et al proposed in [Xie 09] a simple technical solution for syntax header conversion to allow UPnP and IGRS device discovery and interaction. In [Bottaro 06], coexistence concerns has been pointed between UPnP and IGRS since both protocols use the same multicast address. The coexistence has been made possible, since IGRS accepted a multicast modification for the IGRS device discovery.

Figure 4 . 5 :

 45 Figure 4.5: IGRS System Architecture

  Bonjour protocol (formerly known as Rendezvous [RFC 3927]) is the Apple implementation of the Zero Configuration Protocol [Steinberg 05] proposed by the IETF. Bonjour is already supported in the Apple products such as i Tunes and i P* devices. Some printer and network storage devices also support Bonjour. Bonjour is based on the already widely deployed DNS protocol and server. Its is involved in the addressing, naming and service discovery. The multicast-DNS [Apple 11b] handles the addressing and the naming while the DNS Service Discovery [Apple 11a] is used to advertise and discover services. Devices hosting Bonjour announce their hosted services in conformance to the following format Device-Name. protocol. transportProtocol.Domain port number. For instance, an Apple Bonjour printer would announce the following service Apple LaserWriter 8500. ipp. tcp.local. Port 631. The DeviceName is a unique name on the local network. Bonjour specifies how the device should proceed to resolve conflicting names on the same local network. A Bonjour device must also announce its supported protocol and type.

  2 . n o t e =: a d m i n u r l=h t t p : / / L a s e r W r i t e r 8 5 0 0 . l o c a l . / r e n d e z v o u s p a g e . html 3 . p r o d u c t =( L a s e r W r i t e r 8 5 0 0 ) . p d l=a p p l i c a t i o n / p o s t s c r i p t 4 . Punch =3.PaperMax=l e g a l -A4 Listing 4.4: LPR TXT record for a PostScript printer [Apple 05] Bonjour only supports the service discovery and annunciation. Thus, other protocols are used on top of Bonjour for interaction and control. There is no unified control protocol, the interaction protocol used is specific to the device domain and type. For instance, the Media Server iTunes(v4.2) uses the Digital Audio Access Protocol (DAAP[DAA 05]) and the Digital Media Access Protocol (DMAP), while the printers use multiple interaction protocols: IPP (Internet Protocol Printing) [RFC 2567] or/and LPR (Line Printer Daemon Protocol) [RFC 1179]). Each interaction protocol has its own description format, in LPR for example, the description is ASCII-based, set in TXT LPR records[Apple 05]: product=(LaserWriter 12/640 PS), adminurl=http://printer.local./path/configpage.html. DAAP uses another description to exchange information between iTunes (media server) and a client requesting a list songs, for example, daap.songformat, daap.songartist. Thus, Bonjour only supports addressing, announcement and discovery and relies on other protocol layers such as the LPR or DAAP for control and eventing.

  4, plug-n-play protocols define their own underlying protocols each adapted to its own environment. UPnP and IGRS use the SSDP protocol for discovery. While DPWS uses the WS-Discovery along with WS-Transfer, WS-Addressing and WS-MetaDataExchange to discover devices and retrieve additional information. Bonjour employs the multicast DNS and the DNS-SD for the service discovery and annunciation.The four protocols support the peer-to-peer device discovery mode through multicast where devices and applications listen to the arrival and departure of devices on the network. Additionally, DPWS, IGRS and Bonjour support a centralized mode for the device discovery. In such mode, devices arriving on the network register their description in the central registry service, applications searching for devices directly query the central registry. Such mechanism allows to reduce the communications on the local network. In DPWS, the service registry is supported by the Proxy Discovery, in IGRS it is the master-slave mechanism, while in Bonjour it is handled by the DNS server.For the control, UPnP, IGRS and DPWS use the SOAP protocol. IGRS changed the header syntax of the messages when it handles the control and the eventing. Additionally, DPWS uses the MTOM mechanism to optimize the transfer of large data between the devices and the consumers. Bonjour is used only for the device and service annunciation and discovery, thus for the other features, specific protocols depending on the device domain (printing, multimedia) are employed.

  Device and Service Description : During the annunciation each device exposes general device information (id, manufacturer, model etc), supported service interfaces along with associated action signatures and parameters. Each of these protocols use a format description to expose its information and describe its supported services. DPWS and IGRS use the Web Service Description Language to expose the description while UPnP uses the UPnP XML template format. Bonjour uses also a limited proprietary format and dependent on the device type and domain. Table 4.5 resumes the description concepts used between the UPnP, DPWS and IGRS protocols. UPnP and IGRS uses the concepts devices and services. While DPWS has more a web service oriented vision and therefore uses hosting and hosted services concepts. UPnP refers to the capabilities supported by a service as an action while IGRS and DPWS employs the concept of operation from the Web Service Description Language. The UPnP state variables are the parameters handled by the UPnP actions in general are flat without a hierarchical structure, except for the UPnP DM [UPnP 11] parameters. While in the WSDL, a message content can be a simple element or a complex one. A complex element is composed of simple elements and complex ones, as shown in Figure 4.3.

Description

  Content : In addition to the protocol stacks and description format, the description content constitutes another layer of heterogeneity. As mentioned before, those Plug-and-Play protocols target multiple similar domains. Thus, two devices having the same type support different protocols. For instance, an HP printer device supports the UPnP protocol while another Xerox printer device supports the DPWS protocol. And even more, device manufacturers propose the same device type with different plug and play protocols.Even though, two equivalent device types support the same basic functions, the content is expressed syntactically differently. Thus, the functions are semantically equivalent but expressed in different syntax.For instance, on a standard UPnP printer, the action CreateURIJob is equivalent to the association of two actions CreatePrintJob and AddDocument. Another example can be found between the UPnP and DPWS lights. On a DPWS light [SOA4D b], Switch(Token ON/OFF) is semantically equivalent to the SetTarget(Boolean) on a UPnP light[UPnP ].

"Contents 5 . 1

 51 Unlike the human mind, computers do not have such a transparent mechanism for acquiring and representing knowledge internally, just by themselves. They rely on humans to put the knowledge into their memories." -Gašević et al [Gašević 06] Ontologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 5.2 Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.3 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

  Moreover, each protocol defines specific profiles such as light, printer, clock with different representation but similar semantics. For instance, on two light devices, the Switch and the SetTarget actions are syntactically different but semantically similar since they accomplish the same basic task which is turning on/off the light. Thus, as pointed out in the first two roles of Davis et al [Davis 93] definition, the description content captures the reality which is clearly perceived almost equivalently from the semantic point of view. This lack of semantic description adds another layer of non expressivity to the existing plug and play description languages.

Figure 5 . 1 :••

 51 Figure 5.1: Knowledge Representation Techniques

Figure 5 .

 5 Figure 5.2: An Ontology Example

  Figure 5.2 shows a part of a device ontology. The domain is the Plug and Play devices and the concepts of the domain in this example are Device, Service and Action. The relations between the concepts are hasService, hasAction and Is-A. The Light, Switch and SetTarget are instances of these ontology concepts. A Light is a device with a service Switch having an action SetTarget, see Figure 5.2. Thus, the ontology represents the knowledge semantically through relations connecting the main concepts of a domain in a machine interpretable form . Another example is shown in Figure 5.3 which is proposed in [Pierson 09] to represent a context. The Device concept is a general concept in this ontology. The other devices like a TVSet, a Sensor or a ClimStation are represented as sub-concepts of the device concept. The location concept is used to represent information of places such as indoor/outdoor, a general type of room or a hotel room. The physical property is used to represent environmental properties such as temperature, brightness of moisture. These concepts are connected together through properties. For instance a Sensor concept informs a Physical property like the light intensity. This information about intensity is related to a location. Instances are also modeled in this ontology, the HotelRoom number 1345 is an instance of the hotel room concept which belongs to the location concept. This ontology shown in Figure 5.3 allows to represent various elements in an environment and most importantly the relations between. In the appendix B.1, Listing B.1 shows how this ontology is represented in an XML based ontology language OWL which is presented in section 5.1.3.1.

Figure 5 . 3 :

 53 Figure 5.3: A Context Representation Ontology Example [Pierson 09]

Figure 5 .

 5 Figure 5.2, the Is-A relation links Light to Device, while hasService links the two concepts Device and Service.

  [Noy 01], [Devedžić 02] along with surveys such in [Staab 09], [Studer 07] (Chapter 3) and [Gašević 06] (Chapter 2). They all commonly agree on the fundamental rules pointed out by [Noy 01] which are the following:

  Thus, semantic web services according to McIlraith et al [McIlraith 03], is the expressivity augmentation of web services by adding semantics to their description. The semantic augmentation is achieved by adding annotations to the service description through ontologies which represents the knowledge in a machine interpretable form. Thus, reinforcing the description of web services allows to achieve a greater level of service discovery, composition and interaction.5.1.3.1 Ontology LanguagesVarious ontology languages have been designed and proposed for standardization in the literature to semantically express a domain. We outline in the following the most relevant ontology languages based on XML: RDF(S) The Resource Description Framework [W3C 99] is based on XML and XML Schema. It has been proposed to add semantical annotation on top of XML. RDF is expressed using a triplet (Object-Attribute-Value) : The object refers to an identifiable web resource, like a web page, a title or an author. The attribute represents a property linking an object to its value which can be a text or a resource. For instance, the author Bob of a web page www.abc.com/MyPage.html can be expressed in RDF [W3C 99] as shown in Listing 5.1. 1 < r df :D es c ri pt io n rdf:about =" http: // www . abc . com / myPage . html " > 2 < btr:hasAuthor rdf:resource =" x:Bob "/ > 3 </ r df :D e sc ri pt i on > Listing 5.1: RDF Example

  represent the user's view and expectation of a web service functionality and execution. And finally, Mediators are used to resolve incompatibilities between data and terminologies from different ontologies. Such mediators can use ontology alignment results to resolve incompatibilities. Ontology alignment techniques are detailed in chapter 8.

1 ?

 1 device:CLASS , ? serviceX:CLASS , ? a p p l i c a t i o n : C L A S S , ? serviceY:CLASS 2 SELECT ? d e v i c e s u b C l a s s O f p r o v i d e s S e r v i c e some ? s e r v i c e X , 3 ? a p p l i c a t i o n s u b C l a s s O f r e q u e s t s S e r v i c e some ? s e r v i c e Y 4 WHERE ? s e r v i c e X e q u i v a l e n t T o ? s e r v i c e Y 5 BEGIN ADD ? d e v i c e s u b C l a s s O f canBindWith some ? a p p l i c a t i o n END; Listing 5.3: OPPL Query Example Ontologies allow to semantically represent a domain while the rules combined with logics applied on ontologies enable to infer and extract information from what is already represented.

A

  layered model architecture is proposed by the Model Driven Architecture (MDA) [OMG 03] which is promoted by the Object Management Group (OMG) 1 . The MDA defines standard models and languages like the Unified Modeling Language (UML) [OMG 97] to specify concepts at the M1, M2 and the Meta Object Facility (MOF) [OMG 06] to specify concepts at the M3 layer. Additionally, relations between concepts along with the allowed cardinality is specified by each upper layer. The MDA provides a four layer architectural modeling space: M0 as the instance layer, M1 for the model, M2 for the meta-model and the M3 for metameta-model layer.

Figure 5 . 4 :

 54 Figure 5.4: Model Driven Architecture, Generalized Layers Example

  defined semantics in order to remove ambiguity. Additionally, rules represent a powerful mechanism to reason and extract new information from what is already represented. Different rule languages are proposed to query ontologies or to detect predefined patterns and apply transformation and modifications on a ontology. Models also help to capture knowledge on different levels of abstraction. The Model Driven Architecture specified a four layer architecture to abstract information and systems. Additionally, the MDA allows the specification of concepts manipulated at a certain layer only from concepts defined at an upper layer. This allows to express the concepts in a well defined and precise semantics. Combined with the Model Driven Engineering techniques, models offer an automation of software components creation based on high level descriptions through code generation techniques.

  Figure 6.1, points out the three characteristics involved in achieving a complete plug and play device interoperability: Discovery, Control and Eventing. The three characteristics cover the L1, L2 and L3 layers mentioned in chapter 2 where heterogeneity has settled in the protocols stacks, the device and service description format, and the description content. We overview next, the heterogeneity in each layer.

Figure 6 . 1 :

 61 Figure 6.1: Plug And Play Interoperability Layers

Figure 6

 6 Figure 6.2: A Common Application Layer

Frameworks"

  three knowledge representation techniques: ontologies, rules and models. Ontologies are used to capture semantic relations between important concepts of a domain. Rules combined with logics offer formal semantics to extract and reason on information from what is already represented. And finally, models allow to capture information with different details. Models also offer an automation of the software components creation based on high level descriptions through code generation techniques. The rest of the document is organized as follows: the next part overviews related work in the literature proposing different solutions to hide the service description heterogeneity and allow device interoperability through service adaptation. Then part III details our contribution and shows how a combination of the three representation techniques can be used to resolve the plug and play device heterogeneity. Because of the lack of interoperability, we can lose billions of dollars in productivity." -Charles Giancarlo Contents 7.1 Common Ontology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 7.2 Abstract Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 7.3 Uniform Language/Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 7.4 Comparison & Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Figure 7 . 1 ,

 71 shows an example of a common ontology. The Thing concept is the root element in an ontology. The Printer concept for example is a sub-concept of the Device concept.

Figure 7 . 1 :

 71 Figure 7.1: A Common Ontology Example

7. 1

 1 .2 PERSE: PERvasive SEmantic-aware MiddlewareMokhtar[Mokhtar 07] proposes a semantic middleware which provides a semantic service matching, registration and composition. In [Mokhtar 06], the semantic discovery and matching is based on the service description annotation with concepts from one common or various ontologies. In [Mokhtar 06], they proposed a semantic matching algorithm based on the semantic distance between concepts in the ontology, i.e. the levels separating two concepts in a common ontology. In [Mokhtar 08], they proposed a service matching based on Paolucci's semantic algorithm. They also take into account the service non-functional properties such as the QoS during the service matching.

1

  Figure 7.2 shows the internal architecture of the MySIM middleware. It is composed mainly of 4 modules. The Translator module translates OSGi services and properties into the MySIM service model (see Listing 7.2) which holds semantic service description using the OWL-S language. The OSGi service implementation already contains the semantic annotations relating to concepts from the common ontology. The annotation are specified on the OSGi service properties, see section 3.3 and Listing 3.1 for an example of service properties.Based on this translation, the Generator module proceeds with the semantic and syntactic matching of services and also composes services based on their functional services and descriptions. The matching is based on the Java introspection techniques. First the syntactic matching takes place, then, the semantic matching which relies on Paolucci 's algorithm assisted with a semantic reasoner.The Evaluator module verifies that the selected services for composition or adaptation are conform to the requested or provided non-functional properties. Moreover, the Evaluator monitors the arrival and departure of services in order to compose new services. It requests from the Generator module syntactic and semantic compatible services to compose.The Builder is the executor of the adaptation which applies an interface transformation or a composition of one or multiple services. The Builder composes a new bundle from the selected services, it either copies their implementation in the new bundle or simple implement a call to the appropriate service. The Builder then generates and installs the new bundle for the newly adapted service.

Figure 7 .

 7 Figure 7.2: MySIM Middleware [Ibrahim 08]

  Domestic OSGi Gateway [Bonino 08b, Bonino 08a] uses a common ontology as a taxonomy to classify devices and as an abstract model to generate specific code through transformation rules. DOG aims to provide interoperability between basically simple devices (simple light and switches). The DOG framework targets mainly simple protocols such as Konnex [Konnex 04], MyOpen and X10 [Smarthome 04]. The DogOnt ontology is inspired from the European Home System taxonomy which provides a classification for multiple goods in a home environment. Thus, the DogOnt captures a device classification of home equipments. Additionally, functionalities are described apart and then associated to devices in the ontology according to its category.The functionalities are separated into three different abstract categories: control, notification and query. The control functionality models the ability to control a device such as a light. The notification captures the ability of a device to inform about its internal state change. And finally, the query functionality which describes that a device can be queried to retrieve information such as its internal state values. Figure7.3 shows two instances represented in the ontology, a dimmer lamp and an on/off switch and the relations between. The figure also exposes the complexity of representation a device in the DOG approach. An added device such as dimmer lamp must be connected with other existing actions and commands in the abstract ontology.

Figure 7 . 3 :

 73 Figure 7.3: Partial DogOnt Fragment showing Dimmer Lamp and Switch Instances [Bonino 09]

Figure 7 . 4 :

 74 Figure 7.4: (a) EnTiMid Architecture (b) An EnTiMid-UPnP Model Mapping example [Nain 08]

Figure 7 . 5 :

 75 Figure 7.5: PervML Models [Munoz 04]

  Providers model offers the different devices in the system along with their supported operations. The Component Structure model binds the available instances specified by the system analyst in the structural model to the available device instances. And finally, the functional model is used to specify the operations to be executed on a service invocation.After the modeling step, the PervML models are transformed automatically through code generation techniques into an executable OSGi bundles. An OWL ontology also captures the services functionalities and other information to allow a runtime adaptation through reasoning on situations such as the user's location or to predict his next action.Other MDA-based works have been proposed in the literature. Coopman et al [Coopman 10], proposes a common ontology used as an abstract model to represent devices and their functionalities along with their interoperable relations. The ontology in their approach is extended from both the SOUPA [Chen 05] and the DogOnt[Bonino 08b]. The WComp[Tigli 09a] middleware also adopts a high level model to specify service components using an graphical user tools. From the high level specification, specific components are generated and installed in the WComp middleware to allow service composition. The Gator Tech Smart House [Helal 05,Helal 09] also provides the Device Description Language (DDL)[Chen 09] to specify device's information along

Figure 7 .

 7 Figure 7.6: The HomeSOA Architecture [Bottaro 08a]

Figure 7 . 7 :

 77 Figure 7.7: UMB Architecture and a Virtual Device Proxy information [Moon 05]

Figure 7 . 8 :Figure 7 .

 787 Figure 7.8: The DomoNet Architecture [Miori 06]

  2, Figure B.1 shows the PrintTicket element, an input parameter used to invoke the action CreatePrintJob in order to print on a standard DPWS printer.

  resentation mediations. Paolucci do not presents an adaptation realization but focuses only on the service matching. PERSE [Mokhtar 07] and MySIM [Ibrahim 08] both use a common ontology as a taxonomy holding the concepts in a hierarchical classification. MySIM focuses only on OSGi services, thus eliminating protocol use and mediation issues. It uses a translator module to represent OSGi service description in a common model. Then, the content heterogeneity is resolved through reasoning and matching of services' descriptions using the common ontology. MySIM supports the service transformation and composition. The transformation usually adapts parameters while the composition relies on existing services to propose new capabilities. New bundles are created to realize the adaptation. MySIM first introspects the byteCode of the existing services' bundles, then the byteCode is replicated in the new bundle in order to add the translation behavior, the redirection mechanisms or to compose new services. PERSE [Mokhtar 07] targets the UPnP and SLP protocols. It also translates the service descriptions into a common model, then applies reasoning and matching algorithms. PERSE provides in [Mokhtar 07] a detailed

  an expert. Such rules enable the automatic generation of high level concepts into a lower level concepts such

  example a DPWS printer has 2000 lines of WSDL description [Microsoft 07], Figure B.1 in the appendix B.2,shows an input parameter of the print DPWS action. The other drawback resides in the annotation of the plug and play services with concepts from the common ontology. Two options related to the service annotation appear. This first is to impose to competitors such as manufacturers and standardization committees a common ontology where its concepts are used to annotate devices' and services' descriptions. Obviously, this option is too optimistic, there had never been a unified description in the proposed standard profiles (UPnP, DPWS and IGRS) for the same device type. The second option consists in applying a manual annotation on each device description, i.e. the expert manually annotates the description by applying a mapping between the semantics of the description and the concepts of the common ontology. However, annotating the description manually can be difficult and error prone. Besides, new devices holding new concepts not yet represented in the common ontology, require adding a new device concept to the common ontology and connecting it to the other existing entities. The update can produce an incoherent ontology since a new type can have cross semantics with more than one existing concept [Noy 04].

Contents 8 . 1

 81 Matching Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 8.2 Ontology Alignment Tools & Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . 112 8.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Figure 8 . 1 :

 81 Figure 8.1: Integrated Approach

Figure 8 .

 8 Figure 8.2: Federated Approach

Figure 8 .

 8 Figure 8.3 shows an example of an alignment between two ontologies. Each value on the arrows refers to the confidence relation between the concepts. This confidence is referred to as the Similarity value (usually normalized) within an R + [0,1] interval. The similarity value δ is calculated by various algorithms and matching techniques presented in the next section. A high value between two entities indicates a potential match according to the used matching technique. Some techniques calculate the normalized dissimilarity δ which corresponds to 1 -δ.

Figure 8 . 3 :

 83 Figure 8.3: An Alignment Example Between Two Ontologies [Euzenat 07]

1 .< Sense 1 >

 11 WordNet offers different relations between senses such as antonyms (up = down) and synonyms where two words having different syntax refer to the same semantics. WordNet also proposes hypernym relation which is a (superconcept/subconcept) relation. : p r i n t e r , pressman --( someone whose o c c u p a t i o n i s p r i n t i n g ) => s k i l l e d worker , t r a i n e d worker , s k i l l e d workman --( a worker who has a c q u i r e d s p e c i a l s k i l l s ) => worker --( a p e r s o n who works a t a s p e c i f i c o c c u p a t i o n . . . ) => p e r s o n , i n d i v i d u a l , someone , somebody , m o r t a l --( a human b e i n g ; " t h e r e was t o o much f o r one p e r s o n t o do " ) . . . < Sense 2 > : p r i n t e r --((computer s c i e n c e ) an o ut pu t d e v i c e t h a t p r i n t s t h e r e s u l t s o f data p r o c e s s i n g ) => p e r i p h e r a l , computer p e r i p h e r a l , p e r i p h e r a l d e v i c e --( ( computer s c i e n c e ) e l e c t r o n i c equipment c o n n e c t e d by c a b l e t o t h e CPU o f a computer ; " d i s k d r i v e s and p r i n t e r s a r e i m p o r t a n t p e r i p h e r a l s " ) => e l e c t r o n i c equipment --( equipment t h a t i n v o l v e s t h e c o n t r o l l e d c o n d u c t i o n o f e l e c t r o n s ( e s p e c i a l l y i n a g a s o r vacuum o r s e m i c o n d u c t o r ) => equipment --( an i n s t r u m e n t a l i t y needed f o r an u n d e r t a k i n g o r t o p e r f o r m a s e r v i c e ) . . . < Sense 3 > : p r i n t e r , p r i n t i n g machine --( a machine t h a t p r i n t s ) => machine --( any m e c h a n i c a l o r e l e c t r i c a l d e v i c e t h a t t r a n s m i t s o r m o d i f i e s e n e r g y t o p e r f o r m o r a s s i s t i n t h e p e r f o r m a n c e o f human t a s k s ) => d e v i c e --( an i n s t r u m e n t a l i t y i n v e n t e d f o r a p a r t i c u l a r p u r p o s e ; . . . Listing 8.1: A Fragment of WordNet 2.1 Results for the word "printer" Listing 8.1 reproduces the results provided by WordNet 2.1 for the word "printer". There are three senses, the first one refers to the printer profession, the results show that a printer and pressman are synonyms, and a printer is a subconcept of skilled worker, worker and person. WordNet also provides a definition for each concept expressed in a natural language as shown in the Listing 8.1. The senses 2 and 3 refers to the word printer as a device and a machine. Various techniques operates on the results of WordNet. Some techniques only exploit the synonyms relations provided by WordNet and consider that two concepts are equivalent if they are simply synonyms. Other techniques rely on the hypernym structure i.e. the subconcept/superconcept relations provided by WordNet to measure the similarity between concepts. For example, it can take into consideration the number of structural levels separating the two concepts. And some techniques take advantage of the definition provided in a natural language and base their similarity measures on the detection of the searched concepts in such definition. For example, in Listing 8.1, the sense 2 holds the word device in the definition of printer, thus a correspondence between device and printer can be established based on the definition provided by WordNet. A comparison of some WordNet based techniques is provided in [Budanitsky 06].

Figure 8 . 4 :

 84 Figure 8.4: An Internal Structure Alignment Example Between Two Ontologies [Euzenat 07]

  schemes. They apply a down-top matching where the similarity between two elements is computed from the similarity between their children. The COMA also allows users to validate and edit correspondences.

  Unfortunately, COMA++ is not an open source tool and only a prototype version with reduced features can be obtained for a limited time.ROMIE [Elbyed 09] is based on the Ontology Mapping Interactive and Extensible environment (OMIE)[Elbyed 09]. The OMIE framework targets biomedical domain where large ontologies need to be accessed and mapped with other ontologies to retrieve additional information. The OMIE framework applies preprocessing normalization steps and uses a matching strategy combined of syntactical, linguistic, structural and semantic matchers. Each matcher is assigned with a confidence weight. OMIE uses two external resources when applying the linguistic matcher: WordNet and the Unified Medical Language System 2 a sort of a biomedical dictionary with specific vocabularies related to the biomedical domain. OMIE also provides structural matching algorithms with two modes top-down and down-top with different confidence degrees. The structural matching techniques takes into account when calculating the similarity value between two concepts the similarities of their subconcepts and superconcepts. OMIE [Elbyed 09] also supports a semantic matcher which takes into account the semantic relation between concepts such as the equivalent, symmetric and transitive relations. The ROMIE framework extends OMIE by integrating resource instances to enhance the matching process.

"

  Complexity can neither be created nor destroyed; it can only be displaced." -Hurst's LAW Contents 9.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 9.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 9.3 An End To End Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 9.4 Device Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 9.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

Figure 9 . 1 :

 91 Figure 9.1: UPnP as a Common Application Layer

Figure 9 .

 9 Figure 9.2: Architectural View of the UPnP-DPWS Proxy

Figure 9 .

 9 Figure 9.3 outlines the architecture of our approach. The first (MDE) M0 layer represents the heterogeneous plug-n-play devices with their descriptions expressed in different formats. UPnP uses an XML template format while DPWS and IGRS use the standard Web Service Description Language (WSDL). Each description uses different semantics: UPnP uses devices, services, actions and state variables while the WSDL uses hosting and hosted services, operations and messages.

1

  http://upnp.org/sdcps-and-certification/standards/sdcps/

Figure 9 . 3 :

 93 Figure 9.3: Overview Of The Approach

Figure 9 . 4 :

 94 Figure 9.4: The Modular Architecture

  5, one per protocol to generate ontologies based on the devices' descriptions. For example, a UPnP OWL Writer, subscribes to the arrival of UPnP devices. Once a UPnP device is detected by the UPnP base driver on the network, it is represented as a local OSGi service, as shown in Figure 9.5. Then, the UPnP OWL Writer is notified by the OSGi framework about its arrival, then the ontology generation is started. The UPnP OWL Writer parses the UPnP device description and generates an ontology conformed to a predefined meta model to represent the 9.3. AN END TO END ARCHITECTURE 123 device.

Figure 9 . 5 :

 95 Figure 9.5: Ontology Generation by the OWL Writers

Figure 9 .

 9 Figure 9.6: M2 Layer, General Device Model

Figure 9 .

 9 7 shows an example of an intel UPnP Light [UPnP 03] Device description and its ontology representation. Another example of a DPWS Light [SOA4D a] Device provided by the SOA4D project [SOA4D b] and its ontology are shown in Figure B.2, in the appendix B.3. Both ontologies use the same properties to describe and link the concepts. The ontology representation provides information about the concepts and how they are related. For example, the BinaryLight concept is related to the SwitchPower concept through the hasService property.

Figure 9 .

 9 Figure 9.7 shows only the property view between concepts and how they are related. However, during the ontology generation, the taxonomic structure is also represented based on the device description which holds the necessary information to retrieve the concepts type. In Figure 9.8, the taxonomic structure view of the ontologies is shown. The left ontology corresponds to the UPnP light ontology of Figure 9.7 while the right part corresponds to the DPWS light ontology of Figure B.2 in the appendix B.3. Each concept used in the ontology

Figure 9 . 7 :

 97 Figure 9.7: (Simplified) UPnP Ontology Generation from an XML description, (properties view)

Figure 9 . 8 :

 98 Figure 9.8: Part of the Taxonomic Structure of a UPnP (left) and a DPWS (right) Light

Figure 9 . 9 :

 99 Figure 9.9: Part of a Lights Ontology

Figure 9 Figure 9 . 10 :

 9910 Figure 9.10.

  Figure 9.11: DOXEN

Figure 9 .Figure 9 .

 99 Figure 9.12: Global Architecture

Figure 9 .Figure 9 .

 99 Figure 9.13: Sequential Diagram showing DOXEN and the Proxy Interaction

Figure 9 .

 9 Figure 9.15 shows the whole process of the device matching and details the four steps of Figure9.14.The process takes two automatically generated ontologies (Ont1, Ont2) by the OWL Writers. The two ontologies correspond to two equivalent devices' descriptions such as a UPnP and a DPWS printers. First, the ontology "Alignment(1)" techniques and algorithms are applied in order to automatically detect correspondences between equivalent entities from the two ontologies, see Figure9.15. The ontology alignment techniques are detailed in section 9.4.1. This step produces an alignment file (.rdf) containing several tuples < lef tEntity, rightEntity, SimilarityV alue > referring to left and right entities from both ontologies along

Figure 9 .

 9 Figure 9.16: Aligner Simple description

Figure 9 .Sim 1 , 3 Sim 1 , 4 Figure 9 .

 913149 Figure 9.17, shows a part of a basic alignment result between the BinaryLight entity of the UPnP Light ontology O 1 and entities of the DPWS Light ontology O 2 . The similarities showed corresponds to the first line of the similarity matrix. The alignment algorithm using the basic techniques ignores the predefined unified concepts in the ontology such as UPnP Device, DPWS Service.

ListFigure 9 .

 9 Figure 9.18: SMOA++ Matching

Figure 9 .

 9 Figure 9.20: An illustration of the similarity enhancement

(

  service-action) pairs similarities. The Enhancer technique can also be applied on the three concepts types: device, service and action. The output of alignment step is an alignment (.rdf) file containing multiple sets of tuples (leftEntity, Right-Entity, Similarity), see Listing B.3 in the appendix B.5. Each tuple represents an alignment between the left and the right entities of the two ontologies and the calculated similarity values. The expert loading the ontologies and triggering the alignment can choose a threshold value, then all similarity values higher than the threshold value are kept.

Figure 9 .

 9 Figure 9.21: Step 2: Alignment Validation

Figure 9 . 1 Figure 9 .

 919 Figure 9.23 shows a part of the matching between two lights ontologies after the expert validation. Once the mapping is validated, ATOPAI transforms the alignment format containing tuples (leftEntity, rightEntity, similarity) to an OWL Alignment ontology. The Listing B.3 in the appendix B.5 shows an alignment rdf result between a UPnP Binary Light and a DPWS Simple Light. The transformation of the alignment

Figure 9 .

 9 25 shows an example of two fake clocks descriptions. The SMOA++ based alignment result detected only one of the correspondences, (SetClock ≡ SetTime). During the validation, the expert added the alignment between the SetClock and the SetDate actions based on the two devices specifications. The alignment contains then a union composition, the UPnP action SetClock which takes two input parameters (Date, Time) is equivalent to two actions SetTime(Time) and SetDate(Date). In other words, when the UPnP-DPWS Proxy receives the SetClock invocation, it must extract the values from both parameters Time and Date. Then, the UPnP-DPWS Proxy invokes on the DPWS Clock, the two actions SetTime and SetDate.

Figure 9 1 E x p e r t = 1 Figure 9 .

 9119 Figure 9.24: Step 3: Pattern Detection

1

  ? f : CLASS , ? g : CLASS , ? x f : CLASS , ? xg : CLASS 2 SELECT ? f s u b C l a s s O f has UPnP Input some ? xf , 3 ? g s u b C l a s s O f has DPWS Input some ? xg ,

?

  f : CLASS , ? g : CLASS , ?h : CLASS SELECT ? f s u b C l a s s O f S i m p l e M a p p i n g I n p u t some ? g , ? f s u b C l a s s O f S i m p l e M a p p i n g I n p u t some ?h , WHERE ? f != ? g , ? g != ?h // f , g and h ca nn ot be t h e same c l a s s BEGIN ADD ? f s u b C l a s s O f U n i o n 1 t o n I n p u t some ? g , ADD ? f s u b C l a s s O f U n i o n 1 t o n I n p u t some ?h END;

Figure 9 .

 9 Figure 9.27: Sequential Union detected on the Standard Printers

1Listing 9 . 4 :Figure 9 .Figure 9 .Figure 9 .Algorithm 1 1 )

 9499911 Figure 9.28 shows a generalized complex mapping between an action f and a set of union and sequential actions. The has Next relation is modeled with a DAG Directed Acyclic Graph: G since has Next is a partial order relation: irreflexive, asymmetric and transitive. G is sorted to an ordered graph, as shown in Figure 9.29, using a topological sorting algorithm[Frank 66] as shown in Algorithm 1, the proxy can then handle and invoke the ordered action list properly.

Figure 9

 9 Figure 9.30.

Figure 9 .

 9 Figure 9.30: Step 4: Code Annotation

Contents 10 . 1

 101 OWL Writers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 10.2 ATOPAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 10.3 DOXEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 10.4 Experimentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

3 . 1 ,

 31 the Device Matching presented in section 9.4 and carried out by the ATOPAI framework and finally DOXEN overviewed in section 9.3.3 which generates proxies based on the ontology alignments.The reminder of this chapter is structured as follows. First, we present the UPnP and DPWS OWL Writers' implementation details. Then, we suggest a Bonjour OWL Writer architecture since Bonjour is used only for discovery. Second, we present the ATOPAI Framework which allows to perform the device matching and offers a Graphical User Interface to allow alignment validation and code adaptation. Third, we present implementation details regarding DOXEN and the generated proxies. Finally, we provide the realized experimentations with the various plug and play devices.

10. 1 3 4

 13 OWL Writers In our prototype, an OWL Writer module is an OSGi Bundle deployed on a Felix Apache OSGi Framework [Apache a]. Each specific OWL Writer subscribes to receive devices' announcements on the OSGi Framework. When a real Plug and Play device appears on the network, it is reified by a Base Driver, then the framework notifies the OWL Writer about the device arrival. Each OWL Writer relies on the Base Driver API to retrieve the device description and its supported capabilities. In our prototype, we used two base drivers, the UPnP Apache Base Driver [Apache b] and the SOA4D Base Driver [SOA4D b] supported mainly by Schneider Electric. 1 public void s t a r t ( BundleContext Context ) throws E x c e p t i o n { 2 t h i s . c t x t = Context ; // A c t i v e and p a s s i v e d i s c o v e r y o f A l l UPnP D e v i c e s 5 S e r v i c e R e f e r e n c e [ ] s r = c t x t . g e t S e r v i c e R e f e r e n c e s ( UPnPDevice . c l a s s . getName ( ) , n u l l ) ; 6 7 //UPnP LDAP l i k e F i l t e r 8 S t r i n g U P n P f i l t e r = " ( "+C o n s t a n t s . OBJECTCLASS + "=o r g . o s g i . s e r v i c e . upnp . UPnPDevice ) " ; 9 10 // r e g i s t e r a S e r v i c e L i s t e n e r f o r UPnP D e v i c e s , w i l l be n o t i f i e d when a new d e v i c e a p p e a r s 11 c t x t . a d d S e r v i c e L i s t e n e r ( this , U P n P F i l t e r ) ; 12 13 // g e t a l l UPnP d e v i c e s a v a i l a b l e on t h e n e t w o r k 14 f o r ( i n t i = 0 ; s r != n u l l && i < s r . l e n g t h ; i ++) { 15 O b j e c t d e v i c e = c t x t . g e t S e r v i c e ( s r [ i ] ) ; 16 i f ( o n t o l o g y N o t Y e t G e n e r a t e d ( ( UPnPDevice ) d e v i c e ) ) { 17 OB = new OWLBuilder ( ( UPnPDevice ) D e v i c e ) ; // B u i l d t h e o n t o l o g y 18 // add OB. o w l F i l e t o t h e r e p o s i t o r y o f o n t o l o g y f i l e s . . . } } } Listing 10.1: A UPnP OWL Writer Subscribes to UPnP Devices An OWL Writer uses the OWL API 3.0 [OWL ] to automatically generate an ontology based on the device description. The generated ontologies represent the device, the services, the actions and their variables. Each generated entity is a subclass of a well defined concept from the meta-model defined in section 9.3.1. In the lights ontology for example, BinaryLight (resp. SimpleLight) is a subclass of UPnP Device (resp. DPWS Device).

ForFigure 10 . 1

 101 Figure 10.1: Generating Ontologies

10. 1 .

 1 OWL WRITERS when the SwitchPower service is added to the ontology, it is connected to the BinaryLight class with the OWL property has UPnP Service. Additional information are also attached to the ontology such as the version number and the complete UPnP Type (urn:schemas-upnp-org-service:SwitchPower ), Version(1 ) and Id (urn:upnp-org-servideId:SwitchPower:1 ) used later by the generated proxy during description announcement. Listing 10.2 shows a part of the UPnP OWL Writer service generation. The service name, version and type are extracted using the UPnP API. The Listing shows how the service is attached to its type and then related to its device. The owl upnpService refers to the current service entity added to the ontology. Listing B.2 in the appendix B.4, shows the complete generated ontology of the UPnP Binary Light in the OWL syntax. 1 public void W r i t e S e r v i c e ( UPnPService uPnPService ) throws OWLOntologyStorageException { 2 // o w l u p n p S e r v i c e S u b C l a s s o f o w l S e r v i c e , SwitchPower s u b C l a s s o f S e r v i c e 3 o w l u p n p S e r v i c e = f a c t o r y . getOWLClass ( IRI . c r e a t e ( o n t o l o g y I R I +"#"+ S e r v i c e N a m e ) ) ; 4 OWLAxiom axiom = f a c t o r y . getOWLSubClassOfAxiom ( o w l u p n p S e r v i c e , S e r v i c e ) ; 5 manager . addAxiom ( o n t o l o g y , axiom ) ; // add each axiom t o t h e o n t o l o g y 6 7 // Write S e r v i c e Type 8 h a s i n f o=f a c t o r y . getOWLDataProperty ( IRI . c r e a t e ( o n t o l o g y I R I+"#h a s U P n P S e r v i c e T y p e " ) ) ; 9 h a s v a l u e=f a c t o r y . getOWLDataHasValue ( h a s i n f o , f a c t o r y . g e t O W L S t r i n g L i t e r a l ( uPnPService . getType ( ) ) ) ; 10 axiom = f a c t o r y . getOWLSubClassOfAxiom ( o w l u p n p S e r v i c e , h a s v a l u e ) ; 11 // Write S e r v i c e V e r s i o n and S e r v i c e ID . . . 12 13 // Connect upnpDevice w i t h o w l u p n p S e r v i c e , B i n a r y L i g h t h a s S e r v i c e SwitchPower 14 OWLClassExpression h a s S e r v i c e S o m e S e r v i c e=f a c t o r y . getOWLObjectSomeValuesFrom ( h a s S e r v i c e , o w l u p n p S e r v i c e ) ; 15 OWLSubClassOfAxiom Subaxiom=f a c t o r y . getOWLSubClassOfAxiom ( owl upnpDevice , h a s S e r v i c e S o m e S e r v i c e ) ; 16 17 //Now add t h e a c t i o n s , t h e s e r v i c e r e f e r e n c e i s p a s s e d t o a t t a c h t h e a c t i o n s t o i t 18 W r i t e A c t i o n s ( uPnPService ) ; . . . } Listing 10.2: UPnP OWL Writer Service Generation

10. 1 . 1

 11 Figure B.1 in the appendix B.2. The JobDescription element, for example, is used by the following operations : CreatePrintJob, AddDocument, SendDocument and is located at level 3 or 4 from the root element depending on the entry of each action.

Figure 4 . 3 ,

 43 as an example, is represented by a path preserving the whole element structure as follows for each of the actions: Cre-atePrintJob: CreatePrintJobRequest/PrintTicket/JobDescription/JobName, SendDocument: SendDocumen-tRequest/.../JobDescription/JobName, AddDocument: AddDocumentRequest/.../JobDescription/JobName).

10. 1 .

 1 2 The "Bonjour " Exception Devices hosting Bonjour use multicast-DNS 1 for the announcement and the discovery on the network as shown in Tables 4.4 and 4.3. For example, "Apple LaserWriter 8500. ipp. tcp.local. Port 631" is a bonjour printer announcement conformed to the following format DeviceName. protocol. transportProtocol.Domain port number.Bonjour is only used to announce few information compared with the other plug and play protocols. In this example, the ipp protocol is used as an interaction protocol with this printer. Thus, other protocols are used on top of Bonjour for the interaction. For example, the Media Server iTunes(v4.2) uses the Digital Audio Access Protocol (DAAP[DAA 05]) and the Digital Media Access Protocol (DMAP).

Figure 10 .

 10 Figure 10.2: The Bonjour Base Driver compared with other Plug and Play Base Drivers

Figure 10 . 3 :

 103 Figure 10.3: ATOPAI Supported Features

3

  f o r ( S t r i n g syn : synonyms ) { 4 antonyms =wordnet . getAllAntonyms ( syn , pos ) ; 5 i f ( antonyms != n u l l ) l i s t O f A n t o n y m s . add ( antonyms ) ; } 6 return l i s t O f A n t o n y m s ; } Listing 10.4: Second Round Antonyms Search

4 . 3 ,

 43 we specified 12 rules as following: three rules are used to detect the Simple Mapping with its three subcategories (Input, Output and Input Output). Three rules are used to detect the Union Mapping One to N patterns which contains three subcategories (Input, Output and Input Output). The sequential union pattern requires also three rules to detect such patterns along with three additional rules to detect the has Next relation which specifies the sequential relation.As mentioned in section 9.4.3, the N-to-M unions are almost impossible to adapt, therefore, we don't treat the pattern detection and adaptation of such unions. So far, ATOPAI only supports the One-to-N patterns detection and adaptation.

10.2. 3

 3 Expert Adaptation with ATOPAIThree types of adaptation are supported by ATOPAI which can be carried out by an expert. The first type allows to add a default value to an input or output parameter. The second type consists in adding an adaptation behavior as shown in the use case #1, section 9.4.4, between the TV devices. And finally, the last type allows to invoke an external OSGi service on the framework to provide some adaptation. The use case #2 provided in section 9.4.4 relies on an external service to convert a ps file type to a pdf type. We detail next the three adaptation types.10.2.3.1 Add Meta DataThe first type of adaptation is applied on the actions parameters. The DPWS printing ticket used to print a document requires a true or false value for the LastDocument parameter. The DPWS standard printer profile allows to applications to send multiple documents to print for the same printing session identified by the JobId value. Thus, when a DPWS application send several documents, it has to specify using the LastDocument set to true that no other documents will be sent and the printer can process the Job and print the documents. The UPnP Printer does not support such feature and accepts only one document per session.

Figure 10 . 4 :

 104 Figure 10.4: ATOPAI: Meta Data Adaptation

Figure 10 . 5 :

 105 Figure 10.5: ATOPAI: Adaptation Code

3

  out = new OutputStreamWriter ( Outs ) ; 4 Map a c t i v a t o r I n f o r m a t i o n M a p= new HashMap ( ) ; 5 a c t i v a t o r I n f o r m a t i o n M a p . put ( " package " , c f g . G e t C l a s s p a t h ( ) ) ; 6 a c t i v a t o r I n f o r m a t i o n M a p . put ( " u p n p d e v i c e n a m e " , UPnP Device . d e v i c e t y p e ) ; 7 a c t i v a t o r I n f o r m a t i o n M a p . put ( " d p w s d e v i c e n a m e " , DPWS Device . l o g i c a l d e v i c e n a m e ) ; 8 a c t i v a t o r T p l . p r o c e s s ( a c t i v a t o r I n f o r m a t i o n M a p , out ) ; Listing 10.9: Preparing the Activator Information for Template Filling

3Figure 10 . 7 :

 107 Figure 10.7 shows the Java generation dependency between classes. DOXEN starts by filling the Activator template which is the main class of an OSGi bundle, see section 3.3. The Activator.java file holds the compatible

1

  deviceMap . put ( " package " , c f g . G e t C l a s s p a t h ( ) ) ; 2 deviceMap . put ( " u p n p d e v i c e n a m e " , name ) ;3 // Generate t h e d e v i c e ' s s e r v i c e s and r e t r i e v e t h e i r names t o i n c l u d e them i n t h e t e m p l a t e4 A r r a y L i s t <HashMap> s e r v i c e s n a m e s M a p = new A r r a y L i s t <HashMap>( s e r v i c e s . s i z e ( ) ) ; 5 f o r ( UPnP Service s e r v i c e : s e r v i c e s ) { 6 s e r v i c e n a m e s = ( HashMap ) s e r v i c e . g e n e r a t e ( c f g ) ; 7 i f ( s e r v i c e n a m e s != n u l l ) s e r v i c e s n a m e s M a p . add ( s e r v i c e n a m e s ) ; } . . . Listing 10.11: Part of the UPnP Proxy Device Generation Now that the Java files are generated, the final step, detailed next, consists in compiling the code and package it into an OSGi bundle.

1 u r

  l C l a s s L o a d e r = new URLClassLoader ( u r l s O f J a r , d o x e n C l a s s L o a d e r ) ; 2 J a v a S o u r c e C l a s s L o a d e r c l = new J a v a S o u r c e C l a s s L o a d e r ( u r l C l a s s L o a d e r , s r c D i r s , e n c o d i n g , D e b u g g i n g I n f o r m a t i o n .DEFAULT DEBUGGING INFORMATION) ; 3 Map ByteCode = ( HashMap ) c l . g e n e r a t e B y t e c o d e s e x t e n d e d ( c f g . G e t B u n d l e A c t i v a t o r ( ) ) ;4 I t e r a t o r i t b y t e C o d e = Map ByteCode . e n t r y S e t ( ) . i t e r a t o r ( ) ; 5 i n t i =0; 6 F i l e [ ] ToBeJared = new F i l e [ Map ByteCode . s i z e ( ) ] ; 7 while ( i t b y t e C o d e . hasNext ( ) ) { 8 Map . Entry p a i r s = (Map . Entry ) i t b y t e C o d e . n e x t ( ) ;9 S t r i n g C l a s s F i l e n a m e = p a i r s . getKey ( ) . t o S t r i n g ( ) . r e p l a c e ( ' . ' , ' / ' ) ; 10 F i l e F = new F i l e ( c f g . T a r g e t D i r+C l a s s F i l e n a m e+" . c l a s s " ) ; 11 ToBeJared [ i ] = new F i l e ( c f g . T a r g e t D i r+C l a s s F i l e n a m e+" . c l a s s " ) ; 12 OutputStream out2 = new F i l e O u t p u t S t r e a m (F) ; 13 out2 . w r i t e ( ( byte [ ] ) p a i r s . g e t V a l u e ( e a t e J a r F i l e CJF = new C r e a t e J a r F i l e ( ) ; 17 CJF . c r e a t e J a r A r c h i v e (new F i l e ( c f g . G e n e r a t e d B u n d l e T a r g e t ) , ToBeJared , c f g ) ; Listing 10.12: Compile Generated Code Once the classes are packaged in a Jar file, we write the OSGi manifest which contains several kind of information. The general information exposes the bundle name, version and manufacturer. Most importantly, the manifest contains the name of the activator class along with the imported packages needed for the execution of the bundle, and the exported packages which can be shared with other bundles, for example, the UPnP proxy packages can be shared by the other bundles. DOXEN can finally install the generated proxy on the OSGi framework and start the proxy, see section 3.3. When started, the proxy announce its description on the network and searches for a specific DPWS device to interact with. So far, we only support Java/OSGi proxies providing interoperability between the UPnP and the DPWS profiles. In order to support additional protocols, another set of templates must be developed. As shown in the previous section, the templates contain code to call operations of the OSGi framework and the Plug and Play Base Drivers. Thus, their would be three types of templates to generate three types of proxies (UPnP-DPWS, UPnP-IGRS, UPnP-Bonjour) to resolve the heterogeneity between the four protocols. The advantage in using template ontology based code generation consists in writing once the templates and use anywhere with any alignment ontology. The current version of DOXEN generates proxies and installs them on the same OSGi framework as DOXEN due to the code compiling class loader dependency and the bundle resolution. The compilation operation relies on the class loader of DOXEN and other Jars present on the framework. The bundle resolution also depends on other local bundles exporting their packages on the framework. However, a solution for a remote deployment can be based on the ROCS framework [Frénot 10].

10. 3 . 4 . 3

 343 DOXEN Management ServicesDOXEN announces its management services as UPnP Services on the network to share its capabilities with other management applications in the digital home. Additionally, exposing DOXEN as a UPnP service provides the operator with the ability to remotely administrate DOXEN for example through the CWMP protocol mentioned in section 2.4.3. The administration operations vary from updating templates, configuration and ontology alignment files to troubleshooting and diagnostics.

4 . 3

 43 is a standard UPnP service which allows to carry out basic management operations to test the network connectivity through ping and traceroute actions for example. The current version of DOXEN implements the BMS actions such as the ping and traceroute by calling an exec process based on a Windows XP OS. A future version of DOXEN is currently under development and uses the SIGAR 7 API which offers common methods to extract networking information and to perform operations like a ping and a traceroute on different operating systems.

Figure 10 .

 10 8 shows the used devices and the topology. We used each existing DPWS device having an equivalent UPnP Device. We found a DPWS Light device proposed by the SOA4D [SOA4D b] project, a DPWS Clock device proposed by the WS4D 8 and a DPWS Printer, the HP 4515x. We used their equivalent UPnP devices types the intel UPnP Light 9 , the UPnP Felix Clock [Apache b] and the UPnP printer standard profile [UPnP 06a].

Figure 10 . 8 :Figure 10 . 9 :

 108109 Figure 10.8: Experimentation Devices and Topology

Figure 10 .

 10 Figure 10.10: UPnP-DPWS Proxies detected by the MyDevices Application

Figure 10 . 11 :Figure 10 .

 101110 Figure 10.11: Basic Diagnostic Use Case

Figure 10 .

 10 Figure 10.11 also shows the supported services by each device.The Internet Gateway device is connected to all the devices in a star like topology. The Gateway supports the CWMP-UPnP [Lupton 07, Broadband 10b] Device Management Proxy which allows to transfer action invocation from the operator's site to the devices supporting UPnP Basic Management services. The gateway supports the BMS and CMS services. The printer proxy supports the BMS and the extended CMS exposing networking information about its hosted device and the DPWS printer. Finally, the printer application-device (not the UPnP application) supports the BMS and CMS services. Now consider, the following malfunction, the end-user is unable to print, however he was able to print few hours ago. The end-user "Edouard", calls the operator's Hotline for an assistance. The Hotline assistant "Alice", connects to Edouard's gateway through the CWMP protocol. The CWMP-UPnP[Broadband 10b] Bridge allows Alice to forward action invocation to the UPnP Devices. For example, Alice can perform through the CWMP-UPnP Bridge a ping operation on the proxy BMS service or retrieve the information exposed by the extended CMS on the proxy. The extended CMS exposes the IP and MAC address of the hosted proxy device and the DPWS printer. This extension offers Alice an extra visibility and becomes capable to ping the printer and the proxy device through the CWMP-UPnP[Broadband 10b] bridge.

Figure 11 . 1 :Figure 11 . 2 :

 111112 Figure 11.1: UPnP and DPWS OWL Writers Generation

  dependent on the number of actions and arguments. The Fake3 device having 32 actions with various number of arguments has the highest generation time. Thus, we can conclude that having a large number of actions with a multitude of arguments influences the generation time more than a device having a large number of actions with only one argument or a small number of actions with high number of arguments. However, the time difference seems to be acceptable and still fit in the exponential curve of the UPnP OWL Writer time generation shown in Figure11.2.

Figure 11 . 3 :

 113 Figure 11.3: DPWS Writer Generation

applied the alignment on 3

 3 types of devices. The intel UPnP Light [Apache b] and the DPWS SOA4D [SOA4D b] Light, the UPnP Felix [Apache b] Clock and the WS4D 4 Clock, and finally on a DPWS and UPnP standard printers profiles. We tested the alignment on different devices using SMOA and SMOA++. We chose the SMOA [Stoilos 05] technique since it is the most adapted to the entity name selection in the plug and play domain descriptions. As mentioned in chapter 8, SMAO is based on how programmers assign names to variables.

  w3 . o r g /2003/11/ s w r l b#" x m l n s : r d f s=" h t t p : //www. w3 . o r g /2000/01/ r d f -schema#" x m l n s : r d f=" h t t p : //www. w3 . o r g /1999/02/22 -r d f -syntax-ns#" x m l n s : o w l=" h t t p : //www. w3 . o r g /2002/07/ owl#"> <o w l : O n t o l o g y r d f : a b o u t=" " /> <o w l : C l a s s r d f : I D=" B r i g h t n e s s "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#P h y s i c a l P r o p e r t y " /> </ o w l : C l a s s> <o w l : O b j e c t P r o p e r t y r d f : I D=" c h a r a c t e r i z e s "> <o w l : i n v e r s e O f r d f : r e s o u r c e="#i s C h a r a c t e r i z e d B y " /> </ o w l : O b j e c t P r o p e r t y> <o w l : C l a s s r d f : I D=" C l i m S t a t i o n "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#TempServ" /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D=" D e v i c e " /> <o w l : C l a s s r d f : I D=" D i s t r a c t i o n d e v i c e "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#D e v i c e " /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D="Game"> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#D i s t r a c t i o n d e v i c e " /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D=" HotelRoom "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#Room" /> </ o w l : C l a s s> <HotelRoom r d f : I D=" 1345 " /> <o w l : C l a s s r d f : I D=" I n d o o r L o c a t i o n "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#L o c a t i o n " /> </ o w l : C l a s s> <o w l : O b j e c t P r o p e r t y r d f : I D=" i n f o r m s "> <r d f s : d o m a i n r d f : r e s o u r c e="#S e n s o r " /> < r d f s : r a n g e r d f : r e s o u r c e="#P h y s i c a l P r o p e r t y " /> </ o w l : O b j e c t P r o p e r t y> <o w l : O b j e c t P r o p e r t y r d f : I D=" i s C h a r a c t e r i z e d B y "> <r d f s : d o m a i n r d f : r e s o u r c e="#L o c a t i o n " /> <o w l : i n v e r s e O f r d f : r e s o u r c e="#c h a r a c t e r i z e s " /> < r d f s : r a n g e r d f : r e s o u r c e="#P h y s i c a l P r o p e r t y " /> </ o w l : O b j e c t P r o p e r t y> <o w l : C l a s s r d f : I D=" j s d 6 7 8 -mk2"> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#C l i m S t a t i o n " /> </ o w l : C l a s s> <j s d 6 7 8 -mk2 r d f : I D=" j s d 6 7 8 9 8 2 0 0 " /> <o w l : O b j e c t P r o p e r t y r d f : I D=" l o c a t e d I n "> <r d f s : d o m a i n r d f : r e s o u r c e="#D e v i c e " /> < r d f s : r a n g e r d f : r e s o u r c e="#L o c a t i o n " /> </ o w l : O b j e c t P r o p e r t y> <o w l : C l a s s r d f : I D=" L o c a t i o n " /> <o w l : C l a s s r d f : I D=" M o i s t u r e "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#P h y s i c a l P r o p e r t y " /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D=" P h y s i c a l P r o p e r t y " /> <o w l : C l a s s r d f : I D="Room"> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#I n d o o r L o c a t i o n " /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D=" S e n s o r "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#D e v i c e " /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D=" Temperature "> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#P h y s i c a l P r o p e r t y " /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D="TempServ"> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#S e n s o r " /> </ o w l : C l a s s> <o w l : C l a s s r d f : I D="TVSet"> <r d f s : s u b C l a s s O f r d f : r e s o u r c e="#D i s t r a c t i o n d e v i c e " /> </ o w l : C l a s s> </ rdf:RDF> Listing B.1: OWL Description [Pierson 09] of the Ontology in Figure 5.3 B.2 A DPWS PrintTicket Element

Figure B. 1

 1 Figure B.1 shows a PrintTicket element structure. The PrintTicket element is used as an input by the Cre-atePrintJob DPWS operation.

Figure B. 1 :

 1 Figure B.1: A Standard DPWS Print Ticket Element [Microsoft 07]

  <Alignment> <xml>y e s</xml> < l e v e l>0</ l e v e l> <t y p e> * * </ t y p e> <n s 0 : t i m e>1560</ n s 0 : t i m e> <ns0:method>com . f r a n c e t e l e c o m . a l i g n . s m o a S t r a t e g y</ ns0:method> <onto1> <Ontology r d f : a b o u t=" h t t p : //OWLWriter/UPnPWriter . owl "> <l o c a t i o n> f i l e : / C: / h o m e d e v i c e s / UPnPWriter BinaryLight . owl</ l o c a t i o n> <f o r m a l i s m> <Formalism a l i g n : n a m e="OWL2. 0 " a l i g n : u r i=" h t t p : //www. w3 . o r g /2002/07/ owl#" /> </ f o r m a l i s m> </ Ontology> </ onto1> <onto2> <Ontology r d f : a b o u t=" h t t p : //OWLWriter/DPWSWriter . owl "> <l o c a t i o n> f i l e : / C: / h o m e d e v i c e s / DPWSWriter SimpleLight . owl</ l o c a t i o n> <f o r m a l i s m> <Formalism a l i g n : n a m e="OWL2. 0 " a l i g n : u r i=" h t t p : //www. w3 . o r g /2002/07/ owl#" /> </ f o r m a l i s m> </ Ontology> </ onto2> <map> <C e l l> <e n t i t y 1 r d f : r e s o u r c e= ' h t t p : //OWLWriter/

  .6: OSGi LifeCycle State DiagramInstall This operation allows to install a bundle (.jar) on the framework. The installation takes as input the JAR location and deploys the JAR file in the framework's local file system. This step places the bundle in the INSTALLED state. The OSGi framework provides the method InstallBundle(String URL) to install bundles, it takes as input the url location of the JAR file.Start The Start operation allows a bundle to be started. The STARTING state is a transitory one, the bundle implicitly goes from the RESOLVED state into the Active state. The Start() method executed on a bundle object allows it to transit to the Active state upon successful completion of the method execution. However, if the execution throws an exception, it transitions back to the RESOLVED state.Stop An ACTIVE bundle can be stopped suing the Stop() method. Its state goes through the STOPPING state into the RESOLVED state. The STOPPING state is a transitory one like the STARTING state. A stopped bundle goes back to the RESOLVED because its dependencies are still satisfied and there is no need to be resolved again.

	Dependency Resolution Once a bundle is installed, the OSGi framework automatically proceeds to resolve
	its dependencies. Such dependencies represent the imported packages list expressed in the bundle manifest. If
	the imported packages are found on the platform, then the bundles providing those dependencies are already
	installed and resolved. Once the dependencies are satisfied, then the framework exports the packages of the
	bundle and its state goes from INSTALLED to RESOLVED. The framework ensures that all the dependencies of a
	bundle are satisfied before it can be used. The framework will not allow a bundle to transit into a RESOLVED
	state unless all its dependencies are satisfied.
	Update or Refresh The refresh or the update of a bundle automatically leads to its stopping, re-installation
	and resolution. It is active then it will be reactivated again. If the update fails, the bundle can be either in a
	RESOLVED state if the dependencies are still satisfied or in an INSTALLED state if not.

  Plug-and-Play . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.2 Common features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.3 Universal Plug and Play Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.4 Device Profile for Web Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

	"You Plug it in, it runs, you're done"
	-Combs [Combs 02]
	Contents
	4.1 4.5 Intelligent Grouping and Resource Sharing . . . . . . . . . . . . . . . . . . . . . . . . . 66
	4.6 Bonjour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
	4.7 Plug and Play Protocols Divergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
	4.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Table 4 .

 4 1.

	UPnP Vendor Extensions
	UPnP Device Control Protocols
	UPnP Device Architecture
	SSDP Multicast	GENA SOAP HTTP
	UDP	TCP
	IP

Table 4 .

 4

1: UPnP Protocol Stack [UPnP 08]

Table 4 .

 4

2: DPWS Protocol Stack

Table 4

 4

	Protocols	Service Type
	LPR	printer. tcp
	IPP	ipp. tcp
	AppSocket
	PhaserPort JetDirect	pdl-datastream. tcp
	Port 9100

.

3

shows an example of a printer service announcement capabilities. As Bonjour is only used for addressing, discovery and announcement, other protocols are used for the interaction and must be specified in the service announcement in the protocol field.

Table 4 .

 4 3: An Apple Printer Supported Services Example [Apple 05]

Table 4 .

 4

	, Bonjour and

4: Plug And Play Protocols Stack Comparison

  "What Is a Knowledge Representation? " Davis el al[Davis 93] answer this question by defining five different roles of a knowledge representation. The first role is a substitute of a thing from the real world in an intelligent entity. The representation captures some approximation of the reality, thus it is inevitably incomplete. The second role of a knowledge representation is a Set of Ontological Commitments about the reality. Since, a representation is imperfect, then it is about making choices to determine what an intelligent entity "captures" from the real world and what other aspects are ignored. The third role is a Fragmentary Theory of Intelligent Reasoning, in other terms, what information can the system extracts from what it already knows. The reasoning can be based on rules and logics, and it can also benefits from the user's lead or recommendations. A medium for efficient computation is the fourth role, information should be organized in a way for the computation and the reasoning to be the most efficient. And finally, a knowledge representation is also a medium for human expression, humans gather and store knowledge in intelligent entities. Thus, humans have to create medium of communications, to communicate representations to machines.

Humans define the devices' descriptions which represent a knowledge to be shared with other applications and devices. The description expresses such information in various formats and languages. For instance, UPnP uses an XML template format, while IGRS and DPWS use the Web Service Description Language. As pointed out by Davis el al, the representation language is the fifth role which arises questions mainly about expressivity.

  of three parts: the variable definition, the selection and finally the actions. The variable definition part holds before the SELECT, it allows to define the variable type and category in the OWL language. The selection part allows to query the ontology searching for the specified pattern. The action part allows to apply transformation on the ontology by adding or removing elements and properties between. Listing 5.3 shows an OPPL statement example which links a device and an application providing/requesting equivalent service types in the ontology.

	1 SELECT ? a u t h o r
	2 WHERE { h t t p : //www. abc . com/myPage . html b t r : h a s A u t h o r ? a u t h o r }
	Listing 5.2: SPARQL Query Example
	SWRL The Semantic Web Rule Language [Horrocks 04] targets OWL ontologies and supports description

logics. SWRL Rules can be based on the OWL entities. For example, the Atoms of the TBox can be represented using OWL classes C(x), or properties from the ontology P(x,y).

OPPL The Ontology Pre-Processor Language [ Šváb

Zamazal 10]

was initially proposed to manipulate ontologies OWL. The OPPL allows to detect patterns and apply transformations on the ontology by creating new elements or removing others. Moreover, the semantics of OPPL are close to OWL-DL. An OPPL statement is constituted

  Moon 05] proposes a solution to provide interoperability between various protocols: Jino, LonWorks, UPnP. Moon et al propose to represent each device description using a Universal Device Template (UDT) which is inspired from the UPnP representation format.As shown in Figure7.7, the approach uses a UMB adapter for each protocol to represent each device as a virtual proxy device on the universal middleware platform. The virtual proxy exposes the local device description (UPnP, Havi, etc) referred as Local Device Template into a Universal Device Template format. The content semantics are also translated into unified semantics. The translation is carried out by the UDT-LDT mapper which uses a mapping table containing the matching between universal and local device types, functions, parameters and events.In the right part of Figure7.7, a virtual proxy device mapping is showed. The table holds the mapping

between the universal semantics used and the local description of each device. For instance, the table shows that the local device type 0x05 0x01 of the LonMark protocol is exposed as a LAMP in the global device type representation.

Table ,

 ,

	Universal Semantics	Mapping Table,	Universal Semantics
	Required	Not	Required
	Service Registration	Service Discovery,	Service Registration
	Mapping	DomoML
	Mapping	TechManagers TechManagers
	LonWorks	UPnP, KNX,	X10, Jini
		DomoNet

Table 7 .

 7 1: A Comparison between Interoperability Middlewares ( :Supported, ≈: Partially Supported, χ: Not Supported, ?: Information Not Clearly Available)

  However, Figure B.1 in the appendix B.2, shows the input parameter structure of the action CreatePrintJob.Therefore, when dealing with complex devices like a UPnP and a DPWS printers having a large number of parameters for each action, automating the validation and compatibility process decision become essential.

	DPWS (CreatePrintJob)	UPnP (CreateURIJob)
	PrintTicket/JobDescription/JobName	JobName
	PrintTicket/JobDescription/JobOriginatingUserName	JobOriginatingUserName
	PrintTicket/JobProcessing/Copies	Copies
	X 1	SourceURI
	PrintTicket/DocumentProcessing/NumberUp/Sides	Sides
	PrintTicket/DocumentProcessing/NumberUp/Orientation	OrientationRequested
	PrintTicket/DocumentProcessing/MediaSizeName	MediaSize
	PrintTicket/DocumentProcessing/MediaType	MediaType
	PrintTicket/DocumentProcessing/NumberUp/PrintQuality PrintQuality

1

SourceURI has an equivalent variable DocumentURL in the DPWS action AddDocument

Table 9 .

 9

2: Part of a Mapping between a standard DPWS and a UPnP printer action

  Listing 10.3: SMOA++ search for substrings in WordNet Listing 10.3 shows a part of the implementation and the substring search in WordNet. We first test in [line 3] if the string exists in WordNet. If it exists, then, then there is no need to apply the substring trimming, we 2 www.rednoise.org/rita/wordnet/documentation/index.htm directly search for the synonyms and antonyms between the two strings to match. We search in [line 4] only for substrings with a length above than three characters to avoid searching for the propositions, such as to or in. Then, search for the substrings which exist in WordNet and keep the biggest common substring [line 8].

	1 public L i s t <S t r i n g > getSubStringsListFromWordNet ( S t r i n g s t r 1 ) {
	2 i n t s u b S t r i n g i n d e x [ ] = new i n t [ s t r 1 . l e n g t h ( ) ] ; //The s t a r t i n d e x o f v a l i d s u b s t r i n g s
	3	i f ( ! wordnet . e x i s t s ( s t r 1 ) ) {
	4	i f ( s t r 1 . l e n g t h ( ) > 3 ) { // Avoid p r o p o s i t i o n s
	5	f o r ( i n t s t a r t = 0 ; s t a r t < s t r 1 . l e n g t h ( ) -2; s t a r t ++){
	6	f o r ( i n t i = s t a r t + 2 ; i < s t r 1 . l e n g t h ( ) ; i ++){
	7	i f ( wordnet . e x i s t s ( s t r 1 . s u b S t r i n g i n d e x ( s t a r t , i +1) ) ) {
	8	i f ( s u b S t r i n g i n d e x [ i ] < ( i -s t a r t ) ) s u b S t r i n g i n d e x [ i ] = ( i -s t a r t ) ; } } } } }

  consists in compiling and packaging the generated code to build an executable OSGi bundle. We detail in section 10.3.1 the ontology visiting and the information extraction. In section 10.3.2, we explain the code generation and the template filling, then we provide in section 10.3.3 the OSGi bundle packaging.

	1 < Cell >
	2	< CID >3 </ CID >
	3	< UPnPDevice >
	4	< DeviceType > P r i n t e r E n h a n c e d </ DeviceType >
	5	< Services >
	6	< Service >
	7	< Type > PrintEnhanced </ Type >
	8	< Version >1 </ Version >
	9	</ Service >
	10	</ Services >
	11	</ UPnPDevice >
	12	< DPWSDevice >
	13	< DeviceType > P r i n t D e v i c e T y p e </ DeviceType >
	14	< Services >
	15	< Service >
	16	< Type > P r i n t e r S e r v i c e T y p e </ Type >
	17	< Version > 1 . 0 </ Version >
	18	</ Service >
	19	</ Services >
	20	</ DPWSDevice >
	21 < OntologyIRI >/ l o c a l R e p o /UPnP-DPWS -P r i n t e r s . owl </ OntologyIRI >
	22 </ Cell >
		Listing 10.8: Part of the DOXEN Configuration file

4

http://download.oracle.com/javase/6/docs/api/java/lang/reflect/package-summary.html step

  10.3.4 DOXEN's Supported CapabilitiesDOXEN is deployed in the digital home and automatically generates proxies upon the detection of non-UPnP devices equivalent to UPnP standard devices. However, as mentioned is section 2.4.3, the remote administration of devices and applications is essential for telecoms operators and device manufacturers. The remote administration offers a lot of advantages [Lupton 07,UPnP 11,[START_REF]Remote Management of non TR-069 Devices[END_REF] such as applying configuration update and maintenance operations. Moreover, administration operations can be used to carry out diagnostic operations to detect an improper network or connectivity configuration, or even malfunctioning ubiquitous applications and devices.Additionally, DOXEN relies on its configuration containing equivalent devices to generate a proxy, see Listing 10.8. Such generation is based on the templates and the alignment ontologies. Furthermore, DOXEN

is expected to evolve in time to support additional proxy types generation. For instance, the operator can validate a new device matching between two new proposed devices, thus, DOXEN is expected to update its local configuration to support additional device types. Moreover, a new protocol can arise, thus, new templates

Table 10 .

 10 4 are implemented with 6 727 LoC. The implementation of DOXEN is around 2000 LoC, however, when adding the three management services BMS, CMS and DMS, the implementation reaches 11 637 LoC. The templates used to generated the proxy contain 1233 LoC, when the BMS and CMS management services are supported on the proxy, the implementation reaches 6 884 LoC.

	Module	Lines of Code
	UPnP OWL Writer	582
	DPWS OWL Writer	1 119
	ATOPAI	6 727
	DOXEN	2 385
	DOXEN-DM	11 637
	Proxy Templates	1 233
	Proxy-DM Templates 6 884

1: LoC of each module

Table 11

 11 Thus, the description size can give an idea on the ontology generation time, however other elements can influence the ontology generation performance. Therefore, we generated three UPnP fake devices with the same descriptions size as shown in Table11.2, The Fake1 device has 45 actions with one argument for each. The Fake2 device has 16 actions with 16 arguments each. And finally, the Fake3 device has 32 actions with various number of arguments ranging from 1 to 10. The ontology generation time, as shown in Table11.2, seems to be

		.2: UPnP OWL Writer Evaluation of three size equivalent descriptions
	Device Description (LoC) Time (mSec) Comments
	Fake1	542	54.5	45 actions, 1 argument each
	Fake2	542	56	16 actions, 16 arguments each
	Fake3	542	62.5	32 actions with various arguments

Table 11 .

 11 3: Ontology Generation (LoC) per entity type generated lines of codes between the services in Table 11.3. As for the state variable, the generated lines of code in the ontology depend on the variable type, Table 11.3 gives an outline of four types.

	Entity Type	UPnP (LoC)	DPWS (LoC)
	Template	42	32
	Device	9+(6 per Service)
	Service	21+(6 per Action) 3+(6 per Action)
	Action	3+(6 per StateVar.)
	StateVar.	Per Type
	String	6
	Boolean	14
	Enumerated Value	5
	integer	21

3. The first three fake devices

Table 11

 11

	P Strings P P P	P	P P Method P P P	1-Ngrams 1-Jaro 1-Hamming 1-Levenshtein 1-SMOA SMOA++
	(DecrementVolume, IncrementVolume)	0.84	0.93	0.86	0.86	0.92	0.0
	(Particle, Article)	0.9	0.95	0.0	0.87	0.96	0.0
	(SetVolumeUp, SetVolumeDown)	0.7	0.83	0.69	0.69	0.88	0.0
	(Clock, Timer) 3	0.0	0.0	0.0	0.0	0.0	1.0
	(SetClock, SetTime)	0.18	0.64	0.37	0.37	0.37	1.0
	(RetrieveData, FindData)	0.25	0.39	0.0	0.41	0.52	1.0
	(SelectTV, SelectTelevision)	0.6	0.81	0.43	0.5	0.83	1.0
	(SetVolumeUp, IncrementVolume) 1	0.45	0.54	0.0	0.4	0.69	0.4
	(SetVolume, GetVolume) 2	0.85	0.92	0.88	0.88	0.93	0.5
	(PrinterStatus, PrinterState)	0.85	0.94	0.84	0.84	0.95	0.0

.4 similarity values returned by some basic matching techniques applied on two strings. As explained in chapter 8, some techniques return a normalized dissimilarity, δ. The relation between a normalized dissimilarity and a normalized similarity is obtained as follows δ = 1 -δ.

1 "Increment" and "Up" not found in WordNet as synonyms.

Table 11 .

 11 4: Comparison between basic Matching TechniquesWe expose inTable 11.4, three categories of strings. In the first three lines, the first category contains antonyms or non related pairs of strings. The second category compares 4 semantically equivalent pairs of strings. In the final category we chose specific pairs of strings to reveal the drawbacks of the SMOA++ technique.In the first category, our proposed SMOA++ technique, based on the WordNet dictionary detects the antonyms and the non equivalent strings. The substrings Decrement and Increment, Up and Donw exist in WordNet and they can identified as antonyms. Thus, in SMOA++ (see equation9.2) when at least one antonym is detected, the whole similarity is set to zero. The strings article and particle are not related in WordNet thus, the returned similarity is zero.The second category, contains synonyms, SMOA++ is able to detect a valid equivalence between the strings pairs. SMAO++ is able to detect noun abbreviations between two strings like SetTV and SetTelevision. Such abbreviation exists for some nouns in WordNet.The final category reveals two weakness points in the SMOA++ technique. The first point appears when specific domain antonyms and synonyms are not present in WordNet. In Table11.4, the first pair of strings SetVolumeUp and IncrementVolume can be equivalent semantically, however, since the strings Increment is not related to the string Up in WordNet, then the detected similarity is not strong. Moreover, the SetVolume and GetVolume are not detected as antonyms since the strings Set and Get are not related in the dictionary. However, the tokenization is not always accurate as shown in the last pair of strings ofTable 11.4.

Thus, since the SMOA++ similarity is highly dependable on the WordNet dictionary, missing or non related information in WordNet influences the SMOA++ matching result. The chosen strings in Table

11

.4 are specific to the device domain therefore they are not related in WordNet which contains general synonyms relations and antonyms.

The second weakness point resides in the tokenization of strings according to their existence in WordNet.

Moreover, SMOA++ keeps the longest common substring found in WordNet and repeats the operation until no substrings can be identified. For example, the CreateJob string is tokenized into Create, eat and Job. The common longest substring between Create and eat is Create thus the tokenization result of CreateJob is Create and Job.

The PrinterStatus is tokenized into Printer and Status. However, the tokenization of the string PrinterState identifies the following substrings: Printer, State and interState since the three subtsrings exist in WordNet. SMOA++ retains the substring interState which has the longest common substring. Then, the SMOA++ matching applied on {Printer, Status} and {interState} returns zero.

Table 11 .

 11 16: Completing Comparison of Table 7.1

	Used	Protocol Mediation	Presentation Mediation	Content Mediation	Realization	Description Annotation Intervention Human
	UPnP, DPWS, Bonjour, IGRS	Base Drivers	OWL Writers	Device Matching, Alignment Validation	DOXEN	Not Required	Alignment Validation

  <e n t i t y 1 r d f : r e s o u r c e= ' h t t p : //OWLWriter/UPnPWriter . owl#S e t L o a d L e v e l T a r g e t ' /> <e n t i t y 2 r d f : r e s o u r c e= ' h t t p : //OWLWriter/DPWSWriter . owl#S e t L e v e l ' />

	6 6</ measure>
	</ C e l l>
	</map>
	<map>
	<C e l l>
	< r e l a t i o n>=</ r e l a t i o n>
	<measure r d f : d a t a t y p e= ' h t t p : //www. w3 . o r g /2001/XMLSchema#f l o a t '>0 . 7 1</ measure>
	</ C e l l>
	</map>
	<map>
	<C e l l>

UPnPWriter . owl#Tar get ' /> <e n t i t y 2 r d f : r e s o u r c e= ' h t t p : //OWLWriter/DPWSWriter . owl#L i g h t L e v e l T a r g e t ' /> < r e l a t i o n>=</ r e l a t i o n> <measure r d f : d a t a t y p e= ' h t t p : //www. w3 . o r g /2001/XMLSchema#f l o a t '>0 . <e n t i t y 1 r d f : r e s o u r c e= ' h t t p : //OWLWriter/UPnPWriter . owl#GetMinLevel ' /> <e n t i t y 2 r d f : r e s o u r c e= ' h t t p : //OWLWriter/DPWSWriter . owl#G e t L e v e l ' /> < r e l a t i o n>=</ r e l a t i o n> <measure r d f : d a t a t y p e= ' h t t p : //www. w3 . o r g /2001/XMLSchema#f l o a t '>0 . 8 3</ measure> </ C e l l> </map> <map>

Telecoms operators can also provide ubiquitous applications. Thus, playing two roles.

www.oasis-open.org

http://developer.apple.com/opensource/

www.socrades.eu/Home/default.html

www.exceptionalinnovation.com

www.omg.org/mda

http://elite.polito.it/ontologies/simpleHome.owl

mvel.codehaus.org

http://www.nlm.nih.gov/research/umls/

The AddDocument takes the document URL location as an input. The SendDocument can also be used, the document is then sent to the printer as an attachment. In our experimentation, we used the more complex adaptation using SendDocument

Actually, there is a sequential union between the three actions. However, we only consider the union relation in this example in order to explain how a union adaptation can be carried out

In the following matching degrees' definitions the sequential union parameters resolving the non existing equivalence relations is considered. However, it is only removed from the definition for more clarity in the definition presentation

http://oppl2.sourceforge.net

http://freemarker.sourceforge.net/

www.janino.net

http://support.hyperic.com/display/SIGAR/Home

www.ws4d.org

http://software.intel.com/en-us/articles/intel-tools-for-upnp-technologies/

http://itunes.apple.com/us/app/mydevices/id339268052?mt=8

http://www.cybergarage.org

www.globalscaletechnologies.com/t-guruplugdetails.aspx

www.ws4d.org

The left sub-figure11.1 shows only 4 devices since the UPnP Fake TV and UPnP SendPS have similar results

www.ws4d.org

Acknowledgements

List of Tables

difference is positive then the newV olT V 1 > currentV olumeT V 2 , the proxy should increment the volume of the T V 2 , (line 9). Else if the difference is negative then the proxy should decrement the volume of the T V 2 , (line 10).

In order to include the adaptation behavior, the expert adds an alignment between the action a 1 and the three actions a 2 , a 3 and a 4 . Then the expert selects the SetVolume entity on the left ontology and adds the Code. When the SetVolume is invoked on the proxy, it behaves according to the following code.

1 //Select the right entities involved 2 DPWSAction a2 = g e t R i g h t A c t i o n ( "GetVolume" ) ;

3 DPWSAction a3 = g e t R i g h t A c t i o n ( "VolumeUp" ) ;

4 DPWSAction a4 = g e t R i g h t A c t i o n ( "VolumeDown" ) ; 5 //Retrieve the new volume value, "this" refers to the left action 6 i n t v1 = ( I n t e g e r ) t h i s . g e t I n p u t V a l u e ( " newVol " ) ; 7 a2 . i n v o k e ( ) ; // Invoke to retrieve the current volume on TV2 8 i n t d i f f = v1 -( I n t e g e r ) a2 . g e t R e t V a l u e ( " currentVolume " ) ; In this use case, the adaptation requires transforming the ps type files into pdf files. This operation is already implemented and provided by several services. Therefore, the adaptation only requires calling an ps-to-pdf format converter as shown in Figure 9.32. Thus, the expert adding the adaptation behavior of the proxy shown in Listing 9.7, specifies using the high level API to search and bind with an external service using the keyword ExtService. When the a1 is invoked on the proxy, it will invoke the method "File Convert(File ps)", of the external service "conv.ps2pdf.FileConvert" on the OSGi platform to convert a file. Once the file is converted, the proxy invokes the a 2 :SetPDF on the D 2 using the pdf file format.

The added behavior is saved to a file and its reference is attached to the ontology. The added code will be injected in the next step during the proxy generation.

This step returns an ontology holding the equivalent parameters, actions and composition patterns between two devices. It can also include an adaptation behavior. Thus, this ontology captures in a high level language the adaptation and transformation rules to go from one device to another. The next step consists in going from the high level ontology language to the lower level execution code through code generation techniques. The generated code represents the proxy which provides the interoperability between the semantically equivalent devices. 7 // S e t t h e i n p u t v a l u e 8 s 1 . s e t I n p u t V a l u e ( " F i l e " , ( F i l e ) a1 . g e t I n p u t V a l u e ( " ps " ) ) ;

9 // I n v o k e t h e method " Convert " 10 s 1 . i n v o k e ( ) ;

11 // R e t r i e v e t h e c o n v e r s i o n r e s u l t and i n v o k e t h e a2 12 a2 . s e t I n p u t V a l u e ( " p d f " , ( F i l e ) s 1 . g e t R e t V a l u e ( ) ) ; 13 a2 . i n v o k e ( ) ;

Listing 9.7: Adaptation through External Service Invocation

In this section, we detailed the device matching process with its four steps in order to semi-automatically detect correspondences between equivalent devices and services. The device matching allows to find equivalent entities and compositions between actions to achieve a successful substitution and interoperability between the services. Furthermore, since the matching is heuristics-based, we provide a simplified step to achieve an expert validation. Then, based on his validation, we apply rules to detect patterns between several entities. However, since not all the correspondences can be resolved automatically, we offer the expert an adaptation possibility by injecting an adaptation behavior during the code generation.

Concluding Remarks

In this chapter, we detailed our approach to resolve the heterogeneity between two equivalent devices. The approach combines two domains to achieve an interoperability between plug and play devices, the ontology alignment and the model driven engineering. The ontology alignment techniques based on heuristics, semiautomatically detect correspondences between two equivalent devices. Then, rules are applied to annotate the ontology with composition patterns and to report to the expert the non valid action compositions. Thus, the expert checks the device specifications to verify if an adaptation behavior can validate the detected compositions.

The validated ontology alignment contains entities of each device and the adaptation behavior to go from one device to another. Such adaptation behavior is represented in a high level language, the ontology language, independently from the implementation technical details. Matching Concept Once the rules are applied, the ontology is automatically annotated with the new detected patterns properties. The next step consists in finding valid mappings and compositions based on the equivalent input/output parameters as detailed in section 9.4.3.2.

Thus

1 s e q u e n t i a l M a p p i n g <OWLClass , OWLClass>=mergedOntology . g e t C l a s s O f P r o p e r t y ( " S e q u e n t i a l 1 t o n " )

2 d e c i d e M a t c h i n g ( s e q u e n t i a l M a p p i n g ) ; Even though, OPPL fits well into our needs to detect patterns, however it still lacks some expressiveness.

For instance, OPPL does not support a cardinality feature, for example count the number of input parameters an action has or count the number of equivalent parameters two actions share. There is also no support for a Not clause on classes to retrieve for example a UPnP Action class not having any output variable. A Set statement is also needed instead of only adding or removing an information. Loops and condition controls might also be useful. Thus, since OPPL does not support such features, we rely on the OWL API and Java code to detect the matching concepts along with the validity and the matching category between actions. The returned map is then used as an entry of the method decideMatching which applies the definition 9

on each sequential union 1 to n pair. The decideMatching method counts the number of input and output parameter each action has and the number of equivalent parameters. Then based on the decision Table 4.5, the method returns to the expert the decision of the matching.

We apply the same steps on the rest of the patterns. However, since the patterns are dependent on previously detected patterns, for example a Sequential 1 to n is also a Union 1 to n. Therefore, when applying getClassOf-Property("Union 1 to n") on the ontology, it will also return the actions already treated in the previous step, in [line 2]. Thus, we delete from the returned map the list of previously sequential unions and then we apply the deviceMatching on the union mappings.

The detected patterns and the added annotation to the ontology are used as a filter to retrieve the action composition instead of only relying of the equivalentTo relation between the classes of the ontology. The method get-ClassOfProperty("equivalentTo") will return all the equivalent entities in the ontology, devices, services, actions and parameters and then it is up to the developer to test the entities' types in order to carry on with the matching concepts. Using specific relations added by the patterns such as getClassOfProperty("Sequential 1 to n") guarantees that the returned entities are actions which makes it easier to the developer.

The decideMatching returns the decision of the matching concepts and then it is up to the expert to check for the non valid compositions and mappings if an adaptation is possible. The adaptation can require either have the same name but different signature). The returned values can also be retrieved. Listing 9.7 shows an adaptation code example using the External Service API.

The next section details how the ontology alignment is exploited to generate executable code which corresponds to a proxy providing interoperability between the two profiles in the ontology alignment.

DOXEN

In this section, we provide details on the implementation of DOXEN and explain the proxy generation mechanism. We also give an insight on the management services supported by DOXEN and its generated proxies.

Such services allow to carry out some basic management operations such as updating its configuration. Then, we describe through a use case in the experimentations section, how such management services can be used to carry out diagnostic operations. Such operations allow to detect basic malfunctions of applications interacting with the generated proxies in the digital home.

The DOXEN module takes an ontology alignment as an input, walks through the ontology and fills Java templates to generate Java code. The generated code is then compiled on the fly and packaged in an OSGi bundle (see section 3.3). Once started the OSGi bundle searches for the non-UPnP device type contained in the ontology alignment. When the non-UPnP device is found, the proxy first verifies that is supports the same services versions. Then, the proxy announces itself as a UPnP device on the network. When an application invokes the UPnP actions on the proxy, it will adapt the invocations and translates them to the equivalent non-UPnP device on the network.

DOXEN is installed in a home network, on a set-top-box for example. It listens to the non-UPnP device appearance on the network and based on its configuration containing the supported equivalent devices, the code generation can be triggered. We evaluate in this section the OWL Writers in two subsections. First, in subsection 11.1.1, we outline the ontology generation time with regard to the device and service descriptions. Second, in subsection 11.1.2, we provide the description size of the generated ontologies and how much information is needed to semantically annotate the device and service descriptions. 1 Line of Code Table 11.1 shows the time in seconds for the ontology generation per device type and the number of lines of each generated ontology (without comments and spaces). Moreover, Table 11.1 shows the UPnP or the WSDL file description used to generate the ontologies. We carried out each generation four times on a Java/OSGi framework on the PC, thus the time results in the Table 11.1 represent an average of four executions. We also added the generation time for the four fake devices descriptions used to validate the expert's code adaptation, detailed in section 9. Thus, the DPWS ontology generation depends on the complexity of the structural parameters in a description. On the contrary the UPnP ontology generation follows an exponential generation time with regard to the description size. Moreover, the UPnP description is less complex than the WSDL description which allows a structural parameters compositions.

Ontology Generation Time

We evaluate next the description ratio of the generated ontologies with regard to the device and service descriptions.

Annotated Information in the Generated Ontologies

In order to evaluate the ratio of the added information in the generated ontologies, we measured the coefficient of the generated ontology size to the description size used as an input, Overhead = Generated Ontology LoC The overhead DP W S value is less then the overhead U P nP . This is due to the fact that the WSDL description contains redundant and extra information not needed for the ontology generation. In fact, a WSDL description file contains several parts, mainly the portType part which defines a set of abstract operations (actions) and then refers to the input/output messages of each operation. Additionally, the binding part also contains information about the operation and specifies the protocol and data format along with the messages. Therefore, for the DPWS ontology generation, only a part of this redundant information is extracted and used.

As for the Fake4-1537 and Fake3-1537 which have equivalent description WSDL size, we noticed that the overhead of the Fake4-1537 reaches 2.7, while the overhead of Fake3-1537 has a value of 1.03. The difference in the overhead is due to the complex structural elements in the Fake4-1537 device while the Fake3-1537 only uses simple arguments. Thus, the WSDL flattening of complex parameters during the DPWS ontology generation, see section 10.1.1, increases the ontology description and generation time. and false. The "-" symbol refers to undetected mappings. Table 11.5 also shows the false detected matching and outlines the percentage success % of each method using two different threshold values. The success rate represents the number of valid detected mappings divided by all the valid mappings. We applied two hard thresholds t=0.63 and t=0.25, i.e. all the alignment tuples having similarity values higher than the threshold t value are retained. Table 11.5 shows the results without applying a similarity propagation between the entities, i.e t P .

The evaluation result shows that SMOA and SMOA++ have the same success rates 50% and 66% on the lights ontologies. However, the SMOA++ has clearly lower false matches when a higher trimming threshold value is applied. The number of false matches goes from 3 to only 1 with SMOA++. While in SMOA, the number of false matches remains the same.

The Table 11.6 shows the alignment results with the similarity propagation compared with the previous table where no propagation is applied. The propagation expressed in equation 9.4, allows to enhance the similarity value between two entities if their children have a similarity value equal or higher than the threshold value t P .

We show in Table 11.6 the similarity propagation with the following threshold values: t P = 0.4, t P = 0.8, and t P = 1. A hard trimming threshold is applied with t = 0.63. The similarity propagation results shown in Table 11.6 reveal an improvement in the success percentage when the similarity propagation threshold t P is equal or higher than 0.8. Additionally, the number of false matches remains the same. Thus, the similarity propagation allows to improve the matching result when used with an adequate selected threshold. propagation. The SMOA++ based on the WordNet dictionary is able to detect a similarity between the two services, the SimpleClockService and the timer. Thus, the percentage success of SMAO++ with a threshold t = 0.25 reaches 66%. Then, it is up to the epxert to add the 2 remaining matchings uwing ATOPAI to attend a successful mapping between the two clocks.

We show in Table 11.8 the alignment results with a similarity propagation method. The results are improved with two propagation similarity thresholds t P = 0.4 and t P = 0.8. The alignment evaluation on relatively two simple device types lights and clocks revealed high success matching rates reaching up to 66%. A high value hard trimming threshold allows to reduce the errors and remove the false matchings. In Table 11.5, with SMOA++, the number of false matching is reduced from 3

to only 1 with a hard trimming threshold value t = 0.63. However, a high level trimming threshold value also eliminates valid matchings having a weak similarity. Thus, an alternative solution consists in applying a high value hard trimming threshold t with a high level propagation similarity threshold t P to enhance the similarity values of entities based on the similarity of their mapped childs.

We expose next the alignment results between the two standard printers ontologies. The UPnP and DPWS printers represent the most complex standardized devices so far in the UPnP Forum and the DPWS standard profiles. First, we expose in Table C.1 in the appendix, the list of symbols used. Each alignment also indicates the success and false matching percentage rates for two hard trimming thresholds t = 0.63 and t = 0.25. SMOA++ also detects 8 false matches. However, with a high trimming threshold t = 0.63, SMOA++ achieves a lower success rate of 71% with only two false matches.

We also detail in Table C.8, the SMOA++ alignment results assisted with a propagation similarity with three threshold values t P = 0.4, t P = 0.8 and t P = 0.1. The alignment results are represented in Table C.8

without the state variables similarities. Since the similarity propagation applies a down-top enhancement, the state variables are taken into account to enhance the similarities of the actions. Thus, the similarity values of the state variables don't vary when the similarity propagation is applied. In Table C.8, the similarity propagation allows to enhance the similarity value of (PrinterEnhancer, PrinterService) and the GetPrinterAttributes union.

Thus, the similarity propagation increases the success rate of the alignment results.

The following Table 11.9 resumes the alignment evaluation on the three devices (UPnP/DPWS) using SMOA and SMOA++. It exposes the methods used, the alignment time is seconds, the success percentage for the hard trimming thresholds t = 0.63 and t = 0.25 as well as the number of false matches found. Both techniques have similar success rates when the trimming threshold is set to t = 0.63. However, the number of false matches detected in SMOA++ is much lower than those detected in SMAO in all the alignments. Table 11.9 shows that SMOA++ spends more time than the SMOA method in all the alignment results.

In SMAO++, the Tokenization step relies on the WordNet to split a string into several existing substrings in the dictionary. More over, the matching step in SMOA++ relies on the synonyms, antonyms and coordinates search in WordNet. Thus, the huge difference in time is due to the search in WordNet. However, such difference in time remains acceptable since the alignment is treated off line.

SMOA++ provides better results with a trimming threshold value set to t = 0.25. The success rates achieve 78%, however the number of false matches increases from 2 to 8 on the printers alignment for example. SMOA achieves a maximum success rate of 75% on the printers, however, the number of false matches increases from 6 to 11. In the other alignments, SMOA++ success rates are equal or higher than the SMOA's rates and the number of false detected correspondences are equal or lower.

The hard trimming threshold values are set by the expert using ATOPAI. A low threshold value returns high success rates, however it also retains a high number of the false detected matches. Thus, to reduce the number of false detected matches, a higher trimming threshold value is used, since it will remove all the weak similarities lower than the threshold value. However, a higher trimming threshold value also reduces the successful detected 1 For the variables, we counted the various leaves in the complex parameters. The aligned entities for the five devices is greater than 57% (60/104). The detected entities influence the size of the final ontology. In fact, Table 11.12 shows the description size of each generated ontology, the size before and after the ontology alignment. For instance, the printers ontology before the alignment reaches 10655 LoC.

However, after the alignment and validation, the final ontology reaches 11255 LoC, thus, the alignment added 600 LoC. The added information represents the alignment annotation between entities from both ontologies along with the pattern annotation.

During the code generation, DOXEN generates bundles based on the matched entities and the added annotations. DOXEN ignores the entities not having any relation with other entities.

The next section details the evaluation of the DOXEN module, the time spent to generate and start a proxy on the PC and the Set-Top-Box.

DOXEN

In this section, we expose the proxy generation evaluation time, the number of generated files and size of the packaged bundles. We also outline the invocation overhead between an application and a DPWS device receiving invocations through the proxy.

Proxy Generation

Table 11.13 resumes the time spent by DOXEN to (1) generate Java Code, (2) compile it, (3) package it in Java/OSGi Jars and finally (4) install it. We also show the number of Java template files and lines of code (LoC) used (without spaces and comments) as well as the number of automatically generated Java Files and their LoC. The table also shows the cost of adding a basic and a configuration management services. The proxy supporting the two management services (BMS, CMS) is referred to as an Manageable Device (MD) [ UPnP 11].

The values shown in the tables are an average result of three executions. We also show the proxy generation of the two use cases, the fake TVs and the PS-to-PDF conversion described in section 9.4. We detail next, the invocation overhead going through the proxy between a UPnP application and a DPWS device.

Proxy Invocation

We implemented DPWS specific control points to evaluate the invocation overhead through the generated proxies. We also extended the UPnP Felix Control Point to measure the action invocation time. The UPnP Control point and DPWS applications along with the generated proxies are deployed on the same PC and Thus, the proxy parses the returned WSDL structure and retrieves the values, then translates them into UPnP compatible values and returns them to the UPnP application. The difference in time between the two clients ranges from 17 ms to 297 ms. Such difference in time is insignificant to the user or the application invoking the actions. We also measured the time difference for the other devices and proxies, the difference in time is less than 5 ms. humans in their daily tasks. The heterogeneity between plug and play protocols encloses ubiquitous applications in the same protocol area and forces already implemented applications targeting a specific protocol to stay in the same perimeter. We believe that ubiquitous applications should be set free from the technical implementation details and use any available device capable of satisfying the applications needs.

A naive solution to liberate the applications would embed in each application the required implementation to support multiple protocols. However, supporting multiple protocols for each application is time consuming and error prone for the applications' developers. More over, such solution is not easily expendable and requires a manual adaptation.

Therefore, we propose in this thesis a novel approach to provide heterogeneity between equivalent device types supporting different protocols. The approach relies on the intersection of two major domains, the ontology matching crossed with the model driven engineering. The ontology matching allows to semi-automatically detect the correspondences between equivalent devices. Then, the adaptation to support multiple protocols is fully automated and relies on the previously validated ontology alignments. The adaptation is realized by automatically generating proxies. More over, there is no need to adapt the deployed plug and play applications since they will interact with the generated proxies transparently as a standard device supporting the same protocol.

We summarize next the three contribution this thesis provides.

Contributions

The main contributions of this thesis are the following:

• An End To End Adaptation Solution: Our proposed approach provides an end to end solution which consists of three modules to solve the three previously identified plug and play heterogeneity levels. The protocols stacks, the description format and the description content.

To resolve the first heterogeneity level, we rely on the technical solution provided by the service oriented framework OSGi and its base drivers. Each base driver reifies real devices as local OSGi services. Additionally, the base driver offers an API to interact with the real device on the network without going into the technical details of the protocol stacks. Thus, the base drivers refine the protocols stacks heterogeneity level into an API heterogeneity level.

To solve the second heterogeneity level, we propose the OWL Writers which automatically generate ontologies from devices descriptions. Each ontology captures the device description and the relation between its entities. By construction, the generated ontologies are conform to meta model which guarantees a unified representation. Thus, the OWL Writers resolve the heterogeneity level between the description format by exposing the device description in an ontology. The generated ontologies are then sent to the operator's site to detect the correspondences with another equivalent UPnP device.

The third level of heterogeneity, the description content, is handled by ATOPAI which relies on: the semiautomatic alignment techniques, the rules to detect composition patterns and on the expert intervention to validate the detected correspondences. The device matching contains the adaptation behavior between the two devices. Thus, the device matching resolves the content description heterogeneity.

Once the alignment is completed and validated, rules are applied on the ontology to annotate it with composition patterns between the actions. We also proposed the device matching concepts to assist the expert and point out the non valid compositions. Then, the expert refers to the devices specifications to check if an adaptation is possible by adding an adaptation high level code.

12.