Projects Local or Distant Augmented Reality Open Standard Collaboration Context Awareness FP5 STARMATE

Keywords: rotation and scale invariance

L'ingénierie en ligne et les laboratoires distants

Cette étude est centrée sur l'ingénierie en ligne (de l'anglais online engineering) et les laboratoires distants.

Le facteur financier peut être déterminant dans le choix de mettre en place une solution de contrôle distant d'un appareil. Les coûts des appareils à la pointe de la technologie sont souvent trop élevés en en regard de leur fréquence d'utilisation. La solution habituelle consiste à déplacer les opérateurs sur le site de l'instrument, induisant des contraintes administratives et des coûts indirects (logement, temps de déplacement, etc.). Cependant, le coût n'est pas le seul facteur qui peut entrainer un contrôle distant d'appareils. D'autres facteurs comme la sécurité, l'accès àl'instrument,ladisponibilité des personnes, l'observation en sont des exemples. L'instrumentation en ligne est aujourd'hui principalemente m p l o y ép a rt r o i st y p e d'organisation : -l e si n s t i t u t sd ef o r m a t i o np o u rl ' é d u c a t i o ne nl i g n e , -l ' i n d u s t r i ep o u rl am a i n t e n a n c eàd i s t a n c ee tl ' e n t r e p r i s eétendue, -l e sl a b o r a t o i r e sd er e c h e r c h ep o u rl et r a n s f e r td ec o m p é t e nces. Pour illustrer nos propos et nos travaux, nous proposons deuxs c é n a r i o s .L ep r emier concerne Steve, Heather et Jimmy. Steve est enseignant en Norvège et Heather et Jimmy sont deux de ses étudiants actuellement à l'étranger, le premier à Paris, la seconde à Tokyo. Pour pouvoir valider leurs diplômes, Heather et Jimmy doivent valider des cours dans leurs université de départ. Ce cours inclut destravauxpratiquessurun analyseur de spectre et sur un analyseur vectoriel de réseaux.

Le second scénario concerne Alice et Bob. Alice est technicienne dans une manufacture. Suite à une panne sur son appareil, elle contacte Bobq u ie s to p é r a t e u rd a n s une entreprise de maintenance. Pour réduire le coût de la maintenance, Bob effectue tout d'abord une opération à distance sur l'appareil défectueux. Étant donné que Bob effectue des opérations de maintenance sur un large panel d'appareil, il ne connait pas par coeur toutes les procédures à effectuer. Il utilise donc le système d'assistance de son entreprise qui inclut ces procédures.

Ces deux scénarios induisent différentes problèmatiques. En ce qui concerne l'éducation en ligne, ces problèmes sont les suivants :

-une grande fidélité de représentation d'un instrument dans son environnement.

-u n ei n t e r o p é r a b i l i t ép e r m e t t a n td ec o n t r ô l e rl ' i n s t r u m e nt distant de façon pervasive 1 et depuis n'importe quel type de terminal (smartphone, tablette, PC, etc.), -u nc o n t r ô l ed el ' i n s t r u m e n tc o l l a b o r a t i fe tm e t t a n te np l a ce une sensation de groupe, -u n es o l u t i o nl ap l u sr é u t i l i s a b l ee ts i m p l ep o s s i b l e .

1. n'importe où, n'importe quand vi 0.2 État de l'Art L 'INGÉNIERIE en ligne et les laboratoires distants permettent de mettre enplacedes manipulations sur des instruments réels via Internet. Les feedbacks 2 de l'instrument distant doivent donc être représentés pour appréhender l'état de l'opération menée. Côté client, cela peut être permis grâce à la représentation des feedbacks de l'instrument en tant que composants graphiques dans l'IHM de pilotageoualorsentantque composant vidéo de l'instrument filmé. Dans tous les cas, l'IHMp r é s e n t é ed o i tê t r e fidèlement représentative de l'instrument piloté. La réalitém i x t ep e u tê t r eu nm o y e n d'améliorer l'immersion de l'utilisateur et sa compréhension de la scène distante. Àl ' èr edel ' i nf orm a ti queperv a si v eetub i qui ta i r e,l esl a b oratoires distants doivent être accessibles n'importe quand, n'importe où et depuis n'importe quel type de terminal. Cela implique un fort degré d'interopérabilité dans lessolutionsàmettreenplace. La réutilisabilité ainsi que le couplage lâche des composants logiciels sont des points à prendre en compte. La collaboration, i.e. la possibilité d'accomplir une tâche en groupe, ainsi que la conscience du groupe sont également des points importants à prendre en compte.

Dans ce chapitre, nous nous intéressons tout d'abord aux technologies et paradigmes qui permettent de répondre aux problématiques rencontrées au premier chapitre. Dans un second temps, nous présentons les projets de recherche principaux dans le domaine de l'ingénierie en ligne puis nous nous arrêteronssurunesynthèsedeces projets.

Technologies

0.2.1.1 Réalité Mixte La réalité augmentée est la combinaison d'objets virtuels dans un mode réel. La virtualité augmentée est la combinaison d'objet réel dans unmondevirtuel.Puisquela frontière entre ces deux concepts n'est pas direct, nous préférons employer le terme de réalité mixte qui embrasse ces deux notions. Pour réaliser un système de réalité mixte, trois règles fondamentales doivent être respectées : -c o m b i n e rl e sm o n d e sr é e le tv i r t u e l , -p r e n d r ee nc o m p t ed el ac o h é r e n c ee n t r ec e sd e u xm o n d e s , -p r e n d r ee nc o m p t el ' a s p e c tt e m p s -r é e le tl ' i n t e r a c t i v i t édes systèmes de réalité mixte. Pour pouvoir afficher cette combinaison d'objets réels et virtuels dans un monde commun, un environnement propre doit être choisi. Le plus souvent, c'est un environ-2. réactions et retours List of Tables 1D e s c r i p t e u r s d e s c è n e s m u l t i m é d i a . . . . . . . . . . .

Résumé long 0.1 Introduction

T OUT au long des dernières décennies, les architectures logicielles ont évolué de fa- çon significative. Avec l'explosion d'Internet, les systèmesinformatiquessontcapables de traiter des données de plus en plus complexes. Le développement des terminaux utilisateurs a également favorisé les transformationsdesarchitectureslogicielles. 0.1.1 Il était une fois... L'émergence des Applications Internet Riches (RIA, de l'anglais Rich Internet Application) est le résultat de l'évolution des architectureslogicielles,du"mainframe"pendant les années 60 à 80, à l'architecture "deux-tiers" dans les années 80 puis à l'architecture "n-tiers" dans les années 90. Elles sont principalemente x é c u t é e sd a n su nn a v i g ateur Web à l'aide de plug-in et de cadriciels externes. Les plus connus sont Asynchronous Javascript and XML (AJAX), Adobe Flash ou encore Microsoft Silverlight.Leurs principaux avantages sont de proposer :

-u n ee r g o n o m i ea v a n c é e , -la possibilité de modifier l'interface utilisateur en fonction du business, -u n es é c u r i t éa c c r u eg r â c ea u xt e c h n i q u e sd eb a càs a b l e( s a n dbox), -l ap o s s i b i l i t éd ' e m b a r q u e rd e sa n i m a t i o n se td uc o n t e n um u ltimédia.

Les capacités de communication et l'évolution des architectures logicielles ont également rendu possible le contrôle distant d'appareils divers avec leurs représentation dans le monde applicatif. Les relations entre les humains et les machines ont évolué, que cela soit en situation de contrôle d'un appareil ou en situation de simulation. Ce contact n'est pas physique mais est réalisé à l'aide de l'informatique. On parle alors de "systèmes assistés par ordinateur" car l'ordinateur n'est pas seulement responsable de représenter l'Interface Homme-Machine (IHM) de l'instrument contrôlé ou simulé mais également d'assister l'opérateur humain dans sa tâche.

Si l'on met de côté l'aspect intrusif de l'informatique, il est important de remarquer que l'innovation n'est pas seulement technologique. L'informatique peut mettre en relation plusieurs personnes avec plusieurs machines, mettant en place une collaboration pour accomplir des tâches communes. Les télé-opérations, oulacapacitédecontrôler un appareil à travers Internet, permettent de mettre en placedesmanipulationsquine pourraient pas avoir lieu autrement.

Le scénario sur la maintenance à distance permet de soulever les problématiques suivantes :

-l es y s t è m ed o i tp r o p o s e ru nf o r m a l i s m ep o u re x p r i m e rl e sp r océdures, -l al o g i q u em é t i e rq u ip e r m e td ed é t e r m i n e rl e sa c t i o n ss u i v antes doit être découplée le plus possible d'autres composants logiciels pour être réutilisable, -l es y s t è m ed o i tp r o p o s e ru n em é t h o d ep o u ra n a l y s e rl e sa c t i ons effectuées et suggérer des améliorations à l'utilisateur. Nous les compilons dans la liste suivante et soulignons le fait qu'ils impacts de façon significative l'interface homme machine du système :

-u nc o n t r ô l ed ' i n s t r u m e n t sd i s t a n t sq u ip r é s e n t eu n eg r a n d ei m m e r s i o ne tu n e grande compréhension de la scène appréhendée, -u nc o n t r ô l ec o l l a b o r a t i fd el ' a p p a r e i ld i s t a n t , -u n ea d a p t a t i o nd el ' i n t e r f a c ep o u ra i d e rl ' o p é r a t e u r , -u ns y s t è m es c a l a b l ee tc a p a b l ed es ' a d a p t e ràu nl a r g ep a n e ld'instruments.

Organisation du manuscrit

L'ingénierie en ligne permet de contrôler des instruments à distance. Elle est motivée par la volonté de réduire des coûts tout autant que par la volonté de répondre à des problématiques techniques telles que la dangerosité, l'observabilité, la disponibilité, etc. Principalement trois types d'organisations sontc o n c e r n é sp a rl ' i n g é n i e r i ee n ligne : l'industrie pour la maintenance à distance et l'entreprise étendue, les instituts de formation pour l'éducation en ligne, et les laboratoires de recherche pour le partage et le transfert de compétences.

Nous avons présenté deux cas d'usage, l'un concernant l'éducation en ligne et l'autre la maintenance à distance. Ces scénarios nous ont permis de soulever des problématiques.

L'organisation de ce document est la suivante. Le chapitre 2 présente l'état de l'art dans les système d'ingénierie en ligne. Le chapitre 3 présente le cour de notre proposition et son architecture générale. Le chapitre 4 détaille certains points de l'implémentation de notre système. Le chapitre 5 conclut et propose des pistes d'ouvertures et de continuité de nos travaux. vii nement géométrique à trois dimensions. Dans cet environnement, les objets doivent être correctement alignés. Pour réaliser cette tâche, différentes méthodes de suivi d'objets sont mises en place. L'objet suivi peut être l'utilisateur lui-même ou des objets de l'environnement. La plupart des méthodes de suivi sont issues du suivi avec ou sans marqueur.

Le suivi d'objet avec marqueur consiste à placer un objet de référence facilement identifiable dans l'environnement. Ce marqueur peut être visible (comme celui proposé par la librairie ARToolKit), ou invisible (infra-rouge par exemple).

Les techniques de suivi d'objets sans marqueur peuvent prendre différentes formes. Ils peuvent se baser sur des méthodes de système de position (Global Positioning System et boussole), mécanique, ou encore sur des techniques de traitement d'image.

Le suivi basé sur le traitement des images se fonde sur la détection de points d'intérêt dans les images. On peut citer les méthodes FAST [START_REF] Rosten | Fusing points and lines for high performance tracking[END_REF], le détecteur de coins de Harris [START_REF] Harris | A combined corner and edge detector[END_REF] ou encore la différence de gaussienne (DoG) [START_REF] David | Distinctive Image Features from Scale-Invariant Keypoints[END_REF]. D'autres techniques de traitement d'image ajoute à ces points d'intérêts de descripteurs qui permettent de caractériser ces points et faciliter leur identification. Tuytelaars et al. définissent les caractéristiques des descripteurs dans leurs travaux [START_REF] Tuytelaars | Local invariant feature detectors : A survey[END_REF]. Parmi les plus connus, on peut citer Scale-Invariant Feature Tranform(SIFT) [START_REF] David | Distinctive Image Features from Scale-Invariant Keypoints[END_REF],Speeded-Up Robust Feature (SURF) [START_REF] Tuytelaars | Local invariant feature detectors : A survey[END_REF], BRIEF [START_REF] Calonder | BRIEF : Computing a Local Binary Descriptor very Fast.I E E ET r a n s a c t i o n so nP a t t e r n Analysis and Machine Intelligence[END_REF] et Oriented FAST and Rotated BRIEF (ORB) [START_REF] Rublee | ORB : An efficient alternative to SIFT or SURF.C o m p u t e rV i s i o n[END_REF].

Àp a r t i rd e sd i f f é r e n t e sm é t h o d e sd es u i v i ,i le s tp o s s i b l ed ec r é e ru nl a r g ec h o i x d'applications dans des domaines divers tel que la culture[Marshall11], les jeux vidéo [START_REF] David Cheok | Human Pacman : a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing[END_REF], l'armement [START_REF] Julier | BARS : Battlefield Augmented Reality System[END_REF] ou la santé [START_REF] Suzuki | Scorpion shaped endoscopic surgical robot for NOTES and SPS with augmented reality functions.I nP r o c e e d i n g so ft h e 5th international conference on Medical imaging and augmented reality[END_REF].

Collaboration

Les travaux sont souvent naturellement collaboratifs. Cela signifie que plusieurs acteurs interagissent entre eux dans le but d'atteindre un but commun. En revanche, il est important de souligner que collaboration et coopération sont différents. La coopération implique une division des tâches à effectuer, contrairementàl ac o l l a b o r a t i o no ul e s tâches ne sont pas divisées.

La communication prend une part importante dans la collaboration. Différents outils peuvent être utilisés (e-mail, conférence audio-visuel, etc.), chacun apportant des fonctionnalités différentes (messages synchrones/asynchrones, etc.).

Adaptation au contexte et Connaissance Sémantique

En informatique, le contexte représente "toute information qui peut être utilisée pour représenter une situation ou une entité. Une entité est une personne, un endroit, ou un objet viii qui est considérée comme pertinente dans l'interaction entre un utilisateur et une application, incluant l'utilisateur et l'application eux-même." [START_REF] Dey | Towards a better understanding of context and context-awareness.I nC H I2 0 0 0w o r k s h o po nt h ew h a t ,w h o , where, when, and how of context-awareness[END_REF].

Nous n'avons pas l'intention dans ces travaux de redéfinir les systèmes adaptatifs au contexte, mais plutôt de définir un élément clé du contexte del'utilisateur:lapr océdure d'utilisation de l'IHM.

Dans le domaine de la maintenance assistée par ordinateur, l'ingénierie en ligne aide àef fectuerdesopérationsenlignes.Lesopérateursdemaintenance sont responsables d'un large panel d'instruments avec un large panel de procédures propres. Dans ce cadre, un système de contrôle distant d'instruments peut proposer un formalisme pour sérialiser ces procédures et adapter l'interface homme machine du système dans le but d'aider l'opérateur dans sa procédure de maintenance dans lebutdelimiterlenombre d'erreurs humaines.

Le Web Sémantique et ses technologies associées (RDF/RDFS) couplés avec un moteur de règle sémantique permettent le raisonnement nécessaire aux systèmes adaptatifs au contexte.

Description de Scène Multimédia

En informatique, on désigne par média un support d'information. Le terme multimédia désigne généralement des applications capables de générer ou contrôler plusieurs média tels que le son, la vidéo, les images, etc. Nous définissons un multimédia comme "plusieurs média ayant une relation physique, logique, temporel ou sémantique présenté suivant les choix de l'auteur". Pour pouvoir maitriser de telles relations, il est possible d'utiliser les descripteurs de scène multimédia.

Ces descripteurs de scènes permettent de mettre en oeuvre lesinstructionsdegestion des différents média. Les propriétés des descripteurs de scène sont les suivantes [START_REF] Concolato | Descriptions de scènes multimédia : représentations et optimisations[END_REF] :

-l ' e m p l o id em é d i aa u d i o -v i s u e l( n a t u r e lo us y n t h é t i q u e ) , -l ' o r g a n i s a t i o ns p a t i a l ed e so b j e t s( r e l a t i v eo ua b s o l u e )dans un environnement à deux ou trois dimensions, -l ' o r g a n i s a t i o nt e m p o r e l l ed e so b j e t s , -l e sc a p a c i t é sd ' i n t e r a c t i v i t é s . On représente le plus souvent une description de scène multimédia par un arbre de scène (en anglais scene-graph). Les plus connus sont Virtual Reality Modeling Language (VRML), Synchronized Multimedia Integration Language (SMIL), Scalable Vector Graphics (SVG), COLLAborative Design Activity (COLLADA), Adobe Flash, Ja-vaFX, Microsoft Silverlight, HyperText Markup Language (HTML), MPEG-4 BInary Format for Scene (MPEG-4 BIFS et MPEG-4 Lightweight Application Scene Representation (MPEG-4 LASeR).

Les descripteurs de scène multimédia ne sont pas tous capables de LASeR, contrairement à BIFS, n'est pas capable de gérer les contenus en trois dimensions, ce qui fait de MPEG-4 BIFS le descripteur de scène le pluscomplet.

Ingénierie en ligne

Nous avons vu que la réalité mixte est fortement présente dansdifférentsdomaine. C'est aussi le cas dans l'industrie. Nous pouvons par exempleciterlepr ojeteur opéen FP5 STARMATE,leprojetfrançaisTélé Assistance Collaborative ou encore le projet utilisé par l'US Nay Augmented Reality for Maintenance and Repair (ARMAR). Dans l'éducation en ligne, il existe de nombreux cadriciels pour contrôler à distance différents types d'appareils. Ces cadriciels apportent différentes briques. Par exemple, iLab propose une architecture basé SOAP pour gérer l'authentification, l'orchestration ou l'autorisation de l'accès à la ressource [START_REF] Vj Harward | iLab : A scalable architecture for sharing online experiments[END_REF]. VISIR quant à lui propose une interface basée sur Adobe Flash pour représenté l'IHM de l'instrument. Il propose éga-x lement son propre système de gestion d'éducation (Learning Management System en anglais).

Synthèse

Les tableaux 3 et 2 résument l'étude des projets existants. L'étude des projets existant pour l'industrie montre cependant que les projets de réalité mixte ne sont pas ou peu employées dans un usage distant des environnements contrôlés.

Bien que beaucoup de systèmes pour l'éducation en ligne permettent de contrôler des appareils via Internet, peu de ceux-ci utilisent la réalité mixte pour augmenter l'immersion de l'apprenant et encore moins de descripteurs de scènes sont proposés pour gagner en interopérabilité. Peu de systèmes proposent un contrôle collaboratif de l'instrument distant. Le même constat est valable pour les systèmes capables d'adapter leurs interfaces en fonction d'éléments du contexte.

Au vu de l'analyse réalisée, il nous parait important de pouvoir proposer une solution de contrôle d'instruments distants répondant au mieux aux questions suivantes :

- L AS E C T I O Nprécédente a soulevé des problématiques sur l'ingénierie enligneetles laboratoires distants.

Nos travaux se compilent sous la forme d'un cadriciel pour construire des applications de contrôle d'instruments distants pour un usage dans l'éducation en ligne ainsi qu'un usage industriel. L'application cliente est construite à l'aide de la norme MPEG-4 BIFS, pour augmenter le degré d'interopérabilité et utilisel ar éa l i tém i x tepou rr epr ésenter au mieux l'environnement distant.

Ce cadriciel prend en compte le travail collaboratif, i.e la possibilité de contrôler l'instrument distant à plusieurs, tout en reproduisant une conscience de groupe. Pour faciliter cet aspect, certaines modifications ont été apportés à la norme MPEG-4 BIFS.

Nous proposons enfin de modéliser les procédures d'exécution àl'aidedetechnologies du Web sémantique. L'utilisation d'un moteur de règlep e r m e td ed é d u i r el a procédure actuelle et de proposer à l'utilisateur les interactions suivantes possibles.

Cadriciel pour l'ingénierie en ligne 0.3.1.1 Architecture

Nous proposons un cadriciel pour le développement d'applications d'expérimentations distantes. Ce cadriciel comprend deux modes. Le moded'exécution est effectif lors d'un contrôle distant. Il délivre une vue augmentée et collaborative de l'instrument distant. Un méchanisme pour retranscrire la "conscience de groupe" est proposé. Le système permet aussi de proposer une adaptation de l'IHM en fonction de la procédure d'exécution.

Le second mode est un mode d'initialisation.I lp r o p o s ed i f f é r e n t so u t i l sa u t e u r s pour permettre de formaliser la description de la scène distante ainsi que la description des pilotes de l'instrument.

Chaine logicielle

Pour proposer un cadriciel complet de construction de solutions de pilotage d'instruments à distance, il est nécessaire d'étudier la chaine logicielle complète entre les utilisateurs et l'instrument distant. Généralement, une unité de traitement est reliée à l'instrument distant pour améliorer les capacités de communication de l'instrument distant. Cette chaine est principalement composée de trois liens :

-L el i e ne n t r el ' u n i t éd et r a i t e m e n te tl ' i n s t r u m e n tàp i l o t er, -L el i e ne n t r el ' u n i t éd et r a i t e m e n te tl et e r m i n a ld el ' u t i l isateur, xiv -L el i e ne n t r el et e r m i n a ld el ' u t i l i s a t e u re tl e su t i l i s a t e urs.

Pour mettre en place le lien entre l'unité de traitement et l'instrument à piloter, il est nécessaire de maîtriser un grand nombre de standards d'accèsauxinstruments(GPIB, USB, PXI). Cependant, le standard Interchangeable Virtual Instrument (IVI) permet une grande interopérabilité pour des classes d'instruments définies (oscilloscope, analyseur de spectre, etc.).

Le lien entre le terminal utilisateur et l'unité de traitement peut être réalisé grâce àl ' e m p l o id ' u ni n t e r g i c i e l( o um i d d l e w a r ee na n g l a i s ) .C et ype de logiciel permet l'échange de messages entre composants logiciels à travers des réseaux, notamment Internet. Il est important de garder en tête le côté collaboratif de l'architecture que nous souhaitons mettre en place. L'intergiciel à mettre en place doit donc répondre aux critères suivants :

-A s y n c h r o n i s m e : p o u r g a r d e r l ec o n t r ô l e d e l ' a p p l i c a t i o nm ême si le message n'est pas délivré, -R o b u s t e s s ee ts c a l a b i l i t é:p o u rn ep a sl i m i t e rl en o m b r ed epersonnes connectées sur l'instrument distant, -G a r a n t i ed el i v r a i s o n: l ' i n t e r g i c i e ld o i t a s s u r e r q u e l e smessages soient livrés dans l'ordre d'émission, surtout pour assurer le bon comportement de l'instrument à piloter.

L'IHM est le dernier lien de la chaine logicielle. Il nous parait important que l'IHM propose les traits suivants :

-u n ec e r t a i n eg é n é r i c i t ée tu nc o u p l a g el â c h ea v e cl ' i n t e r g iciel pour faciliter la réutilisation, -une grande fidélité de représentation de l'environnement de l'instrument distant, -u n ec o n n e x i o nàl ' i n t e r g i c i e lp o u rc o m m u n i q u e ra v e cl ' i n s trument et avec les autres utilisateurs, -u nl a r g ec h o i xd ec o m p o s a n t sg r a p h i q u e s( i m a g e s ,s o n ,v i d é os, etc.).

-u nh a u tn i v e a ud ' i n t e r o p é r a b i l i t é , -u n ea d a p t a b i l i t ée nf o n c t i o nd ' é l é m e n t sd es o nc o n t e x t e .

Conclusion

L'architecture complète du système est proposée dans la figure3 . 2 .L ac a m é r aI P filme l'instrument distant. Le serveur d'application construit un multimédia interactif basé sur MPEG-4 BIFS et le délivre aux utilisateurs. Lorsque qu'une commande est passée, le serveur d'application détermine les actions possibles suivantes et renvoie les résultats à l'utilisateur. La commande est ensuite passée à l'instrument. Le mode initialisation est illustrée par la figure 3.3. Cette division permet de supporter les quatre contributions majeurs que nous voulons adresser dans ces travaux. Le "Multimedia Builder" construit la descriptions de la scène distante àtraversuneinterfacederéalité xvi augmentée. Les modules "Commands Interpreter" et "Notification Block" permettent le contrôle collaboratif de l'instrument. Le module "Whisper" propose l'adaptation du contenu en fonction de la procédure d'exécution. La combinaison des modules "Commands Interperter" et "Multimedia Builder" permettent de garantir une scalabilité du nombre d'instruments supportés.

Réalité Augmentée

Pour construire l'IHM de l'instrument, il est essentiel d'analyser l'environnement de l'instrument à piloter. Il apparait alors trois types de régions dans la scène :

-u n er é g i o nv i d é oc o r r e s p o n da ufeedback de l'instrument, -u n er é g i o np r o p o s el e sc o m p o s a n t sg r a p h i q u e sd el ' i n t e r f a ce et l'interactivité nécessaire pour piloter l'instrument, -u nf o n d . Ces différents régions sont réalisées grâce aux caractéristiques de la norme MPEG-4 BIFS, à savoir une représentation des médias sous la forme d'un arbre hiérarchique, différents noeuds rendant possible l'interactivité et l'environnement de streaming pour la livraison du contenu multimédia.

Pour assurer la cohésion du monde réel et virtuel dans une application de réalité mixte, il est nécessaire de connaitre à tout moment la position des composants de l'environnement entre eux. Nous choisissons donc d'effectuer unsuividelafaçadedel'instrument à l'aide des descripteurs de points caractéristiques des images. Il s'agit de calculer ces descripteurs sur les images délivrées par la caméraIPetdelescompareravec ceux d'une image de référence enregistrée dans la base de données du système, ce qui permet d'identifier l'instrument présent devant la caméra.

Une fois l'instrument identifiée, les points caractéristiques sont mis en correspondance avec ceux de l'image de référence grâce à l'algorithme Fast Library for Approximate Nearest Neighbor [START_REF] Muja | Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration[END_REF] (FLANN). Il est ensuite possible d'estimer la position de l'instrument dans le flux vidéo à l'aide d'homographies, calculées grâce au calibrage préalable de la caméra en suivant l'algorithme de Zhang[ Z h a n g 0 0 ] -[ Z h a n g 1 0] . L'homographie permet de calculer la transformation d'un solide entre un monde en trois dimensions (l'environnement en réalité mixte) et un monde en deux dimensions (l'IHM de pilotage délivrée à l'utilisateur) à l'aide d'un calcul matriciel.

Gestion de la Collaboration

La collaboration permet de mettre en relation des individus ou organisations dans le but d'accomplir une tâche ou de résoudre une problématiquequin'estpasfacilement achevée seul.

Pour assurer la collaboration, un haut niveau de confiance entre les différentes parties doit être achevé, de même qu'une communication au sein dug r o u p e .A i n s i ,u n système collaboratif ne doit pas non seulement permettre le partage d'une ou plusieurs ressources au sein du groupe, mais également de retranscrire la conscience de groupe (groupe awareness en anglais). La conscience de groupe "est la compréhension de qui travaille avec vous, ce qu'ils font et comment vos actions influent sur les leurs" [START_REF] Gutwin | r o c e e d i n g s o f t h e ACM Conference on Computer-Supported Cooperative Work[END_REF]. La conscience de groupe doit donc être au centre du cadriciel que l'on propose pour proposer un système collaboratif effectif.

Intergiciel

L'intergiciel doit permettre de gérer un très grand nombre depersonnesconnectées sur la plateforme au même moment et répondre aux critères que nous avons énoncés précédemment. Il parait donc qu'un intergiciel orienté message (Message-Oriented Middleware ou MOM en anglais) soit la solution pour assurer ces critères.

L'intergiciel doit suivre le paradigme publier/souscrire expliqué par [START_REF] Th | The many faces of publish/subscribe[END_REF] qui est le suivant :

-l e sc l i e n t ssouscrivent àuncanaldediscussion, -u np r o t a g o n s i t epublie un message sur le canal, -u ncourtier redistribue le message publiée sur le canal aux autres clients.

Pour pouvoir assurer un couplage lâche entre l'IHM de pilotage et l'intergiciel, nous avons créé deux types de canaux pour l'interaction. Le premier canal est le canal de commande. Celui-ci permet d'envoyer les commandes à l'instrument. Le second canal est le canal d'interface. Quand une action est réalisée dans l'IHM, les autres clients doivent être notifiés de cette action.

Interface Multimédia Interactive et Collaborative

Proposer un intergiciel qui supporte la collaboration est nécessaire mais n'est pas suffisant pour transcrire la conscience de groupe aux utilisateurs. Nous proposons ici les outils pour la transcrire.

Chaque utilisateur reçoit une couleur. Cette couleur est envoyée avec chaque action envoyée sur le canal d'interface. Le courtier renvoie donc la couleur associée à tous les clients connectés. Le composant graphique responsable de l'envoi de la commande est alors colorié avec celle de son activateur. Il est donc nécessaire de connaître les correspondance couleur/utilisateur, c'est pourquoi nous proposons un indicateur de télé-présence qui permet de nommer les personnes connectées et leurs couleurs associées.

Nous proposons également un système de messagerie instantanée pour que les utilisateurs puissent communiquer entre eux pendant la manipulation. Nous augmentons encore cette possibilité de communication à l'aide d'un outil de type talkie-walkie.C e t outil permet une communication audio sans faire l'acquisition continue de données au-xviii dio sur le terminal utilisateur, permettant ainsi de limiterl ac o n s o m m a t i o nC P Ue tl a bande passante.

Adaptation de l'interface homme machine

Nous choisissons d'exprimer les procédures d'utilisation àl'aideduResourceDescription Framework (RDF) et du Resource Description Framework Schema (RDFS). Ces formats sont à la base du Web sémantique et permettent de représenter des ordres dans des ressources, grâce aux conteneurs RDFS. Les séquences sont alors des éléments des conteneurs.

Livraison du contenu multimédia

Le format de livraison du contenu multimédia est MPEG-4 BIFS, suite à l'analyse précédente, qui permet de décrire le contenu sous la forme d'un arbre hiérarchique. Il permet d'inclure les différents composants de l'interface, tel que le flux vidéo, les éléments interactifs et le fond.

Nous choisissons de tronquer le flux vidéo pour limiter la bande passante de l'application. Cette troncature correspond à la zone exclusive de feedback d'un instrument, par exemple son écran dans le cas d'un instrument de mesure de type analyseur de spectre.

Pour représenter le fond, l'image de référence enregistrée dans la base de données du système est utilisée. Les trois régions de la scène (vidéo,f o n de tg r a p h i q u e s )s o n t alors multiplexées dans un fichier MP4. Ce format est une autre partie de la norme MPEG-4. Le contenu multimédia complet est ensuite délivré à l'utilisateur à l'aide de technique de streaming broadcast via le protocol Real-Time Streaming Protocol (RTSP).

Conclusion

Cette section présente l'architecture générale du cadriciel que nous proposons pour construire des solutions de pilotage d'instruments distants. Le cadriciel permet un contrôle collaboratif de l'instrument et autorise la communication entre les utilisateurs de façon audio ou via un service de messagerie instantanée. Unsystèmedecolorisation des widgets actifs permet d'améliorer la conscience de groupe.

La partie cliente, réalisée à l'aide de la norme MPEG-4 BIFS, permet de contrôler la scène distante à l'aide de technique de réalité augmentée. Lec o n t e n um u l t i m é d i ae s t délivré aux utilisateurs via le protocole RTSP.

Le contenu est adaptée en fonction de la procédure d'exécution, formalisée dans le cadriciel à l'aide de technologies issues du Web Sémantique.

Dans la section suivante, nous détaillerons les points essentiels de l'implémentation du cadriciel.

Réalisation et Implémentation

Nous allons détailler certains points d'implémentation du cadriciel. Nous commencerons par l'implémentation de l'intergiciel. Nous verronsdansunsecondtempsl'implémentation des pilotes logiciels de contrôle d'instruments. Dans un troisième temps, nous verrons l'implémentation de certains points de l'interface multimédia. Nous verrons enfin l'emploi du Web sémantique pour représenter les séquences de procédure.

Pilote d'instruments

Les pilotes de contrôle ou drivers permettent d'envoyer les commandes spécifiques àl ' i n s t r u m e n t .L e sdrivers suivant la spécification définie par la foundation IVI sont disponibles dans plusieurs langages, notamment C , C++, Labview ou Matlab. Nous avons choisi d'utiliser le langage C étant donnée sa facilité de prise en main. Un autre avantage d'utiliser les drivers IVI est la possibilité d'utiliser le driver dans un mode simulation si l'instrument n'est pas disponible.

Pour autoriser et favoriser la communication entre l'intergiciel et les drivers de l'instrument, une connexion TCP par socket a été mise en place. Ellep e r m e td ' e n v o y e rà l'instrument les messages de commandes, qui sont préalablement interprétées et de renvoyer ses réponses.

Les classes qui permettent de représenter le comportement del'instr umentontété établies de façon à reproduire les menus et sous menus de l'instrument. Le proof of concept de notre cadriciel a été réalisé sur un analyseur de spectre FSL 3 prêté par Rhode &Schwarz.

Implémentation Multimédia

L'application multimédia client est réalisée à l'aide de la norme MPEG-4 BIFS. Le contenu multimédia est interprété à l'aide du projet GPAC 3 qui propose un cadriciel complet pour la gestion de contenu multimédia, notamment de contenu MPEG. Il pro-pose en outre un lecteur multimédia capable d'afficher le rendudecontenuBIFSetun multiplexeur.

L'arbre de scène général du contenu multimédia est composé de4principauxmodules. Ces modules sont les suivants :

-la fenêtre d'authentification, commune à toutes les instruments pilotés.

-l es c r i p tg é n é r a l ,p e r m e t t a n tl ac o m m u n i c a t i o ne n t r el as c ène multimédia et l'intergiciel. Il permet également l'animation de certains composants de l'interface.

-l e sc o m p o s a n t sg r a p h i q u e sg é n é r a u x( w i d g e t s )c o m m u n sàt o us les instruments pilotés (l'outil de communication instantanée, le bouton defermeturedesession utilisateur, etc.). -l ' i n t e r f a c ep r o p r eàc h a q u ei n s t r u m e n t . L'identification sur la plateforme est réalisée au niveau de l'intergiciel. Le client envoie une requête HTTP comportant les champs nécessaires à l'identification. Nous sommes conscient du problème de sécurité que cela entraine. Une amélioration possible de ce problème serait de gérer l'identification en utilisant le protocole HTTPS par exemple.

Les différents widgets de l'interface (généraux ou spécifiques à l'instrument piloté) sont réalisés grâce aux différentes capacités de MPEG-4 BIFS.Encequiconcernel'interactivité, elle est réalisée grâce aux différents noeuds Sensor de la norme qui permettent de reproduire le comportant de boutons, de potentiomètres, etc. Certains sont mêmes capables de "sentir" les touches entrées sur le clavier du terminal utilisateur, permettant la saisie de texte (pour l'identification et pour la communication instantanée). BIFS est également capable d'afficher le résultat de courbes de Bézier,permettantainsidedessiner des courbes à la place de la zone vidéo lorsque l'instrument est en mode simulation.

Pour ajouter des capacités de communication, nous avons développé un module pour GPAC permettant de prendre en compte l'API WebSockets [W3C12] dans les scripts ECMAScript BIFS. Le protocole WebSocket [START_REF]The WebSocket Protocol[END_REF] este nc o u r sd es t a n d a rdisation par l'Internet Engineering Task Force (IETF) et permet de mettre en place une communication bi-directionnelle entre plusieurs clients et un serveur. Cela permet entre autre au server de "pousser" vers les clients les messages du canal d'interface. Ces messages sont ensuite interprétés par le script client qui met à jour l'interface en fonction.

Nous avons également ajouté une API au BIFS ECMAScript permettant de gérer l'outil de communication audio. Cette API permet le contrôledelaprisedesonaussi bien que le contrôle de l'envoi des données vers le serveur quie s tr e s p o n s a b l ed el e s renvoyer en retour aux clients connectés.

Pour faciliter la mise en place facilité de nouvelles interfaces de pilotage d'instruments distants, nous proposons un outil auteur qui facilite la création du contenu multimédia.

xxii 0.4.1.3 Module de chuchotement

Le "module de chuchotement" permet de raisonner sur les procédures d'action actuelles et qui permet d'afficher dans l'interface utilisateurlasuitepossibledesactions àréaliser .

La trace d'utilisation de l'interface est également sauvegardée dans le système en utilisant les conteneurs RDFS. Cela nous permet de facilement extraire la trace d'un ou plusieurs utilisateurs et ainsi d'analyser les résultats. L'analyse porte par exemple sur l'équivalence de séquence. Si deux procédures sont équivalentes que la cardinalité de la première est supérieure à celle de la seconde et que l'opérateur a utilisé la première, le système est capable, toujours grâce au moteur de règles, designalerquelaseconde procédure est moins longue à effectuer que la première.

Un outil auteur est disponible avec le cadriciel et permet d'exprimer les procédures d'exécution. L'interface de ce module est réalisé grâce à MPEG-4 BIFS.

Conclusion

Nous avons dans cette section présenté des points d'implémentation du cadriciel que nous proposons. Les drivers pour piloter l'instrument distant sont réalisés à l'aide d'implémentation C de la norme IVI et communiquent avec l'intergiciel grâce à une socket TCP. L'interface multimédia est réalisée à l'aide des caractéristiques de base de la norme MPEG-4 BIFS. Ils permettent de reproduire le comportement logique d'éléments interactifs de type boutons, potentiomètre, etc.

Nous avons également enrichi la norme MPEG-4 BIFS avec l'API WebSocket dans l'ECMAScript BIFS. Ceci met en place une communication bi-directionnelle entre les clients et le serveur et permet au serveur de notifier les clients de l'utilisation d'un widget par exemple. Nous ajoutons de même une API permettant de contrôler la prise et l'envoi au serveur de données sonore du terminal client pour pouvoir réaliser l'outil de communication talkie-walkie.

Nous avons également présenté comment les technologies du Web sémantique et de moteur de règles permettent de représenter des séquences d'utilisation d'interface et aider l'utilisateur à améliorer sa productivité.

Résultats

Dans cette section, nous présentons les résultats obtenus suite à nos travaux sur le cadriciel présenté pour construire des applications de contrôle d'instruments distants. Nous présentons dans un premier temps les tests utilisateursréalisésdanslecadredu projet MIRA.Nousintroduisonsensuiteleprojetlibrequenousavonsinitialisé.

Projet MIRA

Les projets MIRA 4 ont été crées pour renforcer la coopération dans l'enseignement superieur et la recherche internationale. Nous avons mis en place un laboratoire distant dans le cadre d'un projet MIRA avec l'université Libano-Française de Tripoli au Liban. Cette session a été effectuée en décembre 2011 dans le cadre d'un enseignement sur le traitement du signal.

Les objectifs pédagogiques de ce cours de traitement du signal sont de donner aux étudiants des compétences en acquisition, numérisation, etr e p r é s e n t a t i o ns p a t i a l ee t temporelle de signaux. Ce cours est divisé en cours magistraux et travaux pratiques. Ces travaux pratiques sont réalisés sur des simulations et sur des contrôles d'appareils réels. Les simulations sont réalisées sur Labview. Les étudiants affichent les représentations temporelles et fréquentielles de signaux en faisantvarierdif fér entsparamètr es tels que la bande passante, le span, etc. Les objectifs de la manipulation réelle étaient d'afficher ces mêmes courbes en faisant varier les mêmes paramètres mais avec un signal réel en entrée, ainsi que d'apprendre à manipuler un véritable appareil industriel. La manipulation a été mise en place sur un analyseur de spectreh é b e r g éàT é l é c o m Saint-Étienne (Saint-Étienne, France). Quinze étudiants ont suivi ce cours.

Les données récupérées des étudiants sont encourageantes. Les étudiants ont apprécié l'idée de manipuler un appareil réel de haute qualité qu'ils n'ont pas l'habitude de voir, et cela, en étant à 2000 km de distance. L'enseignant sur place a remarqué que l'IHM proposée était très proche de l'IHM réelle de l'instrument et avait "la sensation de manipuler l'instrument". Il a donné une note de 4 sur 5 sur laf a c i l i t éd ' u t i l i s a t i o n de l'interface. Il a également remarqué que la configuration pour les étudiants étaient très simple, considérant le fait qu'il faille seulement installer un lecteur multimédia et se connecter à une adresse RTSP. La note donnée est de 4 sur 5 sur la simplicité d'installation pour les utilisateurs. L'enseignant a également souligné que le temps de transmission des commandes entre les clients et l'instrument était bon, considérant la distance entre les utilisateurs et l'instrument. Le temps detransmissiondescommandes est évalué à 600ms. La bande passante a été évaluée à 500 kbps. Les utilisateurs et l'enseignant n'ont pas utilisé les outils de communication, étant donné qu'ils étaient tous dans la même pièce pendant la manipulation, mais ont apprécié la notification d'actions par les autres. L'enseignant a souligné l'intérêt d'untelcadricielpourlepartage d'instruments avec un coût d'achat élevé. Toutefois, l'enseignant et les étudiants ont soulevé un inconvénient. La qualité perçue du feedback vidéo délivré par la caméra n'était pas aussi satisfaisant que souhaité. La qualité de la vidéo a été évaluée avec une note de 2 sur 5 sur une échelle MOS. Cela peut venir de la qualité de la caméra employée pour filmer l'instrument ainsi que de la bande passante disponible qui était faible. Une plus profonde analyse MOS et de qualité d'expérience (QoE) peut être envisagée pour améliorer ces problèmes. Toutefois, 4. Mobilité Internationale Rhones-Aples xxiv étant donné que les étudiants n'ont jamais utilisé d'appareil de ce type au préalable, seule l'opinion sur la qualité visuelle des feedbacks peut être prise en compte. L'enseignant est un expert dans son domaine, il est le seul capable de comparer l'utilisation de tels cadriciels.

Un projet Open Source

La désignation logiciel libre, ou logiciel open source,s ' a p p l i q u eàd e sl o g i c i e l sq u i rencontrent les critères définies par l'Open Source Initiative,c ' e s tàd i r el ap o s s i b i l i t é de re-distribution gratuite des sources et des travaux dérivés.

Pour augmenter le développement de notre cadriciel et pour faciliter son évaluation par la communauté scientifique, nous avons décidé de publier notre cadriciel sous la licence Lesser GNU Public General (LGPL). Ce projet est intitulé Open Collaborative Environment for the Leverage of Online insTrumentation (OCELOT).

Conclusion et Perspectives

N OUS avons présenté nos travaux sur l'instrumentation en ligne etproposénosré- sultats sous la forme de tests utilisateurs et d'un projet libre. Nous présentons dans ce chapitre une synthèse des travaux, suivie d'une discussion sur nos résultats. Nous finissons enfin par proposer des pistes de recherche.

Synthèse

Dans ces travaux, nous nous sommes attaché à proposer un cadriciel pour la construction de solutions de pilotage d'instruments distants, utilisant les techniques de réalité mixte pour améliorer la compréhension de la scène distante manipulée, utilisant les standard multimédia de la norme MPEG pour faciliterl ' i n t e r o p é r a b i l i t ée t proposant un contrôle collaboratif de la ressource. La procédure d'utilisation est prise en compte en tant qu'élément de contexte et permet d'adapter le contenu multimédia en fonction des actions suivant la séquence actuelle.

Le cadriciel proposé est divisé en plusieurs composants qui répondent aux problématiques posées au chapitre 2, à savoir :

-l e sd r i v e r sI V I ,r e s p o n s a b l e sd ud i a l o g u ea v e cl ' i n s t r u m e nt distant, -l ' i n t e r g i c i e l ,r e s p o n s a b l ed ec o n v o y e rl e sm e s s a g e se n t r el'IHMetlesdriversIVI. Il est basé sur un MOM, -le centre de notification, responsable d'envoyer les messages d'interface aux clients de l'application. Il est basé sur un server WebSocket, -le multimédia interactif, responsable d'afficher l'IHM de l'instrument aux utilisateurs. Il est réalisé à l'aide de la norme MPEG-4 BIFS, enrichieparl'utilisationdes WebSockets, -le module de chuchotement, capable de définir les actions suivant l'action courante, -u no u t i la u t e u r ,f a c i l i t a n tl ac r é a t i o nd ' I H Mp o u rl ep i l o t age d'instruments.

Discussion

Nous allons maintenant discuter les avantages et limites de notre approche suivant différents critères.

Cadriciels

Comme nous l'avons présenté dans l'état de l'art, la plupart des solutions de laboratoires distants utilisent des technologies de type Flash, Java, Labview, etc. Nous proposons d'utiliser MPEG-4 BIFS (avec des enrichissements)p o u ra m é l i o r e rl ' i n t e r -xxvi opérabilité entre les terminaux utilisateurs tout en mettant en place une solution de réalité mixte.

Ce cadriciel peut être également utilisé pour d'autres typesd'applicationstellesque des services d'IPTV interactifs, vidéo conférence, etc.

Multimodalité

Les travaux sur la multimodalité des IHM souligne le fait qu'elle peut améliorer les processus cognitifs et d'apprentissage et améliorer l'immersion de l'utilisateur. Notre approche se base uniquement sur le terminal utilisateur pouri n t e r a g i ra v e cl ' i n s t r ument distant, i.e. clavier/souris pour un ordinateur classique et interface tactile pour un téléphone ou une tablette et pourrait en effet bénéficier d'interactions multimodales (commande vocale par exemple).

Mais cela pourrait aussi diminuer la compréhension de l'instrument manipulé en ajoutant des modalités qu'il ne supporte pas de base. Nous pensons fortement que pour apprendre à manipuler un appareil réel, les interactions naturelles avec l'instrument doivent être reproduites.

Collaboration

Tandis qu'iLab ne propose pas de contrôle collaboratif dans son cadriciel, la collaboration et la conscience de groupe sont présentes dans notrea p p r o c h e ,m ê m es il e s outils mis à disposition ne sont pas toujours exploités.

Les réseaux sociaux pourraient renforcer également les processus de collaboration. Notre système fournit des outils de communication mais ne fournit pas d'informations sur les utilisateurs à proprement parler (intérêts, localisation, etc.). Un scénario possible serait d'envisager l'intégration d'OCELOT avec un système de type LMS tel que Moodle par exemple. L'intégration des technologies du Web sémantique, notamment le projet FOAF (Friend Of A Friend) peut être une piste de recherche, puisqu'il décrit les utilisateurs et leurs relations respectives.

Limitations

Toutefois, notre système souffre de quelques limitations. Les drivers actuels ne prennent en compte que la norme IVI, ce qui implique un développement non négligeable à considérer lors du déploiement de nouveaux instruments. En considérant les retours des étudiants, la qualité perçue de la vidéo n'est pas suffisante. Cela pourrait être amélioré en employant une caméra de meilleure qualité ouenadaptantlecontenu vidéo en fonction de la bande passante par exemple. L'actuel développement du cadriciel ne prend en compte que des instruments à façade (à deux dimensions). Étant donné que BIFS gère également les contenus à trois dimensions, on peut également considérer un portage des applications vers des instruments à trois dimensions dans une futur version.

Perspectives

Le cadriciel que nous avons présenté pourrait être étendu à d'autres types de cas d'usage, par exemple la chirurgie collaborative à distance,e t c . ,c eq u id o i ti m p l i q u e r des travaux sur l'intégration d'instruments à trois dimensions.

Étant donné que le cadriciel supporte la collaboration, des recherches sur les interactions sociales peuvent être menées.

Des recherches sur l'adaptation de l'application au contexte utilisateur peuvent également être mené pour améliorer la qualité d'expérience du service (bande passante, terminal disponible, taille de l'écran, localisation, etc.).

Des recherches actuelles dans la livraison de contenu multimédia tendent vers le streaming HTTP plutôt que le streaming plus classique RTSP. Le but du streaming HTTP est de faciliter la livraison du contenu en passant par les ports Internet ouverts par défaut, contrairement aux autres solutions de streaming. 8B F a n d F L A N N m a t c h i n g g r a p h s f o r S U R F f e a t u r e s d e s c r i p t o

Introduction

This chapter is the introduction of this thesis. We discover issues of online engineering and remote laboratories. We also present the objectives, our motivations, the scientific perimeter of this study as well as the structure of this document. W E consider modern remote computing as the period after the expansion of the Internet. Over the last five decades, application architectures have evolved significantly. Applications used to provide limited services, such as data access. With the rapid growth of the Internet, systems have become more and more complex and provided more advanced features. Of course, the development of terminal capacities have also improved the application architectures, since terminals, even mobile terminals, are nowadays able to process complex data.

Chapter Plan

Once upon a time. . .

Between the 60s and the 80s, the mainframe architecture was the dominant. In this architecture, all the components are stored on one central computer. Users are connected to the same central unit and share the same data processing. Terminals are passive. Performances of mainframe architectures rely on the centralu n i tc a p a c i t i e s ,a n db ecause of their importances, the teams in charge must be highly qualified. The proprietary evolution of the system implies that changes are complexandcostly .

The evolution of application architecture between the 80s and the 90s brought the "client-server" architecture. The system is distributed between two parts : the server is providing services and the clients are consuming them. Partso ft h eo v e r a l ld a t ap r ocessing can be achieved on the client side. Communication between parts are reached through local network or Internet. In this architecture, client applications are said to be "heavy". This architecture is also known as two-tiers architecture.Themajorproblemarising with this architecture is the evolution of the system. Its modification entails a very large amount of time for configuring all client applications. This leads to the appearance of Web architecture in the middle of the 90s.

Because of maintenance issues with the need to process more and more complex data and the need for better security, applications evolved to three-tiers architectures. The application server is separated into two parts : application and data. The client application requests the application server. The applications e r v e rr e q u e s t sa n do b t a i n s data from the data storage server, processes it and returns results to the client. Web applications are three-tiers application where the client applications are executed in a Web browser. Since all calculations are executed on the server-side, Web applications are referred to as "thin client". Deployment and evolution do not require the configuration time of two-tiers applications.

During the last decade, Web applications have emerged with the same look and behavior as desktop applications. They are usually referredtoasRich Internet Applications (RIA). They are usually delivered via specific browsers and plugins. Often, a framework for executing RIAs is installed in the client system in order to execute it. Some of the most popular framework are Asynchronous Javascript and XML (AJAX), Adobe Flash, Microsoft Silverlight.ThemajoradvantagesofRIAare: Relationships between humans and machines evolved, whetherinr ealcontr olsof devices or simulations. This contact is not physical but happens through computer systems. We talk about computer-assisted systems, since computers are not only responsible for displaying the Human Computer Interface (HCI) that users interact with, but also can help users in their tasks in various ways.

Leaving aside the potentially intrusive aspects of computers, it is important to notice that innovation is not only technical. Information Technology, (IT), can effectively combine multiple people to multiple machines, as opposed to bijective previous relation-ship. This is done to connect mixed processes but also collaborations between people, in order to achieve a shared objective. This scientific area is referred to as Computer Supported Collaborative Work (CSCW), that "focuses on the study of tools and techniques of groupware as well as their psychological, social, and organizational effects"1 .

Tele-operations allow to conduct manipulations that could not occur otherwise. Actors of such systems come from different areas and have together different relationships, as illustrated in Figure1.1. The remote control of devices is nowadays employed in different fields, but especially in :

-L e a r n i n gi n s t i t u t e sm a i n l yf o rd i s t a n c el e a r n i n gc u r r i c u lum. In the field of elearning (meaning online curriculum), online engineering is most of the time referred to as remote laboratories. -I n d u s t r i e s f o r m a i n t e n a n c e o p e r a t i o n s , p r o d u c t i o n c o n t r ols, etc. In the emaintenace area, such systems are usually referred to as Computer-Assisted Maintenance (CAM). -R e s e a r c hi n s t i t u t e st om a n a g ee x p e r i m e n t a t i o n sw i t hr e s e archers from various geographic locations (collaboratories). The evolution of the Internet and the applications architecture allow new paradigms for online engineering. The next section presents the subject of this study.

Online Engineering and Remote Laboratories

The study of this thesis is focused on online engineering and remote laboratories. This section presents the motivations of using remote laboratories and illustrates these motivations through two use case scenario. In a second step, we present the problem arising with online instrumentation. Finally, the organization of this document is presented.

Motivations

Financial factor

The cost of state of the art devices is often very high considering their usage frequency. It also implies the presence of highly qualified operators in-situ in order to manage the machine park. The alternative, consisting in movingdevices,entailstechnical as well as responsibility difficulties (insurance for instance). The usual solution consists in moving the operators to the site of the devices. This solution entails other arising issues, such as administrative tasks, access demands for the site, accommodation, etc. Thus, remote controlo fd e v i c e sc a nl e a dt o saving direct as well as indirect money (time2 for instance). Despite the possible outcomes, we must however consider the development time and costforonlineengineering solutions.

In our work, concerned devices come mainly from research laboratories. Such devices have a high cost of purchase as well as of maintenance. They are bulky and present fragility of transport.

Overcome constraints

The financial cost is not the only constraints that could lead to an online engineering solution. Other factors, such as the moving of people or devices are fundamental. Laboratory sites are not always suitable for the hosting of such groups. Problems arising come from different criteria :

-

Software Chain

In this subsection, we review the main software architectureofr em oteex per i m entations. Figure 1.2 shows the end to end software chain used nowadays. This chain, derived from n-tiers software architectures, is composed ofthreemainlinks:

-T h el i n kb e t w e e nt h el o c a lc o m p u t e ra n dt h ed e v i c et oc o n t r o l.

-T h el i n kb e t w e e nt h ed i s t a n tc o m p u t e ra n dt h el o c a lc o m p u t e r. It is the middleware responsible for the application discussion between the remote computer and the local computer. -T h el i n kb e t w e e nt h ed i s t a n tc o m p u t e ra n dt h ed i s t a n tu s e r:the Human Computer Interface.

Local Computer to Device Link

The link between the local computer and the device entails mastery of standards and of access protocols. This is complex due to a) the heterogeneity and the number of technologies and b) the number of material interfaces allowing interfaces with devices : PXI, USB, GPIB, etc.

But since some devices from different constructors or from different series allow the same kind of functionality, some efforts were made in order toproposecommondrivers through standards. Those drivers define classes of devices, in order to ease interoperability between devices and to decrease the development time and cost.

Unfortunately, those standards suffer from limitations. [START_REF] Landragin | [END_REF] exposes some limitations (translated from French source) :

-" I ti sc l e a rt h a tr e s u l t sw i l ln o tb ei d e n t i c a lb e t w e e nd e v i ces as measurement performances are very different, even if each seems to work perfectly in the environment". -"It is obvious that devices that combine specific functionalities, unique in their category [...] do not ensure interchangeability".

Remote Computer to Local Computer Link

This link is referred to as middleware.Thegoalofthemiddlewareistoconvey ,using the Internet, commands or requests from remote users to devices. It is also responsible for transmitting data from the device to users.

[Bernstein96] defines a middleware as a link between the application layer and its operating system. Its main goal is to convey messages betweenapplications,beingthe most transparent both for the user and the developer. The middleware is then in charge of conveying the commands from the remote user to the device, and also of giving a behavior of the collaborative platform, i.e. conveying the messages from the devices to all the users.

Human Computer Interface

The Human Computer Interface is the software component that enables users to interact and communicate with the system. HCIs must present items to send commands to the remote device as well as items to receive feedbacks fromtheremotedevice.

Online Engineering and its usages 1.2.3.1 Collaboratories

Collaboratories can be defined as follows : "[a] center without walls, in which the nation's researchersc a np e r f o r mt h e i rr esearch without regard to geographical location, interacting with colleagues, accessing instrumentation, sharing data and computational resources, and accessing information in digital libraries" [START_REF] Vinton | National collaboratories : Applying information technology for scientific research[END_REF].

In order to spread results and experimentations, to exchangeideasortoeasecommunication between researchers, tools are needed. We can also notice that issues in the scientific world involve more and more interdisciplinary skills. A network of researchers can then occur in collaboratories. Online engineeringa n dth er em otec on tr olof devices can help and improve exchanges in the scientific world.

Remote Maintenance

Industrial processes, such as maintenance can also benefit from online engineering. Online engineering help industries to execute maintenance procedures on remote devices. Production and industrial maintenance sites rely on highly qualified technicians and staff. The latter are responsible for many assembly/disassembly operations, calibration, etc. These human operators are expected to understand and memorize a large panel of procedure steps, in a specific order. Moreover, moving to the production site presents several issues (cost, risk, etc.). As a consequence, a first maintenance operation is usually performed remotely.

Learning and being able to reproduce the full catalog of maintenance operations can be a difficult task. The complexity of this task is also rapidly exploding since the number of devices operated by a single human operator increases when these devices can be operated remotely.

e-learning

Alotofabroadexchangeshappenduringcollegestudies,especially in the engineering field. In Europe, several exchange programs exist betweenu n i v e r s i t i e s .A m o n g them, the Erasmus program3 is probably the most famous. This program was launched in 1987 and enables students to spend part of their studies in a partner university in Europe. More than 1.5 million students took benefit from this program. Students are asked to follow courses and pass their exams in the exchange university. Foreign courses must fit their home studies. But sometimes, it is necessary to validate home courses. This could happen if the host institution does not provide a particular lecture. Then students need to validate their home courses online.

In distance learning, lectures, exercises or projects are easily brought online. It is more difficult with practical laboratories. Meanwhile, they are a compulsory learning material in engineering studies, since they bring students practical skills and knowhow. Bringing laboratories online aims at offering learnerst h ep o s s i b i l i t yt om a n ipulate devices from different locations. It allows reducingg e o g r a p h i c a ll a b o r a t o r i e s constraints : "If you cannot come to the lab, the lab will come to you" [START_REF] Vj Harward | iLab : A scalable architecture for sharing online experiments[END_REF].

Industries and research laboratories can benefit from the possibility to manipulate online devices. They both could use research and training service delivery. Research as well as training service deliveries are punctual, and the financial cost of devices could be reduced to a minimum when manipulating devices online.

Use Case scenario

The previous subsection presented the ins and outs of tele-operations. Indeed, we saw that the remote control of devices is not only a financial issue, but brings a technological answer to manage otherwise intractable situations. Stakeholders make up a trio composed of laboratories, learning institutes and industries. In order to illustrate our works and proposals, we put forward two use case scenario in this subsection. They help illustrating needs and issues of the remote control of devices. The first use case scenario focuses on online engineering for e-learning while thesecondfocusesonremote maintenance.

e-Learning

Steve is a teacher in Norway. Heather and Jimmy are both Steve's students. As part of their engineering curriculum, Heather and Jimmy are studying abroad for the current year. Heather is in Tokyo and Jimmy is in Paris. But in order to graduate, they still need to validate Steve's lecture, which includes practical sessions on a spectrum analyzer and on a vector network analyzer.

Industrial Maintenance

Alice is a technician working in a manufacture. Her device presents several malfunctions. She calls Bob who is an operator in a company performing maintenance services. In order to decrease the maintenance service cost, Bob first runs remote operations on the device to avoid in situ maintenance, if possible. But since Bob conducts maintenance on several devices for different companies, he does not know by heart all the procedures for all the machines. He uses the CAM system provided by his company to run the supervision that includes all the procedures for the many devices he is in charge of.

Arising Problems

We saw that online engineering and remote laboratories try toanswerissuesthatare not easily solvable when controlling device locally. They also raise issues that need to be taken into account. Previous use cases raised some of these issues, which are presented in this subsection.

e-learning

The e-learning use case illustrates the need of online engineering. But it implies several issues as follows :

-Human Computer Interface.D u r i n gt h ep r a c t i c a ls e s s i o n s ,H e a t h e ra n dJ i m m ya r e meant to learn how to use both analyzers in order to use their skills after graduation. Giving a high fidelity representation of the HCI of the analyzers improves Heather and Jimmy's understanding of the device. -Interoperability and Pervasive Computing.W i t ht h ee x p a n s i o no fd i f f e r e n tk i n d so f terminals and platforms (smartphones, tablets, computers,e t c . ) ,S t e v ew a n t st o provide a system with the highest interoperability, so that he does not have to worry about Heather and Jimmy's terminal. Since distance entails latency (Heather, Steve and Jimmy are in different time zones), Steve alsoneedstopr ovidea solution that limits the bandwidth and that is accessible anytime and anywhere. -Collaboration and group awareness. In-situ practical sessions are often done by groups in order to achieve more complex tasks. Bob wants online practical sessions to work in a collaborative manner, so that Heather and Jimmy can achieve together the same goals as their in-situ comrades. Therefore, in order to reproduce the in situ collaboration, Steve's system needs to provide notifications of actions done by each users to give the group sensation. Notifications alone are not sufficient to reproduce a real collaboration, the system also needs to provide communication tools. -Reusability and simplicity.F o rh i sp r a c t i c a ls e s s i o n s ,S t e v ei su s i n gt w od i f f e r e n t devices. Therefore, the system must be as much reusable as possible with the least efforts to build new online practical sessions. He alsow a n t st h es h o r t e s t initialization.

Industrial Maintenance

In addition to the above-mentioned issues, the industrial maintenance use case raises others :

-Procedure formalism.TheCAMsystemmustprovideaformalismtodescribeprocedures.

-Business Logic.Thebusinesslogicdetermineswhichprocedureisbeingperformed

and what the next possible actions are. This logic must be loosely-coupled with any programming language in order to be fully reusable. -Analysis.TheCAMsystemshouldprovidetoolstoanalyzeactionsthathave been made by the operators, in order to propose improvements.

Conclusion

Online engineering is motivated by the need to reduce financialc o s t sa sw e l la s the need to answer technical issues such as visibility, dangerousness, availability, etc. It is mostly used by three kind of organizations : industries,l e a r n i n gi n s t i t u t e sa n d laboratories, with their own needs and usages.

We presented two use case scenarios that illustrates online engineering situations. The first one is centered on e-learning. The second one focuses on industrial maintenance. Both of them raises issues.

From our point of view, these issues are of the utmost importance when designing online engineering. The work we conducted tries to answer them and both scenarios will be use in this document in order to illustrate our proposition. We can notice that these issues strongly impact the Human Computer Interface presented to the end user and are compiled as follows :

-ar e m o t ec o n t r o lo ft h ed e v i c et h a tp r e s e n t s ah i g hl e v e lo fi mmersion and understanding of the remote scene. -ac o l l a b o r a t i v ec o n t r o lo ft h er e m o t ed e v i c e .

-a na d a p t a t i o no ft h eH C Ii no r d e rt oh i n tt h eo p e r a t o r .

-as c a l a b l es y s t e ma b l et oc o n t r o law i d er a n g eo fd i f f e r e n td evices.

Organization of the manuscript

This chapter introduced the issues of online engineering. Weh a v ep r e s e n t e dh o w the Internet and its mutations made it possible to bring devices online. We have presented the needs of bringing devices online and the actors of online engineering. In the second part of this chapter we have presented two use case scenarios that helped illustrating issues of remote control of devices, especially in the e-learning and the industrial maintenance area.

The need of high fidelity of the representation of the device hasl e a du st oc o n s ider mixed reality techniques. Interoperability, meaning the possibility to manipulate the device from any kind of terminals has lead us to the usage ofm u l t i m e d i as c e n e descriptions.

This document is organized as follows. Chapter 2 presents thes t a t eo ft h ea r to f technologies and projects in the online engineering field. Chapter 3 presents the overall architecture of the system we propose in this work. Chapter 4 details implementation points of our solution and presents the results we obtained. Chapter 5 concludes the work and provides prospects.

It is easier to move a problem around (for example, by moving the problem to adifferentpartoftheoverallnetworkarchitecture)thanitistosolveit. It is always possible to add another level of indirection.

Ross Callon -RFC1925, "The Twelve Networking Truths"

2

State of the Art

While the previous chapter has raised the need for remote laboratories, we make in this section a review of online engineering and industrial maintenance systems. R EMOTE Laboratories and Online Engineering bring the possibility to perform mani- pulations on real devices over the Internet. With the rapid growth of the Web and its associated technologies, controlling remote devices has become to reality.

Instruments' feedbacks need to be represented to operators in order have an understanding of the operation state. This can be achieved on the client-side by representing feedbacks with graphical components in the Human Computer Interface or by displaying a video capture of the remote device. Anyhow, HCIspresentedtooperators require a high fidelity. Mixed reality could be a way to improve the user immersion and understanding of the apprehended remote devices.

In the pervasive computing area, remote laboratories are expected to be accessible anytime and anywhere. This implies a high level of interoperability in order to control devices from any kind of terminal, from personal computers tom o b i l ed e v i c e sa b l e to display multimedia content. Reusability, i.e. the ease ofdeploymentofnewdevices online, should also be considered when building Remote Laboratories frameworks. The difficulty of HCI construction is a story deadlock for remote laboratories development due to resources costs. Loose coupling between the instruments hardware commands and the HCI presented to operators is then necessary and client-side applications need to rely on standards to ensure interoperability, for both graphical and communication aspects.

Collaboration, i.e. the possibility to perform remote operations within a group, is a key component of online engineering. Sharing resources and interacting with them at in the same time is not enough, the "group awareness" needs to be taken into account.

In the previous chapter, we have raised four main issues of remote experimentations. The first part of this chapter presents notions and technologies in order to overcome these issues.

The second part of this chapter presents the state of the art research activities of online engineering and remote laboratories usage in industry and learning areas. We focus our analysis on the four criteria defined previously.

The third part of this chapter summarizes the analysis.

Technologies

In chapter 1, we have presented the four main issues we try to answer in this thesis :

1. remote control of the device that presents a high level of immersion and understanding of the remote scene.

2. a collaborative control of the remote device.

3. an adaptation of the HCI in order to hint the operator.

4. a scalable system able to control a wide range of different devices.

In this section, we review the technologies and notions that lead to in the resolution of such issues.

The first issue can be answered using augmented reality technologies. Therefore, section 3.2 presents the augmented reality paradigm and its associated technologies. Section 2.1.2 presents the issues that need to be taken into account when designing collaborative systems. Context-aware systems are systems ablet or e a c ta n da d a p tt h e i r behaviors according to context items. In section 2.1.3, we present such systems. Finally, in order to address the wider range of different devices, a description of the device environment has to be chosen. Therefore, a formalism to describe the device environment, such as multimedia scene description is needed. Section 2.1.4 presents multimedia scene descriptions.

Augmented Reality

Since the advent of Personal Computers, WIMP graphical interfaces (Windows, Icons, Menus and Pointing devices) have established themselves as the dominant interaction style. But WIMP interfaces are nowadays proving their limits in numerous situation (critical work, simulation, disability, etc.). Furthermore a virtual world is created, in complete opposition to the real world.

The real world is the world in which users evolve whereas the virtual world encompasses storage, communication and data processing capabilities. Since about twenty years ago, computer scientists have tried to bring together the real and the virtual worlds, leading us today to the mixed reality paradigm.

Mixed Reality Continuum

Av i rtua lr ea l i tysystemi sac om putedm odeli nwhi c hgra phi c sa r eu sedtoc r ea te aw o r l dt h a ts e e m sr e a l i s t i c .T h i sm o d e l i n gi sn o ts t a t i cs i n ce it answers users' orders (gesture, speech, external commands, etc.). The virtual reality notion exists since 1957 with the invention of the Sensorama Simulator by Morton Heilig. It was composed of a3 Dv i deosystem ,f a n sa n dav i b r a ti n gc h a i r .Fa n sw er el oc a ted near the user's head Virtual reality is a simple way for humans to visualize, manipulate and interact with complex data with the help of a computer. Augmented reality regroups all the techniques allowing to associate a real world to a virtual world. It is a view of a real scene where computed information is added. Added Information can consist of texts, sounds, graphics, etc.

Augmented reality was born with Sutherland's works on virtual reality head mounted device in the middle 70's [START_REF] Sutherland | Ah e a d -m o u n t e dt h r e ed i m e n s i o n a ld i s p l a[END_REF]. The "See through System" displayed simple superimposed polygons on the view of the real world. The use of such a head mounted device displaying superimposed virtual objects on ar ea lw or l dv i ewa l l ow s users to keep the contact with the real world while being in a virtual world.

Three fundamental rules must be respected in order to build aneffectiveaugmented reality application [START_REF] Swan | Survey of User-Based Experimentation in Augmented Reality[END_REF] :

-

C o m b i n et h er e a lw o r l da n dt h ev i r t u a lw o r l d . -T a k ei n t oa c c o u n tt h ec o n s i s t e n c ya n dt h ec o h e r e n c eb e t w e e nt h er e a la n dt h e virtual worlds. -H a n d l et h ec o n s t r a i n t so fr e a l -t i m ea n di n t e r a c t i v i t y .

This definition excludes simple 2D superpositions that do not respect the 3D coherence as well as post-production composition, which is not real-time. On the other hand, no constraint is set the photometric realism of compositions. For instance, virtual objects could be overlaid. Augmented reality is a view of a real scene where computed information is added. By extension, augmented virtuality refers to the merging of real objects to a virtual scene. Augmented reality and virtuality can be expressed in amoreglobal:termmixed reality. Mixed reality encompasses both augmented reality and virtuality [START_REF] Milgram | AT a x o n o m yo fM i x e dR e a l i t yV i s u a l Displays.I E I C ET r a n s .I n f o r m a t i o nS y s t e m s[END_REF], as shown in Figure 2.2.

Different kinds of devices and sensors are employed to realize augmented reality systems, and by extension, mixed reality systems. Of course,t h ed i s p l a ys y s t e m sa r e mandatory in augmented reality applications. They enable visual rendering and the generation of virtual scenes. We can distinguish between different classes of such display systems :

-Head Mounted Devices : users hold a system on the head, such as helmets. They are divided in two sub-classes. The first ones are composed of anL C Ds c r e e n coupled with a semi-transparent mirror and the mix between real and virtual objects is done by the eyes. The second sub-class are the videosee-throughhead mounted devices. The mix is done with a combination of computed graphics and images produced by cameras. -C l a s s i c a ls c r e e n s:ac a m e r ad e l i v e r st h er e a lo b j e c t sa n dacomputer mixes real and virtual items. -P r o j e c t o r s:t h ev i s u a l i z a t i o ni sd o n eo nas u r f a c ei nt h er e al world. This surface can be a table, a wall, etc.

Before displaying the mix between virtual and real objects, ap r o p e re n v i r o n m e n t model has to be created. This model helps placing real and virtual objects in the scene. Most of the time, it is a 3D geometrical model. As well as proposing a proper system for visualizing data and a proper environment model, augmented reality systems must take into account the reality-virtuality alignment. Both real and virtual items must be correctly superimposed so that users feel that virtual and real worlds coexist.

Tracking Sensors

In order to realize such alignment, spatial references objects are needed in the scene. These objects are very important because they enable the computation of the spatial representation of the scene, needed at anytime. In the augmented reality fields, the tracking of objects consists in following their orientationandposition.T rackedobjects can be the user himself or particular objects in the scene. Mosttechniquesarederived from either marker-based tracking or marker-free tracking.

2.1.1.2.a Marker-based Tracking

Visual Makers

The tracking with markers consists in placing markers, i.e. atargetobjects,inascene and detect their positions and orientations in every image. Markers must be easily identifiable, i.e. with minimum calculations, in order to increase the system efficiency. If the relative position between an object (real or virtual) and them a r k e ri se s t i m a t e d ,o n e can determine the absolute position of this object in the scene. Zhang proposes in [Zha02] a review of marker-based tracking.T h eH O Mm a r k e r system is developed at Siemens AG for photogrametry purposes. A 6 bits encoding side bar is added to the main square in order to improve the detection robustness. These HOM markers are used in "a Windows-based application for creating, updating and managing 3D computer models of factories from input digital images" [START_REF] Navab | Cylicon : A Software Platform for the Creation and Update of Virtual Factories.InPr oc.7thIEEEInt'lConf[END_REF]. He also presented two other kind of markers developed with the ARVIKA 2 project, which aims to "research into and to realize Augmented-Reality-Technologies (AR) which will support development, production and servicing with relation to complex technical products in a user-oriented and application driven manner". The first marker system is the IGD marker. An IGD marker "is a square divided into 6×6squaretilesof equal size. The inner 4×4tilesareusedtodeterminetheorientationandthecodeofthe marker". The second marker system is the SCR marker, also coded with inner matrix. They are designed as 4×4gridswithfeaturespointsinsidetheboxes.

Markers may be in other forms than squares. Cho et al. presentedin [START_REF] Cho | i n t e r n a t i o n a l workshop on Augmented reality : placing artificial objects in real scenes : placing artificial objects in real scenes[END_REF]ring shaped fiducial markers. Naimark et al.

Invisible Makers

Another approach is to place invisible markers in the scene toanalyze.In [START_REF] Burnett | Using Infra-Red Beacons as Unobtrusive Markers for Mobile Augmented Reality[END_REF], Burnett et al. propose to using infra-red markers. Such markers are interesting because they are invisible for users but can be tracked easily with infra-red cameras and can be used in varied light conditions.

Active radio frequency identification (RFID) chips can be usedforaugmentedreality systems [START_REF] Zhang | RFID-assisted assembly guidance system in an augmented reality environment.I n t e r n a t i o n a lJ[END_REF]. They can be positioned inside structures to allow in situ positioning.

The arising technology Near Field Communication (NFC) is even used in order to create indoor navigation. NFC "is a bidirectional short range, wireless communication technology" between two devices, based on RFID [START_REF] Ozdenizci | 2 0 1 1F o u r t hI n t e r n a t i o n a lC o n f erence on Information and Computing[END_REF]. Nowadays, more and more mobile devices allow the NFC technology.

2.1.1.2.b Marker-free Tracking

Mechanical tracking

The first head mounted device designed by Sutherland [START_REF] Sutherland | Ah e a d -m o u n t e dt h r e ed i m e n s i o n a ld i s p l a[END_REF] used a mechanical tracking method. "Either of two head position sensors, one mechanical and the other ultrasonic, is used to measure the position of the user's head". The users movements were limited to a "working volume of head motion about six feet in diameter and three feet high", and "beyond these limits, head positionc a n n o tb em e a s u r e db y the sensor". Such method, with improvements, are still in usenowadays.

Positioning Systems

Some tracking methods use the Global Positioning Systems (GPS), gyroscopes and accelerometers. They have been widely used in mobile augmented reality applications since the appearance of nowadays' smart phones and their capabilities. This method uses satellites signal in order to compute the geographical location of the user. Gyroscopes and accelerometers determine the vision line of the user [START_REF] Erifiu | Hardware Sensor Aspects in Mobile Augmented Reality[END_REF].

Interest Points Detection

The visual marker-free tracking methods are more contemporary than the markerbased ones. They are mostly based on image analysis and understanding and can be classified into two different classes. [START_REF] Rosten | Fusing points and lines for high performance tracking[END_REF] is an other famous corner detector. A point p is considered as a corner if there exists an arc of at least nine pixels that are all brighter or darker than p in a circle around p.

The second class of detectors uncludes the Blob detectors. They use local extrema of responses of certain filters as interest points. For instance, the Difference of Gaussian (DoG) detector uses differences of Gaussians as filter [START_REF] David | Distinctive Image Features from Scale-Invariant Keypoints[END_REF].

Feature Descriptors

In addition to interest points detection, some methods add descriptors. Such descriptors are descriptions of visual features of images such as color, shape, texture, motion, etc. Tuytelaars et al. [START_REF] Tuytelaars | Local invariant feature detectors : A survey[END_REF] describes the ideal local feature with the following properties :

-R e p e a t a b i l i t y:ah i g hp e r c e n t a g eo ff e a t u r e sm u s tb em a t c h ed between two images of the same scene taken from different viewing conditions (point of view, light, etc.). The features must be invariant to image deformations, noise, compression artefacts, etc. -D i s t i n c t i v e n e s s:d e s c r i p t o r ss h o u l dd i s t i n g u i s hf e a t u r es efficiently.

-L o c a l i t y:t h el o c a lp r o p e r t y" r e d u c et h ep r o b a b i l i t yo fo c clusion" and "allow simple model approximations of the geometric and photometric deformations between two images taken under different viewing conditions( e . g . ,b a s e do na local planarity assumption)". -Q u a n t i t y:T h en u m b e ro fd e t e c t e df e a t u r e sm u s tb eh i g he n o u gh to detect small objects and match them between images of different viewing conditions. -A c c u r a c y:" T h ed e t e c t e df e a t u r e ss h o u l db ea c c u r a t e l yl o c alized, both in image location, as with respect to scale and possibly shape". -Efficiency : The detection of features should take into account "time-critical applications". Fayolle et al. have presented in [START_REF] Fayolle | Robustness of a multiscale scheme of feature points detection[END_REF] a multi scale detection of feature points using wavelets. "Multiscale edges are defined using smoothed versions of the signal and a first-or a second-order derivative". Feature points are detected studying "the behavior across scales of wavelet transform coefficients of the gradient phase signal". When detected, Fayolle et al. characterize feature points byth el en gtha n dth ea m pl itude of the phase transition of gradient phase signal. Edges routes are calculated using the Freeman coding [START_REF] Fayolle | Application of multiscale characterization of edges to motion determination[END_REF]. In order to determine the correspondence between two images, a similarity measure is computed. It is based on computing the distance between the gradient vectors of the feature points. "The distance used is a combination of of the phase difference and the modulus difference of the vectors".

Among others, the Scale-Invariant Feature Transform (SIFT)b yL o w e[ L o w e 0 4 ]i s probably the most famous one. The first step of the algorithm is to detect circular areas around key points. In order to determine key points, the image is filtered at different scales with Gaussian filters. Key points are then local extrema of the DoG-blurred images. Descriptors are then computed by attributing key points local orientations determined from gradient orientation in the key point neighborhood. The SIFT method is used in a large variety of applications. In Figure 2.4, a box isrepresentedaloneandina scene. The SIFT key points are extracted and displayed.

In 2008, Bay et al. [START_REF] Bay | Speeded-Up Robust Features (SURF)[END_REF] presented the Speeded-Up Robust Features (SURF) descriptor. Tuytelaars et al. categorize this descriptor as efficient, because its implementation privileges the execution speed, SURF being 5 times faster than DoG [START_REF] Tuytelaars | Local invariant feature detectors : A survey[END_REF]. SURF is partially inspired by SIFT. In order to determine the interest points of an image, its integral is calculated. The interest points are determinants of the Hessian matrix applied to 2D Haar wavelets responses in scale space. The descriptions of each interest point is computed from the local distribution of the gradiento r i e n t a t i o n .T h eS U R F descriptor is presented in the form of a 128-dimensional vector. Similar to figure 2.4, Figure 2.5 shows the extracted SURF key points.

Calonder et al. [START_REF] Calonder | BRIEF : Binary Robust Independent Elementary Features[END_REF] proposed in 2010 a binary representation of descriptors. The goal is to decrease the size of descriptors. The original is firstly smoothed in where I(P, x) is the pixel intensity of the filtered patch P of size S×Sa tx =( u, v).

Then "choosing a set of n d (x, y)-location pairs uniquely defines a set of binary tests". The BRIEF descriptor is the n d -dimensional bit string that corresponds to the decimal counterpart of :

∑ n d i=1 2 i-1 • τ(P; x i , y 1 )
BRIEF has limited invariance to in-plane rotation. In order to overcome this limit, some derivation of BRIEF were achieved [START_REF] Calonder | BRIEF : Computing a Local Binary Descriptor very Fast.I E E ET r a n s a c t i o n so nP a t t e r n Analysis and Machine Intelligence[END_REF]. Figure 2.6 shows the extracted BRIEF key points.

In 2011, Rublee et al. presented an alternative to SIFT and SURF [START_REF] Rublee | ORB : An efficient alternative to SIFT or SURF.C o m p u t e rV i s i o n[END_REF]. The key point detector is based on the FAST detector while the descriptor is based on BRIEF. Its name is ORB for Oriented FAST and Rotated BRIEF. Since FAST "does not produce am e a s u r eo fc o r n e r n e s s " ,R u b l e ee ta l .e m p l o y e d" aH a r r i sc o rner measure to order the FAST key points". They also produced FAST features on a pyramid of the original image, filtered by Harris, in order to have multi-scale features. In order to bring rotation invariance to BRIEF, Rublee et al. steered BRIEF according to the orientation of keypoints. However, ORB is not scale invariant. This means that the descriptor is efficient, but only on a defined and constant distance between the camera and the object.

2.1.1.2.c Conclusion on Tracking Methods

In order to propose an efficient reality-virtuality alignment, spatial references objects in the scene are needed. Such objects can be the user itself, textures, rigid objects, etc. In order to determine the position of such objects in the scene, a augmented reality system should provide an efficient tracking method.

We have divided the tracking methods into two classes. The firstclassencompasses marker-based tracking, while the second one includes marker-free tracking.

Marker-based tracking can be realized using visual or invisible markers, such as infra-red markers. Visual markers are patterns easily detected using the minimum image processing. Tracking methods with markers are robust and efficient, but they suffer from an essential drawback : the marker itself. Adding markers in the scene could lead to a "pollution" of the scene. In some application environments it may not be possible to know measured locations of fiducial markers and so any approach that relies on them could not be used. Military battlefield applications or,g e n e r a l l y ,u n f a m i l i a r outdoor environments come to mind when considering this limitation.

Marker-free tracking methods is an alternative to marker-based tracking. Some methods uses mechanical or magnetic sensors, GPS and gyroscopes or also image processing. Image processing-based methods propose key pointsdetectionandsometimes key points descriptors. Descriptors permit to increase the matching of objects between two images of different viewing conditions but come at a time-consuming price.

When designing augmented reality applications, a trade-offbetweentrackingand real-time has to be make made. Choosing the "good" tracking method depends on the application to develop.

Augmented Reality Applications

Augmented reality is a recent research area and is becoming more and more popular. Several applications are using augmented reality in different fields, from games to medicine. The overall objective of augmented reality is toi m p r o v et h eu s e r' sp e rceptions and interactions. We are now presenting augmented reality applications in different fields.

Entertainment

Entertainment and culture industries are more and more attracted by augmented reality applications. [START_REF] Chien | ARToolkit-based augmented reality system with integrated 1-D barcode : combining colorful markers with remote servers of 3D data for product promotion purposes[END_REF] propose a colored-marker augmentedrealitysystemwith 1-D barcode reader. The markers used in their application is acompositionofacolored marker and the barcode of products. If the system detects the marker in the scene, a 3D graphics model is displayed in the scene, representing theproducttoanalyze.This system is intended for advertisement and promotion.

Chien et al. in

In order to create a museum display platform, Chen et al. [START_REF] Chen | Establishing a Museum Display Platform by Using Combination of Reflection Holograms and Tangible Augmented Reality[END_REF] propose the Augmented-Holograms Display Platform (AHDP). This system is a combination of optical holographs and tangible user interfaces. It makes use of a Web camera to track the user's head movement using computer vision in order to project augmented images on holograms according to the head's position.

Marshall proposes in his thesis report an Android applicationtona v i ga tethr ough Stockholm to discover Nobel's prize related places outside the walls of the museum [Marshall11]. The augmented reality tracking system is basedo nt h ep o s i t i o n i n gs y stems of mobile devices and uses their cameras and displays to add pieces of informations in the scene.

Takacs presentes a panoramic broadcasting for cultural presentation and entertainment services [START_REF] Takacs | Immersive interactive reality : Internet-based ondemand VR for cultural presentation[END_REF]. This project aims at bringing augmented streaming panoramic video onto mobile platforms. This project has been testedforvariousapplications such as sport, bridge walkabout with added information3 ,etc.Thisapplicationisbased on Flash technologies.

Games

The games industry also perceives the benefits of augmented reality applications. They enable the users' better immersion. [START_REF] David Cheok | Human Pacman : a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing[END_REF] present Human Pacman, an augmentedrealitygamebased on the famous Pacman. Users evolve as teams, both as the characters of the Pacmen and Ghost. In this game, "players are provided with custom-built wearable computers" and head mounted devices to visualize the game world.

Cheock et al. in

Not only mobile games are taking the benefits of augmented reality. Yamabe et al. proposed an augmented reality system for "traditional" games such as table games (poker, Go game, etc.) [Yamabe11]. The system is composed of ap r o j e c t o ra n daW e b camera linked to a computer. The projector superimposes the feedbacks on the table (score, player turn, etc.) while the Web camera detects the position of game elements (cards, Go stones, etc.).

Serious gaming also comes with augmented reality benefits. TheA R P u z z l ei sa "simple game that consists of six equal parts that represent parts of City University campus in London" [START_REF] Liarokapis | AC a s eS t u d yo fA u g m e n t e d Reality Serious Games[END_REF]. The components of the CityC a m p u sa r es u p e rimposed onto the table-top environment. Users receive information about the campus while trying to resolve it through text, audio or graphics feedbacks.

Military

In the military area, simulation benefits through augmented reality have been proved. Gelenbe et al. in 2003 [Gelenbe03] presented a simulation environment for a military simulation area. A multi-agent system is coupled with the augmented reality application in order to provide intelligent behavior of injected artificial entities.

The Battlefield Augmented Reality System (BARS) presented in 2000 by the Naval Research Laboratory in Washington DC (USA) is a famous example of augmented reality application for military usage [START_REF] Julier | BARS : Battlefield Augmented Reality System[END_REF]. BARS uses a head mounted device to display data, a wearable computer and a wireless network. In order "to prevent information overload", BARS proposes a mechanism to prioritize the information to add in the user's view depending on the location, the mission, etc.

Due to the confidential aspects of military systems, it is not easy to find a deeper analysis of existing solutions in the literature.

Health Care

Virtual surgery training implies a realistic simulation. Chen et al. in their works [START_REF] Chen | Development of an Augmented Reality Force Feedback Virtual Surgery Training Platform.I n t e r n a t i o n a lJ o u r n a l of Automation and Smart Technology[END_REF] studied in their works interactions involving vision and tactile sensations. They came up with force feedback modules for cutting functions. They modeled this behavior with collision detection algorithms. The 3D model implemented in this study is based on Computed Tomography images and was implemented for cranium surgery.

In endoscopic surgery, Suzuki et al. [START_REF] Suzuki | Scorpion shaped endoscopic surgical robot for NOTES and SPS with augmented reality functions.I nP r o c e e d i n g so ft h e 5th international conference on Medical imaging and augmented reality[END_REF] wanted to improve the "general function of the surgical robot system". The robot embeds a camera. The endoscopic images are displayed on a monitor and are augmented with additional information such as haptic sense information, patient information and navigation data.

In health care, augmented reality is used to "evaluate and appreciate the angioarchitecture of the nidus with regards to position and spatial relationship of critical feeders and draining veins" [START_REF] Ng | Surgical planning for microsurgical excision of cerebral arterio-venousmalformations using virtual reality technology[END_REF], or "assisted skull surgery with ultrasound tracking" [START_REF] Alusi | Three dimensional tracking with ultrasound for augmented reality applications in skull base surgery[END_REF] or even in order to "understand the blood vessel network of a patient" [START_REF] Hutchison | Medical imaging and augmented reality[END_REF].

Conclusion

Augmented reality is the combination of real and virtual objects in a scene. It can improve the understanding of the apprehended environment bya d d i n ge x t r ao rr e ndering invisible information to the user.

In order to build effective augmented reality applications,t h r e ef u n d a m e n t a l constraints have to be taken into account : a) combine the realandthevirtualworld,b) consider the consistency and the coherence of the real and thevirtualworldandc)take into account the real-time and the interactivity of augmented reality applications.

In order to build an augmented reality application, a proper modeling environment has to be chosen as well as a tracking method to place objects inside this environment. Most of the time, a 3D environment is preferred. Marker-based tracking methods consist in placing targets that are easily detectable in the environment. Targets can be infra-red markers, RFID ships, fiducial markers, etc. Marker-free tracking methods represent an alternative to marker-based tracking. Some methods use mechanical or magnetic sensors, GPS and gyroscopes or also image processing.

Augmented reality can be applied in many areas. In the culturea r e a ,a u g m e n t e d reality applications enable users to have a better interaction, such as in museum applications. In the games area, augmented reality improves the immersion of players. In the military area it helps soldiers to have invisible data on the mission field while in health care it can give assistance to surgeons. The learning and maintenance areas are not represented in this short review on augmented reality applications, since they are the subject of a deeper analysis in following sections of thischapter .

Collaboration

Work is often naturally collective and collaborative. This means that it often makes many actors interact with each other in order to reach a commong o a l .I ti sh o w e v e r important not to mistake collaborative work and cooperative work. Collaborative work is done from the beginning to the end without tasks division. Its results depend on the motivation of people to collaborate, onthenumberofactorsand on their skills. To ensure collaboration, a high level of trust has to be achieved and a frequent communication has to take place.

Of course, when applied to online engineering, collaboration also depends on the time and on the location of the actors. Collaboration can be synchronous (collaboration occurs at the same time for every actors) or asynchronous. It also can be centralized or distributed if people are located in the same place.

In learning theories, socio-constructivism [START_REF] David H Jonassen | Objectivism versus constructivism : Do we need a new philosophical paradigm[END_REF] is the theory that tries to make students learn from their own observations and from the interactions with others, both teachers and other students. In distant collaborative work, that entails that each actor is aware of the others. In online engineering that implies taking into account the "group awareness". As said in [START_REF] Gutwin | r o c e e d i n g s o f t h e ACM Conference on Computer-Supported Cooperative Work[END_REF], "Group awareness is the understanding of who is working with you, what they are doing, and how your own actions interact with theirs".

Aforumcanbeseenasadistantasynchronouscollaborativesystem. Users are able to communicate, and each actions is recorded and seen by otherusersintheformofa history.

The Google documents 4 are examples of synchronous distant collaborative systems. In order to reproduce the group awareness, a tele-presence indicator is presented to users. A history of each actions can be seen in order to understand "who did what".

Communication takes a great part of collaborative systems. In order to achieve communication, different tools are used, such as Bulletin board, e-mail, chat, audio/video conferencing, etc. [Bafoutsou02]. Each communication toolbringsitsdifferentfunctionalities and services. For example, e-mails allow an asynchronous possibility to communicate. Chats allow a synchronous way to communicate. E-mail notifications allow the possibility to notify an actor of changes in a project, without the possibility to answer. Whiteboards "allow two or more people to view and draw on as h a r e dd r a w i n g surface. This may be used for discussing or describing objects, which are difficult to verbalize". Bafoutsou et al. classify communication tools in four categories. The first one is textitgroup file. It involves working with files. Actors share a view of files with a possibility (or not) to edit, manage and store the shared documents in the system database. The second category is computer conferencing that provides tools to communicate in a synchronous or asynchronous manner. Whiteboards or forums are examples of those tools. Electronic meeting systems represent the third category. Most of the time, this category entails real-time communication such as audio/visual. The Electronic workspace is the last category. The basic idea underlying it is the possibility to "offer a common space to coordinate and organize" the group work. Anyway, some tools can be classified in different categories.

It is clear that collaborative work is based on information and resources sharing. Depending on the situation, different tools can be used in order to produce the group awareness. In the field of online engineering, a collaboratives y s t e ms h o u l da l l o wt h e possibility to control remote devices at the same time. It entails that real-time communication is required for this situation, but is not enough. Communication tools does not reproduce the group awareness, i.e. the knowledge of "who didwhat".

Context Awareness and Semantic Knowledge

In this subsection, we expand our vision on remote laboratories to remote laboratories able to adapt their HCI depending on their environments.Systemsabletopropose an adaptation depending on their context are referred to as context-aware applications or context-aware systems.

Context and Context aware systems

A definition of context, extracted from a dictionary, is the following : "the circumstances that form the setting for an event, statement, or idea, and in terms of which it can be fully understood and assessed"[Press12].

This means that around a fact gravitates a group of elements that explicit it. The context is then the interpretation of a fact through a filter composed of elements. This means that each fact has its own context that allows to interpret it.

In the scientific literature, each work dealing with context starts with defining its own definition of the context. Definitions of context are relative to the goal of the application. This practice leads to a large diversity of definitions [START_REF] Bazire | hI n t e r n a t i o n a la n dI n t e r d i s c i p l i n a r yC o n f e r e n c e on Modeling and Using Context[END_REF],allpreservinga common pattern, that leads to the definition of the context by Dey [START_REF] Dey | Towards a better understanding of context and context-awareness.I nC H I2 0 0 0w o r k s h o po nt h ew h a t ,w h o , where, when, and how of context-awareness[END_REF] : "Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves". This definition is still the reference, even if it does not take into account the temporal aspect of context. Among these specifications of context, the notion of "context-aware computing" [START_REF] Schilit | Context-aware computing applications[END_REF] devides the context of systems into three categories :

-C o m p u t i n gc o n t e x t( b a n d w i d t h ,t e r m i n a lr e s o u r c e s ,e t c . ) .

-User context (profile, geo-location, etc.).

-P h y s i c a lc o n t e x t( n o i s e ,l i g h t n i n g ,e t c . ) . We do not intend to redefine context-aware systems or to providead i f f e r e n ta pproach to handle context-aware systems, but rather to define a key element of the context of remote laboratories we tried to answer in this thesis. This element can be classified within the Computing context :theprocedureofcontrolofthedevice.W ethink the orders of the actions taken by the operator, i.e. the user's activity on the HCI, is an element to take into account in the context of remote laboratories. In the following subsections, we will analyze our intention to propose HCI adaptation depending on the user activity.

Adaptation depending on the procedure

In the field of Computer-Aided Maintenance, online engineeringhelpindustriesto execute maintenance procedures on remote devices. Production and industrial maintenance sites rely on highly qualified technicians and staff. Thelatterareresponsiblefor many assembly/disassembly operations, calibration, etc. These human operators are expected to understand and memorize a large panel of procedure steps, in a specific order. Moreover, moving to the production site presents several issues (cost, risk, etc.). As a consequence, a first maintenance operation is usually performed remotely. Learning and being able to reproduce the full catalog of maintenance operations can be a difficult task. The complexity of the task is also rapidly exploding since the number of devices operated by a single human operator increases. CAM systems must provide a formalism to express such procedures in order to ease operators actions and increase their productivity. As a consequence, on the Human Computer Interface side, the CAM systems have to provide indications on which action has been made and "whisper" the operator the next possible actions in the maintenance procedures. The aim is to limit the number of human errors, as well as to increase the efficiency( t i m es p e n t ,q u a l i t y , etc.) of the maintenance process.

The scenario of remote maintenance with Alice and Bob showed the necessity of proposing a formalism to express sequences of actions. It also showed that the business logic determines which procedure is being performed and whatt h en e x tp o s s i b l ea ctions are. This logic must be loosely-coupled with any programming language in order to be fully reusable. In order to achieve this loose-couplingr ulesenginessystemshall be used. After a review of related works, rules engines will bei n tr odu c ed.Fol l ow i n g, we present Semantic Web technologies and reasoning capacities when coupled with rules engines. In the last paragraph, we present the necessity to develop a formalism to express maintenance procedures.

Related Works

Works were conducted in the literature in order to propose adaptation. In [Kock11], the authors propose a system to monitor and interpret sequential activities in e-Learning environments. Their system is based on pattern mining and data clustering. The goal is to offer a personalization of e-learning servicesb a s e do nt h er e s u l t so f the analysis. Sequences are represented using Discrete Markov Models. This approach is interesting for e-learning processes since the system is able to determine if users have successfully completed the problem. However, this approach is not suited for remote maintenance since the sequences of actions are not pre-determined but computed afterwards.

The authors of [START_REF] Schmidt | An user interface adaptation architecture for rich internet applications.I nP r oceedings of the 5th European semantic web conference on The semantic web : research and applications[END_REF] propose an architecture for RIA user interface adaptation. The system uses ontologies to annotate the application, and to describe the user model. It also takes advantages of the Semantic Web for mininga da pta ti onr u l es.Th e adaptation rules are located on the client side and are designed using OWL-DL and SWRL.

In [START_REF] Iung | Conceptual framework for e-Maintenance : Illustration by e-Maintenance technologies and platforms[END_REF], the authors propose a complete conceptual framework for remote maintenance. This framework helps designing e-Maintenance applications using five abstraction levels such as strategic vision, business process,o r g a n i z a t i o n ,s e r v i c e ,d a t a structure, and IT infrastructure. The sequences are designed as workflows. The framework does not describe the formalism of the sequences and the way to help operators in maintenance operations.

In [START_REF] Lindstaedt | Getting to Know Your User -Unobtrusive User Model Maintenance within Work-Integrated Learning Environments[END_REF], the authors propose a user model based on ontologies. This model is used to characterize users in order to adapt Web application using rules. They designed and implemented a rule-driven agent, which tries tocaptureinterestingpatterns of user-system interactions. This approach is user-centered but not on processes of pre-defined actions.

In , the authors provide a framework for a personalized e-learning system. The goal is to adapt the system depending on user context. This framework is using ontologies and rules.

Rules Engine

Ab u s i n e s sr u l e se n g i n ei sas o f t w a r ec o m p o n e n t ,t h er o l eo fw hich is to execute pattern matching between facts and business rules. Businessrulesis"Anatomicpiece of re-usable business logic, specified declaratively" [START_REF] Ross | Building business solutions : Business analysis with business rules[END_REF]. This definition contains four important words :

-Atomic :ab usi nessr ul ec a nnotb edi v i dedunderthepena l tyofl ossofsemantic value. -Re-usable : a business rule is not specific to one application but can be shared by multiple applications. -Business :r u l e sa r eb a s e do nb u s i n e s s ,t h e r e f o r et h e ya r ei n t e n d e dt obusiness experts. The latter has to manage the rules. A non-technical representation model is required. -Declaratively :rulesarepresentedinadeclarativeway . An example of business rule can be the following : Ac u s t o m e rw h oo r d e r sm o r et h a n1 0p r o d u c t sd u r i n ga calendar year is a good customer.

Abusinessruleismadeofconditions,e.g."Acustomerwhoorders more than 10 products during a calendar year", and decisions, e.g. "is a good customer". Its purpose is to automate a decision.

Rules engines enable a loose-coupling between business rules and programming components. They entail a high level of portability and reuse.

In the literature, many e-Maintenance systems are built on topo fS e m a n t i cW e b technologies [START_REF] Muller | b i l i t yE n g i n e e r i n g &S y s t e mS a f e t y ,v o l .9 3 ,n o .8[END_REF]. They provide a high level abstraction and description of resources. Coupled with rules engines, they offer reasoning capabilities. In the following paragraphs, we explain our motivations for the use of Semantic Web technologies.

RDFS/RDF and RDFS Reasoning.

The Resource Description Framework (RDF) is a graph-based model designed to describe in a formal way resources on the Web, along with theirmetadatatoallowthe automatic treatment of such descriptions. RDF is a W3C specification [W3C04a] and is the skeleton of the Semantic Web. One of its serializations uses XML format. RDF Schema (RDFS) is also a W3C specification [W3C04b]. It furnishes bases vocabulary to structure RDF resources.

We can find several RDFS reasoners. Jena is a popular Java framework for building Semantic Web applications. It proposes an API for reading, processing and writing RDF data in XML. It also includes a RDFS rule-based reasoner based on Datalog and RETE [START_REF] Forgy | Rete : A Fast Algorithm for the Many Pattern/Many Object Pattern Match Problem[END_REF]. This rules engine allows to write rules and to obtain reasoning on RDF resources.

4rs [4rs06], 4store reasoning, integrates 4store's SPARQL and SPARQL/Update implementations with RDFS backward chained reasoning.

IRIS [START_REF] Reasoner | [END_REF] is another RDFS reasoner also based on Datalog.

OWL and SWRL.

OWL stands for Ontology Web Languages. It is the most widely spread specification for ontologies and is divided into different levels of expressiveness OWL Lite, OWL-DL and OWL Full, ordered by increasing expressiveness. OWL-DL is the most widely used sub-language of OWL, because of its decidability. Expressing rules over ontologies axioms can be achieved using Semantic Web Rule Language.

Multimedia Scene Descriptions

In order to address a wide range of different devices, a formalism to describe the remote device environment has to be chosen.

In Latin language, media is the plural of medium.Incomputerscience,amediumis any kind of support of information. The term multimedia grew at the end of the 80's with the appearance of Compact Disc -Read Only Memory (CD-ROM). It designated applications able to generate or control together differentkindofmediasuchassound, video, images, etc. We prefer defining a multimedia as : Definition (Multimedia). Any kind of media related to each other with digital, logical, temporal or semantic relationship presented according to the author's choices.

In order to carry on such relationships, it is necessary to master the media's presentation. It is achieved by incorporating media in multimedia scenes. Multimedia scene descriptions enable the instructions for staging media. Concolato, based on Scherp et al. works on scene description [START_REF] Scherp | Paving the Last Mile for Multi-Channel Multimedia Presentation Generation.I n i n P r o c e d i n g s of the 11th Multimedia Modeling Conference[END_REF], defines the properties of scene descriptions [START_REF] Concolato | Descriptions de scènes multimédia : représentations et optimisations[END_REF] :

-t h ee m p l o y m e n to fa u d i o -v i s u a lm e d i ao ro b j e c t s( n a t u r a lo fsynthetic).

-t h es p a t i a lo r g a n i z a t i o no fo b j e c t s( r e l a t i v ea n da b s o l u t e) in 2D or 3D environments -t h et e m p o r a lo r g a n i z a t i o no fo b j e c t s( o v e rt i m ee v o l u t i o n ) -i n t e r a c t i v i t yc a p a b i l i t i e s Am u l t i m e d i as c e n ed e s c r i p t i o ni su s u a l l yr e p r e s e n t e da sas cene-graph. Different scene-graphs and scene description languages have been created. With the rapid growing Internet phenomenon, interoperability is needed in order to ease the broadcast of multimedia content.

Scene Description Formats

Multimedia scenes descriptions enable the instructions for staging media. In this section, we will present some of the most common description languages and formats.

VRML

The Virtual Reality Modeling Language (VRML) is one of the first description languages. Its second version is standardized within the standard organization ISO and is called VRML 2.0 or VRML97 [ISO/IEC-1477297].

This description language is oriented to the creation and visualization of 3D scenes. VRML proposes a hyperlink mechanism to create files and to refer to external ones. Interactivity is achieved using special nodes named Sensors and scripts supports in the Script node.

But VRML lacks the support for 2D graphics as well as compression. Streaming content is not allowed either. A third version of VRML has been published. This version enables to create VRML content in a XML manner and adds various improvements in 3D primitives. It is called Extensible 3D (X3D). SMIL is based on the XML syntax allowing the presentation of different media objects such as images, video, audio and text. It provides support for 2D graphics but not for 3D graphics. Like VRML, SMIL lacks compression support.

SVG

The Scalable Vector Graphics (SVG) is also a W3C specification [W3C11b]. The second edition of its version 1.1 was published in August 2011. SVG is used to describe 2D scenes and graphical applications by using 2D primitives and vector graphics. It entails that objects can be viewed on any resolution with the best possible quality. Figure 2.9 shows two examples of SVG 2D graphics. SVG supports ECMAScript. This characteristic enables to manipulate all objects of the scene and to perform animation on them.

As opposed to SMIL and VRML, SVG can be compressed using the gzip 5 standard, producing the SVGZ format. No streaming support is provided.

COLLADA

Sony Computer Entertainment released in 2004 the initial version of COLLAborative Design Activity (COLLADA). Its current version, 1.5, is released by Khronos [Khronos08]. This description language is widely accepted, especially in the game industries.

5. http ://www.gzip.org/index-f.html COLLADA also has an XML syntax. It supports both 2D and 3D primitives but does not support video or sound media. It does not support interactivity, nor streaming of data. Hence, COLLADA is mainly used for storing data.

Flash

Flash is a proprietary solution by the Adobe Systems company [Adobe12]. A lot of Web sites are using the Flash format in order to present elements in the page, such as advertisement or games. It enables to improve the interactivity, the dynamism and the animation of Web pages. That is why Flash is very popular.

Flash designates the authoring tool, the data formats to thistoolaswellasthepublishing format. The Flash publishing format is a binary format.

Flash supports 3D scenes since it 11th version, published in October 2011, but this support is only restricted to personal computer for the time being. Flash supports scripting through ActionScript, a derivative of ECMAScript and proposes a compression of the scene.

Flash is probably still the most widespread format on the Web, even if HTML and its fifth version could possibly dethrone it, according to W3C and companies behind it.

JavaFX

JavaFX 6 is a software platform developed by Oracle. The current release version is 2.0 and has been distributed since October 2011. Java FX is intended for Rich Internet Applications and the scene representation is also a scene-graph. Since its second version, JavaFX is implemented as a native Java library. Therefore, it allows developers to embed Java packages easily in JavaFX applications. JavaFX also supports FXML as an XML-based description language.

JavaFX provides supports for 2D and 3D graphics, interactivity through java code, atimeline.JavaFXscenesar ecompiled.JavaFXsupportsonlyonevideocodec,which is the VP6. Concerning audio support, only the MP3 codec is supported. There is no streaming support.

Silverlight

Silverlight is a product distributed by Windows 7 .Itscurrentversionis5.Silverlight targets mobile applications as well as Web applications. It uses the XAML description.

In terms of video and audio, Silverlight supports several codecs such as MPEG-AVC, Theora QuickTime, MP3, etc. Interactivity is supported through .NET languages scripts. Silverlight proposes a support for 2D and 3D graphics. Like JavaFX, Silverlight is compiled.

HTML

The HyperText Markup Language (HTML)[W3C11a] is probably the most famous W3C specification. It is a markup language designed to describe Web pages. Its fifth version is currently under construction and brings a lot of new features to the language such as syntax alleviation, encoding, new nodes. An interesting node of the HTML 5 specification is the canvas node. This node enables the support of 2D graphics through a dedicated API. The WebGL specification [Khronos11] is an efforttobring3DAPIstothe HTML 5 canvas tag. This specification is not currently supported by all Web browsers. Figure 2.10 shows an animated WebGL scene in the Google Chromebrowser . The video tag adds video support to HTML. Currently, no video codec is associated with the video element, each Web browsers proposing its own implementation. Interactivity is achieved through JavaScript. BIFS provides the support for 2D, 3D or mixed 2D/3D scenes. As opposed to VRML, BIFS provides tools to change scenes in a dynamic way by replacing, deleting, adding nodes. BIFS proposes an XML version of its language called Extended MPEG-4 Textual format (MPEG-4 XMT). Through the Object Description framework, BIFS is able to manage other kinds of media such as images, videos, etc. Figure 2.11 shows an animated 3D scene displaying a PNG image and an MP4 video in 3D boxes.

BIFS and XMT

MPEG

LASeR

Lightweight Application Scene Representation (MPEG-4 LASeR) is another scene description language standardized in MPEG-4 [START_REF] Dufourd | An MPEG Standard for Rich Media Services[END_REF]. Iti sb a s e do nS V GT i n y 1.2 for spatial representation and on SMIL for animations, temporal support and media management. SVG Tiny 1.2 is a sub specification of SVG designed for terminals with weak capacities, mainly with removing items to SVG such as 3D graphics, etc. LASeR also has streaming mechanisms allowing the compression of scenes.

Analysis

Scene descriptions represent methods for creating, presenting and sharing interactive and animated multimedia content. The previously presented scene descriptors are not suited for any kind of applications. They are not capable of managing the same Scene descriptors can help developers design remote laboratories since they allow a spatial and temporal representation of scenes, as well as ensuring interoperability between terminals. The choice of a scene descriptor must be carefully analyzed in term of capabilities. As observed from table 2.1, only the MPEG scene descriptions are capable of :

Scene Descriptor

Graphics

Audio and Video

-p r e s e n t i n ga l lk i n d so fm

Conclusion

Remote Laboratories and Online Engineering bring the possibility to perform manipulations on real devices over the Internet. We believe augmented reality technolo-gies can help remote experimentation, providing a better understanding of the remote scene. Thus, the augmented reality paradigm and its associated uses have been presented. We have presented collaboration and group awarenessanddifferentassociated tools in software systems. We have introduced Semantic Web technologies as a key component of Context-Aware System. Multimedia scene descriptors are a way in order to formalize the remote device environment. Thus, they also were introduced and discussed.

Remote Laboratories and Online Engineering

The orevious section discussed technologies and paradigms that can answer the four main issues raised in Chapter 1. In this section, we review and analyze the use of the presented technologies and in Remote Laboratories andOnlineEngineering.W e have noticed that in the scientific literature, augmented reality applications are widely used for industrial usages ; online engineering is mostly used for educational purposes. That is why we present a review of scientific contributions divided into two parts. The first part of this analysis focuses on e-Learning. The second part of this analysis focuses on industrial usage and Computer-Assisted Maintenance.

e-Learning

In this subsection, we intend to present the major online engineering projects, especially remote laboratories frameworks. This study is not intended to be exhaustive, but rather presents an overall picture of the online engineeringp r o j e c t sp r e s e n t e di nt h e scientific literature.

The iLab 8 project, developed by the Massachusetts Institute of Technology (Cambridge, MA, USA), is probably one of the best known online laboratory framework still in development. The overall objective of this framework is toproposeasoftwarearchitecture for the development of online courses. This architecture is based on the Service Oriented Architecture, SOA. Services are components for accessing online devices, such as authentication, scheduling, experiment storage or authorization. Any kind of client application can be used in order to control the device as the software developer follows the iLab Shared Architecture API 9 based on SOAP Web Services 10 [START_REF] Vj Harward | iLab : A scalable architecture for sharing online experiments[END_REF]. In 2012, the first iLab Android client application was presented [START_REF] Deaky | The First Android Client Application for the iLab Shared Architecture[END_REF]. iLab does not provide any tool for collaboration. All the feedbacks from the controlled online devices are designed as software components.

Inside the SOCRATES European programs, the "Minerva action aims to promote European collaboration in the fields of Open and Distance Learning (ODL) and Information and Communication technology (ICT)" 11 (2006-2008). This action leads into a noticeable project : eMerge. eMerge is an educational network structure whose purpose is to disseminate online laboratories experiments, especially in Europe. This project lead to the creation of the remote laboratory platform eLab in Bordeaux (France) [START_REF] Lang | Pedagogical evaluation of remote laboratories in eMerge project[END_REF]. The client application is a Web application based on the classical Linux Apache MySQL PHP technologies as well as ASP.NET, HTML and SVG for the clientsi dea ppl i c a ti on. Av i d e of e e d b a c ko ft h er e m o t ei n s t r u m e n tc a nb ei n s e r t e di n t ot h eW e ba p p l i c a t i o n The LabShare project is an initiative from Australia. The originality of this framework is the hybrid architecture they use in order to interact with the remote devices. A stand-alone application runs on its own virtualized server and the user accesses this application via a remote desktop client. The overall framework is constructed with ASP.NET technologies, PHP, JavaScript and Virtual Network Computing Tools for the client application access. This framework proposes video capture of the remote scene. No support for collaborative works is mentioned nor mobile application [START_REF] Murray | Experiences with a hybrid architecture for remote laboratories[END_REF].

Te l e l a b s12 is a project from the University of Western Australia. This remote laboratory is intended for students in the mechanical engineering curriculum. It allows students to remotely control different kinds of robots. This project is focused on the will to measure learners' performance and to optimize the organization between tutored time and non-tutored time of the learners while keeping an optimallearning.Ontheclient side, a video capture of the remote robot is delivered to the learner. Some remote experimentations also use augmented reality techniques [START_REF] Friz | Design of an Augmented Reality User Interface for an Internet based Telerobot using Multiple Monoscopic Views[END_REF]. The desktop application, called Labs-On-Line, is developed using ActiveX and LabViewt e c h n o l o g i e s .T h e r ei s no mention about a mobile client application [START_REF] Trevelyan | Lessons Learned from 10 Years Experience with Remote Laboratories[END_REF].

The PEARL project, Practical Experimentation by Accessible Remote Learning, includes several European institutions from England, Scotland, Ireland, Greece, and Portugal [START_REF] Cooper | Remote Controlled Experiments for Teaching over the Internet : A Comparison of Approaches Developed in the PEARL Project.I n 1 9 t h A n n u a l C o n f e r e n c[END_REF]. This project started in 2000 and ended in 2003. Its first goal was to propose a unique system for accessing remote experiments but they finally divided their solution into two approaches. One is based on LabView, the other is based on CORBA. This project proposes communication tools between learners and also focuses on bringing laboratory accessibility to disabled learners.Theclientapplicationisbased on Java applets and proposes a video feedback of the devices.

WebLab 13 is a project conducted in Spain [Zubía12]. The particularityo ft h i sp r oject is that it offers different possibilities for the clientapplicationforaccessingther emote laboratories such as virtual world or web applications.F o rm o b i l ed e v i c e s ,W e b applications are used. On the client side, WebLab provides libraries in three technologies : Java Applets, JavaScript and Flash [START_REF] Orduna | Enabling mobile access to Remote Laboratories[END_REF]. Virtual world experiments can be achieved using the Second Life software development kit. In this Second Life world, apseudo-collaborationispr oposedusing3Dchatr oomsand3Davatarsinthevirtual world. A learner can control a micro-robot and "other students can be in the same room in Second Life watching the same results without interactingwiththeexperiment".

Mixed reality techniques for remote experiments can also be integrated in virtual learning environments (VLE). This enables to use both virtual instrumentation (i.e simulation) and real devices. The proposed system in [START_REF] Pereira | Integrating Mixed-Reality Remote Experiments Into Virtual Learning Environments Using Interchangeable Components[END_REF] has been developed as part of the RExNet Consortium, financed by the Alfa II European project. A noticeable point of this project is the integration of the Moodle learningmanagementsystem.The client application of this project has been developed using Java Applets technologies and has been improved with VRML support for 3D environment [START_REF] Schaf | Collaborative learning and engineering workspaces[END_REF]. Institutions from Brazil and Germany are involved in the development of this remote experiments framework. A "complete computer-supported collaborative environment" is under development. There is no mention about a mobile application.

The objectives of the CORELA project (Collaborative Remote Laboratories), are to propose a "low cost learning environment for collaborative learning" [START_REF] Muller | o s tL e a r n i n gE n v ironment for Collaborative Engineering[END_REF] and to spread "mixed-reality environments in which real devices interact with simulation models and 3D Virtual Worlds". This project is mainly intended for mechatronics remote laboratories. A video capture of the remote device is available in the client Web application, developed with PROCESSING 14 .PROCESSINGisaprogramminglanguageand ad e v e l o p m e n te n v i r o n m e n tb a s e do nJ a v aA p p l e t se n a b l i n g .T he 3D environment is based on OpenSim and integrates PROCESSING. This project uses Moodle in order to reproduce the collaboration and has been influenced by the MARVEL project.

The MARVEL 15 project is a remote laboratory projects network focused on virtual laboratory in mechatronic. The funding for this project comefromtheEuropeanCom-13. https ://www.weblab.deusto.es/web/ 14. http ://processing.org/about/ 15. http ://www.informatik.uni-bremen.de/ mueller/en/projects/marvel.html mission, through the Leonardo Da Vinci program. This projectincludesseveralremote laboratories in Europe such as the Remote Accessible Full-Scale Solar Plant in Germany (Delmenhorst), the Remote Accessible Pilot Solar Lab in Cyprus (Nicosia), the Robot Training Workshop for Remote Tutoring in Greece (Athens), the Remote Workshop for Robotics and Automation in Scotland (Edinburgh), the Mixed Reality Web Service for Mechatronics in Germany (Bremen), the Remote Workbench for Electronics in Portugal (Porto) and the Remote Mechatronics in Switzerland (Sion) [START_REF] Bruns | Engineering Future Laboratories[END_REF]. The Mixed Reality Web Service for Mechatronics (artecLab) is noticeable since it is the base of the client application of the RExNet project.

In Italy, the Internet Shared Instrumentation Laboratory (ISILab) proposes remote experiments in digital electronic circuits and digital signal processing board. This project started in 1998. In 2001, it started being used in engineering courses 16 .Inpractical sessions, a "startup source code is provided and students must complete, compile and test it online" [START_REF] Buschiazzo | RodolfoZ u n i n o& Anna Marina Scapolla[END_REF]. The feedbacks from the remote controlled devices are presented as software components in the user interface. The HCI is built on top of HTML and AJAX technologies and allows users communication through a video conferencing component. The ISILab is build using SOA and grid infrastructure of online devices [START_REF] Buschiazzo | A service oriented framework to set up flexible and distributed data acquisition and control systems[END_REF]. It provides a plugin for controlling webcams.

The Remote Lab at EPFL in Swiss (Lausanne) is still under active development and is part of different European projects such as STELLAR 17 (2009) or ROLE 18 (2009) [START_REF] Gillet | Remote Laboratory at EPFL[END_REF]. The particularity of the proposed framework is to divide the HCI into reusable components in the form of widgets. Currently those widgets are being built using Java Applets and Flash and a migration to HTML 5 and SVG is being conducted. This would enable to easily build a mobile version of the Web application since mobile browsers are more and more compliant with this specification. Video streaming of the remote devices is achieved using MJPEG. A video conferencing tool is available for students collaborative communications as well as a Laboratory Journal that reports experimental data, graphs or parameters produced by the learnersintheeMersion cockpit [START_REF] Salzmann | Remote lab : online support and awareness analysis[END_REF]. The client application is adapted depending on the available bandwidth.

Another remote laboratory framework was proposed in Australia at the University of South Australia, the NetLab project. Like the eMersion project, NetLab proposes a "collaborative environment for effective interaction between users and the equipment, as well as between users themselves". The user-to-user communication is based on a chat room in the client application [START_REF] Machotka | 1st WIETE Annual Conference on Engineering and Technology Education[END_REF]. This application is developed with Java technologies and presents a video feedback of the remotedevices.Nomobileapplication is proposed for this framework.

The DIESEL project (Distance Internet-Based Embedded System Experimental Laboratory) is funded by the UK Engineering and Physical Sciences Research Council and 16. http ://isilab.dibe.unige.it/how-it-works.aspx 17. http ://www.stellarnet.eu/ 18. http ://www.role-project.eu/ is located in Derry (Ulster, Northern Ireland) [START_REF] Michael | Paradigms in Remote Experimentation[END_REF]. This project proposes video conferencing tools for collaboration as well as presentation/slide-show, a shared remote desktop, a shared whiteboard and a tele-presence indicator. A video feedback of the remote microprocessor is available in the desktop interface. This interface is developed using ActiveX and Flash technologies.

The LiLa19 project (Library of Labs) is funded by the eContentplus program of the European Community. It involves institutions from Germany, Switzerland, UK, Spain, Sweden, Greece and Netherlands. This project started in May 2009 and ended in May 2011. The goal of this project is to aggregate and disseminater e m o t ee x p e r i m e n t si n Europe and to provide a common architecture for remote experimentations similarly to the iLab framework [START_REF] Bohringer | LiLa : A European Project on Networked Experiments[END_REF], defining metadata on collaboratories. The virtual world Wonderland supports the LiLa architecture by providing a framework for collaboration. This virtual world enables constructing virtual "three-dimensional representations of real experiments that are close to look and feel". Some remote laboratories focus more on augmented techniquesf orpr esen ti n gr emote laboratory interfaces. This is the case of the work conducted in [START_REF] Manuel Andujar | Augmented Reality for the Improvement of Remote Laboratories : An Augmented Remote Laboratory[END_REF]. The goal of this work is to control a remote robot. The client application is built using Java Applet. Odeh et al. present in [Odeh09] a remote laboratory with a distributed architecture applied to electrical courses. It is a Web based application which overlays real kits with virtual objects. This application is developed using C and •NET technologies.

We presented in this section the major remote laboratories for e-Learning. One can notice that a lot of solutions are proposed in order to remotely control devices. Most of the time, the same kind of architecture is presented : a clienta p p l i c a t i o nt h a tc a nb ea stand-alone application or a Web application, a middleware (in charge of relaying commands) and a device connected to a local computer (for hardware communications). We consider the Java Applets based solutions as stand-alone clients since they require the use of the Java Virtual Machine on the client side in order torun.W ealsocannotice that only the Remote Lab at EPFL project presents context adaptation. The following section will summarize and analyze this literature review.

Industrial Maintenance

We list here the major projects identified in the scientific community for the industrial usage. Again, this list is not intended to be exhaustiveb u tf oc u sesonm a j orpr ojects.

The majority of projects in the area relate to the production and maintenance industry. Production sites and industrial maintenance personnel rely on highly skilled technicians. The latter are responsible for many assembly/disassembly operations, ca-libration, etc. These human operators are expected to understand and memorize a large panel of procedure steps, in a specific order. Moreover, moving to the production site presents several issues (cost, risk, etc.). As a consequence, a first maintenance operation is usually performed at distance. In order to address those issues (Remote Computer-Assisted Maintenance), several projects have emerged.

The FP5 STARMATE project is supported by the Thalès Optronique company. The project has raised the issue of assisted maintenance and training on site. This project allows an operator to bring with him the documentation related to maintenance equipments. It helps establishing training courses for in-situ operations of assembly and disassembly of equipment partners. The STARMATE project employs augmented reality technologies in order to access the documentation with voicecommandsandacursor . The project ended in 2000.

The ARMAR (Augmented Reality for Maintenance and Repair) system is usedby the US Navy. It uses augmented reality techniques so as to assist the human operator in the realization of a procedural task or repair [START_REF] Henderson | Exploring the benefits of augmented reality documentation for maintenance and repair[END_REF]. This project requires to wear head-mounted devices and is based on movements detections to provide additional information to the human operator. Additional information can be presented as labels on components, real-time components diagnosis, etc.Itallowsdeportingmaintenance operation states in the virtual world by virtualizing the maintenance procedure so that remote operators can monitor and interact with the in-situ human operator using audio communication. Pictures of the system ARMAR situations are available on the project website 20 .

The Chocolate Factory results from a collaboration within the Silicon Valley between TCHO and the FX Palo Alto Laboratory [Back10]. TCHO is a chocolate products manufacturer located in San Francisco, California (US). Thesystemconsistsinavirtual world visualization on smartphones or personal computers ofmanufacturingprocesses used within the factory. Hundreds of sensors feed the chocolate factory engine 3D virtual world engine, that is based on the 3D graphics engine Unity 21 .Theresultingvirtual world is used for simulation, visualization and new staff training. It is mostly used in order to know the location of each item involved in the production chain. Although the system was centered on the monitoring and control of production elements, the authors I met in the International Conference on Multimedia & Expo 2011 in Barcelona confirmed that only monitoring is effective.

The T.A.C project (Télé Assistance Collaborative) is a French projecti n t e n d e dt o combine remote collaboration and industrial maintenance [START_REF] Bottecchia | Système de réalité augmentée pour la Télé-Assistance Collaborative dans le domaine de la maintenance.I nA s s o c i a t i o nF r a n ç a i s e de Réalité Virtuelle[END_REF]. The in-situ operator wears head-mounted devices in order to capture the scene. A video stream is sent to a remote expert. The remote expert is responsible for augmenting the scene through a dedicated interface. The additional data are sent back to the in-situ opera-20. http ://graphics.cs.columbia.edu/projects/armar/index.htm 21. http ://unity3d.com/ tor who visualizes them through his head-mounted device. Collaboration is achieved using audio-communication tools between the remote and the in-situ operator.

The ARVIKA project is a German project started in 1999 and ended in 2003. This project is especially intended for the automotive and aerospace industries. The operators also wear head-mounted devices in order to have the augmented view of the world, with contextualized data. Some tracking markers were developed during this project as explained in section 2.1.1.2.a. It seems like this project isabletodelivercontextualized data to the operator, according to [START_REF] Bottecchia | Système de réalité augmentée pour la Télé-Assistance Collaborative dans le domaine de la maintenance.I nA s s o c i a t i o nF r a n ç a i s e de Réalité Virtuelle[END_REF].

The ARITI project was initiated in 1996 [START_REF] Boudoin | To w a r d s m u l t i m o d a l h u m a nrobot interaction in large scale virtual environment[END_REF]. Early work led to the establishment of the first robot tele-operation system using augmented reality over the Web in France. The ARITI project aims at controlling and commanding a remote robot. Unlike the T.A.C and ARMAR projects, the Human Computer Interface uses a 3D virtualized renderer 22 .Thehumanoperatormustmakethecognitiveeffortofsynchronizing this world with a video capture of the remote scene. Part of the results of this project constituted preliminary results for the ANR RIAM Digital Ocean.Aparticularity of this work is that it allows a collaborative remote operation on the robots using a multi-agent system.

The ANR RNTL AMRA (2001)(2002)(2003)(2004) was intended to implement an augmented reality system running on mobile devices for industrial use, especially in the industrial maintenance field. More specifically, the project focused on the training phase of maintenance operations [START_REF] Didier | AMRA : Augmented Reality assistance in train maintenance tasks[END_REF]. The originality of this project is the study of contextual helps of inexperienced operators and the bringing-in of relevant information to maintenance staff on their workstation (maintenance documentations, operation processes, assembly video, etc.), the pillar of the project being augmented reality techniques in order to increase the availability of information on the sitemaintenance.

In Italy, VRMedia 23 ,as p i no f ff r o mS c u o l aS u p e r i o r eS a n t ' A n n ad eP i s e ,m a r k e t s REAL-AR.I ti sa na u g m e n t e dr e a l i t ya p p l i c a t i o nf o c u s i n go nl e a r n i n g. The product provides real-time tutoring on devices maintenance. Tutorsc a ns e n di n f o r m a t i o nt o learners who receive them on a portable computer. Italians from Pisa are also involved in the FP7 FLEXA project (2008-2012). Even if the main objective of this project is to develop robotics for motor production in the aeronautic industry, they intend to use this product for in-situ maintenance at the University of Pisa. The remote dimension is not taken into account.

In the ANR CONTINT program, we can mention the ANR RAXENV project (2007-2010). The aim of this project is to demonstrate the feasibility of an outdoor augmented reality system in environment area, in terms of technology but also of adoption by end users. It focuses on reality enrichment with contextualizedi n f o r m a t i o nu s i n gg e o g r aphical sensors. The system provides no interaction since it was not the objective of the 22. http ://ariti.ibisc.univ-evry.fr/doku.php ?id=manipulation 23. http ://www.mindthebridge.org/wp-content/themes/d_natives/VRmedia.html project.

Other projects are ongoing concerning teleoperation on robots or devices using augmented reality techniques, especially the FP7 Digital Ocean 2 (2011-2012). It is the follow-up of the FP7 Digital Ocean (2006)(2007)(2008)(2009). It concerns the whole chain of digital content of the seabed from the creation, the production, the edition, the interfaces, serious gaming screenwriting to its distribution on dedicatedterminalsormobiledevices. It will allow to geo-localize and contextualize multimedia contents thanks to submarine robots.

Another solution to address remote maintenance operations consists in using remote desktop sharing tools such as Virtual Network Computing (VNC). This is often al a s tr e s o r tf o rb a s i cm a i n t e n a n c e .U s i n gV N C se n t a i l si n s t alling the entire device interface on a system able to receive such configuration. This kind of system does not have genuine collaborative working arrangements. A unique cursor also promotes the "scrollwars" [START_REF] Lukosch | Reusing Single-User Applications to Create Multi-user Internet Applications[END_REF]. Any communication protocol is available and the interface cannot be contextualized. This review of augmented reality systems used in the industryareashowsthatmost of the proposed solutions are not conducted online. Few of them propose a collaborative control of the remote device and a lot of them require wearing devices such as head mounted devices. Also, few of them also propose contextualized data to end users.

Synthesis

Analysis

In this section we analyze the solutions presented in the scientific literature for controlling devices. The proposed analysis focuses on four main points : the distant control of devices, the augmented reality paradigm, multimedia scene descriptions and collaboration. A lot of frameworks for the remote control of devices are proposed in the literature. It is a difficult task to review all of them, knowingthatalotofthemarereused in a variety of different projects. Table 2.3 and table 2.2drawasyntheticviewofour analysis.

In the e-Learning area, we can make the same kind of remarks. While some of remote laboratories propose augmented reality solutions, fewofthemproposeacollaborative control of the remote device. Again, most of the time a video feedback is delivered to the remote learner. The video feedback allows the visualization of command results on the remote device, such as a new plot displayed by the device. But it does not allow to know which command was sent to achieve this. The consequence is the inability to transcribe the "group awareness", i.e. to make the user feel that he is in a group of people connected and to enable him to determine the author of a command, as well as to communicate with the people of his group.

In the industrial maintenance area, we can notice that very few of the analyzed solutions propose remote operations. The in-situ operator often wears a head-mounted device in order to have the "augmented information". The remote expert is either confined to an advisory role through an integrated audio system or interacts with the remote scene via a virtualized model of the remote scene, which is notthemostimmersivesolution. At best, a captured video is provided and the remote expert, who is a stranger to the local reality, has to close the gap between the simulated model and the video feedback control. In this case, the in-situ operator perceives only the results of the actions made by the remote expert.

Ve r y f e w o f t h e p re s e n t e d w o r k s a re u s i n g m u l t i m e d i a s c e n e d e scriptions, especially those with augmented reality techniques. As presented in the previous sections, multimedia scene descriptions rely on standards. As a consequence, they ease interoperability between terminals. At the pervasive computing area, remote laboratories must be controlled by the highest number of terminals. From our point of view, interoperability is an utmost issue for remote laboratories. We can also notice that very few systems take into account some elements of the user or system context.

Towards Collaborative and Contextualized Online Instrumentation

Our study on remote laboratories in e-Learning shows that thesolutionspresented do not try to reproduce enough the interactions between learner and machine. Some solutions are using video feedbacks in order to reproduce the reality of the remote scene. But from our point of view, it does not offer a high fidelity of therepresentationofthe apprehended scene. Remote laboratories should go towards augmented reality in order to give a better representation of the remote scene. In the industrial maintenance area, augmented reality is widely used but very few solutions enable the remote control of devices. We can also notice that very few of the solutions proposed in e-Learning are used in industry, even if the need for remote control is the same.

Interoperability is an utmost issue in remote laboratories.W h e nb u i l d i n go n l i n e engineering solutions, we need to take into account the possibility to control remote devices with any kind of terminal. Multimedia scene descriptions contribute to interoperability by providing standards to rely on when designingmultimediaapplications. The study on multimedia scene descriptions shows that not allofthemhavethesame capabilities. It shows that MPEG-4 BIFS is the only scene descriptor answering the problems of streaming, compression and media presentation.

Collaboration generally refers to individuals or organizations working together to address problems and bring results that are not easily achieved by working alone. Collaboration entails the sharing of the same resource at the same time. Collaboration is a key component in remote tele-operation, especially in the field of engineering learning. In learning theories, socio-constructivism [START_REF] David H Jonassen | Objectivism versus constructivism : Do we need a new philosophical paradigm[END_REF] is the theory that tries to make students learn from their own observations and from the interactions with others, both teachers and other students. So far, a remote laboratory solution should manage the "group awareness" [START_REF] Gutwin | r o c e e d i n g s o f t h e ACM Conference on Computer-Supported Cooperative Work[END_REF] mechanism, so that collaboration that occurs in traditional local sessions can be reproduced online. To ensure collaboration, a high level of trust has to be achieved and a frequent communication has to take place. Thus, collaborative systems should not only allow the sharing of a resource between the different protagonists, but it should also maintain group awareness. As stated in [START_REF] Gutwin | r o c e e d i n g s o f t h e ACM Conference on Computer-Supported Cooperative Work[END_REF], "Group awareness is the understanding of who is working with you, what they are doing, and how your own actions interact with theirs". Our study shows that few of the proposed systems enable a collaborative control of the remote device. When in doubt, use brute force.

In this thesis, we try to answer the following questions : -H o wt op r o p o s ear e m o t ec o n t r o lo fd e v i c e st h a tc a nb eu s e di nindustry as well as in e-Learning ? -H o wt op r o p o s eaf u l l yi n t e r o p e r a b l ea u g m e n t e dr e a l i t ys o l ution ? -H o wt op r o p o s eaR I At h a tl i m i t st h eb a n d w i d t h ? -H o wt oh a n d l ec o l l a b o r a t i o ni na u g m e n t e dr e a l i t ya p p l i c a t ions ?

Butler Lampson

3

Proposal Chapter 1 raises the four main issues of online engineering wetrytoanswerinthisthesis.These issues are the remote control of devices through augmented reality views, the collaborative control of devices, the adaptation of the interface depending on the context of use and the scalability in terms of different controlled devices. Chapter 2 presents the state of the art approaches that can overcome these issues. In this chapter, we present our contribution in the online engineering field, serialized in form of a framework. C HAPTER 1raisesthefourmainissuesofonlineengineeringwetrytoanswer in this thesis. These issues are the remote control of devices through augmented reality views, the collaborative control of devices, the adaptationoftheinterfacedependingon the context of use and the scalability in terms of different controlled devices. Chapter 2 presents the state of the art approaches that can overcome these issues.

Chapter Plan

Augmented reality solutions provide a better understandingo fa p p r e h e n d e d scenes. We have observed that in the industrial maintenance area, few of the proposed systems enable a remote control of the device. In the e-Learning area, a lot of systems to control the remote devices are available, but few of them grant for an augmented view of the scene. In both fields, very few systems propose a collaborative control of devices, nor an adaptation of the interface depending on the context. Finally, most of the time, reusability is not often taken into account.

In this chapter, we present a framework for building online experimentations, both for industrial maintenance and e-Learning areas. This framework uses augmented reality to present the interface to the remote users. It enables acollaborativecontrolofthe devices and creates a group awareness in order to reproduce "the sense of being there". The framework also proposes an adaptation of the interface depending on the users' activity. Finally, it eases re-usability, providing the different tools to set up a wide range of devices. Section 3.1 presents this framework and its main architecture. Section 3.2 presents how augmented reality views are created. Section 3.3 explains how collaboration is handled, both in the middleware and at the client side. Section 3.4 introduces the adaptation of the multimedia HCI, based on user's activity. Section 3.5 presents how the multimedia content is delivered to the end users.

Online Engineering and Computer Aided Maintenance Framework

Architecture

As previously explained, figure 1.2 presents the basic architecture of online experimentations. We can notice that the Human Computer Interface is closely bound to the device to control, since a specific interface is designed for a specific device. The same observation can be made for the driver implementation (the link between the local computer and the device to control). The architecture used nowadays implies that for a new device to be remotely controlled, a new instance of the schema has to be created. Moreover, the middleware in those situations is just a message conveyer between both computers.

In this thesis, we try to overcome those issues. Then, we propose the architecture illustrated in figure 3.1. Moving intelligence on the middleware side allows to add an FIGURE 3.1 -Remote experimentation architecture. abstraction layer to the drivers. The middleware is then not just a messages conveyer, it also allows to process commands. This feature is importantf o rd e v i c es c a l a b i l i t y , i.e. the possibility to support a wide range of different instruments without creating a new instance of the system when running new experimentations. The communication middleware is designed in order to support collaboration. The Device Context component permits to formalize the description of the remote device. Firstly, it allows to design the augmented reality interface delivered to the end users. Secondly, it delivers the adaptation of the interface, depending on the context of use.

In this section, we present the main contributions of our worki nt h eo n l i n ee n g ineering field. Those contributions are implanted in an overallframeworkforbuilding and running online experimentations in the e-learning and intheremotemaintenance areas.

The framework proposes two operating modes. The first mode is the runtime mode. This mode is effective when running online experimentations. It delivers an augmented view of the remote device. It allows a collaborative control of the remote scene. Users can interact in the same time with the remote scene. Since group awareness is at the center of collaboration, we propose a mechanism and tools to transcribe the "sense of being there" [START_REF] Gravier | Adaptive System for Collaborative Online Laboratories[END_REF]. In order to achieve these goals, some improvements are proposed in the BIFS standard. The runtime mode also adapts HCIs of online engineering client applications based on the user's activity. When a widget of the interface is activated, the system proposes to the operator the next possible actions. The aim is to limit the number of human errors, as well as to increase the efficiency( t i m es p e n t ,q u a l i t y , etc.) of the process.

The second mode concerns the set-up of remote experiments. It is basically composed of authoring tools that enables the creation and the deployment of new online experimentations. It creates the remote scene description,basedonmultimediascene descriptions and enables business rules experts to express sequences of actions in HCIs.

Software Chain

In order to propose a framework for building remote experiments using augmented reality techniques, we identify some factorable elements ofthesoftwarechain,presented in section 1.2.2 and illustrated in figure 3.1. In this section, we study the possibility to factor the links of the software chain.

Devices drivers and Processing Middleware

The link between the local computer and the device entails themasteryofstandards and of access protocols. This is complex due to a) the heterogeneity and the number of technologies and b) the number of material interfaces allowing interfaces with devices : PXI, USB, GPIB, etc.

As presented in section 1.2.2, some efforts were made in ordertoproposecommon drivers through standards. Those drivers define classes of devices, in order to ease interoperability between devices and to decrease the development time and cost. The Virtual Instrument Software Architecture 1 ,a l s ok n o w na sVISA,i sa nA p p l i c a t i o n Programming Interface (API) widely spread in the instrumentation area. This standard was developed following the Standard Instrument Control Library from Agilent Technologies. VISA was the starting point of the IVI standard.

The Interchangeable Virtual Instrument standards 2 (IVI), define instrument classes such as Oscilloscope, Spectrum analyzer, Switch, etc., in order to provide interchangeability between instruments of different constructors. The hardware functions, once designed for a class of instruments, are fully compatible with other instruments of the same class, assuming they are IVI compliant. Those drivers also provide simulation modes, enabling to simulate the device when it is not accessible. Unfortunately, those standards suffer from one limitation added to the previously announced ones [START_REF] Landragin | [END_REF] :

-" T h ea d d i t i o n a ls o f t w a r et h a te n t a i l st h ea d o p t i o no ft h eI VI standard is likely to slow down the system speed". We can add that a lot of online engineering solutions are from "home made" developments. We cannot guarantee the standards are respected. Yet,t h i sr e p r e s e n t san o nnegligible number of remote laboratories in the reality.

Ac om pl etegeneri ca r c hi tec tur ef ordev i c eswoul db ev erydi fficult to achieve. Almost systematically, a dedicated software development is needed in order to perform this link. The proof of concept developed in this thesis uses a spectrum analyzer FSL 3 from Rhode & Schwarz, which is fully IVI-compliant and was lent by the company in order to conduct our experiments. The IVI foundation proposes an implementation of the IVI drivers through Windows libraries. Although limited, the IVI standards seem the best compromise for the development of our proof of concept.

Message Middleware

We now focus on the middleware. The middleware is in charge of conveying the commands from the remote user to the device but also of giving ab e h a v i o ro ft h e collaborative platform, i.e. conveying the messages from the devices to all the users. Two different approaches exist in order to realize a middleware : Object-Oriented Middleware (OOM) and Message-Oriented Middleware (MOM). The difference between those approaches relies on the remote service access methods. OOMs instantiate a local representation of the remote object and use it as the remote object. On the contrary, MOMs do not directly expose access methods to remote objects but rely on a messaging mechanism in order to exchange messages containing operationstoexecute.

As Bernstein [START_REF] Bernstein | Middleware : a model for distributed system services[END_REF] proposes, a middleware (OOM or MOM), can be perceived as a layer between the application layer and the operating systems, due to its conveying mission. The middleware is then an essential pieceinalldistributedapplications. Our framework for remote operation is necessarily adi str i b u teda r c h i tec tu r e, since remote users are not connected using the same terminal.W eneedtool sa ndm echanisms to ensure the messages' integrity.

From our point of view, a middleware, as a basis of collaborative architectures, must have the following features :

-A s y n c h r o n i s m .As y n c h r o n o u sm i d d l e w a r ei sc h a r a c t e r i z e dby a blocking call for the message reception method. As no response is received yet, the program process is waiting. An asynchronous middleware allows to keep control on the application even if the message is not yet received. -R o b u s t n e s sa n ds c a l a b i l i t y .T h er e m o t ec o n t r o lo fd e v i c e sdoes not limit the number of users connected to the experiments. -S c h e d u l i n gw a r r a n t y . T h em i d d l e w a r en e e d s t oe n s u r e t h a tm essages are received in the order they are emitted. This is a compulsory feature of online engineering middlewares, since the order of commands on the devices is crucial for the good behavior of experiments, even for the integrity of the system.

Human Computer Interface

We saw in the previous chapters that the Human Computer Interface used in online engineering and industrial maintenance are not enough interoperable and especially not for augmented reality applications.

In order to propose a generic HCI for online engineering we propose the following features :

-A high fidelity of the representation of the device environment.

-I n t e r a c t i v i t ya sc l o s ea sp o s s i b l et ot h ed e v i c ei n t e r a c t i vity.

System at runtime

The overall architecture of the system at runtime we propose is illustrated in figure 3.2. An IP camera is capturing the remote scene and sending an MPEG-4 AVC stream to the middleware, hosted on an application server. The middleware computes and delivers the interactive multimedia to the end user. Users send commands to the middleware through the multimedia HCI. Depending on the whisper,thestreamisadapted and the scene is updated on the users' HCI. The interpreter is responsible for sending the corresponding hardware fonctions.

Conclusion

In this section, we have defined the factorable elements of the software chain. This chain can be divided into three main components : the driver instruments (the processing middleware), the message middleware and the HCI. Driveri n s t r u m e n t sa r er e sponsible for sending hardware messages to the device, corresponding to the messages sent from the HCI. The message middleware is responsible for conveying messages between users and the device and should allow a collaborativec o n t r o lo ft h ed e v i c e . The HCI is the link allowing remote users to interact with devices. The HCI should be accessible from any kind of terminal and should use mixed reality techniques in order to propose a better understanding of the remote apprehended scene. We can notice that this division enables to support the four main contributions we try to answer in this thesis. The Multimedia Builder (thanks totheset-upofthesystem) constructs the multimedia scene descriptions of the remote scene using augmented reality techniques. The Commands Interpreter and the Notification Block enable the collaborative control of the device. The Whisper module is the engine responsible for the adaptation of the interface. The combination of the Commands Interpreter and the Multimedia Builder enable to create a scalable system able to control a wide range of devices.

This section has introduced the overall architecture of the system proposed in this thesis work. The next section presents the multimedia scene reconstruction as well as the multimedia delivery chain to the end users.

Augmented Reality

Augmented reality is the combination of real and virtual objects in a common environment. As presented in previous chapters, three fundamental rules must be respected in order to build an effective augmented reality application -H a n d l et h ec o n s t r a i n t so fr e a l -t i m ea n di n t e r a c t i v i t y . This entails computing a proper environment (a 3D environment) as well as disposing reference objects in order to satisfy the first two constrains.O fc ourse,whenb ui l di ng remote laboratories, the device itself will be the referenceobject.

In this section, we present how we reconstruct the instrumentf eedb a c k,seena sa natural video for the end-user and integrated in the multimedia interface. After analyzing the environment of the remote device, we present the algorithms employed to determine the device identification and its pose estimation inthescene.

The multimedia delivery chain is illustrated in figure 3.4 and serves as the reference in the following subsections.

The IP camera delivers encoded video. The processing block identifies the instrument present in the video stream by comparison with availabledevicesinthedatabase and computes the position of this instrument in the images. The instruments storage in the database is based on the instrument's image features. The processing block also adapts the video by cropping the region of interest, which in general corresponds to the device's screen. Let us note that the image processing operations in the processing block are effectuated in the non-compressed domain, therefore a video decoder and encoder are integrated. Once the instrument is recognized, thec o r r e s p o n d i n gg r a p h i c a l interface is streamed using the broadcaster block.

Scene Analysis

Figure 3.5 represents the spectrum analyzer we have used in order to develop our proof of concept.InordertoconstructthemultimediaHCIoftheinstrument,the different kinds of existing regions in the scene need to be identified. We encountered 3 types of regions for our use case :

-T h ev i d e or e g i o n sw h e r eo b j e c t sa r el i k e l yt oc h a n g ed u r i n gtime, corresponding to the feedback from the instrument (the display of the instrument in figure 3.5 for instance). -T h ei n t e r a c t i v er e g i o n se n a b l i n gf u n c t i o n a l i t i e so ft h ei nstrument, (the buttons in figure 3.

for instance). -T h eb a c k g r o u n d( s t a t i cu n c h a n g e dp i x e l se l e m e n t so v e rt h esession).

In order to limit the bandwidth when transmitting the multimedia, only the video region is transmitted. For this kind of devices, only the display is a relevant video region. All the others regions are computed using graphics or static pixels. In order to improve the interoperability between terminals we choose multimedia scene descriptors to present the augmented scene. Based on our review on multimedia scene descriptions, MPEG-4 BIFS is the better compromise in order to deliverr em otem i xedr ea l i ty FIGURE 3.5 -A Spectrum Analyzer. applications.

Device Identification and Pose Estimation

The processing block presented in figure 3.4 decodes the compressed video delivered by the IP camera. It then identifies the device in the video stream and computes its pose estimation. The video feedback is then extracted and encoded, broadcasted to the remote users. In this subsection, we present how we compute the identification and the pose estimation of the device in the video stream. The first section presents the general principle. The second paragraph presents the camera calibration, a must phase in every image processing phase. The third part explains how we determine homographies that help computing the 3D environment needed for mixed reality.

General Principle

To clarify why we need to recognize the instrument and to calculate its position in the video stream, let us consider the scenario of Steve, Heather and Jimmy, defined in section 1.2.4.

Steve streams the video of the instrument to our system with his smartphone. The system identifies the instrument and delivers the proper interface to Heather and Jimmy. Since Steve's hand is shaking a little, the video is notstable,sothesystemkeeps calculating the instrument positions in order to crop the good region of the stream to broadcast it to the students. They can then realize the remotee x p e r i m e n t a t i o nw i t h as t a b l ev i d e of e e d b a c kf r o mt h ei n s t r u m e n tb ya c t i v a t i n gw i dgets in the multimedia interface. While the marker-dependent methods offer important robustness and efficiency in computational terms as compared to the marker-free methods,thelatterdoesnotcompel aprioriintervention in the scene. Such markers would also entail a very high calibration phase because the system would have to know all the elements' relative positions and orientations in the scene from the marker.

All the devices from the database have associated descriptors. When the camera captures a device, the descriptors are extracted and compared with those in database. Due to the computation time and the real-time constraint, thedesc r i ptori sc om pu ted for one image out of two.

Once the device is identified, local features are used in order to calculate the pose estimation. The local feature points detected are matched with the local feature points of the reference image stored in the database using the Fast Library for Approximate Nearest Neighbor [START_REF] Muja | Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration[END_REF]. It is then possible to estimate thepositionoftheinstrument in the video stream thanks to the Zhang algorithm [Zhang00], [START_REF] Zhang | Estimating Projective Transformation Matrix (Collineation, Homography)[END_REF]. Knowing the position of the device in the video stream allows us to crop thefeedbacksoftheinstrument, encode and stream them to the final user with the whole interactive multimedia content. In the spectrum analyzer scene, the feedback area isrepresentedbythedisplay of the device, as illustrated by figure 3.6. This allows limiting the bandwidth, since only the necessary area is a streamed video. The rest of the multimedia scene is construc-FIGURE 3.7 -Checkerboard with detected corners. ted using graphics. We highlight the fact that the device is identified frequently. The importance of this feature is discussed in section 3.5.2.3.

In Chapter 5, a test benchmark of local features and their usage for remote laboratories are presented.

Camera Calibration

Before being able to solve the pose estimation of the device inthescene,weneedto determine camera parameters. These parameters are presented as a matrix, commonly know as the camera matrix.I ti sa3 ×4m a t r i xt h a td e s c r i b e st h em a p p i n go fp i n h o l e camera from 3D points in the real world to 2D points in a image.

In order to compute the camera matrix, the algorithm we describe here needs the coordinates of known points of a scene taken from different point of view. Zhang recommends using multiple images of a model whose characteristics we know. A checkerboard seems the best choice, since we know the size of squares.

In order to detect squares corners, we based our realization on the Vezhnevets algorithm [Vezhnevets06]. This algorithm first computes a binary version of the image using an adaptive threshold. Black objects squares are extracted from this binary image and their coordinates are extracted. Figure 3.7 is an image of thet a k e nc h e c k e r b o a r da n d its detected squares. The Zhang algorithm allow computing the camera matrix using homographies and detected points. Let M int be the camera matrix.

Let B and b matrixes such as :

B = M -t int • M -1 int b = B 11 B 12 B 22 B 13 B 23 B 33 t
Let h i be the matrix representing a homography. We have :

h i = h i1 h i2 h i3 t
then :

h t i • B • h j = v t ij • b with : v ij = h i1 h

j1 h i1 h j2 + h i2 h j1 h i2 h j2 h i3 h j1 + h i1 h j3 h i3 h j2 + h i2 h j3 h i3 h j3 t

if n images are captured, then the last equation can be shorten as :

V • b = 0 V is a 2n × 6dimensionmatrix.
Thesolutionofthislastequationisgiven by the eigen vector associated with the smallest eigen value of the V t • V vector. When b is estimated, all the intern matrixes can be determined. We finally obtain the camera matrix estimation using a minimization of the previously determined matrixes.

Homography Between Planar Model and its Image

An homography is a transformation that allows the correspondence of pointsfrom a plan to another, as illustrated by figure 3.8. It is represented by a 3×3m a tr i x .Ith a s 8degreesofliberty(onecomponentofthematrixisonlyanarbitrary scale factor). We then need four pairs of corresponding points to determine an homography.

The homography is determined using the RANdom SAmple Consensus,RANSAC [START_REF] Fischler | Random sample consensus : a paradigm for model fitting with applications to image analysisa n d automated cartography[END_REF]. RANSAC is an iterative methods that helps classifying items of a statistical population. We choose 4 pairs of points determined by the FLANN,FastApproximate Nearest Neighbor, algorithm. FLANN determines corresponding points in high dimension vectors using a hierarchical tree. When the 4 points are determined, we compute the homography using them. We determine a distance d for each correspondence. This operation is repeated until obtaining the best homography.

The device pose estimation is an important step, since it allows to determine the position of the video feedback streamed to the remote user. Let P 3D,i,i=1..4 be points of the device in a reference image in homogenous coordinates and p 2D,i,i=1..4 the corresponding points in images captured by the camera in homogenouscoordinates.Thanks to the Zhang algorithm that is robust and simple to implement,f o re a c hP 3D,i,i=1..4 of the reference image, we can write :

s • p 2D,i,i=1..4 = M int • M ext • P 3D,i,i=1..4
with s being a scale factor.

M int and M ext represent respectively the camera matrix and the extrinsic matrix of the system. M ext can be represented by : M ext = R, T with R = r 1 r 2 r 3 R and T are the rotation and translation matrixes mapping the objectl a n d m a r k (points coordinates in the reference image) to the camera landmark (points coordinates captured by the camera). M int is defined as follows :

M int =   α u 0 u 0 0 α v v 0 001   u 0 v 0
being the coordinates of the principal point, i.e. the centeroftheimageinthe camera landmark. α u and α v are scale factors following the u and v axes of the image.

We have constructed our prototype using a spectrum analyzer.Thisdevicehasa2D font. When representing its coordinates in a mathematical model, we can easily write that their z-coordinates are null. Written in the homogenous coordinates, we can write :

  s • u s • v s   = M int • r 1 r 2 r 3 T •     X Y 0 1     = M int • r 1 r 2 T •   X Y 1  
Ap o i n tP 3D,i is related to its image p 2D,i by an homography H defined with a scale factor by :

s • p 2D,i = H • P 3D,i
with :

H = λ • M int • r 1 r 2 T
The homography being a 3×3dimensionmatrix,wecanwritethat:

h 1 h 2 h 3 = λ • M int r 1 r 2 T
This last equation can be solved with an analytical method, knowing all the parameters of the system.

r 1 = λ • M -1 int • h 1 r 2 = λ • M -1 int • h 2 r 3 = r 1 • r 2 t = λ • M -1 int • h 3 with λ = 1/N(M -1 int • h 1 )=1/N(M -1 int • h 2 ) where N(X)isthenormoftheXvector .
We can then determine points coordinates in the scene of pre-defined points in the image reference. For the spectrum analyzer, only the displayisthefeedbackofthedevice. This method is used to determine the position of the display and crop it in the video stream. Only the video feedback as natural video is broadcasted to the remote user, as presented by figure 3.6.

Conclusion

In order to create an augmented view of the remote scene, we need to create a computed 3D environment of the scene. We identify 3 areas in the remote scene, a feedback area, a background and an interactive area (encompassing interactive elements).

The device is first identified in the video streamed delivered by the IP camera using image features and descriptors. Then, an homography betweent h e3 De n v i r o n m e n t and a reference image is computed in order to determine the pose estimation of the device. Homographies are determined thanks to Zhang algorithms.

Handling Collaboration

Collaboration generally refers to individuals or organizations working together to address problems and bring results that are not easily achieved by working alone. Collaboration entails the sharing of the same resource at the same time.

To ensure collaboration, a high level of trust must be achieved and frequent communication has to take place. Thus, collaborative systems should not only allow the sharing of a resource between the different protagonists, but it should also maintain group awareness. As said in [START_REF] Gutwin | r o c e e d i n g s o f t h e ACM Conference on Computer-Supported Cooperative Work[END_REF], "Group awareness is the understanding of who is working with you, what they are doing, and how your own actions interact with theirs".

In remote control systems, teachers are concerned by group awareness since it eases the management of student's achievement. To manage collaboration and thus group awareness in remote laboratories, different approaches areu s e d .I n[ T u d d e n h a m 1 0 ] , Tuddenham et al. present a shared visual workspace for tabletop. Their system reproduces protagonists' arms when working on the remote tabletopandspeakerphonesare used to ease communication between the users. Pereira et al. in [START_REF] Pereira | Integrating Mixed-Reality Remote Experiments Into Virtual Learning Environments Using Interchangeable Components[END_REF] propose a "learning-material module". Depending on the logged user, different kind of operations are allowed. Different "learning modes" are suggested by the system, such as "active learning" or "team learning".

Communication tools, such as video, audio, chat and shared workspaces are not the only key issue when designing collaborative remote laboratories, especially in distance learning curriculum, as stated in [Lowe09] : to support supervision, they decided to display multiple devices at the same time, like "a tutor beinga b l etow a l ka r ou n dth e laboratories and observe each student or group of students".

Previous works in the LT2C laboratory made by Gravier et al. [Gravier06] proposed to enforce the group awareness with a communication tool based on a chat, VoIP and a "group awareness enforcement service" which provides atele-presenceindicator and allocates a color to each user. When an interaction occurs, the widget's color of the front of the HCI of the remote laboratory changes to the color of the user responsible for this interaction. In addition to those tools for computing the group awareness, a system based on ontologies and semantic rules is applied to manage the access to the resources [START_REF] Gravier | Adaptive System for Collaborative Online Laboratories[END_REF]. This system was also adapted to run inavirtualword,W onderland [START_REF] Fayolle | g s o f t h e 1 0 t h W S E A S i n t e r n ational conference on applied informatics and communications, and 3rd[END_REF]. Virtual worlds can help increasing group awareness, since all users are virtually present in the virtual world, which already provides group awareness [21]. In addition, "the spatial layout of the 3D world coupledwiththeimmersiveaudio provides strong cognitive cues that enhance collaboration" [START_REF] Fayolle | g s o f t h e 1 0 t h W S E A S i n t e r n ational conference on applied informatics and communications, and 3rd[END_REF].

Different approaches can be considered in order to achieve group awareness in collaborative applications. Communication tools are the mandatory components, but other tools can be added in order to enforce collaboration. Anyhow, in order to handle FIGURE 3.9 -Publish/Subscribe paradigm. collaboration, the system, both on server and client side, should be able to send and receive messages from different users. The following subsections will present how we handled collaboration at the level of the middleware and in the multimedia HCI.

Middleware

Apossiblewayofbuildingmiddlewareforremotelaboratories is using RPC 3 .RPC enables the remote release of operations on an application server and to manipulate objects (in the Object Oriented Programming paradigm).

Because of its multi-platform capacities, RMI 4 was studied. RMI adds object orientation to RPC. It is issued from the Java programming world. However, RMI is synchronous, thus until the client receives data, the executionofthepr ogra mi sb l oc ked. Moreover, RPC was originally constructed for point-to-pointcommunication,andthen is not adapted for multi-user applications.

An alternative to RPC is the Java Messaging Service (JMS) standard. JMS is an asynchronous MOM, following the publish/subscribe paradigm explained by [START_REF] Th | The many faces of publish/subscribe[END_REF] and illustrated in figure 3.9. The procedure is the following :

1. Clients subscribe to a shared discussion channel.

2.

A publisher emits a message on the channel.

The broker redistributes the published message on the channel to all users.

MOMs provide important features when designing collaborativer emotelaboratories: -G u a r a n t e e so fs c h e d u l i n gt h a te n s u r et h a tm e s s a g e sa r er e c eived in the order they were emitted. We propose to reproduce the publish/subscribe paradigm for multimedia services. The will to keep a loose coupling between the client and the server, pushed us to the creation of two kinds of channels for interaction. Both channels rely on the publish/subscribe paradigm. The first channel is the commands channel. Commands are sent to the middleware for processing. The second channel is the interface channel. This channel is responsible for updating part of the BIFS scene graph. It is one of the main components to present the group awareness in the scene. Figure 3.10 illustrates how collaboration is handled within our framework. All these components are part of the Command Interpreter presented in section 3.1.1.

The Command Channel

When a widget of the multimedia interface is activated, a message is sent to the middleware. This message contains information required forthemiddlewaretoconvey the command, such as user's name, widget ID, the interaction's nature, etc. The message is formatted as JavaScript Object Notation (JSON). The middleware serializes it in the message queue. A message broker retrieves messages according to the scheduler and pushes them to the interpreter. The interpreter processes them in order to send to the device the corresponding hardware function.

The Interface Channel

The same kind of mechanism is employed for the Interface Channel. A JSON message containing the widget ID, user's name, etc. is publishedinadedicatedqueue.The broker, according to the scheduler, pushes the message to the notification center. This module then pushes to users the scene updates.

The next subsection will present the solution we have adoptedi no r d e rt or e a l i z e the notification center and to create the group awareness on theclientside.

Collaborative Interactive Multimedia Interface

Proposing a middleware that supports collaboration is necessary but not yet efficient for users to sense the group awareness. In this subsection we present how group awareness is supported in our system. This work is the follow-up of [START_REF] Gravier | Coping with collaborative and competitive episodes within collaborative remote laboratories[END_REF] ; this previous work did not rely on multimedia but on a stand-alone Java application. Group awareness functionalities are a chat and VoIP tool for communication and changing colored based widgets for seeing "who is doing what".

For handling group awareness, each user receives a differentcolor .Theclientsends widget information (user login, user color, widget ID) with the WebSockets protocol. The WebSocket server broadcasts this message to all the connected users, and the scene updates are handled in the ECMAScript code corresponding to the message received. The widget changes its color for every connected clients, depending on the user. This feature allows each user to know what action is made on the interface and who made it. Atele-presenceindicatoristhennecessaryinordertorepresent the association between auserandacolor .

In order to improve group awareness, we ease the communication between users by using an audio communication tool. Receiving multimedia content (audio, video, BIFS) and streaming audio content to the server at the same time all over the session could require too much CPU capabilities, especially on mobile devices. In order to limit this, we have implemented a walkie-talkie tool, which offers audio communication without continuously acquiring audio data on the client sideandstr eamsthemtothe middleware. The user experience may not be optimal in this situation but the CPU consumption is limited.

Implementation details of the BIFS scene are presented in Chapter 4.

Adaptation of the Human Computer Interface

In the industrial maintenance area, learning and being able to reproduce the full catalog of maintenance operations can be a difficult task. The complexity of this task is also rapidly exploding since the number of devices operated by a single human operator increases when these devices can be operated remotely. Computer-Aided Maintenance (CAM) systems must provide a formalism to express suchproceduresinorder to ease the operators' actions and increase their productivity. From our point of view, on the HCI side, the CAM systems must also provide indicationsonwhichactionhas been made and "whisper" to the operator the next possible actions in the maintenance procedures. The aim is to limit the number of human errors, as well as increase the efficiency (time spent, quality, etc.) of the maintenance process. We propose, as part of the presented framework, to express maintenance proceduresbasedonSemanticW eb technologies. A rule-based engine provides the context logic to indicate the operator which next possible actions can be made according to the current procedures. In this subsection, we present the architecture of what we call the "whisper module" and its uses.

Whisper Module

The whisper module is illustrated in figure3.11. The procedureexperteditsthesequences of actions that are serialized in the Knowledge Base (KB). The operator sends actions to the middleware through the HCI's widgets. The Processing Blocks (PB) interprets the actions. It requests the KB which the current possible sequences are, based on previous inferences. PB also places the new action as the current one in the KB. The Reasoner Bloc (RB) infers which the next possible interactions are, according to sequence rules and the Notification Block (NB) pushes the results to the HCI. We take the following example : Let A i be interactions made on a HCI through widgets. Let S j be sequences of interactions pre-determined and serializedinthesystem.S j is an ordered list of actions Ai. Let's assume that :

S 1 = {A 1 , A 2 , A 3 } (3.1)
and

S 2 = {A 1 , A 2 , A 4 }.( 3 . 2 )
The operator sends the action A 1 .T h es y s t e mw h i s p e r st h a tt h en e x tp o s s i b l ea c t i o n recorded is A 2 in sequences S 1 and S 2 .ThewidgetcorrespondingtoA 2 is highlighted in the HCI. The operator then sends A 2 .InS 1 and S 2 ,thenextactionsarerespectively A 3 and A 4 ,andthecorrespondingwidgetsarehighlightedintheHCI.

Expressing Sequences.

By sequences we imply an ordered list of actions on an HCI that compose a procedure. The usage of Semantic Web technologies to describe sequences of actions for e-Maintenance is motivated by the interoperability they provide, the detachment from specific programming language and mainly the reasoning they provide through rule languages.

RDFS and SWRL reasoners both follows the Open World Assumption (OWA), i.e. the assumption that the truth-value of a statement is independent of whether or not it is known by any single observer. Both rule engines propose a Human readable syntax for rule editing, which is in the following form :

antecedent => consequent

As a consequence, those criteria cannot be used to choose one solution over an other. RDFS proposes in its core components rdfs :Container such as rdf :Alt, rdf :Bag, rdf :Seq. They are used to describe groups of things and are summarized in Table 3.1.

OWL has no support for ordering. rdf :Seq is not illegal but it has no logical semantics accessible to a DL classifier. Some propositions have been made in order to

Class

Container Description rdf :Bag Bag container indicates that the container is intended to be unordered rdf :Seq Sequence container indicates that the numerical ordering of the container membership properties is intended to be significant rdf :Alt Alternative container indicates that typical processing will be to select one of the members of the container. The first member of the container, i.e. the value of the rdf :_1 property, is the default choice TABLE 3.1 -RDF containers implement sequences such as [START_REF] Drummond | Putting OWL in Order : Patterns for sequences in OWL[END_REF]. But they are not a natural way to deal with sequences in OWL whereas rdfs :Container can express sequences with its core components.

Considering the better expressiveness of RDFS for this kind of problem, we made the choice of representing the sequences using RDFS/RDF.

Use Trace

Back to our use case, Bob has now finished to perform maintenanceoper a ti on son Alice's device. Let's T be the trace of Bob's actions. An example of T could be :

T = {A 3 , S 2 , S 5 , A 3 , A 3 , S 1 ,...} (3.3)
where A i is an action on the CAM system's HCI and S j is a sequence of actions recorded in the system. Let's assume that in the Business rules of his company :

S 5 ≡ S 3 (3.4)
and Card(S 5 )

Card(S 3 ) (3.5)
The system should notify Bob that he performed one or several useless and/or suboptimal actions in the maintenance procedures. In order to realize it, the full use's trace of Bob has to be serialized in the system before being analyzed.

Since the system could be used by somebody else than Bob, the use's trace needs to contain the name of the operator. The system could also be collaborative, i.e. Bob could be using the system at the same time as James who has otherc om petenc i esf or the task. The analysis of the trace can thus give details on thefullremoteoperationbut also details on a single operation. FIGURE 3.12 -MPEG-4 BIFS graph example.

Multimedia Delivery

Multimedia Composition using MPEG-4 BIFS

BIFS was introduced in section 2.1.4.1, therefore this section presents in more detail its features. MPEG-4 BIFS defines two major components : visual and audio. The audio component can be a mix between various audio sources and the visual component can be 2D, 3D or a mix between 2D and 3D objects. Using MPEG-4 BIFS, one can represent as c e n ea sah i e r a r c h i c a ld y n a m i ct r e e ,w h e r el e a v e sa r ea u d i ovisual objects, as illustrated in figure 3.12. This property is derived from VRML. When designing complex behaviors, branches of this tree can be created, modified or deleted as demonstrated by Tran et al [START_REF] Sm Tran | New proposal for enhancing the interactive capability in MPEG-4[END_REF]. Currently, more than 200 nodes are supported and implemented in the BIFS standards and every node can be assigned with a "name"inor dertoidentify it. Each node can be characterized with fields that define the nodes, like attributes in XML tags.

In order to support both 2D and 3D scenes, BIFS defines the Layer2D and Layer3D nodes. This allows having a 2D scene rendered on top of 3D scene.

The Shape node is defined in BIFS in order to include a visible entity inside the scene, using its main fields appearance, which specifies the visual appearance of the entity (texture, color, etc.), and geometry, which specifies the shapes of the entity (square, circle, complex shapes, etc.).

BIFS is also capable of referencing other scenes inside the tree, using the Inline nodes. It is then possible to include scenes in a BIFS scene inside another BIFS scene. This feature eases adding/deleting/changing content. BIFS derives nodes from VRML in order to support interactivity. I n t e r a c t iv it y is achieved through Sensor nodes such as TouchSensor, TimeSensor or Proximity Sensor. BIFS also embeds ECMAScript nodes. This feature brings the possibility to manipulate the media objects of the scene using JavaScript. It provides the ability to store state information in local variables, such as mouse position in thesceneforinstance.

In order to propagate events, BIFS uses the VRML concepts of ROUTEs.Theyar e connections of values of one field to another. The syntax of a route is the following : ROUTE <OutNodeID>.outFieldName TO <InNodeID>.inFieldName meaning that values of the field "inFieldName" of the node "InNodeID" is assigned with the value of the field "outFieldName" of the node "OutNodeID". MPEG-4 defines channels named Elementary Streams (ES). These channels contain afullyorpartiallyencodedrepresentationofasingleaudio-visual object, scene description, or control information. ES are distinguished using Object Descriptors (OD) stored in the ES. OD is a "structure similar to a Uniform Resource Locator (URL), containing pointers to ES" [START_REF] Signes | MPEG-4's binary format for scene description.v o l .1 5 ,n o . 4 -5[END_REF].

The scene description and the stream description are considered as ESs and are fully separated. In order to point to object descriptors, thes c e n ed e s c r i p t i o n su s eO D identifier, the OD_ID.T h i sa l l o w st om u l t i p l e xs o m eo t h e rm e t a -i n f o r m a t i o ni nt h e scene, such as images, video, etc.

The transport layer of MPEG-4 systems is "media unaware and delivery aware" [START_REF] Signes | MPEG-4's binary format for scene description.v o l .1 5 ,n o . 4 -5[END_REF]. No specific deliver layer is defined by the standard, and MPEG-4 media can be transported on existing transport layers such as RTP, RTSP, MPEG-2 TS and more recently Hypertext Transfer Protocol (HTTP) streaming. MPEG-4 systems and MPEG-4 BIFS scenes provide all the required tools in order to create multimedia remote laboratories HCI using mixed reality. The scene descriptions enable a high level of interoperability between terminals. Every kind of media object can be inserted in the dynamic scene and can be multiplexed inside a single format, the MP4 format, without having to deal with multiple formats. Moreover, MPEG-4 provides the ability to use classical transport protocol in order to deliver the multimedia content, like any kind of media. Other existing scene descriptors are designed in a way that the whole scene has to be downloaded before being rendered by the terminal.

Multimedia Delivery

In the previous subsections, the MPEG-4 BIFS standard was presented. This standard enables to create the multimedia content without havingthehindranceofdealing with multiple formats. The multimedia HCI delivered to the remote user is built using MPEG-4 BIFS features. We have also presented how to estimate the pose of the device in the video stream delivered by the camera and how the video feedback of the remote device can be presented. In this subsection, we present the streaming technologies in order to deliver the whole multimedia content to end users.

The term streaming designates a principle for real-time (or slight delay) delivery of multimedia content. It is in opposition to the download delivering where the whole multimedia content has to be fully downloaded before being rendered. The streaming is still a download of multimedia content because of the data exchange between the client and the server. Data are downloaded continuously in RAM and are analyzed on the fly by the computer and quickly transferred to the media player for rendering.

Unicast, Broadcast and Multicast Streaming

Unicast is a point-to-point connection protocol between the client and the server. A connection is established for each and every client requesting the multimedia content, as illustrated by figure 3.13. In order to compute the bandwidth, all the bandwidths between all the clients and the server are added. Unicast is useful for on-demand media services. Each client is able to consume delayed media services.

The Unicast Broadcast is designed to consume real-time media services. Media content is transmitted on the media server, as illustrated by figure 3.14. The server sends packets that all terminals in the network receive. In order tocomputethebandwidth, all the bandwidths between all the clients and the server are added.

The Multicast transmission is a routing process that consists in sending a packet to many clients simultaneously. It lightens the network's load. Packets are sent to a group without necessarily knowing the clients. Data is sent to all subscribers of the group. It implies the modification of the networks routers in order to access the multicast transmission.

The multicast streaming seems the best option in order to deliver multimedia content, especially for multiple users. However, multicastentailsmasteringallnetwork elements in order to deliver media. Since it is impossible to control all the network items in our case (think of a student in a foreign country), we decided to use a broadcast solution in order to deliver the multimedia content to the end user.

Streaming Protocols

The streaming methods mentioned above rely on streaming protocol to ease the transmission of data.

The Tr a n s m i s s i o n C o n t r o l P r o t o c o l (TCP) is a protocol for sending packets on a network. A dialog between the client and the server is established and copies of packets are kept on the server. The server waits for client agreement for sending data, thus if a packet is lost on the network, the server can send the copy to the client.

The User Datagram Protocol (UDP) protocol does not make a dialog between the sender and the receiver. If a packet is lost, the sender will not know it. The advantage of using UDP is its efficiency for real-time applications, if the loss of packets is acceptable.

The Real-time Transport Protocol (RTP) is a protocol for streaming real time data over the Internet. Usually, RTP packets are sent via UDP. RTP does not address resource reservation and does not guarantee Quality of Service for real-time services. For each multimedia session, a RTP Session is established, using the Session Description Protocol (SDP). The SDP file provides information about the RTP packets,suchascodec, timestamp, etc. underlying data delivery protocol since RTSP doesn't deliver the data itself. RTSP is a verbal protocol, it uses messages in order to control the media.

We presented here the streaming protocols. These protocols are included in the delivery of the multimedia content.

Delivery

The broadcaster block illustrated in figure 3.16 is realized using the Darwin Streaming Server, an open source server able to deliver multimediac o n t e n tv i aR T S Pa n d able to cast MP4 files. It is used as a relay of the RTP packets transmitted by the RTP packet generator (RPG). The BIFS scene contains a link to the SDP file describing the RTP packets delivered by RPG.

As highlighted in section 3.2.2.1, the device is identified frequently. This allow us to change the BIFS scene depending on the device captured by the camera. Changing the device interface is then a simple operation : changing the device in front of the camera. This features strengthens the reusability of the system.

On the client side, the multimedia player is responsible for decoding and displaying the RTP packets of the live video. The end user then just has to connect to the RSTP address of the multimedia content to play and interact with it. RTSP clients are implemented in almost all media players.

Conclusion

This chapter presented our contributions to the online engineering field. These contributions are serialized in a framework for building andr u n n i n go n l i n ee x p e r imentation in the industrial maintenance and e-Learning areas.

We presented how augmented reality techniques are used to compute the 3D environment of the remote scene. We also presented how we extract the minimum video feedback from the stream (the display of the device in our example) in order to limit the bandwidth. We also presented how the interface is modified when a new device is placed in front of the camera.

In order to handle collaboration, we have presented how a middleware can be used on the server side. The will to keep a loose coupling between the client and the server, pushed us to the creation of two kinds of channels for interaction. The first channel is the commands channel. Commands are sent to the middleware for processing. The second channel is the interface channel. This channel is responsible for updating parts of the BIFS scene graph.

On the client side, group awareness is an utmost issue. In order to reproduce it, the multimedia interface needs to present the users' actions to each others. A tele-presence indicator is then necessary. Each user receives a different color and widgets are highlighted with one's color when activated. An audio communication tool is also proposed in order to create real communications between participants.

We have presented how the interface is adapted on the client side depending on the user's activity. We use Semantic Web technologies in order tor e p r e s e n ts e q u e n c e so f actions.

Finally, we have presented how MPEG-4 BIFS is employed to present and create the multimedia content delivered to the end user. The broadcaster block is built on top of the RTSP protocol.

The next hapter of this work presents implementation detailsoftheframeworkwe propose.

When the only tool you have is a hammer, everything begins to look like a nail.

Anonymous

4

Realization and Results

While Chapter 2 presents issues of remote laboratories, Chapter 3 presents the overall architecture of the system resulting from our work. This Chapter details noticeable implementation points of the framework. We also introduce a first evaluation test and present the framework in the form of an open source project.

I N the previous chapter, we have presented the overall architecture of the online engi- neering framework. The framework can be divided into three main components : the driver is responsible for the communication with the device,themiddlewar econveys messages from clients to devices and the multimedia client isresponsibleforreproducing the device's HCI. This chapter presents noticeable implementation points of the proposed framework and is organized into two major parts. The first one explicits instrument drivers construction and choices. MPEG-4 BIFS use for remote laboratories interface design are detailed as well as the ECMAScript API we proposed to enrich theBIFScapacitiesfor remote laboratories. A local features and descriptors analysis is presented to determine the best compromise when tracking a device in the remote scene. We also present the user's interaction procedure modules implementation. The second part presents the overall realization of our system through a first evaluation test and an open source project realization.

Realization and Implementation

The realization of our proof of concept has been constructed using a spectrum analyzer. In this section, we detail implementation points of the proposed system. First, we present how the middleware is realized.Secondly, we presenthowinstrumentdrivers can be implemented. We then explicit how MPEG-4 BIFS features can be employed to construct interactive interfaces for remote laboratories.Afterwardswepresentabenchmark of local features and descriptors. In the last part of this section, we present how semantic Web and RDF technologies can be used to model users' interaction procedures and how they can be analyzed consequently.

Middleware

Due to its multi-platform capacities and the previous expertise of the SATIN team in the LT2C laboratory, the Java programming language was chosen in order to implement the middleware.

An application server, based on the Java Enterprise Edition (JEE) is coupled with the MOM in order to provide mechanisms for :

-U s e r ss e s s i o n .

- -G e r o n i m of r o mt h eA p a c h eF o u n d a t i o n 5 . The choice was made for the Java Open Application Server (JOnAS). The first reason is the previous work conducted with it. The second reason of this choice is the implementation of the JMS APIs through the Java Open Reliable Asynchronous Messaging 6 (JORAM). JORAM can be distributed as a standalone JMS server but it is also packaged inside the JOnAS distribution, ensuring a high level of compatibility.

Instrument Driver

Programming Languages

Ac om pl etegeneri ca r c hi tec tur ef ordev i c eswoul db ev erydi fficult to achieve. Almost systematically, a dedicated software development is needed in order to perform this link. The proof of concept developed during this thesis uses a spectrum analyzer FSL 3 from Rhode & Schwarz, which is fully IVI-compliant and was lent by the company in order to conduct our experiments. The IVI foundation proposes an implementation of the IVI drivers through Windows libraries.

Those libraries are proposed for the following languages : C++, C and Visual Basic.NET, LabView, LabWindows/CVI, MATLAB, Measure Foundry, Visual Basic 6.0 and Agilent VEE Pro. Considering our programming language knowledge, the choice was made for C++ and C .

The middleware we proposed in this thesis is implemented using the Java programming languages. In order to complete the link between the local computer and the local device, we firstly considered using C++ programs through Java Native Interface 7 (JNI). JNI enables developers to access native code (throughC++functionsforexample) using Java functions. The steps in writing and running JNI programs are the following : 4. Write the C++ implementation of the native method.

5. Compile C++ code and generate native library. 6. Run the Java program using the Java interpreter (using java).

This procedure is long and difficult to debug for obvious reasons. In order to facilitate the development of the instrument driver, we decided to move to C programming languages, even if a full driver could have been developed only using C++.

It is not easy to choose programming languages. While some claim that C++ is faster that C ,othersclaimthatC is a more user-friendly programming language. We found that the C syntax is closer to the Java syntax and the IVI documentation is more suited for C developers. Using the C brought the problem of communication between the middleware and the instrument driver, given the different chosen languages. We think that in order to achieve a better loose coupling between the middleware and the device, both parts need to be separated and should exchange messages. Since messages from the client application are stored in a message queue and delivered according to scheduling, there is no need to propose such a component for the message delivery between the middleware and the driver. A simple TCP connection between the two entities exchanging the messages is enough in this case.

Since IVI libraries are only available for Windows platform,t h eC programming language seems to be the easiest solution in order to write instruments drivers. In order to enable middleware and driver communication, a connectioni sr e q u i r e d .W eh o sted our middleware in the same network as the device for our prototype. The access of the middleware is exposed to the Internet (using a public IPa d d r e s s )i no r d e rt o use the platform, as opposed to the device. So, even if the security is not optimal, we constructed a TCP connection between the middleware and the device. In this connection, those two entities exchange messages. The following paragraph presents the messages content as well as driver implementation details.

Driver Classes

When designing the driver program, we choose to implement classes that match best the interactions with the spectrum analyzer. We have identified in the device's HCI modes, sub-modes and validation widgets as presented in figure 4.1. The modes of the spectrum analyzer correspond to the following functionalities of the device :

-F R E Q:r e p r e s e n t st h ef r e q u e n c i e sa c q u i s i t i o nm o d e .T h es u b-modes of FREQ are the frequencies parameters possible to set for the signal acquisition such as "Center Frequency", "Start Frequency", "Stop Frequency", "Frequency Offset". -S P A N:r e p r e s e n t st h ef r e q u e n c ys p a na c q u i s i t i o np a r a m e t e rs. In this mode, it is possible to parameter the "Frequency Span", the "Sweep time", as well as start and stop frequency. -AMPT : represents the amplitude parameters for the drawing oft h ea c q u i s i t i o n curves. Sub-modes of AMPT enable the configuration of the amplitude. Messages between the middleware and the driver component are exchanged using aTCPconnection.Thesemessagesmustcorrespondtothedriver implementation. They are composed of the mode, the sub-mode and the value set by the operator and are presented as a JSON object. The driver component parses messages, determines the mode and the sub-mode and then sends the corresponding hardware function to the device.

Another interesting feature of IVI drivers is their capacityt or u ni nas i m u l a t i o n mode. This enables to run the driver component even if the instrument resource is not available.

Simulation

The IVI drivers offer the possibility to run the driver in a simulation mode. Simulation drivers are "virtual instruments" able to generate simulated data. The code developed can be reused without change when swapping to particular instruments. This feature allows to use IVI drivers for remote as well as virtual laboratories. But this feature is not fully standardized. This entails some differences in the drivers implementations. For example, the driver for the FSL3 spectrum analyzer enable a running driver simulation but every signal delivered from the drivergivesazerovector.Inorder to conduct our tests, a very small simulation mode has been added to the middleware application. This mode is just developed in order to validateourapproach.

Multimedia Implementation

In this subsection, we detail the implementation of the multimedia HCI using MPEG-4 BIFS. In Chapter 3, most of BIFS features for building remote laboratories clients are presented. The GPAC8 project [START_REF] Lefeuvre | Experimenting with multimedia advances using GPAC.I nP r o c e e d i n g so ft h e1 9 t hA C Mi nternational conference on Multimedia, MM '11[END_REF] is a multimedia framework that provides multimedia creation tools and a multimedia player capable of displaying BIFS scenes. The client application is realized using the GPAC framework. GPAC provides the BIFS Text (BT) format, that "is an exact textual representation of the MPEG-4 BIFS scene" in order to ease multimedia scene creation.

The general scene graph of the scene is illustrated in figure 4.2. The BIFS scene is composed of four main components. The Authentication Screen enables the user to log in the platform. This screen is composed of a background image, two text fields and avalidationbutton.

The General Script component includes Script features used for the communication between the client and the server through ECMAScript functions. It contains global variables, mouse positioning, etc.

The General Widgets component includes widgets that are common to all remote laboratories HCI such as the "Presence indicator", the Chat tool, and the "Information Box". This box is necessary for displaying messages in the interface, such as network issues, etc. The Device's HCI component is composed of all necessary widgets to present the device's HCI to the remote user. It includes background image(s), video area(s) and interactive widgets reproducing the interactions with the device.

In this subsection, we firstly presents how identification is achieved in the platform. The second paragraph illustrates how widgets are designed in the multimedia interface. The third paragraph illustrates the realizationo fa na u t h o r i n gt o o lt oc r e a t e remote laboratory HCIs.

Authentication on the Platform

Authentication is needed on the platform to ensure that the user is allowed to interact with the remote device. The middleware and the JEE application server are responsible for hosting user information contained in databases and Lightweight Directory Access Protocol (LDAP) directories.

Authentication can be achieved using two mechanisms : when retrieving the multimedia content with the RTSP protocol and when using the multimedia content. Even if security is beyond this thesis' scope, we need to keep in mind that security is important.

The RTSP protocol provides tools to support authentication through "digest authentication" or "clear authentication". RTSP "digest" authentication [START_REF] Franks | An Extension to HTTP : Digest Access Authentication[END_REF] comes from Web security mechanism. A hash function is applied on the password before sending it over the network. The "clear" authentication consists in retrieving the multimedia content through its URL and including the login and the password inside the URL. In this case, the password is sent in plain text over the network. While digest authentication provides a secured mechanism, it can be difficult to implement this protocol. Implementing digest authentication would entail a communication between the middleware and the RTSP server and a large amount of development ontheDarwinStr eaming Server to enable communication with the middleware has to be considered. We keep in mind this issue, but we decided not to implement digesta u t h e n t i c a t i o n ,b u t rather authentication through the multimedia interface. Ino u ru s ec a s e ,s o m e b o d yi s In order to achieve authentication on the platform, the multimedia client application needs to communicate with the middleware and send the user's login and password. The GPAC project implements a very interesting feature in order to ease client/server communication : Asynchronous Javascript and XML 9 (AJAX). This feature enables to send HTTPRequests to the server. It is implemented as XMLHttpRequest in the BIFS ECMAScript. Using AJAX, we can send the entered user's login and password to the remote server for processing. Again, this entails sending user's login and password clear over the network. We could improve the AJAX implementation of GPAC to support the HyperText Transfer Protocol Secure (HTTPS), which is a composition of HTTP and Secure Sockets Layer (SSL), but it is beyond this thesis' scope.

On the client side, an authentication screen is proposed to the client. This screen is illustrated in figure 4.3. The middleware is responsible for getting the user's login and password and find the matching in the databases and LDAP directories. When the log in is successful, the device HCI is presented.

Widgets

A Widget is a contraction of the words window and gadget.I td e s i g n a t e sag r aphical interface component such as button, list, scroller, etc. The BIFS graphics and interactive capacities enable to design and use widgets in remote laboratory HCI. In this paragraph, we detail the implementation of noticeable widgets. It is important to notify the reader that the plan's origin in the BIFS system is located in the middle of the A button widget is an interactive component that is characterized by its touchable feature, its position, its appearance, its color, its labels( a n dt h e i rp o s i t i o n ,c o l o r ,f o n t style, value, size). In order to design buttons in the BIFS interface, we propose the BIFS pseudo-graph represented by figure 4.4. Button are represented as Tr a n s f o r m 2 D nodes having three children, one for handling the button appearance, one for handling the button's label appearance and the last one for handling Script values. The Transform2D node is characterized by its translation field that gives its coordinates in the scene. Both button and label appearance nodes are illustrated in figure 4.5. Buttons are coupled with BIFS TouchSensor,inordertoreproducethe"touch"interactionofthedevice's real buttons. All buttons receive an identifier through their Script node. This identifier helps representing the interactions modes of the spectrum analyzer in the same way it does for driver developments. In order to enable interactivity, we link those buttons using BIFS ROUTEs with the General Script node in order to send their identifier to the middleware for processing.

A knob widget is a widget generally with a circular shape. Its goal istor epr oduce real circular controllers such as potentiometers and is characterized by its rotation. This widget is constructed using the same schema, but using the DiscSensor node, that gives the possibility to access its rotation angle. The widget needs to rotate with user's interactions. This animation is realized using ECMAScript.

In order to let users enter text for login and password or chat messages, we also must reproduce Te x t e d i t widgets. Text edit widgets are realized with the InputSensor node, which "is used to add entry points for user inputs into a BIFS scene. It allows user events to trigger updates of the value of a field or the value of anelementofamultiple field of an existing node" [START_REF]International Standardization Organization (ISO) and International Electrotechnical Commission (IEC)[END_REF]. Input devices are modeled as"devicesthatgenerate frame of user input". The frame consists in a list of values determined by its interface and their values are used to update the scene. The StringSensor device is the device we use to produce the Text edit widgets. Its interface is as follows :

StringSensorDataFrame [ [ SFString enteredText SFString finalText ]]
enteredText events produce characters when the keyboard is pressed. The values of this characters is a UTF-8 string. When a keystroke is pressed, thee n t e r e d T e x ti sm o v e d to the finalText and is then set to the empty string. The StringSensor is available in the scene thanks to the OD framework : In order to display the video streamed from the middleware, i.e. the video feedback of the remote device, the VideoTexture feature of BIFS contains the URL of the SDP files :

MPEG-4 BIFS ECMAScript API and Group Awareness

In order to propose group awareness tools, the BIFS scene has to be updated for each and every user, every time a user has done a single action.Themainissuehereis to propose a mechanism for updating BIFS scenes using middleware messages.

Different approaches can be considered in order to realize it. The first one would be to implement a BIFS-Commands server. BIFS-commands can update any part of the scene using three basics commands : Insert, Delete, Replace.

The second one would be using the MPEG-U standard [START_REF] Concolato | Overview of the ISO/IEC 23007 MPEG-U[END_REF]. MPEG-U is a standard for rich-media user interface and has been built as an extension of the W3C Widget standard 10 (WPC). Widgets defined in the MPEG-U standard are capable of communicating with each other through UPnP/DLNA services orl o c a ls e r v i c e so f other widgets.

However, we strongly believe that one of the major feature of HTML celebrity is JavaScript due to its user friendly programming language anditsstrongAPIforinteractivity and data management. So, we decided that the push/pull fonctionalities of the notification center should be accessible from the BIFS ECMAScript. This would allow an easy way to develop and modify any BIFS scene. ECMAScript is integrated into the scene by wrapping the script into Script nodes. It allows to have enriched interaction in the scene. Scripts can be activated by different means, including timed events and user actions. Then we have chosen this to enrich the ECMAScript capabilities of BIFS.

WebSockets

The WebSockets protocol [START_REF]The WebSocket Protocol[END_REF] is currently under standardization by the Internet Engineering Task Force (IETF). This protocol permits to open a permanent bidi-10. http ://www.w3.org/TR/widgets/ rectional connection between the clients and the server. Thegoa lofth i spr otoc oli sto develop real-time multi-user applications.

After the handshake between a client and the server, i.e. if the server allows the connection with that client, the client and the server can freely send data to each other via this protocol. They are sharing a HTTP port with the available content, avoiding proxy or firewall problems. The WebSocket protocol provides real push/pull functions.

Since MPEG BIFS can embed ECMAScript, the usage of the WebSocketAPI[W3C12] could be added. The WebSockets API is currently under standardization by the World Wide Web Consortium (W3C).

In order to add push/pull functionalities to the framework, we have added the WebSocket API to the MPEG-4 BIFS ECMAScript API of the GPAC framework. GPAC is built on top of modules that help extending the player capacities. We then designed aG P A Cm o d u l ef o ri n t e r p r e t i n gt h eW e b S o c k e tA P Ii n s i d eB I F Sscripts. The current implementation in the GPAC module does not take into account the whole WebSocket interface11 but is inspired from it and is as follows : The constructor takes two arguments. The first one specifies the URL to connect with. The second one specifies a subprotocol that the server provides if any. The url attribute returns the URL specified in the constructor and the protocol attribute returns the subprotocol specified in the constructor. The readyState attribute specifies the current state of the WebSocket connection between the client and the server using the values :

- The onopen function is called after the handshake between the client andt h es e r v e r has been established. The onerror function is called when the connection is interrupted, following network issues for example, and the onclose function is called when the connection between the client and the server has to be closed.Theonmessage function is called whenever a protagonist receives a message.

The send function sends the message to the WebSocket server. The message is the SFString argument data.ThebufferedAmount attribute then returns the number of bytes of application data that have been queued using the send function. The close function calls the end of the connection. This GPAC module has been realized using the libwebsockets 12 that is a C library available with the LGPL license.

Tele-Presence indicator

We have designed a chat tool inside the BIFS interface. Messages are printed in the dedicated window with the color corresponding to the writer of the message. Figure 4.8 shows the chat tool and a telepresence indicator designed in the multimedia interface using WebSockets. Each user's name is displayed with its given color. Below, the chat window shows the messages exchanged by users.

VoIP tool

For the VoIP tool, the FMOD 13 libraries combined with the FFMPEG 14 libraries are used to realize the acquisition of the voice of users at thec l i e n ts i d e ,p a c k e t i z e d and to stream it to the application server. The application server then multiplexes the digitalized voice within the BIFS scene, which is delivered to the end-user. The current available API is the following : The constructor takes two arguments. The first one specifies the URL to connect with. The second arguments is the server port to stream the audio content. The attributes url and port return respectively the URL and the port specified in the constructor. The acquire function is responsible for acquiring audio data from the terminal microphone and for streaming the audio content to the middleware. The close function stops the acquisition of audio data.

Feature descriptors Benchmark

In Chapter 2 we have presented some local features and descriptors algorithms along with their usage in mixed reality applications. In chapter 3, we have presented the general algorithm of our proposed solution in order to construct the 3D environment for building mixed reality applications. We showed that the solution uses local features and descriptors in order to compute the device identification and its pose estimation. In this subsection, we present a benchmark of local features andtheirdescriptorsapplied This analysis shows the extraction time of local features, the extraction time of feature descriptors, the matching time between the reference image and the image and the sum of all those steps. This study is realized using ten devices, captured in the same conditions. The images' resolution is 640×340. There are more than 360 images per devices. This benchmark is realized on a MacBook 2006 with a 2.13 GHz Intel Core 2 Duo processor and a 4 Go and 800 MHz DDR2 SDRAM memory. This observation is made for all the devices we analyzed. There is a high difference of computation speed between descriptors. BRIEF is the fastest one in our benchmark, SIFT being the slowest. ORB is close to BRIEF, while SURF is a bit faster than SIFT. All the charts for the ten devices are presented in Annex 7. Table 4.1 presents the scale and rotation invariance capacities of the analyzed descriptors. Only the SIFT and SURF descriptors are both scale and rotation invariant. Scale invariance ensures the recognition of objects in the scene when the distance between the camera and the object is changing over time. Rotationinvarianceensuresthe recognition of objects when the angle between the object and the camera is changing. These are both important features to take into account when designing mixed reality applications. Since we want our system with the fewest calibration time, i.e. the angle and the distance between the camera and the object are not known and could change during the session, only the SURF and the SIFT descriptors can fit our requirements.

Extraction of features and descriptors is not the only time consuming operation to take into account. The matching between the reference image and the captured scene is also critical. In order to achieve matching between descriptors, we also used the interfaces provided in the OpenCV library. We compare here a brute force matching against the FLANN matching, after computing the SURF descriptors. The charts represented in figure 4.11 show the results of computing time for both matchings on the same device. The other graphs are presented in Annex 8.

Of course the extraction time of features and descriptors is strictly the same. The charts show that the FLANN matching decreases the computation time from 750 to 100 ms. The overall process takes about 800 ms.

These results show that computing SURF features and descriptors and matching in a real-time application using 640×340 image resolution is not possible on the tested computer. In order to solve this issue and to keep the real-time aspect of the application, the SURF extraction is not done for every images but for one outof5.Thisimpliesthat the relative position between the camera and the object does not move too fast. A more powerful computer could decrease this process.

Whisper Module

The whisper module hints users to next possible actions. It was presented in Chapter 3. The implementation of this module is a component of the middleware and is illustrated in figure ??.

In order to ease loose-coupling, the whisper module is accessible through a REST Web service architecture over HTTP requests.

The Processing Block (PB), that interprets HTTP requests, queries the KB through SPARQL Protocol and RDF Query Language (SPARQL) queries, which are the current possible sequences based on previous inferences. PB also updates the current system states (the current action) to the Knowledge Base (KB) through SPARQL/Updates (SPARUL) requests. The RDFS reasoner infers which the next possible interactions are according to sequences rules and the Notification Center (NC) pushes the results to the HCI. The NC is the same as presented previously, since its goali sa l s ot op u s hH C I messages to remote users.

In order to achieve this module, a formalism to express sequences of interactions must be chosen.

Actions as RDF resources

Considering the better expressivity of RDFS for this kind of problem, we made the choice of representing the sequences using RDFS/RDF, as presented in section 3.4.2 Sequences are modeled using rdfs :Container. This allows thegranularityofexpressing compositions of ordered lists and choices of actions, with the power of Semantic Web tools. We define an action as an RDF resource, as presented in figure 4.13. The resource is identified with an Uniform Resource Identifier (URI)intheformofanUniform Resource Locator (URL). This resource has 4 properties :isCurr ent,isNext,isV erified and hasName. The three first ones are XML Schema boolean datatypes, whereas the fourth one is an XML Schema normalizedString datatype.

-isCurrent : defines if this action is the current one (i.e. if this action is the one being applied). -isNext : defines if this action is following another. Its valueisdeterminedbythe rules engine, as explained in following sections. -isVerified : defines if an action has already been called in the sequence.

-hasName : defines the name of the action. The full sequence is then a composition rdf :Seq, which represents the sequences of actions to perform and rdf :Alt which represents a choice between actions (e.g. A 3 or A 4 ). A literal value is predicated to the rdf :Seq in order to identify the device.

As a framework, Apache Jena seems a good compromise with its RDF graph framework and its reasoner capacities. Jena is a Java framework that includes APIs to process RDF or OWL data. The RDF graphs are stored in Joseki [START_REF] Hp | Joseki -A SPARQL Server for Jena[END_REF], an HTTP engine that supports the SPARQL protocol and a protocol for SPARQL/Update.

Reasoning for Interactions Hints

The main goal of the rule engine is to determine the possible actions following the current one and depending on the previous ones. Rules can be described as the following pseudo-code inside a sequence : if the item (m+1) in the sequence has rdf :Alt type, we need to put all its elements at next also. In order to verify the triple described above, we implemented custom built-ins in the Jena rule-engine. This built-in is in the form hasNext( ?seq, ?x, ?y). It verifies that {seq rdf:_(n) x} and {seq rdf:_(n+1) y} and that the previous actions in the sequences have been validated.

But the Open World Assumption is closely related to the monotonicity of first-order logic. It means that the reasoning system may accept new factst h a tm i g h tc o n t r a d i c t previous ones. Then, the Jena Rule engine may deduce contradicting facts, such as : {item_{m}, widg:isNext, 'true'^^xsd:boolean} and {item_{m}, widg:isNext, 'false'^^xsd:boolean} KB revision could be a solution in order to keep consistency. Some works have been conducted in this field such as in [START_REF] Grosof | The silk system : Scalable higher-order defeasible rules[END_REF] and . In [START_REF] Grosof | The silk system : Scalable higher-order defeasible rules[END_REF], the authors introduce an annotation approach for prioritizing rules over the remaining FIGURE 4.14 -RDF Graph of an action in the trace. ones, in order to prevent contradiction clashes. The authorso f[ H a l a s c h e k -W i e n e r 0 6 ] submit a system that maintains the internal state of DL reasoners. But this topic is beyond the scope of our work. In order to revise the KB and keep consistency, we therefore decided to remove old assertions and replace them with new deducted ones when aclashoccurs.

Use Trace

Use trace analysis helps operators by proposing them improvements in their procedures. We also have decided to record traces using rdf :Seq for the same abovementioned reasons. Each recorded action is identified by its name and by the person who made that action. The Friend of a Friend (FOAF) project15 aims at providing description of persons and links between them. The full trace is then a composition of RDF resources represented by Figure 4.14.

The use trace can thus be easily analyzed using pattern matching algorithm such as SuffixTree [START_REF] Ukkonen | Constructing Suffix Trees On-Line in Linear Time[END_REF] in order to determine which actions and/or sequences have been made, and by whom.

But the RDF resources representing the sequences need a property to express equivalency in order to propose improvements to the operator. A disadvantage using RDF against OWL is that RDF does not provide support for symmetricpr op er ti es.Asymmetric property is a bidirectional relationship. OWL provides such a support with owl :SymmetricProperty that is a subclass of owl :ObjectProperty. Whence, when designing sequences, we need to explicitly express that :

S i ⇒ S j (4.1) and S j ⇒ S i (4.2)
The RDF Graph of the sequence predefined in the system can be summarized in figure 4.15. If the system determines that S 5 appears in the use trace of Bob, the system can notify Bob that he should be considering using S 3 instead. Again, the rules engine is responsible for doing those assumptions. The analysis of use traces is available in our system through a REST Web Service architecture. Once the rules engine infers new deductions and finds the next possible interactions, a notification has to be pushed to the client application.

Another option would have been to use the OWL sameAs built-in.H o w e v e r ,t h i s OWL built-in is designed to be used between individuals that capture the very same observable. This would have been an ontological mis-engineering issue, as the two sequences are not identical, although they are equivalent.

Authoring Tools

We have presented the overall multimedia scene graph of the interactive multimedia remote laboratory. We have also presented how we implement the whisper module. In this subsection, we present the two authoring tools proposed in the framework. The first one creates the multimedia content for users. The second tool enables the creation of sequences of actions. It also enables to express the equivalence of sequences.

Multimedia Content Creation

The user needs to upload an image representing the backgroundofthescenecomposed of static pixels unchanging with time over the session.W ec a l lt h i si m a g et h e reference image.I ti ss t o r e di nt h es y s t e md a t a b a s e .T h eu s e rc a na d dw i d g e t sin the interface and chooses their color, label and size. Those widgets can be linked with already defined device functions stored in the database of the system. The user also defines the streaming area that is the area where the images captured from the camera will be displayed in the scene. This area is also stored in the database. Once the user finishes creating the interface, the BIFS scene is generated and multiplexed within an MP4 file as a media track. This is achieved thanks to the GPAC project. Figure 4.16 shows the authoring tool of the system, employed to design the multimedia content of the spectrum analyzer. In that case, only the display of the device needs to be a feedback zone. This authoring tool has been developed by a engineer trainee,L é oC h e w ,i nt h e LT2C laboratory that I supervised. It is a Flex application developed using the Bla-zeDS16 ,a"serv er -b a sedJ a v ar em oti nga ndwebm essa gi ngtec hnol ogy" that eases the Web client application integration with the system's middleware. I supervised the trainee for a 6 month period. Asequenceeditorisprovidedintheframework.Itenablesusers to create sequences and store them in the system's knowledge base, as well as to define which sequences are equivalent to others. The interface of the sequence editor is to be found in figure 4.17.

Sequences are displayed in the BIFS interface using the operators " ", "<", "(" , ")" and the widgets labels. When stored in the KB, the " "o p e r a t o ri n d i c a t e sr d f: A l te l ements and the "<"i n d i c a t e sr d f: S e qe l e m e n t s .F o re x a m p l e ,t h ed i s p l a y e ds e quence FREQ < (kHz||MHZ) < ENTER in the interface means that the first element of the sequence is FREQ,thesecondelementisachoicebetweenkHz and MHz (represented as a rdf :Alt element in the sequence) and the third element of the sequence is ENTER. When using the sequence editor, the operator directly uses the device interface. Activating a widget adds it to the edited sequence.

The second part of the tools proposes all the stored sequences. The user can then choose one of them and indicate equivalence between sequences. The sequences and their equivalence rules are stored in the Knowledge Base. Thissetupstepistobedone only once for a given device.

Conclusion

In this section, we have presented implementation details ofthesystemwepropose for remote laboratories. On the driver side, we propose to usetheIVIdriverinorderto ease interoperability, through their C libraries. We have designed programming classes in order to represent best the interaction modes of the device. On the client side, we use MPEG-4 BIFS and have presented how widgets can be designed in order to recreate the "real" device interactions. The ECMAScript API addition which proposes the WebSocket API and the audio communication has been presented. A benchmark of local features and descriptors has been realized. This benchmark lead us to the choice of the SURF algorithm in order to track the device in the video stream delivered by the IP camera, even if its computation time is high. We presented how the semantic Web and RDF technologies can be used in order to model sequences of actions and how users' actions can be analyzed. Finally, two authoring tools are introduced. They ease the creation of the multimedia content for online engineering and the creation of sequences of actions in the processes.

Results

An online engineering framework has been introduced. This framework proposes to build collaborative remote laboratories using interactive multimedia based on MPEG-4 BIFS and mixed reality. We propose here to present the first evaluation tests conducted to validate our approach. This evaluation has been conductedaspartofaMIRA project. We finally present the Open Source project resulting from our works.

MIRA Project

The MIRA projects 17 aim at "strengthening cooperation in international higher education and research". As part of a MIRA project, we set up a remote laboratory session with the Université Libano-Française of Tripoli (ULF) in Lebanon. This session took place in early December 2011, as part of a course on signal processing.

The pedagogical objectives of the ULF's course on signal processing are to give students skills on signal acquisition, digitalization and temporal and frequency representation. The course was divided into lectures and practical sessions, which were divided into simulation exercises and real manipulations. The simulation exercises were realized with Labview. The students had to display temporal and frequency representations of a given entry signal by varying parameters such as bandwidth, span, etc. The objectives of the manipulations were also to display temporal and frequency representations by varying parameters but on a real entry signal and to learn how to manipulate a real industrial device. The manipulations were realized using a spectrum analyzer hosted in Télécom Saint-Etienne (Saint-Etienne, France). 15 students followed that lecture.

The feedbacks from the students and the teacher were encouraging. The students liked the idea of manipulating a high quality device they are not used to see and more than 2000 kilometers away. The teacher pointed to the fact that the HMI presented was "really close to the real one" and really had "the feeling of manipulating the device". He gave a score of 4 out of 5 on the usability of the interface. He stated the configuration on the user side was really simple, given the fact that it consists of installing a video player and connecting to a RTSP address with it. The score is also 4 out of 5. He also said the transmission rate of commands between the client and the instrument was good, given the distance between the students and the device.Thetransmissionrateof commands was evaluated at an average of 600 ms. The bandwidth was evaluated at 500 kbps. The students and the teacher did not use the communication tools the framework provides since they all were in the same room when manipulating the device but appreciated the notifications of actions from others. The teacher emphasized the utility of such framework for the sharing of very expensive instruments. 17. Mobilité Internationale Rhône-Alpes However, both teacher and students pointed to a drawback. Theper ceivedquality of the video feedbacks from the IP camera was not as good as expected. The quality was not "good enough". The quality of the video feedback was evaluated at 2 on a MOS scale. That could come from both the IP camera quality and the available bandwidth in the ULF, which was not optimal. A deeper Mean Opinion Scale (MOS) and Quality of Experience (QoE) analysis would certainly be an interesting avenue of research. However, since students never used this kind of device before, only their opinion on the "visual quality" of feedbacks can be relevant for our analysis. The teacher is an expert in this domain, he is the only one able of comparing the usability of such devices.

An Open Source Project

The Open source designation applies to softwares whose license specifically meets the criteria established by the Open Source Initiative 18 ,thatistosaythepossibilityof free redistribution of the source code and derivative works.

In order to ease the software development of our system and to ease its evaluation by peers, we decided to open the source code of our system. Thispr oj ec ti sth eOpen Collaborative Environment for the Leverage of Online insTrumentation 19 (OCE-LOT). OCELOT is a complete solution framework for projects ofr e m o t ec o l l a b o r ative interactions. Using OCELOT, you can bring remote instrumentation solutions to your end-users faster. The possibilities range from online laboratories, remote industrial maintenance, cross-reality, etc.

OCELOT is distributed under the GNU Lesser Public License version 3 20 ,LGPLv3. This license, contrary to the GNU General Public License, limits the copyleft, i.e. the heritage of the license when deriving works. It is then possible to derivate the work and use other licenses. This project is divided into four parts. The first one is the multimedia part, that is the BT file used to create the multimedia content. The second part is the player part. It includes the different plugins and modules for the GPAC player. It is mostly written using the C programming language. The third part is the image processing part. This part is responsible for decoding images, device pose estimation and RTP streaming. The programming language used for this part is C++. The fourthpartistheinstrument driver, written with C languages. The last part is the middleware, mostly written in Java. Table 4.2 shows the different developed parts with their programming languages and some noticeable external libraries. O NLINE instrumentation and the remote control of devices must answer new chal- lenges : collaborative and computer-assisted work as well as usage context in the HCI of remote devices. Nowadays, access modalities are mainly applicative, giving abetteravideofeedbackofther emoteappr ehendedscene.Theobjectivesofthisthesis are to break those limited approaches by proposing an interactive and collaborative multimedia interface of remote laboratories, using a contextualized multimedia stream and considering ubiquitous and pervasive computing.

In this Chapter, we first make a synthesis of the proposition of this thesis, resuming the main ideas of our approaches. The second section tries to discuss the advantages and limitations of our solution, through different criteria. The third section introduces the prospects following conducted works.

Synthesis

Online manipulation of devices is nowadays used in three different areas :

-L e a r n i n gi n s t i t u t e sm a i n l yf o rd i s t a n c el e a r n i n gc u r r i c u lum. -I n d u s t r i e sf o rm a i n t e n a n c eo p e r a t i o n s ,p r o d u c t i o nc o n t r ols, etc. -R e s e a r c hi n s t i t u t e st om a n a g ee x p e r i m e n t a t i o n sw i t hr e s e archers from various geographic locations (collaboratories). Bringing devices online in the applicative world allows introducing new paradigms that were not possible on physical devices. The two main aspects in this domain are the collaborative computer-assisted work and the contextualization,i . e .p e r s o n a l i z a t i o n and screenwriting, of manipulation sessions of remote devices.

The collaborative computer-assisted work tries to allow operators to manipulate the same device while in different geo-locations to exchange competencies and knowledge in a common and shared task. In the industrial area,w ec a nm e n t i o nt h e Computer-Assisted Maintenance. When a maintenance operatorreacheshiscompetencies threshold, he calls a remote expert (phone, mail, etc.).Thisallowsnotonlytorealize the maintenance but also to bring the operator new competencies or knowledge. Collaborative online instrumentation permits to solve issues with a consolidation of competencies and performances by an enforcement of the dialogue between the operator and the remote expert, visualizing in real-time the apprehendeds c e n e .I nt h ee -l e a r n i n g area, it amounts to reproducing online practical exercises that are often done by groups in local manipulation.

The personalization of instrumentation tries to adapt applications following the capacities/habilitations of users or tasks to realize.

The state of the art analysis conducted in Chapter 2 showed that mixed reality applications can improve the representation's fidelity of online manipulation. Seeing the real scene and interacting with the remote device leads to having abetterunderstandingof the manipulated device and improves the "sense of being there", which is an utmost issue in distance learning. Mixed reality systems demonstrate their performance and benefits in various fields and applications, such as remote maintenance, telemedicine, education, serious gaming, or culture. Most of the time, additional information is overlaid on the viewed scene, although the subject of study is in the same place as the user, the activity is limited to observation, and the application is single user. Online engineering need to go beyond : the subject of study is distant, interaction with it is compulsory, and collaboration with peers leads to a better metaphor of laboratories on the Web. to others. This part is realized using WebSockets technologies. -T h ei n t e r a c t i v em u l t i m e d i a ,p r e s e n t i n gt h ei n t e r f a c et ot he end users. This component is built with MPEG-4 BIFS features and mixed reality techniques. -T h ew h i s p e rm o d u l e ,e n a b l i n gt h ea w a r e n e s so fu s a g ep r o c e d ures and the adaptation of the multimedia interface. -A na u t h o r i n gt o o l ,e n a b l i n gt h ec r e a t i o no ft h em u l t i m e d i acontent for new online devices. This framework is serialized in the form of an open source project, called OCELOT. This project is hosted by the OW2 forge and is distributed in the LGPL license. Its evaluation has been introduced with remote practical exercises conducted in the UFL in Tripoli, Lebanon. Fifteen students manipulated the device hosted in Saint-Étienne during the session.

The next section discusses the advantages and limitations ofourproposition.

Discussions

We presented how a complete solution for collaborative remote laboratories could be achieved using mixed reality techniques and interactive multimedia based on MPEG-4 BIFS. We also discuss the limitations and advantages ofsuchanapproach.

Remote Laboratories Frameworks

As presented in the state of the art, most of Remote Laboratories frameworks and HCIs are built on top of technologies such as Java, Flash, LabView, etc. In our work, we propose to build interfaces using MPEG-4 BIFS, enabling to easily bring mixed reality and ensuring interoperability. Given the feedbacks from the first tests conducted, we can say that our approach gives a real sensation of manipulating the remote device. The mixed reality paradigm allows a high fidelity of the representation of the device to manipulate. Our approach gives a high level of interoperability since every terminal capable of displaying multimedia content can be used in ordert oc o n t r o lt h er e m o t e device. The MPEG-4 BIFS standard is not bounded to any specific platform or operating system and proposes an efficient compression schema of thesc en e,u pto3ti m es according to our tests on different devices.

The middleware and the multimedia client allow to use the framework not only with remote laboratories but with any kind of collaborative multimedia applications, such as IPTV services, video conferencing, etc.

Multimodality.

Related works on multimodality highlight the fact that it canenhancecognitiveand learning processes and increase learner immersion. Our approach relies on the learner's terminal to interact with the remote device, i.e. keyboard and mouse for a classical computer, a touchscreen for smartphones or tablets, etc. and could benefit from multimodal interactions. But it could also decrease the understanding of the manipulated device by adding extra modality, such as voice control on a spectrum analyzer. When it comes to remote laboratories, we strongly believe that the interactions on the remote device should be as close as possible to the ones on the in-situ device.

Collaboration.

While iLab does not provide collaboration within its framework, collaboration and group awareness can be achieved using the framework we propose, even if the communication tools are not used when all the users are located inthesameroom.

Social networks could enforce the collaboration processes.O u rs y s t e mp r o v i d e s communication tools for people connected and interacting with the same device but does not provide information on users such as location, interests, etc. One possible scenario for improving this point could be the integration in a Learning Management System such as Moodle, which provides community tools for learning. The integration of Semantic Web technologies, especially the Friend of a Friend( F O A F ) 1 project could also be a track search since its goal is to describe "people, the links between them and the things they create and do".

Limitations.

However, our approach has some limitations. On the instrument side, if one wants to create a remote laboratory with a not IVI-compliant device, there is a non-negligible development time to consider. This problem could be addressed with driver introspection for example. Considering students feedbacks, the perceived video quality from the IP camera is not sufficient. This problem could be achieved using an IP camera with better quality and also by adapting the image resolution depending on the available bandwidth. This work will be a part of our future research on the system. The current developments on the framework allow only to build remote laboratories for devices with a 2D front, such as measurement instruments (oscilloscope, spectrum or vector network analyzer, etc.). Since MPEG-4 BIFS scenes can embed 3Dscenesandgraphics, we plan on supporting 3D instruments in a future version of theframework.

Perspectives

Perspectives for the work conducted in this thesis can be sought from the broader viewpoint of online mixed reality. We presented here an architecture for online engineering using mixed reality techniques and interactive multimedia. The framework resulting from our research could be extended to different use cases, such as collaborative IPTV services, collaborative remote surgery, etc. This would entail deeper works on 3D scenes on the client side.

Since the framework supports collaboration, deeper research on social interactions could be led. In particular, the FOAF project for managing users and operators could also be a track search since its goal is to describe "people, the links between them and the things they create and do". Learning Management System (LMS), such as Moodle for instance, are systems built in order to assist anyone involved in a learning process in its management of learning paths. In e-learning situations, it could improve the pedagogical objectives of the courses.

Current researches in the context-aware system area could also bring improvements to the framework. For now, the client application is the same on every kind of terminal. We could imagine adapting the multimedia user interfacedependingondif fer ent criteria such as the terminal, available bandwidth, geo-location, etc.

Of course current research on streaming over HTTP are also trends and could improve the framework. The goal of streaming over HTTP is to propose technologies that use the natural opened port for internet connections, as opposed to other technologies such as RTSP that uses by default the 554 port. Such ports are also often blocked by routers, firewalls or proxies. The basic idea behind HTTP streaming is to split files into segments and present these segments in a playlist file. HTTP streaming protocols can be used using HTTP Live Streaming from Apple or Dynamic Adaptive Streaming over HTTP (DASH) from the MPEG committee. Always code as if the guy who ends up maintaining your code willbeaviolent psychopath who knows where you live.

Rick Osborne

BF and FLANN matching graphs for SURF features descriptors

This annex shows the performance results of the brute force matching against the FLANN matching for SURF feature descriptors for ten devices that are available at Télécom Saint-Étienne and in the LT2C laboratory.

  C o m m e n tp r o p o s e ru ns y s t è m eu t i l i s a b l eàl af o i sd a n sl ' i n d ustrie et dans l'éducation ? -C o m m e n tp r o p o s e ru ns y s t è m ed er é a l i t ém i x t ei n t e r o p é r a b l e? -C o m m e n tp r o p o s e ru n eR I Aq u il i m i t el ab a n d ep a s s a n t e ? -C o m m e n tg é r e rl ac o l l a b o r a t i o na us e i nd ' u n ea p p l i c a t i o nd eréalitémixte? -C o m m e n tp r o p o s e ru n es o l u t i o nl ap l u sr é u t i l i s a b l ep o s s i b le pour diminuer le coût de développement des laboratoires distants ? -C o m m e n tp r o p o s e ru ns y s t è m eq u ip r e n de nc o m p t el e sp r o c é d u re d'exécution des opérateurs et qui adapte le contenu de l'IHM en fonction ?

FIGURE 1 -

 1 FIGURE 1-Architecturegénéraleenmoded'exécution.

FIGURE 2 -

 2 FIGURE 2-OverallArchitectureatsetup.

xx 0. 4

 4 Réalisation et RésultatsL AS E C T I O N précédente a présenté l'architecture principale de notre cadriciel avec ses principaux composants. Dans ce chapitre nous détaillonsdespoi n tsd' i m pl émentation particuliers du cadriciel dans un premier temps. Dans un second temps, nous présenterons les résultats obtenus à travers de tests utilisateurs. Le cadriciel est également réalisé sous la forme d'un projet libre (open source en anglais).

FIGURE 1 . 1 -

 11 FIGURE 1.1 -Tryptic of online engineering.

  S e c u r i t y:o fp e o p l ea n dg o o d s ,i n s u r a n c ec o n t r a c tr e s p e c t ,etc. -A c c e s s i b i l i t y:l i m i t e dn u m b e ro fp e o p l ei nt h es a m er o o m ,d isabled people, etc. -A v a i l a b i l i t y:o fp e o p l e . -Observability : infinitesimal or infinitely large. Online engineering solutions are not bounded to decrease the financial constraints of instrumentation.

FIGURE 1 . 2 -

 12 FIGURE 1.2 -Remote experimentations software chain.

FIGURE 2 . 1 -

 21 FIGURE 2.1 -Sensorama Machine by Morton Heiling (1957).

FIGURE 2 . 2 -

 22 FIGURE 2.2 -Mixed Reality Continuum [Milgram94].

FIGURE 2 .

 2 FIGURE 2.3 -A marker extracted from the ARToolKit library.

Figure 2 .

 2 Figure 2.3 is an example of a marker extracted from the famous ARToolKit library 1 .ARToolKit is a C library, which includes several models of two-dimensional fiducial markers. This library is based on the work of Kato & Billinghurst[START_REF] Kato | g s o f t h e 2 n d I E E E a n d A C M I n t e r n a t i o n a l Workshop on Augmented Reality[END_REF].

  [START_REF] Naimark | Circular Data Matrix Fiducial System and Robust Image Processing for a Wearable Vision-Inertial Self-Tracker[END_REF] suggested a "2D barcode fiducial design that can generate thousands of different codes and canb eu s e df o rw i d ea r e a tracking". But as mentioned by Zhou et al. in [Zhou08], after 2002, "therewerenonewgeneral marker based systems presented". Their assertion is based onaanalysisofthepapers published in the International Symposium on Mixed and Augmented Reality. Their article has been published in 2008.

  The first class of detectors concerns the corner detectors. In 1988, Harris et al. presented the corner detector known as the "Harris Corner Detector" [Harris88]. The Harris Corner Detector is based on the Moravec [Moravec80] corner detector. The idea behind the Moravec Corner Detector is to analyze a pixel neighborhood( aw i n d o w )a n dt o determine the changes of intensity in this neighborhood whent h ew i n d o wm o v e si n different directions. The FAST corner detector by Rosten et al.

FIGURE 2 . 4 -

 24 FIGURE 2.4 -SIFT examples.

FIGURE 2 . 5 -

 25 FIGURE 2.5 -SURF examples.

Figure 2 .

 2 7 shows again the box, with its extracted ORB key points.

FIGURE 2 .

 2 FIGURE 2.6 -BRIEF examples.

FIGURE 2 . 7 -

 27 FIGURE 2.7 -ORB examples.

  C o o p e r a t i v ew o r ki s ac o o p e r a t i o nb e t w e e na c t o r si no r d e rt or e a c hac o m m o ng o a lw hile sharing tasks.

  SWRL, Semantic Web Rule Language, "is based on a combination of the OWL-DL and OWL Lite sublanguages of the OWL Web Ontology Language theUn a r y/ Bi n a r y Datalog sublanguages of the Rule Markup Language. SWRL allowsu s e r st ow r i t e Horn-like rules expressed in terms of OWL concepts to reason about OWL individuals" [O'Connor05]. It uses first order logic in order to perform reasoning. Different frameworks are also available to deal with SWRL. Among others we have : Pellet [Pellet12] that is an open source Java OWL-DL reasoner that supports SWRL, Hoolet [Hoolet06] or SWRLTab that supports editing andexecutingSWRLrules in a Protege plugin-in. Context-aware systems propose schema in order to achieve adaptation. Most of them rely on Semantic Web technologies. Semantic Web technologies allow describing resources and their links. Coupled with rules based engines,reasoningcanbeachieved.

Figure 2 .

 2 8 shows two examples of X3D scenes.

FIGURE 2 . 8 -

 28 FIGURE 2.8 -X3D examples.

FIGURE 2 .

 2 FIGURE 2.9 -SVG examples.

  FIGURE 2.10 -WebGL examples.

7 .

 7 FIGURE 2.11 -BIFS examples.

  -4 BInary Format for Scene (MPEG-4 BIFS) [ISO/IEC-1449608] is specified in chapter 8 of the standard ISO/IEC 14496-1 released in 1999 andthenrevisitedasaPart of the standard ISO/IEC 14496-11 in 2005 and in 2008. BIFS is basically an extension of VRML and is a binary compressed format.

  e d i ao b j e c t s( e x c e p tf o rt h el a c ko f3 Ds u p p o r tf o rL A -SeR). -s t r e a m i n ga l lk i n d s o fm e d i a o b j e c t s( e x c e p t f o rt h e l a c ko f3D support for LASeR). -c o m p r e s s i n gt h es c e n ei no r d e rt or e d u c et h es i z eo ft r a n s m i tted data over networks.

FIGURE 2 .

 2 FIGURE 2.12 -VISIR client application.

Figure 2 .

 2 12 illustrates thec l i e n ta p p l i c a t i o no ft h e VISIR platform. No video capture of the remote scene is displayed in the client application. No mobile application has been mentioned so far. Someworkswereconducted in order to use the VISIR client application together with theiLabSharedArchitecture[START_REF] Garbi | A VISIR Lab Server for the iLab Shared Architecture[END_REF]. The VISIR platform does not provide a collaborative control of devices.

-

  H o wt op r o p o s eas o l u t i o na sm u c hr e u s a b l ea sp o s s i b l ei no r d er to decrease the development time of remote laboratory construction ? -H o wt op r o p o s eas y s t e mt h a tt a k e si n t oa c c o u n tt h eu s e rp r o c edure, as an element of context ? How to adapt the user interface depending on this element of context ?

  -C o n n e c t i v i t yt ot h em i d d l e w a r ef o rs e n d i n ga n dr e c e i v i n gc ommands/results as well as collaborative messages (users actions, text messages, audio, etc.) -Ar i c hg r a p h i c sa n dw i d g e t s p a n e l( p o s s i b i l i t yt oe m b e d2 D / 3D graphics, images, animations, etc.). -Ah i g hl e v e lo fi n t e r o p e r a b i l i t y ,e n a b l i n gu s e r st or e m o t e ly control devices from any kind of terminals. -Al o o s ec o u p l i n gb e t w e e nd e v i c e sa n dt h ea r c h i t e c t u r e ,i no rder to be as reusable as possible (independence to device). As presented in section 2.1.4.2, the MPEG-4 BIFS multimedia scene description is the only one capable of : -p r e s e n t i n ga l lk i n d so fm e d i ao b j e c t s( 2 D ,3 D ,a u d i o ,v i d e o ,etc.). -s t r e a m i n ga l lk i n d so fm e d i ao b j e c t s . -c o m p r e s s i n gt h es c e n ei no r d e rt or e d u c et h es i z eo ft r a n s m i tted data over networks. Therefore, the description of the remote scene delivered to the remote user in our framework is based on MPEG-4 BIFS.

FIGURE 3. 2 -

 2 FIGURE 3.2 -Overall Architecture at runtime.

  FIGURE 3.3 -Overall Architecture at set-up.

  [Swan05]: -C o m b i n et h er e a lw o r l da n dt h ev i r t u a lw o r l d . -T a k ei n t oa c c o u n tt h ec o n s i s t e n c ya n dt h ec o h e r e n c eb e t w e e nt h er e a la n dt h e virtual worlds.

FIGURE 3. 4 -

 4 FIGURE 3.4 -Multimedia Chain Delivery.

FIGURE 3 . 6 -

 36 FIGURE 3.6 -Display extraction.

FIGURE 3. 8 -

 8 FIGURE 3.8 -Homography principle.

3. Remote Procedure Call 4 .

 4 FIGURE 3.10 -Collaboration and Messages delivery.

FIGURE 3 .

 3 FIGURE 3.11 -Whisper module.

FIGURE 3 .

 3 FIGURE 3.13 -Unicast streaming.

FIGURE 3 .

 3 FIGURE 3.14 -Broadcast streaming.

FIGURE 3 .

 3 FIGURE 3.15 -Multicast streaming.

The

  FIGURE 3.16 -Delivery.

  N e t w o r kl o a d . -C o n c u r r e n c y . -P e r s i s t e n c y . The JEE framework offers a large panel of application servers. A choice can be made in the main list composed of -W e b S p h e r ef r o mI B M 1 . -Glassfish from Oracle 2 . -J B o s sf r o mR e d H a t 3 . -J O n A Sf r o mO W 2 4 .

FIGURE 4 . 1 -

 41 FIGURE 4.1 -Interactions modes.

FIGURE 4 . 2 -

 42 FIGURE 4.2 -BIFS Scene composition.

FIGURE 4 .

 4 FIGURE 4.3 -Authentication Screen.

FIGURE 4 . 4 -

 44 FIGURE 4.4 -Button pseudo-graph.

FIGURE 4 . 5 -

 45 FIGURE 4.5 -Button details pseudo-graph.

FIGURE 4 . 7 -

 47 FIGURE 4.7 -BIFS Interface.

  C O N N E C T I N G:t h ec o n n e c t i o ni sn o te s t a b l i s h e d .T h i si st h edefault value. -O P E N:t h ec o n n e c t i o ni se s t a b l i s h e da n dc o m m u n i c a t i o ni sp ossible. -C L O S I N G:t h ec o n n e c t i o ni sc l o s i n g . -C L O S E D:t h ec o n n e c t i o nh a sb e e nc l o s e do ri n t e r r u p t e d .

FIGURE 4 .

 4 FIGURE 4.9 -ORB and BRIEF extraction.

Figure 4 .

 4 Figure 4.9 and figure 4.10 show the results of feature extractions for the E4407B Agilent spectrum analyzer. BRIEF and ORB are faster in the extraction of both features and descriptors than SURF and SIFT.

FIGURE 4 .

 4 FIGURE 4.10 -SURF and SIFT extraction.

FIGURE 4 .

 4 FIGURE 4.11 -Brute Force and FLANN matching.

FIGURE 4 .

 4 FIGURE 4.12 -Whisper module implementation.

FIGURE 4 .

 4 FIGURE 4.13 -RDF Graph of an action.

FIGURE 4 .

 4 FIGURE 4.15 -Sequence representation.

FIGURE 4 .

 4 FIGURE 4.16 -Authoring tool.

FIGURE 4

 4 FIGURE 4.17 -Sequences Editor

Figure 4 .

 4 Figure 4.18 presents the final interface realized using the OCELOT framework on the spectrum analyzer, running on an Android tablet and on a laptop, with its different interface components. Figure4.19 summarizes the functioning of the application.

Figure 4 .

 4 Figure 4.18 presents the final interface realized using the OCELOT framework on the spectrum analyzer, running on an Android tablet and on a laptop, with its different interface components. Figure4.19 summarizes the functioning of the application.

FIGURE 4 .

 4 FIGURE 4.18 -OCELOT running on an Android tablet and on a laptop.

FIGURE 4 . 5

 45 FIGURE 4.19 -Sequence diagram.

  In this work, we answer the questions raised in Chapter 2 with af r a m e w o r kf o r online experimentations that :-e n a b l e st h er e m o t ec o n t r o lo fd e v i c e s . -h a n d l e sc o l l a b o r a t i o na n dg r o u pa w a r e n e s s . -a d a p t si t sH C Id e p e n d i n go nt h eu s a g ep r o c e d u r e ,t h a n k st oS emantic Web technologies. -i m p r o v e st h ec o m p r e h e n s i o no fm a n i p u l a t e dr e m o t es c e n e s ,thanks to mixed reality techniques. -i si n t e r o p e r a b l e ,t h a n k st om u l t i m e d i as c e n ed e s c r i p t o r s . -i sr e u s a b l e . The framework is composed of different components : -T h ed r i v e r s ,r e s p o n s i b l ef o rt h ed i a l o g u ew i t hd e v i c e s .T h is part is realized using IVI standards. -T h em i d d l e w a r e ,r e s p o n s i b l ef o rc o n v e y i n gm e s s a g e sf r o mu ser side to the remote device. It is designed on top of MOM features. -The notification center, responsible for pulling back interface messages from users

FIGURE 7 . 1 -

 71 FIGURE 7.1 -Agilent E4407B.

FIGURE 7 . 2 -

 72 FIGURE 7.2 -Agilent E4407B : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.3 -Agilent E4407B : SIFT and SURF extraction.

FIGURE 7 . 4 -

 74 FIGURE 7.4 -Agilent 54622D.

FIGURE 7 . 5 -

 75 FIGURE 7.5 -Agilent 54622D : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.6 -Agilent 54622D : SIFT and SURF extraction.

FIGURE 7 . 7 -

 77 FIGURE 7.7 -Anritsu 37397A.

FIGURE 7 . 8 -

 78 FIGURE 7.8 -Anritsu 37397A : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.9 -Anritsu 37397A : SIFT and SURF extraction.

FIGURE 7 .

 7 FIGURE 7.10 -HP 8561A.

FIGURE 7 .

 7 FIGURE 7.11 -HP 8561A : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.12 -HP 8561A : SIFT and SURF extraction.

FIGURE 7 .

 7 FIGURE 7.13 -HP 54510A.

FIGURE 7 .

 7 FIGURE 7.14 -HP 54510A : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.15 -HP 54510A : SIFT and SURF extraction.

FIGURE 7 .

 7 FIGURE 7.16 -HP 54600B.

FIGURE 7 .

 7 FIGURE 7.17 -HP 54600B : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.18 -HP 54600B : SIFT and SURF extraction.

FIGURE 7 .

 7 FIGURE 7.19 -HP E5060A.

FIGURE 7 .

 7 FIGURE 7.20 -HP E5060A : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.21 -HP E5060A : SIFT and SURF extraction.

FIGURE 7 .

 7 FIGURE 7.22 -LeCroy WaveJet 324A.

FIGURE 7 .

 7 FIGURE 7.23 -LeCroy WaveJet 324A : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.24 -LeCroy WaveJet 324A : SIFT and SURF extraction.

FIGURE 7 .

 7 FIGURE 7.25 -Hameg HM407-2.

FIGURE 7 .

 7 FIGURE 7.26 -Hameg HM407-2 : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.27 -Hameg HM407-2 : SIFT and SURF extraction.

FIGURE 7 .

 7 FIGURE 7.28 -Tektronix TDS 210.

FIGURE 7 .

 7 FIGURE 7.29 -Tektronix TDS 210 : BRIEF and ORB extraction.

FIGURE 7 .

 7 FIGURE 7.30 -Tektronix TDS 210 : SIFT and SURF extraction.

FIGURE 8 . 1 -

 81 FIGURE 8.1 -Agilent E4407B.

FIGURE 8 . 2 -

 82 FIGURE 8.2 -Agilent E4407B : brute force and FLANN matching.

FIGURE 8 .

 8 FIGURE 8.3 -Agilent 54622D.

FIGURE 8 . 4 -

 84 FIGURE 8.4 -Agilent 54622D : brute force and FLANN matching.

FIGURE 8 . 5 -

 85 FIGURE 8.5 -Anritsu 37397A.

FIGURE 8 .

 8 FIGURE 8.6 -Anritsu 37397A : brute force and FLANN matching.

FIGURE 8 . 7 -

 87 FIGURE 8.7 -HP 8561A.

FIGURE 8 . 8 -

 88 FIGURE 8.8 -HP 8561A : brute force and FLANN matching.

FIGURE 8 .

 8 FIGURE 8.9 -HP 54510A.

FIGURE 8 .

 8 FIGURE 8.10 -HP 54510A : brute force and FLANN matching.

FIGURE 8 .

 8 FIGURE 8.11 -HP 54600B.

FIGURE 8 .

 8 FIGURE 8.12 -HP 54600B : brute force and FLANN matching.

FIGURE 8 .

 8 FIGURE 8.13 -HP E5060A.

FIGURE 8 .

 8 FIGURE 8.14 -HP E5060A : brute force and FLANN matching.

FIGURE 8 .

 8 FIGURE 8.15 -LeCroy WaveJet 324A.

FIGURE 8 .

 8 FIGURE 8.16 -LeCroy WaveJet 324A : brute force and FLANN matching.

FIGURE 8 .

 8 FIGURE 8.17 -Hameg HM407-2.

FIGURE 8 .

 8 FIGURE 8.18 -Hameg HM407-2 : brute force and FLANN matching.

FIGURE 8 .

 8 FIGURE 8.19 -Tektronix TDS 210.

FIGURE 8 .

 8 FIGURE 8.20 -Tektronix TDS 210 : brute force and FLANN matching.

présenter les Section 0.2. État de l'Art ix

	Descrip-		Compo-	Audio et	Streaming Compression Interactivité
	teurs	de	sants	Vidéo
	scène		Gra-
			phiques
	VRML		3D	oui	non	non	oui
	SMIL		2D	oui	non	non	oui
	SVG		2D	non	non	version SVGZ oui
	COLLADA 3D	non	non	non	non
	Flash		2D/3D	oui	oui	oui	non
	JavaFX		2D/3D	MP3 & VP6 non	compilation	oui
						bytecode
	Silverlight	2D/3D	plusieurs	non	compilation	oui
				codecs
	HTML		2D	oui, mais	oui	non	oui
			and 3D	partiel-
			through	lement
			WebGL	standardisé
	BIFS	et	2D/3D	oui	oui	oui	oui
	XMT
	LASeR		2D	oui	oui	oui	oui

TABLE 1 -

 1 Descripteursdescènesmultimédia

mêmes types de contenu. Le tableau 1 résume les capacités des différents descripteurs de scène. Seuls les descripteurs de scène proposées par le comité MPEG, i.e. MPEG-4 LASeR et MPEG-4 BIFS, sont capables de fournir des flux de la totalité de leurs contenu.

Ta b l e o f C o n t e n t s Foreword i

  Thank-you note . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i Résumé long . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii 0.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii 0.2 État de l'Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi 0.3 Proposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii 0.4 Réalisation et Résultats . . . . . . . . . . . . . . . . . . . . . . . . . . ... x x 0.5 Conclusion et Perspectives . . . . . . . . . . . . . . . . . . . . . . . . ... x x v Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxix 1I n t r o d u c t i o n 1

  Problems, objectives and structure of this thesis 1.1 Once upon a time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

		3
	1.2 Online Engineering and Remote Laboratories . . . . . . . . . . ......	6
	1.3 Organization of the manuscript . . . . . . . . . . . . . . . . . . . . . ... 1 3

2S t a t e o f t h e A r t 15 Related works : principles and formalization

  2.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2 Remote Laboratories and Online Engineering . . . . . . . . . . ...... 4 3 2.3 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

51 3P r o p o s a l 57 The

	heart of the proposal
	3.1 Online Engineering and Computer Aided Maintenance Framework . . . 59
	3.2 Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
	3.3 Handling Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 3
	3.4 Adaptation of the Human Computer Interface . . . . . . . . . . . .... 7 7
	3.5 Multimedia Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
	3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

xxx TABLE OF CONTENTS 4R e a l i z a t i o n a n d R e s u l t s 89 What

  we have so far 4.1 Realization and Implementation . . . . . . . . . . . . . . . . . . . . ... 9 1 4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

116 5C o n c l u s i o n a n d P e r s p e c t i v e s 121 Next

  step in Online Engineering 5.1 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 5.2 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 5.3 Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

127 6S p e c t r u m A n a l y z e r i m p l e m e n t a t i o n : F R E Q m o d e 1 2 9 C class for IVI driver 7F e a t u r e s a n d d e s c r i p t o r s g r a p h s 133 Charts of features detection

r s 145 Charts of descriptors matching Glossary List of Figures

  Tryptic of online engineering. . . . . . . . . . . . . . . . . . . . . . .... 1.2 Remote experimentations software chain. . . . . . . . . . . . . ...... 2.1 Sensorama Machine by Morton Heiling (1957). . . . . . . . . . . . . ... 1 2.2 Mixed Reality Continuum [Milgram94]. . . . . . . . . . . . . . . . . . .. 2 2.3 A marker extracted from the ARToolKit library. . . . . . . . . . ...... 2 2.4 SIFT examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 SURF examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 BRIEF examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 ORB examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 X3D examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 SVG examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 WebGL examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Homography principle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Publish/Subscribe paradigm. . . . . . . . . . . . . . . . . . . . . . . ... 7 3.10 Collaboration and Messages delivery. . . . . . . . . . . . . . . . ..... 7 3.11 Whisper module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.12 MPEG-4 BIFS graph example. . . . . . . . . . . . . . . . . . . . . . . . . .

	1A r c h i t e c t u r e g é n é r a l e e n m o d e d ' e x é c u t i o n . . . . . . . . . . . . . ..... x v 2O v e r a l l A r c h i t e c t u r e a t s e t u p . . . . . . . . . . . . . . . . . . . . . . . . .. x v 1.1 2.11 BIFS examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
	2.12 VISIR client application. . . . . . . . . . . . . . . . . . . . . . . . . .... 4
	3.1 Remote experimentation architecture. . . . . . . . . . . . . . . ...... 6
	3.2 Overall Architecture at runtime. . . . . . . . . . . . . . . . . . . . ..... 6
	3.3 Overall Architecture at set-up. . . . . . . . . . . . . . . . . . . . . ..... 6
	3.4 Multimedia Chain Delivery. . . . . . . . . . . . . . . . . . . . . . . . . . .6
	3.5 A Spectrum Analyzer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	3.6 Display extraction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
	3.7 Checkerboard with detected corners. . . . . . . . . . . . . . . . . ..... 6
	3.8 3.13 Unicast streaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
	3.14 Broadcast streaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 8
	3.15 Multicast streaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8

TABLE 2 .

 2

	Compression Interactivity
	Strea-
	ming

1 -Scene Descriptors review kind of media content in the same way. Table

2

.1 summarizes some of the properties of the presented scene descriptors. All the presented scene descriptors propose animation, though this feature does not appear in the table. By fulls t r e a m i n g ,w em e a nt h e capacity to stream all kind of media objects (2D or 3D graphics, audio/video objects, animation, etc.).

  3.1 Online Engineering and Computer Aided Maintenance Framework . 59 3.1.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 3.1.2 Software Chain . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 3.1.3 System at runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.1.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.2 Augmented Reality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.2.1 Scene Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.2.2 Device Identification and Pose Estimation . . . . . . . . . . .. 6 7 3.2.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 3.3 Handling Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.3.1 Middleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.3.2 Collaborative Interactive Multimedia Interface . . . ...... 7 6 3.4 Adaptation of the Human Computer Interface . . . . . . . . . . . ... 77 3.4.1 Whisper Module . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.4.2 Expressing Sequences. . . . . . . . . . . . . . . . . . . . . . . . 78 3.4.3 Use Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.5 Multimedia Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.5.1 Multimedia Composition using MPEG-4 BIFS . . . . . . . . . 81 3.5.2 Multimedia Delivery . . . . . . . . . . . . . . . . . . . . . . . . 83 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

  Chat tool and Tele-presence indicator.

	FIGURE 4.8 -readonly attribute SFString url;
	readonly attribute SFString port;
	// networking
	attribute Function? onerror;
	void acquire();
	void close();
	};
	Class AudioTool {
	AudioTool(SFString url, SFInt32 port);
	12. http ://git.warmcat.com/cgi-bin/cgit/libwebsockets/
	13. http ://www.fmod.org/
	14. http ://ffmpeg.org/

TABLE 4 .

 4

	Features Descriptor	Rotation Invariance	Scale Invariance
	SIFT	yes	yes
	SURF	yes	yes
	ORB	yes	no
	BRIEF	limited	yes

1 -Features Descriptors : rotation and scale invariance

TABLE 4 .

 4

	To o l	Language	Number	of	Number	of	External
			files		code lines		Libraries
	Multimedia	BT	1		5869		none
	content
	Player plugins C	4		1524		FMOD,	lib-
							websocket,
							FFMPEG
	Image proces-	C++	35		6742		OpenCV,
	sing						FFMPEG,
							Xerces
	Instrument	C	5		661		IVI librairies
	driver
	Middleware	Java	104		16348		Apache Jena,
							Commons
							Exec, BlazeDS

2 -Open source realization

  Beware of bugs in the above code ; I have only proved it correct,nottriedit.

	Donald Knuth
	6
	Spectrum Analyzer implementation : FREQ
	mode
	using System;
	using System.Collections.Generic;
	using System.Linq;
	using System.Text;
	using Ivi.SpecAn.Interop;
	using RohdeSchwarz.RsSpecAn.Interop;
	namespace IVISpectrumAnalyzer
	{
	class IVIFreq
	{
	public IIviSpecAnFrequency _freq;
	/ *
	* Constructor
	* /
	public IVIFreq(IRsSpecAn rs)
	{
	IIviSpecAn specan = (IIviSpecAn)rs;
	_freq = specan.Frequency;
	}

Glossary

  AJAX Asynchronous Javascript and XML. 3 API Application Programming Interface. 59, 99 CAM Computer-Assisted Maintenance. 5, 10 CD-ROM Compact Disc -Read Only Memory. 35 Collaboratories [a] center without walls, in which the nation's researchers can perform their research without regard to geographical location, interacting with colleagues, accessing instrumentation, sharing data and computational resources, and accessing information in digital libraries. 8 COLLADA COLLAborative Design Activity. 37 CSCW Computer Supported Collaborative Work. 5

	Glossary
	KB Knowledge Base. 75
	LDAP Lightweight Directory Access Protocol. 94
	MOM Message-Oriented Middleware. 60
	MOS Mean Opinion Scale. 114
	MPEG-4 BIFS MPEG-4 BInary Format for Scene. 38
	MPEG-4 LASeR MPEG-4 Lightweight Application Scene Representation. 39
	MPEG-4 XMT MPEG-4 Textual format. 39
	NFC Near Field Communication. 22
	OD Object Descriptors. 79
	OOM Object-Oriented Middleware. 60
	ORB Oriented FAST and Rotated BRIEF. 26
	QoE Quality of Experience. 114
	REST REpresentation State Transfer. 105
	RFID Radio Frequency IDentification. 22
	RIA Rich Internet Application. 3
	DoG Difference of Gaussian. 23 RMI Remote Method Invocation. 72
	ES Elementary Streams. 79 RPC Remote Procedure Control. 72
	RTP Real-time Transport Protocol. 82 GPS Global Positioning Systems. 22 RTSP Real Time Streaming Protocol. 82
	HCI Human Computer Interface. 4 SDP Session Description Protocol. 82 HTML HyperText Markup Language. 38 SIFT Scale-Invariant Feature Transform. 24 HTTP Hypertext Transfer Protocol. 79 SMIL Synchronized Multimedia Integration Language. 36 HTTPS HyperText Transfer Protocol Secure. 95 SPARQL SPARQL Protocol and RDF Query Language. 105
	IETF Internet Engineering Task Force. 99 SPARUL SPARQL Update. 105
	IVI Interchangeable Virtual Instrument. 59 SSL Secure Sockets Layer. 95
	SURF Speeded-Up Robust Feature. 24 JEE Java Enterprise Edition. 89 JMS Java Messaging Service. 72 SVG Scalable Vector Graphics. 36
	JOnAS Java Open Application Server. 90 TCP Transmission Control Protocol. 82
	JORAM Java Open Reliable Asynchronous Messaging. 90 UDP User Datagram Protocol. 82
	JSON JavaScript Object Notation. 73 ULF Université Libano-Française. 113

http ://gpac.wp.mines-telecom.fr/

sources : wikipedia : http ://en.wikipedia.org/wiki/CSCW

"Time is money". Benjamin Franklin.

European Region Action Scheme for the Mobility of University Students -http ://www.europeeducation-formation.fr/erasmus.php

http ://www.panocast.com/

http ://docs.google.com

http ://telerobot.mech.uwa.edu.au

http ://www.lila-project.org/about/index.html

http ://gpac.wp.mines-telecom.fr/

http ://www.w3.org/TR/websockets/

http ://www.foaf-project.org/

http ://opensource.adobe.com/wiki/display/blazeds/BlazeDS

http ://www.foaf-project.org/

CORELA

Distant yes no (OpenSim) yes

Graphics capabilities of BIFS scene enable any kind of other widget families, such as scrollpane, slider, label,etc.usingfeaturessuchasLayer2D, Layer3D, XCurve2D,etc.

When the driver of the device is in simulation mode, the middleware is notified and forwards this notification to the client application. The feedback area of the multimedia HMI switches from the video mode to a graphics mode. The cloud points the driver delivers in simulation mode are then displayed in the multimedia, as shown in figure 4.6. This can be achieved thanks to the XCurve2D node of the MPEG-4 BIFS standard, which allows the drawing of Bézier curves. Buttons are composed of three children nodes. The first one represents the shape of the button. The second one represents the label of the button and the third one is javascript functions that enable communication with the middleware using HTTP requests.

Annex 6. Spectrum Analyzer implementation: FREQ mode

7

Features and descriptors graphs

This annex shows the performance results of the extraction ofthefourfeaturedescriptors BRIEF, ORB, SIFT and SURF for ten devices that are available at Télécom Saint-Étienne and in the LT2C laboratory.