Keywords: Tangible interaction, tabletop, dual reality, design, user performances, RFID retail shelf planning . . . . . . . . .

In everyday life, new interactions are gradually replacing the standard computer keyboard and mouse, by using the human body gestures (hands, fingers, head, etc.) as alternatives of interactions on surfaces and in-air. Another type of interaction resides within the manipulation of everyday objects to interact with digital systems. Interactive tabletops have emerged as new platforms in several domains, offering better usability and facilitating multi-user collaboration, thanks to their large display surface and different interaction techniques on their surfaces, such as multi-touch and tangible. Therefore, improving interaction(s) on these devices and combining it (respectively them) with other concepts can prove more useful and helpful in the everyday life of users and designers.

The topic of this thesis focuses on studying user interactions on tangible interactive tabletops, in a context of use set in a dual reality environment. Tangible User Interfaces offer users the possibility to apprehend and grasp the meaning of digital information by manipulating insightful tangible representations in our physical world. These interaction metaphors are bridging both environments that constitute the dual reality: the physical world and the virtual world.

In this perspective, this work presents a theoretical contribution along with its applications. We propose to combine tangible interaction on tabletops and dual reality in a conceptual framework, basically intended for application designers, that models and explains interactions and representations, which operate in dual reality setups. First of all, we expose various works carried out in the field of tangible interaction in general, then we focus on existing work conducted on tabletops. We also propose to list 112 interactive tabletops, classified and characterized by several criteria. Next, we present the dual reality concept and its possible application domains. Second, we design our proposal of the framework, illustrate and explain its composing elements, and how it can adapt to various situations of dual reality, particularly with interactive tabletops equipped with RFID technology. Finally, and as application contributions, we show case studies that we designed based on our proposal, which illustrate implementations of elements from our proposed framework. Research perspectives are finally highlighted at the end of the manuscript.

Résumé

De nouvelles modalités d'interactions reposant sur les postures et les gestes complètent progressivement les modalités couramment employées par les ordinateurs de bureau, les tablettes et les surfaces interactives. Ces modalités peuvent être enrichies par l'adjonction d'objets tangibles, directement tirés de la vie quotidienne ou représentant de manière symbolique des concepts abstraits de l'interface. Les tables interactives, de par leur horizontalité et leurs cadres d'utilisation, souvent collaboratifs voire conviviaux, sont un territoire privilégié d'exploration des usages des objets tangibles et de la manière dont ils sont capables d'enrichir les modalités classiques d'interaction avec ces tables que sont le pointage et le toucher.

Le sujet de cette thèse porte sur l'étude des interactions utilisateur avec des tables interactives tangibles, dans un contexte d'utilisation en environnement de réalité duale constitué de deux mondes symétriques, interconnectés et d'influence mutuellement. Les interfaces utilisateur tangibles offrent aux utilisateurs la possibilité d'appréhender et de saisir la signification des informations numériques en manipulant des représentations tangibles judicieuses de notre monde physique. Ces métaphores d'interaction établissent un pont entre les deux environnements qui constituent la réalité duale : le monde physique et le monde virtuel.

Dans cette perspective, ce travail présente une contribution théorique, ainsi que ses applications. Nous proposons de combiner l'interaction tangible sur table interactive avec la réalité duale dans un cadre conceptuel, essentiellement destiné aux concepteurs d'applications, qui modélise et explique les interactions et les représentations, qui fonctionnent dans des configurations de réalité duale. Nous exposons tout d'abord différents travaux réalisés dans le domaine de l'interaction tangible en général, puis nous nous concentrons sur des travaux menés sur les tables interactives. Nous proposons également de recenser et répertorier 112 tables interactives, classées et caractérisées selon plusieurs critères. Ensuite, nous présentons le concept de la réalité duale et ses domaines d'application possibles. Ensuite, nous proposons un framework de conception, illustrons et expliquons ses éléments constitutifs, et comment il peut s'adapter à diverses situations de réalité duale, notamment avec des tables interactives équipées de la technologie RFID. Enfin, quant à nos contributions applicatives, nous montrons des études de cas que nous avons conçues sur la base de notre proposition, qui illustrent les mises en oeuvre des éléments de notre framework proposé. Les perspectives de recherche sont enfin mises en évidence à la fin du manuscrit.

Mots clés : Interaction tangible, table interactive, réalité duale, conception, performances utilisateur, RFID.

First of all, I would like to express my deep gratitude to my thesis supervisors prof. Christophe Kolski, prof. Antonio Krüger, and Dr. Alexis Héloir for giving me this chance to do my PhD thesis and for their supervising. I would like to thank them from a scientific perspective for the work done, and from a human and cultural perspective for the moments and experiences we lived and explored together. Particularly I thank Christophe for his availability and his quick replies for any enquiry; also Alexis for his technical assistance and guidance when needed.

I would like also to express my gratitude to prof. Antonio Krüger for giving me the opportunity to spend a research stay of few months in his lab in DFKI, Saarbrücken, Germany; also I thank prof. Gloria Mark from University of California in Irvine, USA, for giving me the chance to visit her lab and work with her for a short time.

Then, I would like to thank the jury members for accepting to evaluate my work and to participate in my thesis defense: prof. Franck Poirier and prof. Jean Vanderdonckt for reporting my thesis, and prof. Nadine Couture for examining my work.

I also thank the automation department of the Laboratory of Industrial and Human Automation control, Mechanical engineering and Computer Science (LAMIH), particularly Marie-Pierre Pacaux-Lemoine and Patrick Millot for lending us the Lego robots (used in SUCRé project). I also thank Lydia Habib for her help with the robots' programming and communication.

I thank Yosra Rekik for her collaboration and help in two studies (exposed in Chapter 3 and Chapter 4) that we run in our laboratory.

I thank all the people who agreed to participate in the different user experiments, undertaken as part of my PhD thesis; a thank you to the different colleagues in LAMIH laboratory; my friends as well, with whom it was good to escape the research environment occasionally.

Finally, I warmly thank my parents, from the depths of my heart, who have always supported me and believed in me for all these years since my childhood, for their continuous encouragement and blessings, and without whom I wouldn't be where I am today at all. I hope that this modest work shows the fruit of their efforts, and will make them proud. My thank goes to my sisters as well, for adding joy to my life; and to my fiancee for her company during the last few months. ChainFORM hardware configurations and modules, b: reconfigurable display, c: shape changing stylus, d: animated character, e: haptic glove [START_REF] Nakagaki | ChainFORM: A Linear Integrated Modular Hardware System for Shape Changing Interfaces[END_REF]. 1.9 Different possible usages and arrangements of Zooids [START_REF] Le Goc | Zooids: Building blocks for swarm user interfaces[END_REF]. . . . . . . . . 1.10 (a) Teegi [START_REF] Fleck | Teegi, he's so cute: example of pedagogical potential testing of an interactive tangible interface for children at school[END_REF]. (b) A user observing and analyzing his own brain activity in real-time while manipulating Teegi [START_REF] Frey | Teegi: tangible EEG interface[END_REF]. . . . . . . . . . . . . . . . . . 1.11 (a) BendDesk tabletop [START_REF] Weiss | Benddesk: dragging across the curve[END_REF]. (b). ELC2 tabletop [START_REF] Elecom | Table Tactile ELC2[END_REF]. . . . . . . . . . . . 1.12 (a) Cooking ideas on TangiSense tabletop (version one) [START_REF] Lebrun | Management of Distributed RFID Surfaces: A Cooking Assistant for Ambient Computing in Kitchen[END_REF]. (b). A musician using mixiTUI tabletop [START_REF] Pedersen | mixitui: a tangible sequencer for electronic live performances[END_REF]. . . . . . . . . . . . . . . . . . . . 1.13 (a) Users collaborating around The reacTable tabletop [START_REF] Jordà | The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces[END_REF] allowing fingers and/or tangible objects detection (adapted from [START_REF] Kubicki | Contribution à la prise en considération du contexte dans la conception de tables interactives sous l'angle de l'IHM, application à des contextes impliquant table interactive RFID et objets tangibles[END_REF]). . 1.17 An example of distributed application architecture (Cooking Ideas [START_REF] Lebrun | Management of Distributed RFID Surfaces: A Cooking Assistant for Ambient Computing in Kitchen[END_REF]). 1.19 Fundamental mappings between the real (left) and the virtual (right) [START_REF] Lifton | Dual reality: an emerging medium[END_REF]. 1.20 Representation of a virtuality continuum, adapted from [START_REF] Milgram | A taxonomy of mixed reality visual displays[END_REF]

Median values for the user-perceived attention demand of displacement

of set of objects measured as absolute rating (top) and relative ranking (bottom), highlighting the used interaction technique, the size of object to move and the the cardinality. Note: In both graphs, displacement conditions are ordered by ascending rating values; higher numerical values show larger attention demand. . . The touch version of the application the Tangisense Tabletop. . . . . . . 5. [START_REF] Akhras | Dominotes[END_REF] The gloves that simulate the touch feature on the tabletop, with the RFID tags in the index fingers. (a) The left hand glove. (b) The right hand glove.110 5. 4 Possible types of disaster where our system could be used. (a) Contaminated zones [START_REF] Christensen | 30 years after the chernobyl meltdown, why is the ukrainian government pushing nuclear energy[END_REF]. (b) Fire zones [START_REF] Pedneault | Fire inside an abandoned convent in massueville, quebec, canada[END_REF]. (c) Battlefields; adapted from [START_REF] Cnn | The battle of aleppo in 20 photos[END_REF]

Introduction

Context and motivations

In recent years, personal computing units became pervasive. This follows Mark Weiser's observation in 1999, when he defined ubiquitous computing as a computing that is always present, providing service but invisible [START_REF] Weiser | The computer for the 21 st century[END_REF]. However, it is not just small devices that are making ubiquitous computing more present in every day's life, in recent years. Relatively big devices are also integrating into our everyday environment, and are making computing more ubiquitous. Large interactive surfaces and screens are good examples, as we can find them installed in shopping centers, offices, smart homes, laboratories, etc. Interactive surfaces encompass tabletops, that have horizontal representation, making it is possible to interact with applications of different domains, for different purposes. Recent advances in capture and tracking technologies allow different interaction modalities on tabletops surfaces, such as through multi-touch and tangible techniques. The emergence of Augmented Reality and Virtual Reality, permits to imagine situations types of interaction; beyond simply combining the tactile and tangible interaction on one platform. Furthermore, humans do not interact with machines exclusively, but also with other humans and objects in their environment(s). Such interactions can be achieved and enriched by combining different types of interactions performed in different environments. Therefore, combining tangible interaction on tabletops with dual reality (a setup where two symmetrical world are interconnected and can mutually influence each other [START_REF] Lifton | Dual reality: Merging the real and virtual[END_REF][START_REF] Lifton | Dual reality: an emerging medium[END_REF]) is a new interesting concept. It gets more motivating when we combine this concept with the usage of several distributed tabletops and a multi-user environment.

The topic of this thesis explores the possibility to formalize interactions and representations, by designing a novel theoretical framework that models and explains interaction and representation metaphors, operating in dual reality spaces. These metaphors should faithfully convey the actions of local and remote users. The framework should help dual reality applications designers in their work, as their designs can be based on it, making interactions bridging both of the physical and the simulated (virtual) environments of dual reality, in a purpose of performing tasks and solving given problem(s) in a well defined domain.

Research questions

Research in the field of information and communication technologies has always been challenging and thriving. Researchers have then conducted works in different areas of this domain, resulting in different recommendations, methods, products, and technologies, which in turn lead to new research questions.

Taking into consideration the advances in Human-Computer Interaction domain, more precisely advances in tabletops interactions and the appearance of dual reality, we address their combined use and their convenience; and which can have a significant effect on the users' daily life interactions. This does not affect only users of applications and systems in dual reality, but also designers who design and build them. Therefore, it led us to seek answers to the following research questions:

• What may combining tabletops and dual reality lead to, in terms of new perspectives in Human-Computer Interaction?

• What benefits can integrating tangible interaction into a dual reality environment bring to the users of a system in dual reality?

• How can designers better represent and model applications in dual reality?

• Do tangible interaction improve user experience1 on distributed tabletops setup in dual reality?

Our contributions

To answer our research questions, we have carried out this thesis. It presents our research work on dual reality and interactive tabletops, particularly on tangible interactive tabletops. In a first step, we aim to identify and characterize interactive tabletops that are available commercially today (for business and industrial usage) and also in laboratories, for research purposes. The goal of this classification is to facilitate the work of future researchers on tabletops, through proposing characterisation of the tabletops according to their capture technologies, interaction mode, and many other criteria. We then propose a comparison between Virtual Reality, Augmented Reality, Mixed Reality, and dual reality based on several criteria. Second, we propose a design framework that accounts for interactions and representations in dual reality setup. The design framework we propose takes also into account the tangible interaction on tabletops, which have appeared as alternatives to the classic personal computers (that use WIMP type of interaction) in many domains. We use UML structural and behavioral diagrams to model the representations and interactions in this framework.

Third, we propose to validate our framework through four steps. We conducted two preliminary studies, in laboratory, to better understand tactile and tangible interactions on the tabletops surfaces. The outcome of these two preliminary studies is used in two other studies, which we conducted as proof of concept of our proposed framework. We designed the two studies, based on our proposal and the outcome of the preliminary studies, in two different domains. The first study consists of remote control of robots using one tabletop, and the second one consists of remote monitoring of patients with Alzheimer's disease using two distributed tangible tabletops.

Structure of the thesis

This thesis manuscript is organized as follows.

In the first chapter, we present the background of our research topic. First we propose a literature review of tangible interaction, encompassing its definition, applications and benefits, and its interaction forms. Then, we propose a literature review about tabletops with a particular focus on tangible tabletops, as they stand for our interaction platform. We also identify, present and characterize interactive tabletops, based on the following criteria: name and reference, capture technology, display, size, communication, and supported types of interaction. Finally, we end this chapter with a literature review of dual reality, consolidated by several application examples of this concept and a comparison to Augmented Reality, Virtual Reality, and Mixed Reality.

The second chapter of this manuscript exposes our contributions and proposal. We start by considering systems' design in dual reality and we propose to model the structure using UML structural diagrams. We then show how it is possible to integrate tangible interaction into our model (of dual reality), while also considering other types of interaction, such as tactile. We finish this chapter by explaining interactions and dependencies between different actors and/or components of a system in dual reality, based on our previously proposed model. We use UML behavioral diagrams for this purpose and we propose interactions models.

In the third chapter, we expose our first preliminary study, designed to understand differences between three different objects sizes (small, medium, and big), of the same shape, in terms of performances and user preferences. At the beginning of this chapter we present our material support of our research, the interactive tabletop called TangiSense 2. We use this tabletop in all our user studies in the following chapters, therefore we present briefly the project behind its production, then we describe it from both hardware and software perspectives, along with its functioning principles and architecture. We also provide some details about the software library customization according to our needs and the implementation of tactile feature on its surface.

Next, we describe our motivations for this study and its design; we expose our findings and we conclude this chapter with recommendations regarding the objects' size to use in the coming studies.

In the fourth chapter we present our second preliminary study, which consists of understanding the user performances and attention demand on a tabletop. We expose our motivations and the study design, then we present our findings and design guidelines derived from this study. We conclude this chapter by indicating how our findings can help us design our studies to validate our model proposed in Chapter 2.

The fifth chapter exposes a study aiming to validate our model proposed in Chapter 2. The study is focused around an application, for remote robot control, implemented in dual reality setup and which uses tangible tabletop as an interaction platform. The design of this study is based on our findings of the previous preliminary studies, and is considered as an implementation of the proposed model. We end this chapter by exposing our results and recommendations, and we show our research perspectives.

The last chapter of this manuscript presents a second study, in which we show another proof of concept of our model presented in Chapter 2. We expose an application prototype that may help to monitor patients with Alzheimer's disease remotely, using distributed tabletops. It is designed in dual reality setup and it instantiates our model, using distributed tabletops architecture, with a tabletop on each side of the dual reality.

Finally, we conclude this work with a general conclusion in which we recall our contributions, highlight the novelty of our work and expose our perspectives.

Chapter 1

State of the art

Introduction

Interactive tabletops are very different from the personal computer, currently widely used in day-to-day life. Indeed, the notion of interactive tabletops implies a collaborative and co-located workspace, allowing to involve several users at the same time. Nowadays, some tabletops offer also remote and distributed collaboration between users, such as in [START_REF] Bouabid | Design and evaluation of distributed user interfaces between tangible tabletops[END_REF]. Researchers and developers have developed applications and platforms that support multi-user collaboration simultaneously, such as multi-pointing (used in document editing for example) and multi-modal interactions in real time (the usage of round tangible gadget to rotate in menu list, then validating an item with a touch interaction for example.

In recent years, interactive tabletops are becoming more ubiquitous than ever before, and available for public usage after being restricted to research only, in their early years of appearance. They also currently offer different interaction modalities and multimodal interactions, such as multitouch, tangible, gestural, etc. With the emergence of new technologies and concepts such as Augmented Reality and dual reality, Human-Computer Interaction is given new horizons and perspectives, where researchers can innovate and create new concepts by merging these concepts and technologies with tabletops in an interactive environment.

In this chapter we introduce what is tangible interaction, expose its applications and possible interaction forms. Next, we present tabletops and we particularly focus on tabletops that support tangible interaction, along with their emergence as new interaction platforms; we also expose a census of 112 available tabletops nowadays, classified according to different criteria and technical characteristics. Next we present a literature review of dual reality and we provide several examples of systems implanted in dual reality setup. We finally end this chapter with a brief comparison between dual reality, Augmented Reality, Virtual Reality, and Mixed Reality.

Tangible interaction

Tangible interaction is an interdisciplinary area that describes design approaches and related research. It started to emerge in the 90s then rapidly became an interest for designers and researchers, like in Human-Computer Interaction (HCI) domain and interaction design. Different disciplines such as IoT, robotics and ubiquitous computing have contributed to the development of tangible interaction.

Definition

Tangible User Interfaces (TUIs) bridge the physical world and the digital world by manipulating physical artifacts (known also as tangible objects) to interact with digital representations [START_REF] Hornecker | Getting a grip on tangible interaction: A framework on physical space and social interaction[END_REF][START_REF] Ishii | Tangible bits: beyond pixels[END_REF][START_REF] Ullmer | Emerging frameworks for tangible user interfaces[END_REF]. These artifacts are used for representations and for controls at the same time, they can be static like in [START_REF] Terrenghi | Affordances for manipulation of physical versus digital media on interactive surfaces[END_REF] or dynamic such as in [START_REF] Rubenstein | Programmable self-assembly in a thousand-robot swarm[END_REF]. Dynamic tangible objects are mobile and usually equipped with motor(s), sensor(s), screen(s), etc. Ishii and Ullmer in [START_REF] Ishii | Tangible bits: towards seamless interfaces between people, bits and atoms[END_REF] introduced and considered TUIs as an alternative to the Graphical User Interfaces (GUIs) that makes greater use of physical space and real-world, day-to-day, objects as interface tools.

Benefits and applications

Different benefits of TUIs have been shown in several user studies and researches, these benefits include natural affordances of tangible objects [START_REF] Fitzmaurice | Bricks: laying the foundations for graspable user interfaces[END_REF], bimanualism [START_REF] Fitzmaurice | An empirical evaluation of graspable user interfaces: Towards specialized, space-multiplexed input[END_REF][START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF], enjoyment and programming self-beliefs [START_REF] Horn | Comparing the use of tangible and graphical programming languages for informal science education[END_REF][START_REF] Melcer | Bots & (Main)Frames: Exploring the Impact of Tangible Blocks and Collaborative Play in an Educational Programming Game[END_REF], positive impact on learning such as in [4,[START_REF] Manches | Physical manipulation: evaluating the potential for tangible designs[END_REF][START_REF] Marshall | Do tangible interfaces enhance learning?[END_REF][START_REF] Melcer | Bots & (Main)Frames: Exploring the Impact of Tangible Blocks and Collaborative Play in an Educational Programming Game[END_REF], attention demand [START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF] and various aspects of embodiment [START_REF] Hornecker | Getting a grip on tangible interaction: A framework on physical space and social interaction[END_REF]. Through these TUIs studies and others, researchers have outline several guidelines to assist practitioners in different aspects of design. We cite here guidelines related to better collaborating [START_REF] Kubicki | RFID interactive tabletop application with tangible objects: exploratory study to observe young children' behaviors[END_REF][START_REF] Schneider | Benefits of a Tangible Interface for Collaborative Learning and Interaction[END_REF], learning [START_REF] Fitzmaurice | Bricks: laying the foundations for graspable user interfaces[END_REF][START_REF] Manches | Physical manipulation: evaluating the potential for tangible designs[END_REF][START_REF] Marshall | Do tangible interfaces enhance learning?[END_REF], better illustrate the information [START_REF] Everitt | Two worlds apart: bridging the gap between physical and virtual media for distributed design collaboration[END_REF][START_REF] Ullmer | The metadesk: models and prototypes for tangible user interfaces[END_REF][START_REF] Leithinger | Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration[END_REF]. Other studies have also shown that TUIs enhance the level of engagement of users [START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF], facilitate exploration and promoting action and interaction [START_REF] Price | Using 'tangibles' to promote novel forms of playful learning[END_REF], enhance performance in a problem-solving activity [START_REF] Schneider | Benefits of a Tangible Interface for Collaborative Learning and Interaction[END_REF], improve the user experience during manipulation tasks [START_REF] Merrad | Tangible tabletops and dual reality for crisis management: case study with mobile robots and dynamic tangible objects[END_REF][START_REF] Pangaro | The actuated workbench: computer-controlled actuation in tabletop tangible interfaces[END_REF][START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF][START_REF] Richter | Remote active tangible interactions[END_REF][START_REF] Riedenklau | An integrated multi-modal actuated tangible user interface for distributed collaborative planning[END_REF] and offer accessible interaction design for people with impairment [START_REF] Ducasse | Botmap: Non-visual panning and zooming with an actuated tabletop tangible interface[END_REF][START_REF] Mcgookin | Clutching at straws: using tangible interaction to provide non-visual access to graphs[END_REF]. A study shows that, like for touch interaction, in some cases users do not use bimanualism in tangible interaction [START_REF] Tuddenham | Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks[END_REF] even though it is possible to use both hands, while another study report positive results for bimanualism in tangible interaction on tabletops [START_REF] Terrenghi | Affordances for manipulation of physical versus digital media on interactive surfaces[END_REF]. Like for touch interaction, this could be affected by many factors affecting, including the design of the system and the nature of tasks. We briefly note that TUIs outperformed multi-touch interfaces in many tasks like acquisition/manipulation [START_REF] Tuddenham | Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks[END_REF], layout manipulation [START_REF] Lucchi | An empirical evaluation of touch and tangible interfaces for tabletop displays[END_REF], grouping [START_REF] North | Understanding multi-touch manipulation for surface computing[END_REF] and sorting [START_REF] Terrenghi | Affordances for manipulation of physical versus digital media on interactive surfaces[END_REF].

TUIs systems can be implemented on different material supports. For instance, we can cite interactive surfaces whose applications among many are for controlling robots [START_REF] Guo | Utilizing physical objects and metaphors for human robot interaction[END_REF][START_REF] Guo | Touch and toys: new techniques for interaction with a remote group of robots[END_REF][START_REF] Krzywinski | Robotable: a tabletop framework for tangible interaction with robots in a mixed reality[END_REF][START_REF] Merrad | Tangible tabletops and dual reality for crisis management: case study with mobile robots and dynamic tangible objects[END_REF], creating and editing music [START_REF] Arfib | Prototyping musical experiments for tangisense, a tangible and traceable table[END_REF][START_REF] Jordà | The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces[END_REF], education and learning [START_REF] Anastasiou | A questionnaire-based case study on feedback by a tangible interface[END_REF][START_REF] Kubicki | Evaluation of an interactive table with tangible objects: Application with children in a classroom[END_REF][START_REF] Kubicki | RFID-driven situation awareness on TangiSense, a table interacting with tangible objects[END_REF][START_REF] Schneider | Benefits of a Tangible Interface for Collaborative Learning and Interaction[END_REF][START_REF] Tabard | The elabbench: an interactive tabletop system for the biology laboratory[END_REF]. Interactive surfaces include interactive walls (vertical support) such as the work of Detken et al. [START_REF] Detken | The search wall: tangible information searching for children in public libraries[END_REF] and Buur et al. [START_REF] Buur | Hands-only scenarios and video action walls: novel methods for tangible user interaction design[END_REF], and tabletops which are -horizontal-interactive displays that emphasize collaboration, planning, organizing, and other spatially-situated activities [START_REF] Mandryk | Display factors influencing colocated collaboration[END_REF][START_REF] Rogers | Collaborating around large interactive displays: which way is best to meet?[END_REF][START_REF] Scott | System guidelines for co-located, collaborative work on a tabletop display[END_REF]. Among the technologies used for capturing objects on these tabletops we find RFID as in [START_REF] Bouabid | Design and evaluation of distributed user interfaces between tangible tabletops[END_REF][START_REF] Kubicki | TangiSense: présentation d'une table interactive avec technologie RFID permettant la manipulation d'objets tangibles et traçables[END_REF]173], camera-based detection like in [START_REF] Wall | Demo i microsoft surface and the single view platform[END_REF], acoustic and infrared like in Tviews [START_REF] Mazalek | The TViews Table for Storytelling and Gameplay. Concepts and Technologies for Pervasive Games: A Reader for Pervasive Gaming Research[END_REF], fiber and optical like in [START_REF] Baudisch | Lumino: tangible blocks for tabletop computers based on glass fiber bundles[END_REF][START_REF] Weiss | Madgets: actuating widgets on interactive tabletops[END_REF] and light sensors like in [START_REF] Hauert | Instant city[END_REF]. More examples and details about capture technologies will be given later in a dedicated section (Section 1.3.3.2).

Tangible interactive interfaces (such as tangible tabletops) are considered as mixed systems [START_REF] Dubois | Conception, Implémentation et Evaluation des Systèmes Interactifs Mixtes: une Approche basée Modèles et Centrée sur l'Interaction[END_REF], this is because of the interaction technique using tangible objects as input in the real world to manipulate digital information in the virtual world. More details about this notion will be given in Section 1.4.3.3.

Interaction forms

Tangible Interaction encompasses user interfaces and interaction approaches that emphasize the following [START_REF] Hornecker | Tangible interaction[END_REF]:

• tangibility and materiality of the interface,

• physical embodiment of data,

• whole-body interaction,

• the embedding of the interface and the users' interaction in real spaces and contexts.

Based on these statements, tangible interaction could happen on different supports and could use different devices and/or objects; hereafter we mention some forms along with application examples and descriptive figures.

Tangible Interactive Surfaces

Users can interact with tabletops using physical objects such as for controlling robots [START_REF] Guo | Utilizing physical objects and metaphors for human robot interaction[END_REF][START_REF] Guo | Touch and toys: new techniques for interaction with a remote group of robots[END_REF][START_REF] Krzywinski | Robotable: a tabletop framework for tangible interaction with robots in a mixed reality[END_REF][START_REF] Merrad | Tangible tabletops and dual reality for crisis management: case study with mobile robots and dynamic tangible objects[END_REF], creating and editing music [START_REF] Arfib | Prototyping musical experiments for tangisense, a tangible and traceable table[END_REF][START_REF] Jordà | The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces[END_REF] (see Figure 1.1 (b)), education and learning [START_REF] Anastasiou | A questionnaire-based case study on feedback by a tangible interface[END_REF][START_REF] Kubicki | Evaluation of an interactive table with tangible objects: Application with children in a classroom[END_REF][START_REF] Schneider | Benefits of a Tangible Interface for Collaborative Learning and Interaction[END_REF][START_REF] Tabard | The elabbench: an interactive tabletop system for the biology laboratory[END_REF] (see Figure 1.3 and Figure 1.4), serious gaming [START_REF] Lepreux | Jeu sérieux pour le tri des déchets sur table interactive avec objets tangibles : étude exploratoire[END_REF] (see Figure 1.1) and other usages like "The Actuated Workbench" [START_REF] Pangaro | The actuated workbench: computer-controlled actuation in tabletop tangible interfaces[END_REF], "The metaDESK" [START_REF] Ullmer | The metadesk: models and prototypes for tangible user interfaces[END_REF], "DataTiles" [START_REF] Rekimoto | Datatiles: a modular platform for mixed physical and graphical interactions[END_REF]. Interactive surfaces also include interactive walls such as the works of Detken et al. [START_REF] Detken | The search wall: tangible information searching for children in public libraries[END_REF] (see Figure 1.2) and Buur et al. [START_REF] Buur | Hands-only scenarios and video action walls: novel methods for tangible user interaction design[END_REF].

Shape-changing interfaces and transformable shapes

A Shape changing interface is usually an interactive surface that changes physical shape to convey certain properties of digital data, it changes its topology so that users can feel and manipulate data with their hands and body. Researches and works in this context have been done by Nakagaki and his colleagues in MIT 1 . Some of their works are "inForce" [START_REF] Nakagaki | inforce: Bi-directionalforce'shape display for haptic interaction[END_REF], "Materiable" [START_REF] Nakagaki | Materiable: Rendering Dynamic Material Properties in Response to Direct Physical Touch with Shape Changing Interfaces[END_REF] (see Figure 1.5), "SoundFORM" [START_REF] Colter | Sound-FORMS: Manipulating Sound Through Touch[END_REF], "Conjure" [START_REF] Levine | Conjure[END_REF] (see Figure 1.6), "Tangible CityScape" [START_REF] Tang | Tangible cityscape[END_REF] (see Figure 1.7), "Physical Telepresence" [START_REF] Leithinger | Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration[END_REF][START_REF] Leithinger | Shape Displays: Spatial Interaction with Dynamic Physical Form[END_REF], "Programmable Droplets for Interaction" [START_REF] Umapathi | Programmable droplets for interaction[END_REF][START_REF] Umapathi | Programmable Droplets for Interaction[END_REF].

Figure 1.5: Manipulating and simulating landscapes with flexible, elastic and viscous rendered material properties [START_REF] Nakagaki | Materiable: Rendering Dynamic Material Properties in Response to Direct Physical Touch with Shape Changing Interfaces[END_REF]. Other similar works and researches have been done by Hardy et al. such as ShapeClip" [START_REF] Hardy | Shapeclip: towards rapid prototyping with shape-changing displays for designers[END_REF] and by Villar et al. such as "Project Zanzibar" [START_REF] Villar | Project Zanzibar: A Portable and Flexible Tangible Interaction Platform[END_REF]. Also, one noticeable work is that of Nakagaki et al. [START_REF] Nakagaki | ChainFORM: A Linear Integrated Modular Hardware System for Shape Changing Interfaces[END_REF] where they presented "ChainFORM", a linear, modular, actuated hardware system. Its form is inspired by modular and serpentine robotics and can be constructed and customized into different interactive applications (see Figure 1 Furthermore, in another article [START_REF] Nakagaki | Designing linebased shape-changing interfaces[END_REF] they highlight the new trend in HCI of shapechanging lines like [START_REF] Nakagaki | Lineform: Actuated curve interfaces for display, interaction, and constraint[END_REF] and encourage researchers and designers to further investigate this novel direction for HCI.

Robots and mobile devices

Among other supports that tangible interaction could be based on are robots and mobile devices. One noticeable work in this matter is Zooids [START_REF] Le Goc | Zooids: Building blocks for swarm user interfaces[END_REF]: the authors designed a platform that consists of a collection of custom-designed wheeled micro robots (each 2.6cm in diameter), a radio base-station, a high-speed DLP structured light projector for optical tracking, and a software framework for application development and control. This user interface is thus comprised of many autonomous micro robots that represent both of the display and the interaction medium. These authors consider it as a Swarm User Interface and say that "Zooids can be held as tokens, manipulated collectively or individually, behave as physical pixels, act as handles and controllers, and can move dynamically under machine control (see Figure 1.9). Figure 1.9: Different possible usages and arrangements of Zooids [START_REF] Le Goc | Zooids: Building blocks for swarm user interfaces[END_REF].

Other works include Teegi [START_REF] Fleck | Teegi, he's so cute: example of pedagogical potential testing of an interactive tangible interface for children at school[END_REF][START_REF] Frey | Teegi: tangible EEG interface[END_REF][START_REF] Frey | Scientific Outreach with Teegi, a Tangible EEG Interface to Talk about Neurotechnologies[END_REF] which is an interactive robot, designed to be used in educational context. Its purpose is -to enable children-to discover the relation between the brain activity and the human body functions in an easy, engaging and informative way (see Figure 1

Synthesis

We have presented in this section the concept of tangible interaction and detailed some of its forms and supports, along with examples of applications and/or systems for each support. In the next section we focus on interaction on tabletops; then we go deeper to focus on tangible interaction on tabletops.

Tabletops

In this section we focus on tabletops -interaction medium-and describe their principles. A part of this section is dedicated to present tabletops available nowadays in the market and/or in labs (made only for research purposes) and describe them regarding several criteria. We also focus on tangible tabletops and their application domains as this thesis is based on tangible interaction around tabletops.

Main interaction principles

The concept of an interactive tabletop gives a first idea or impression of a collaborative and co-located workspace, allowing several users to be involved at the same time in the same space and on the same or on different task(s). There exists nowadays many interactive tabletops distinguished by several criteria, some of them are based on multitouch user interface such as BendDesk [START_REF] Völker | BendDesk: Seamless Integration of Horizontal and Vertical Multi-Touch Surfaces in Desk Environments[END_REF][START_REF] Weiss | Benddesk: Seamless integration of horizontal and vertical multi-touch surfaces in desk environments[END_REF][START_REF] Weiss | Benddesk: dragging across the curve[END_REF] and ELC2 [START_REF] Elecom | Table Tactile ELC2[END_REF]. Some other tabletops are based on tangible user interfaces, allowing interaction with physical objects, among others we can mention TangiSense [START_REF] Arfib | Prototyping musical experiments for tangisense, a tangible and traceable table[END_REF][START_REF] Kubicki | TangiSense: présentation d'une table interactive avec technologie RFID permettant la manipulation d'objets tangibles et traçables[END_REF], the drift table [START_REF] Gaver | The drift table: designing for ludic engagement[END_REF] and mixiTUI [START_REF] Pedersen | mixitui: a tangible sequencer for electronic live performances[END_REF][START_REF] Pedersen | Tangible bots: interaction with active tangibles in tabletop interfaces[END_REF]. A musician using mixiTUI tabletop [START_REF] Pedersen | mixitui: a tangible sequencer for electronic live performances[END_REF].

Meanwhile there exist some tabletops which are based on mixed user interface, allowing interactions with both virtual and tangible objects such as the ReacTable [START_REF] Jordà | The reactable: a tangible tabletop musical instrument and collaborative workbench[END_REF][START_REF] Jordà | The reactable[END_REF][START_REF] Jordà | The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces[END_REF] (see Figure 1.13) and Microsoft Surface Studio 2 [START_REF]Microsoft. Meet the surface studio 2 -the ultimate creative studio -microsoft surface[END_REF]. After having an idea about what a tabletop is, we can proceed to a classification of tabletops, according to different criteria as shown in the next section.

Technical characteristics and classifications

In this section we show a census and a classification of available tabletops nowadays. Our classification method is based on Kubicki's characterization of tabletops is his PhD thesis [START_REF] Kubicki | Contribution à la prise en considération du contexte dans la conception de tables interactives sous l'angle de l'IHM, application à des contextes impliquant table interactive RFID et objets tangibles[END_REF]. We have chosen technical specifications that distinguish tabletops the most and are shown in columns in Table 1.1. The columns names and their description are defined as follows:

• Names & references: indicates the name of the tabletop and its reference(s). We indicate N.C (Not Communicated) if the authors/manufacturer did not give it a name.

• Capture technology: this column describes the used technology to capture user(s) and/or object(s) interactions on the tabletops' surfaces. The used technology is specified, otherwise N.C or N.A when it is respectively Not Communicated or Not Applicable.

• Display: we specify with this characteristics if the platform has a visualization part. We distinguish two types: screens (LCD, LED, ...) and video-projections. In this latter case, we specify whether this is from above (Overhead) or below (Underneath) the table.

• Size: we mention the size of the table, when this information is specified by the authors/manufacturer. However, in some cases it is not mentioned if it is the size of the table itself or the size of the interaction surface. We also note that when the display is of screen type, the size refers to the screen dimensions and thus interaction surface. This measure is expressed in inches or centimeters as Length×Width or diagonal of the screen.

• Communication: some tabletops can communicate with each other and some others can make objects communicate with them or -objects-with each other.

Tabletops that do not have this feature are indicated by a No. For other tabletops, it is specified which technology is used or N.C when we do not know about it.

• Tactile: we determine whether the tabletops have the option of being used in a tactile way, i.e. allowing interactions related to touch. However, we have relied solely on the authors'/constructors' usages of their interactive It may happen that tabletops share some characteristics, particularly if they are issued by the same constructor. In this case, there exist other technical details that make the difference, such as the CPU, the screen resolution and quality, storage, memory, shape and/or -possible adjustable-height. We judged it unnecessary to mention them all as our work context is not related to them. Relative dissatisfaction with user interfaces based on traditional screens, but also with virtual reality (what was considered as a distance from the people of the "real world") motivated the proposal of the first prototypes of Tangible User Interfaces, while technological innovations (such as RFID technology) have allowed their construction and design. In parallel, industrial product design has come to the fore to also engage in tangible interaction with new markets for devices, and/or objects, containing electronic components, exploiting digital technology, and that become interactive, communicative, even intelligent nowadays. For designers of interactive systems, this has created new challenges and opportunities [START_REF] Djajadiningrat | Tangible products : redressing the balance between appearance and action[END_REF][START_REF] Djajadiningrat | Augmenting fun and beauty: A pamphlet[END_REF].

Name & reference

In recent years, different ways of interacting with interactive surfaces and tabletops have became possible, such as using fingers in touch interfaces, an electronic pen, or even sheets of paper (if they can be identified automatically). The use of real, physical objects with user interfaces is another interaction modality that is becoming more popular. It focuses on actually switching from WIMP to post-WIMP and not just changing the display plan by carrying WIMP interfaces on interactive tabletops [START_REF] Van Dam | Post-wimp user interfaces[END_REF].

Therefore, TUIs offer to its users the possibility to manipulate data in a way that is as close as possible to reality. Users can manipulate physical objects from day-to-day real life and can react in a natural way [START_REF] Kubicki | Utilisation en classe d'un jeu sérieux sur table interactive avec objets tangibles pour favoriser l'activité des élèves : une évaluation comparative en cours préparatoire[END_REF]. Moreover, designing a Tangible User Interface is a user-centered design since it generally starts from the analysis of two important factors that are directly related to the users:

1. Their particular needs with respect to the application.

2. How it resolves these needs in real life.

The main objective in designing a TUI is to materialize the data, i.e. to pay attention to actions and perceptions carried out in real life and to integrate them into a digital representation [START_REF] Ishii | Tangible bits: towards seamless interfaces between people, bits and atoms[END_REF][START_REF] Sugimoto | Design of Systems for Supporting Collaborative Learning Augmented with Physical Artefacts[END_REF].

Objects' recognition techniques

As shown in Table 1.1 there exist several technologies that allow to recognise and identify tangible objects on tabletops' surfaces. We can mention the following technologies and techniques:

• One technique is RFID detection; it consists of tagging physical objects with RFID tags and when these objects get close a RFID reader or antenna they are read/captured. Each RFID tag (and hence each object) has a unique ID number that allows it to be identified and triggers the corresponding interaction on the tabletop surface. • Laser Light Plane (LLP) technology uses the infrared light IR of a laser just above the surface. The plane of light is about 1mm thick and is placed just above the surface. When the finger touches the laser beam, the beam hits the fingertip and is then recorded as a drop of infrared light by the camera installed below. Most camera systems use the ReacTIVision framework [START_REF] Bencina | Improved topological fiducial tracking in the reactivision system[END_REF], implying the use of a tag under objects (see Figure 1.16) with a better performance.

Figure 1.16: Illustration in general of a camera capture system (here, below the table) allowing fingers and/or tangible objects detection (adapted from [START_REF] Kubicki | Contribution à la prise en considération du contexte dans la conception de tables interactives sous l'angle de l'IHM, application à des contextes impliquant table interactive RFID et objets tangibles[END_REF]).

• Some recent multi-touch tabletops allow interactions with tangible objects on their surfaces. They recognize objects using triangulation of 3 touch points for each object and (x, y) coordinates for locations.

Application domains of tangible tabletops

Tabletops are becoming more and more present in many life areas, thanks to the different developed applications. In this subject we can mention among others applications in the domain of education, training and learning such as the waste sorting application [START_REF] Havrez | A design model for tangible interaction: case study in waste sorting[END_REF] that simulates microbiological waste sorting in a practical educational setup; another application is the eLabBench [START_REF] Tabard | The elabbench: an interactive tabletop system for the biology laboratory[END_REF] a system that supports experimental research in biology laboratory using tangibles on an interactive tabletop. In the domain of music and media editing, we can mention the ReacTable tabletop which can be used as a musical instrument and as a tabletop application allowing the construction of different audio topologies.

In domotics and home applications like the cooking assistant called Cooking Ideas [START_REF] Lebrun | Management of Distributed RFID Surfaces: A Cooking Assistant for Ambient Computing in Kitchen[END_REF], which can suggest recipes and helps with the preparation, based on real ingredients put on the surface of an interactive tabletop.

In planning and simulation like [START_REF] Lebrun | Interaction Between Tangible and Virtual Agents on Interactive Tables: Principles and Case Study[END_REF] which allows road traffic simulations and road intersections configuration. Another recent application is ShapeUD [START_REF] Tang | ShapeUD: A Real-time, Modifiable, Tangible Interactive Tabletop System for Collaborative Urban Design[END_REF], a real-time, modifiable and tangible interactive tabletop system for participatory urban design. In medicine we can mention for instance the PaperLens system [START_REF] Spindler | Paperlens: advanced magic lens interaction above the tabletop[END_REF] which uses a tracked sheet of paper to navigate volumetric, layered, zoomable and temporal spaces with regard to the Z-dimension (height above the tabletop surface); in this context, it can for example show different layers of the human body (bones, muscles, ...) depending on the paper's height from the tabletop's surface. In robotics and remote control of machines and mobile robots like in [START_REF] Merrad | Tangible tabletops and dual reality for crisis management: case study with mobile robots and dynamic tangible objects[END_REF], an application that allow users to remotely control mobile robots by displacing tangible small toys on a small-scale map on the tabletop's surface.

Beside these domain-oriented applications, some applications are designed for older persons because they provide more comfort and ease of use than traditional platforms, such as desktops, laptops, tablets and smartphones (because of their small font sizes and small screen sizes). Some others are designed to meet the requirements of people with special needs; several research and studies have aimed to design and study the feasibility of systems that run on tabletops and meet two fundamental criteria: usability and satisfaction of the needs, as seen by the concerned users [START_REF] Guerreiro | Blind people interacting with large touch surfaces: Strategies for onehanded and two-handed exploration[END_REF][START_REF] Li | A tangible tabletop game supporting therapy of children with cerebral palsy[END_REF][START_REF] Marco | Bringing tabletop technology to all: evaluating a tangible farm game with kindergarten and special needs children[END_REF].

Pros and cons of tangible interaction on tabletops

Among pros of tangible interaction technique we can mention the identification of users and who did what on the interaction surface, since each user and/or object is identified by a tag or associated to certain tangible objects. Also, it demands less attention and focus as users can count on their peripheral vision [START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF] and haptic feedback from physical objects.

Moreover, depending on the context, tangible interaction can be faster, easier to learn/understand and more intuitive than other interaction techniques such as touch or in-air interaction techniques.

Meanwhile, it also has some cons like the rigidity of physical objects besides of being static. This feature limits their utility comparing with, for instance, digital objects which are malleable and easy to create, replicate and edit [START_REF] Pangaro | The actuated workbench: computer-controlled actuation in tabletop tangible interfaces[END_REF]. However, nowadays there exists some shape-changing objects and dynamic objects, as shown in previous sections (Section 1.2.3.2 and 1.2.3.3), that relatively overcome these shortcomings of tangibles.

Distributed architecture of applications on tabletops

The concept of distribution consists of having more than one interaction support for the same application or system. The work PSyBench of Brace, Ishii and Dahley [START_REF] Brave | Tangible interfaces for remote collaboration and communication[END_REF] was among the first ones to suggest a possibility of distributing a TUI. They proposed to replace traditional video/audio conferencing with tangible interfaces. Researches in HCI have shown how it is possible to distribute the user interfaces, for one single application, into different supports and/or devices. For instance, Cooking Ideas [START_REF] Lebrun | Management of Distributed RFID Surfaces: A Cooking Assistant for Ambient Computing in Kitchen[END_REF] is a distributed application on a tabletop (TangiSense) and a screen. On the first interaction support users can grab ingredients together, select which type of meal to prepare and see what ingredients to mix or exclude, whilst on the second support (screen) they can see the recipe preparation guide, which ingredients are missing and if they wish to order them online. Figure 1.17 shows this aspect of distribution in Cooking Ideas. Another example of a distributed application or system is the work of Bouabid et al. [START_REF] Bouabid | Design and evaluation of distributed user interfaces between tangible tabletops[END_REF] where they designed a distributed application for one or several children on one hand and one or several supervisors on the other hand, on two interactive tabletops. It consists of children learning colors by placing colorless objects in colored frames on one tabletop, while the supervisor assists and validate their actions on the other tabletop. Figure 1.18 shows the two tabletops and the application running on them.

Synthesis

In this section we have seen how tabletops constitute a good medium for -large-surface computing. A census of available tabletops was given along with their main technical characteristics and most distinguishing criteria. One thing we notice is that there is a tendency in recent years that tabletops are made with LCD or LED screens and offer both of tactile and tangible (object recognition) interaction. Also, they offer a great variety of communication means such as Bluetooth as Wi-Fi, and can host a wide range of applications.

1.4 Dual reality: a paradigm relaying the real and the virtual

1.4.

Definition

Lifton in his PhD thesis [START_REF] Lifton | Dual reality: an emerging medium[END_REF] defined the concept of dual reality as "an environment resulting from the interplay between the real world and the virtual world, as mediated by networks of sensors and actuators. While both worlds are complete unto themselves, they are also enriched by their ability to mutually reflect, influence, and merge into one another". He continues later on that same thesis that "sensor networks will turn the physical world into a palette, virtual worlds will provide the canvas on which the palette is used, and the mappings between the two are what will make their combination, dual reality, an art rather than an exact science. Of course, dual reality media will in no way replace other forms of media, but rather complement them". Finally Lifton [START_REF] Lifton | Dual reality: an emerging medium[END_REF] adds that "a complete consideration of dual reality must also include the possibility of sensor data from the virtual world embodied in the real world. Insofar as technically feasible, dual reality is bi-directional". According to Lifton in his thesis [START_REF] Lifton | Dual reality: an emerging medium[END_REF], a direct mapping between the two worlds (mapping of the real to virtual and vice-versa) maybe not be the most appropriate in all situations. Hence he proposes a mapping strategy shaping the virtual world according to our subjective perceptions of the real world (see Figure 1.19), whereas Figure 1.20 shows a simplified representation of continuum virtuality [START_REF] Milgram | A taxonomy of mixed reality visual displays[END_REF]. We note that Molina et al. have extended this model, offering a wider view of UIs, considering not only 3D UIs, but also 2D GUIs and command-based interfaces (1D UIs) [START_REF] Molina | A space model for 3d user interface development[END_REF]. Lifton and Paradiso in their paper [START_REF] Lifton | Dual reality: Merging the real and virtual[END_REF] added that both worlds are enhanced by the ability to mutually reflect, influence, and merge by means of sensor/actuator networks deeply embedded in everyday environments. In this same paper and to pursue with the idea, they presented a system made under the Dual Reality paradigm: using a plug sensor node, the system demonstrates the information flow from the real world to a virtual environment, this latter one is implemented in the Second Life Online Virtual World [156,[START_REF] Rymaszewski | Second Life: The Official Guide[END_REF], where the data sensed from a real object (such as light, temperature, motion, sound and electrical current) influences the corresponding digital representation.

Dual reality application domains and implementation examples

The dual reality concept can be applied in different domains, particularly in technical fields. In this section, we present examples of dual reality implementation that we judge to be expressive and best describe an illustration of what dual reality is. We explain each example along with eventual Figures when available. Back et al. also developed mobile and web-based collaboration environments, that could be used simultaneously and interchangeably, allowing users to collaborate distantly in industrial settings, such as factories in one country and managers in another country.

Example two: Shadow Lab

In [START_REF] Lifton | Dual reality: Merging the real and virtual[END_REF] Lifton and Paradiso have showed another example called "Shadow Lab", which is a virtual space in Second Life that reflects a real lab. The real lab has the Plug sensor network [START_REF] Lifton | A platform for ubiquitous sensor deployment in occupational and domestic environments[END_REF][START_REF] Lifton | Tricorder: A mobile sensor network browser[END_REF][START_REF] Lifton | Dual reality: an emerging medium[END_REF] deployed; it exemplifies their real space to virtual space mapping. Figure 1.22 shows the implementation of "Shadow Lab" where an avatar is standing; their labspace -in the foreground-is rendered in detail while the rest of the building -like in the background-was represented by a map. The authors says that "The primary feature of Shadow Lab is the to-scale twodimensional floor plan of the third floor of our building. Only a small portion of the entire space is modeled in three dimensions. In part, this is due to the difficulty and resource drain of modeling everything in three dimensions" [START_REF] Lifton | Dual reality: Merging the real and virtual[END_REF].

Figure 1.22 shows the map of the lab in the virtual space (Shadow Lab), with approximately 30 data ponds. These data ponds are placed accordingly to their positions of their corresponding Plugs in the real lab [START_REF] Lifton | Dual reality: Merging the real and virtual[END_REF], demonstrating the dual reality paradigm.

Example three: A Visual Monitoring and Management Tool

Kahl in [START_REF] Kahl | A visual monitoring and management tool for smart environments[END_REF] proposed a monitoring tool for monitoring smart spaces, which are composed of a real and a virtual component. In each component, sensors and actuators are installed in order to visualize data, exchange data and control the physical counterpart. The real part is the smart space itself, while the virtual one is a 3D model of this space. According to the dual reality paradigm [START_REF] Lifton | Dual reality: an emerging medium[END_REF], these two interconnected worlds can influence each other. In fact, the management dashboard can visualize a real smart space in an 3D interactive model; and thus, the virtual model can reflect changes in the real world while information are transferred from the virtual space to the real space. Benefits of systems like this can be seen in urban management control stations and smart factories.

In addition to this work, Kahl and Bürckert proposed an event-based communication infrastructure [START_REF] Kahl | Architecture to enable dual reality for smart environments[END_REF] which allows interconnection between different services in an instrumented environment. This work also fits in the definition of the dual reality paradigm [START_REF] Lifton | Dual reality: an emerging medium[END_REF].

Example four: retail shelf planning

In this example [START_REF] Raber | The comparison of performance, efficiency, and task solution strategies in real, virtual and dual reality environments[END_REF], the realistic task of "shelf planning" from retail domain is replicated in virtual world. This is a daily task that is being performed in retails in order to optimize the profit, which consists of ordering and positioning the products in a shelf. A real and a virtual environment have been designed where users can place real and virtual products at desired positions on shelves. The influence is mutual between the two environments, which are kept always "synchronized".

In a complete dual reality setup, if a product placement is done in one side (virtual or real) it should be automatically replicated in the other side (real or virtual, respectively). To workaround this automation in the experimental setup, a hidden -person-assistant ensures the work of a robot grabbing and placing tangible products in the shelves according to the virtual counterpart. This dual reality setup was part of a study comparing the efficiency of dual reality, its performance and its task solution strategies to virtual and real setups (see Figure 1.23). In this same context of retail, Kahl et al. [START_REF] Kahl | Management dashboard in a retail scenario[END_REF] have worked on virtual dashboard that offers a real time visualization of an actual supermarket in an interactive 3D model, including simulators and communication channel between the two worlds. It reflects changes in the real world instantly to the virtual world, and information from the virtual world are also interpreted in the real world. This work in linked to the living lab project Innovative Retail Laboratory [START_REF] Spassova | Roots for the Future of Ambient Intelligence[END_REF] of the DFKI3 in collaboration with German retailer GLOBUS SB-Warenhaus Holding in St. Wendel.

Another similar work in retail is that of Khan et al. [START_REF] Khan | Pervasive application evaluation within virtual environments[END_REF], which consists of a virtual supermarket. A user can interact with this environment by moving his/her head and then according to the orientation the virtual scene is rotated, allowing the user to explore the area. A shortcoming of this system is that is uni-directional, hence it does not fit into the dual reality paradigm.

Comparison with other related concepts

In order to a have clearer vision of dual reality, we describe in what follows related terms and concepts that dual reality is often confused with, then we highlight differences between them in a comparative table.

Augmented reality (AR)

Augmented Reality (AR) is an environment in which virtual objects are superimposed upon or composited with the real world objects. Therefore, it would be seen as if the virtual objects coexist with the real objects in the same space. AR is then considered as a supplement of reality itself, rather than a replacement, where users can see the real world and interact with both of real and virtual objects [START_REF] Azuma | A survey of augmented reality[END_REF].

One of the recent applications developed in AR, which has gained a world wide popularity, is Pokémon Go game [START_REF] Pokémon | [END_REF]. It is a location-based mobile game in which players locate, battle, capture, and train Pokémons. These Pokémons are virtual creatures, which appear on the player's mobile phone screen as if they are in the real-world location. This can be achieved using the mobile phone GPS and camera;

Virtual reality (VR)

Virtual Reality (VR) is regarded as a computer-generated environment (therefore it is completely virtual) in which users are completely immersed to interact with -virtual-objects. Therefore VR replaces the users' world with a virtual one, and while immersed they cannot see the real world around them [START_REF] Azuma | A survey of augmented reality[END_REF]. Users in this environment interact through controllers, such as a set of wired or wireless gloves/controllers and a position tracker; they also wear a head-mounted stereoscopic display or headset for visual output [START_REF] Steuer | Defining virtual reality: Dimensions determining telepresence[END_REF].

Mixed reality (MR)

Mixed Reality (MR) is a concept in which the two approaches of AR and VR merge and cohabit together in a continuum, in between them [START_REF] Milgram | A taxonomy of mixed reality visual displays[END_REF] (see Figure 1.20). Therefore, MR takes place not only in the physical or the virtual world but in a mix of them, taking characteristics from each world and making each one of them a potential user or system input and/or output. Users can thus manipulate objects in both physical and virtual environments, and while immersed they have one foot in the real world and the other one in the virtual one. Some smartglasses such as Microsoft HoloLens [180] are a good example of mixed reality systems (see Figure 1.26).

Comparison

One of the differences between dual reality and mixed systems (mixed reality systems) is that the latter merges both of real -physical-and virtual components into one world [START_REF] Dubois | Conception, Implémentation et Evaluation des Systèmes Interactifs Mixtes: une Approche basée Modèles et Centrée sur l'Interaction[END_REF]. Whilst in a dual reality system, the two worlds are kept separate from each other but linked with sensors and actuators. The following If we compare the real and the virtual environments, along with the emergence from and to each one. The emergence concerns space (S), time (T), objects (O) and people (P), also called the STOP metaphor, in each environment as depicted in Figure 1.27. The main difference that stands between dual reality and the others is that in dual reality, at a given time instant t, STOP are available in both environments and synchronized through bidirectional interconnections as shown in Figure 1.28. Figure 1.28: STOP metaphor in both environments and interconnection between real and virtual side of dual reality.

Synthesis

We have presented in this section the dual reality and its concept, which is a representation of abstract and concrete aspects of a given problem in a shared interactive interface. We have also presented some application domains of dual reality which include, among others, simulation, monitoring and planning. We also highlighted the main differences between AR, VR, MR and dual reality from several perspectives as described in Table 1.2.

Conclusion

After having introduced the tangible interaction, its applications and interaction forms, we have presented tabletops, which are eventually considered as interaction platforms, with a focus on tangible interaction on their surfaces. We have also provided a classification based technical characteristics of tabletops such as the capture technology, the display, the size, communication technology, and possible interaction types (tactile, multitouch, and tangible). We have presented in details these criteria as well as the different values that they can take, and then we exposed a census of 112 tabletops according to our characteristics and criteria.

Next, we have presented a literature review of dual reality, including its definition and several examples of systems in dual reality. We ended this section by brief review of Augmented Reality, Virtual Reality, and Mixed Reality; followed by a comparison of these three concepts with dual reality.

In the next chapter, we propose to define a framework that takes into consideration the theoretical concept of dual reality and its paradigm, interactive tabletops and tangible interaction. This framework can be useful for different usages, in different contexts.

Chapter 2

Design principles for dual reality applications supported by tangible tabletops

Introduction

We have presented in Chapter 1 the many use cases of tabletops as new interaction platforms. They make it possible to collaborate in divers situations, particularly with applications using touch and/or tangible interaction techniques and objects, thus bringing together the advantages of both techniques. This combination however challenges both the design and the evaluations of resulting interactive setups and presents therefore new challenges, not only within the tabletop itself as a platform but also for adapting the developed applications to the dual reality paradigm.

Our objective in this chapter is to introduce design principles for designers and developers of applications in dual reality, and using tangible tabletops as interaction support. We therefore propose to model, in a generic manner, dual reality applications design. These applications use tangible tabletops and tangible interaction on the virtual side of dual reality. Hence, our framework allows to bridge in a generic manner two separate environments based on the dual reality paradigm as described in Section 1.4.

We keep both sides of the dual reality separated and symmetrical. The user does not belong to any side of the dual reality; instead, s/he is modeled in a dedicated part, along with personal details, associated tasks, and activities. This separation is derived from the dual reality principle as we have seen in Chapter 1 and is depicted by UML structural diagrams (we use package and class diagrams). We also integrate the notion of tangibility -on tabletops -into the interaction between the user and the system or application. The interaction between the user and the system/application and between components of the system/application (without having the user doing anything) is described by UML behavioral diagrams; we use sequence diagrams.

Taking into consideration the duality

We propose to model the interaction between the two sides of dual reality by a set of UML diagrams. We propose to start with a high level modeling using package diagrams, then progressively expose details of packages and dependencies between their contained classes.

Package diagrams for dual reality applications modeling

Our start point of modeling is based on separating the real world and the virtual world into two packages, while the user is modeled in a third dedicated package. Every package of these three contains other packages that we describe in details later on in this same section. This structure allows to model a given system in dual reality and keep the system components well organized. Figure 2.1 illustrates our basic modeling of a system in dual reality. As shown in Figure 2.1, a basic notion is to keep the symmetry and consistency between the two sides of dual reality. In fact, each package on one side of the dual reality has its counterpart on the other side, as indicated by names (Platform Package is the counterpart of Environment Package), thus making it possible to have a mutual influence and a bi-directional interaction. This latter is modeled by the bi-directional dependency (arrow) between VirtualSide Package and RealSide Package. The User Package has two unidirectional dependencies to both sides packages, as a user can interact with virtual objects or with real objects.

The symmetry between the two sides can guarantee that some objects and components must be represented in both sides, real and virtual, in order to keep both sides synchronised. Although, it is not compulsory to have exactly the same components in both sides (eventually packages), as some components are exclusive for only their side(s). Moreover, we consider our modeling as basic and generic for the dual reality modeling; therefore it is possible to add other package(s) in one or in both sides of the dual reality, depending on the context of usage and case study.

The Communication Packages are used to establish and run the interconnection between both sides, i.e. from and to virtual and real sides. The communication technology which is usually wireless differs from a platform to another, therefore it can be Wi-Fi, ZigBee, xBee, Bluetooth, or others. The topology of such network connection is similar to a computer network in master-slave topology (see Figure 2.2), whether with one or many slaves, or in star topology as depicted in Figure 2 The CoreApplication Packages represent the application itself and its tools/objects if there are any, its content is therefore different from an application context to another. However, some basic and generic components are required for every application running on a dual reality setup. They are considered as the core components and they ensure the symmetry and the mutual influence between the two sides components, i. Meanwhile, the User Package handles the user and his/her activities around the tabletop (virtual side) and/or on the real side. The user can interact with both sides, such as by moving objects, and cannot be influenced by the objects or the system, i.e. the system/application cannot change some or all of his/her properties. This is modeled by unidirectional dependencies, as shown in Figure 2.4. This latter illustrates that the user(s) can interact only with the CoreApplication Package of either VirtualSide Package or RealSide Package, in order to effect and/or change other packages. This is because the user can only interact with objects (real or virtual) of the application. Furthermore, there can be many users collaborating around one tabletop simultaneously; yet, the previous modeling still valid for multi-user situations.

Every package in the VirtualSide Package, in RealSide Package and the User Package is composed of several classes; these classes are listed and packaged (grouped) respectively in The CoreApplication Package encompasses classes related to the functional aspect of the system. The main class in this package is SoftwareApplication, which is the managing entity of all the other classes. The package contains also MappingAndCalculation class, which ensures all the conversions and calculations to maintain the correct ratio for representing the real world on a small scale; PresentElementsOnRealSide, a class with a list of elements and objects available on the real world; PresentElementsOnVirtualSide, another class with a list of elements and objects available on the virtual (digital) world; VirtualEnvironmentProperties, a class defining all the properties and characteristics of the virtual part of dual reality. VirtualObject, a class representing an object (digital or physical) in the virtual part of dual reality, its position on the tabletop is defined by Position class, of the same package.

The Platform Package models the physical support that the user interacts with. It contains Platform class which represents the tabletop hardware or device; its display is then modeled by the class Display; Finally, some platforms may also have an embedded AI, which can be centralized or distributed and can prove helpful with, for example, the use of a Multi-Agent System [START_REF] Lebrun | Architecture multi-agent pour la gestion d'objets tangibles et virtuels sur Table Interactive[END_REF].

Several technical characteristics exposed in Section 1.3.2 are found in this package as different attributes of different classes. This basic modeling can eventually be extended with additional classes as we illustrate in Section 2.3. The User Package classes are user-centered and encompass the user profile and his/her activities around the tabletop. This package is then linked directly and only to the Core-Application packages of both sides, as the user(s) can interact with the system/application by directly manipulating objects, and/or menus on the application interface (this one is on the virtual side). Furthermore, this modeling is also in adequacy with The Artifact, User, Task, Organization, Situation (AUTOS) Pyramid HCD Framework described in [START_REF] Boy | Orchestrating human-centered design[END_REF] and in [START_REF] Boy | The handbook of human-machine interaction: a human-centered design approach[END_REF]. In the next section we describe these classes in details using class diagrams.

Class diagrams

We have described in the previous section the usefulness of our defined classes, we describe in this section the dependencies between them. The VirtualSide Package is described by class diagram in Figure 2.8.

The SoftwareApplication class is at the heart of the modeling and is associated with several other classes. Since the application can run only on one physical platform, the SoftwareApplication class is then associated with only one Platform class; i.e., multiplicity 1. The Platform itself is composed of Display, as depicted by the aggregation dependency, and is associated with AI class.

The SoftwareApplication class is also associated with MappingAndCalculations class, with a multiplicity of "1..*" depending on the needs. It is also associated with Presen-tElementsOnVirtualSide class and PresentElementsOnRealSide class with a multiplicity of "1..*" both. This multiplicity can prove useful in a situation of managing several real sides of dual reality using the same application (for example several production units/areas of a factory). All of these classes are associated with only one SoftwareApplication as they belong to only one application of dual reality.

Properties of the virtual side of dual reality are depicted in a dedicated class, Vir-tualEnvironmentProperties, which is also associated toSoftwareApplication class with multiplicities of "1..*" on each side of the dependency; This is for the same reason of managing several real sides of dual reality using the same application. The Server class is strongly associated to the SoftwareApplication class (multiplicity of 1 on both edges of the dependency), meaning that each application has a Server instance managing the commands and the data exchange, and inversely, each instance of Server can only belong to one SoftwareApplication. The Server organizes the commands to send in queue (FIFO, to keep both sides synchronized and commands ordered) modeled by DataQue-ueToSendVirtualSide class which contains 0 or many commands (0..* ), as depicted by CommandVirtualSide class.

The user(s) may interact with the application or system using digital or physical objects on the tabletop surface. VirtualObject class takes into consideration this aspect, it represents objects that the user may use. Therefore it is associated with at least one SoftwareApplication (1..* ). Inversely a SoftwareApplication instance may use (or not) several VirtualObject, hence the multiplicity "0..*". Furthermore, each VirtualObject has one and only one (multiplicity of 1) position on the tabletop surface, modeled by Position class. More details about using digital or tangible (physical) objects will be provided in Section 2.3. The Client class has a dependency with RealEnvironmentProperties class, which contains properties and characteristics that serve to maintain the symmetry and consistency between the two sides of dual reality. This RealEnvironmentProperties class has, among many other properties, a location; which is depicted by Location class. A real side of dual reality has then only one physical location (as shown by multiplicity of 1 in Figure 2.9). It can meanwhile be influenced and changed by external elements and factors that are not -expected-part of the real side, this is modeled by DisruptiveElement class.

These disruptive elements can also influence objects used on the real side of dual reality, hence the dependency with RealObject class (with "0..*" multiplicity on each edge). This latter is managed by only one Client class; inversely, a Client class can manage many RealObject. As every RealObject on the the real side of dual reality is characterized by its own relative location in the area, it is then associated with one Position class.

The RealObject class is very generalized, the objects manipulated by the user are then depicted in RealElement class, which inherits from RealObject class. Objects manipulated by the user may have sensors implanted on them, therefore we model this by the Sensor class associated to RealElement with multiplicity of "0..*" on each edge of the dependency.

Integrating tangibility

We have presented so far how a system can be globally modeled in dual reality, without taking into account how the user interact with it. In this section we explain how tangible interaction can be integrated in the modeling of a system. The user can interact with the application objects on the virtual side to perform tasks and do activities; these objects are elements of the application and differ from one to another, depending on the application context. They can be graphical objects (such as a digital image representing something on the interaction surface for example), which the user(s) can manipulate via touch technology. This is modeled by the VirtualElement class, that inherits from VirtualObject class.

Users can also interact with physical objects (such as small toys for example, having 3D shapes and tangibles as shown in Figure 2.13), which the users can directly grab with hand(s). Each tangible object is then represented by TangibleElement class in our model that is a specialization of VirtualObject class. Depending on the objects identification and tracking technique used, tangible (or physical) objects on the surface of the tabletop are equipped with tags; this can be for example RFID tags or QR code. Our model proposes to associate each TangibleElement with a tag, represented here by Tag class with a composition dependency. Our proposal is also in adequacy with Kubicki's modeling [START_REF] Kubicki | Contribution à la prise en considération du contexte dans la conception de tables interactives sous l'angle de l'IHM, application à des contextes impliquant table interactive RFID et objets tangibles[END_REF] of interactive tabletops, that takes into consideration tangible objects. Several network techniques such as RFID and Bluetooth allow the tabletop to interact with tangible objects on its surface. It is therefore possible to capture and track several objects at the same time. The class Interaction shown in Figure 2.12 models this need and provides at least one interaction type for each tabletop. There can be several technologies used for this matter as presented in Table 1.1, such as RFID, camera detection and IR detection of objects.

Consequently, several classes can be specialized from this Interaction class in the same Platform Package, depending on the interaction type and/or technology. The interaction with the tabletop can be through several interaction techniques; some tabletops also support more than one interaction technique at the same time. For instance, a user can use a pen with one hand and his/her other hand's finger(s) on the tabletop surface to perform a task. Tangible class, inherited from Interaction class, depicts possible tangible interaction technique supported by the tabletop. The same thing applies to MultiTouch class and Mid-Air class; they inherit from Interaction and represent possible interaction techniques supported by the tabletop. Other interaction types can eventually be specialized from the Interaction class, including the ones that may appear in the future. This makes our model extensible and adaptable across time.

If a system is based on Multi-Agent architecture and/or using Multi-Agent platform such as Jade1 , each VirtualObject is associated with one Agent at most, as represented by the multiplicities in Figure 2.15. In return, an Agent can be associated to one and only one class, and it contains all of the agent properties such as its role and objective. More details about MAS for tangible and virtual objects interaction on tabletops are given in Lebrun's thesis [START_REF] Lebrun | Architecture multi-agent pour la gestion d'objets tangibles et virtuels sur Table Interactive[END_REF].

Modelling the interaction

We use UML sequence diagrams to describe the interaction between different parts of our modeling. We present interactions between the user and the system in the following sections.

Interacting with virtual object

In a dual reality setup, the user can interact with a virtual object (hence, it is in the virtual side) and consequently manipulate a real object in the other side (real side), and/or making changes to the other side environment. Figure 2.19 shows a sequence diagram for a user manipulating a real object via interacting with a virtual object. The sequence that is to the left of Server lifeline, including this latter and excluding the User, describes the interaction(s) and the exchange(s) made in the virtual side of the dual reality. Meanwhile, what is to the right of the Server lifeline, excluding this latter, describes the interaction(s) and exchange(s) on the real side of the dual reality. We notice that the server and client lifelines are in charge of bridging the two sides of dual reality.

Changes made by the user are applied to the physical object (depicted by RealObject in Figure 2.19), through the applyChanges() method. The physical object (RealObject) has to be equipped with actuators in order to physically make changes in real life, whether on itself or on its environment.

A user can also make changes on the real side environment by interacting with a virtual object. Figure 2.20 describes this interaction. The message sent to the user at the end of the interactions is a notification; it can take several forms such as a message on the tabletop screen, an audible sound, an haptic feedback, etc. In both diagrams in Figure 2.19 and Figure 2.20, the user interacts with VirtualObject, aiming to generalize interactions with TangibleElement and VirtualElement as they are specialized and inherit from VirtualObject.

Interacting with real object

As indicated in the general dual reality class diagram in Figure 2.16, a user can also interact with a real object on the real side of the dual reality, which in return affects the virtual part of dual reality. In this sequence the interaction starts from the user, goes to the real side of dual reality (RealObject and Client lifelines), then goes to the virtual side of dual reality (Server, SoftwareApplication and VirtualObject lifelines). Like when interacting with a virtual object, the message sent to the user at the end of the interactions is also a notification for the user, and it can also take several forms such as a message on the tabletop screen, an audible sound, an haptic feedback, etc.

Disruptive element affecting the system

Changes on the real side that are reflected on the virtual side can also be caused by elements belonging to the real environment, rather than the user. Examples of these -disruptive-elements can be an unexpected obstacle on the path of a robot, or external weather changes (such as snow or ice) that can affect the functioning of the system. A disruptive element class models real things and/or conditions on the real side of the virtual reality that can eventually affect both sides of dual reality. This affection interaction is modeled in the sequence diagrams in As a disruptive element can influence the real side environment (hence make changes to RealEnvironmentProperties), changes are reflected to the VirtualEnvironmentProperties passing through respectively Client, Server, and SoftwareApplication. At the end of this interaction sequence, an update confirmation is then returned to SoftwareApplication and a notification is sent to the user. Changes in the RealEnvironmentProperties can also influence a real object and consequently changes are reflected into the virtual side of dual reality. This interaction sequence starts from DisruptiveElement and goes respectively through RealEnvironmentProperties, RealObject, Client, Server, SoftwareApplication, to finally arrive to VirtualObject. An update confirmation is then returned to SoftwareApplication and an update notification is sent to the user. This interaction is modeled in Figure 2.24.

Conclusion

In this chapter we have proposed to a framework of dual reality applications design, using tangible tabletops and tangible objects on the virtual side of dual reality. Very little research is being done on mixing dual reality and tangible interaction, or bridging two separate environments based on the dual reality paradigm as described in Section 1.4.

We first proposed, in a generic manner, a separation of real side and virtual side of dual reality. The user has also been modeled separately, using a packaging model based on the definition of the dual reality proposed by Lifton [START_REF] Lifton | Dual reality: an emerging medium[END_REF]. Then we proposed a modelling using class diagrams, detailing every package and proposing a class for every necessary element on this setup. When it comes to interaction modelling, we proposed to model the generic interactions such interacting with virtual and real objects. This modelling was done using UML sequence diagrams.

Our modelling takes also into consideration the interaction platform and its related characteristics, such as its display, interaction technology and the possibility to have an AI. In the coming chapters, we present proof of concept of our modelling on two systems (Chapter 5 and Chapter 6), and we describe them according to the class diagram model and the package diagram model proposed in this chapter.

Chapter 3

Preliminary study 1: determining optimal objects' size for performing elementary tasks on tangible tabletop surface

Introduction

After having presented our proposal in the previous chapter, it comes now to implement it and evaluate it on applications. To do this, we first propose to present two preliminary studies 1 . The first one aims to determine the optimal objects' size to be used for our dual reality applications design; we present this preliminary study in this chapter. The second one focus on understanding the user performance and attention demand using both hands on a tabletop that we present in the next chapter.

In all our studies, we propose to use the TangiSense 2 tabletop, which, using RFID technology, allows its user(s) to interact with tangible and with virtual objects. We first present the interactive tabletop, its interaction principles, its technical details and how we implement a touch interaction on its surface. We will present hardware and software aspects of this tabletop. Finally, we present our preliminary study about the optimal objects' sizes for evaluating our model with tangible and virtual objects.

Our motivation behind this preliminary study is that before starting our experiments and studies, we want to know what is the optimal size of objects to be used on the tabletop, particularly the size of tangible objects. As these latter may have different 3D shapes, we choose cylindrical shaped objects since our tangible objects used on the tabletop are cylindrical or have a similar shape to a cylinder. We end this chapter with a conclusion of objects' size to be used in our next studies.

Presentation of the "TangiSense 2" tabletop

Description of the tabletop

The TangiSense tabletop was designed and developed by RFIdées company2 , as part of the ANR TTT and then IMAGIT projects. It is available in two versions, the first version of this tabletop is described in [START_REF] Kubicki | TangiSense: présentation d'une table interactive avec technologie RFID permettant la manipulation d'objets tangibles et traçables[END_REF]. Contrarily to most interactive tabletops which offer tactile interactions, this tabletop offers only tangible interactions on its surface by means of tangible -physical-objects.

TangiSense uses RFID technology to capture and identify physical objects on its surface. This technology works on the principle of a reader and a transmitter [START_REF] Finkenzeller | RFID handbook: fundamentals and applications in contactless smart cards, radio frequency identification and near-field communication[END_REF]. RFID tags (put on each tangible object) are composed of a chip that can contain a small-sized memory, connected to an antenna that reacts to radio waves (see Figure 3.1). These tags are usually read by a reader connected to a computer. The communication between the tag and the reader is established by radio frequency and not by optical reading (such as for barcode reading). The major difference between a barcode and an RFID tag is that the former allows to identify a family of products while the RFID tag allows the unique identification of an object with a tag. RFID tags can be read remotely (1 -2cm above this tabletop surface); moreover, several tags can be read at the same time by one single or many reader(s)/antenna(s). The antennas work on a radio frequency of 13.56 MHz. Figure 3.1: Overview of RFID technology (here by a matrix of antennas) [START_REF] Kubicki | Contribution à la prise en considération du contexte dans la conception de tables interactives sous l'angle de l'IHM, application à des contextes impliquant table interactive RFID et objets tangibles[END_REF] The "TangiSense 2" has all the features of the "TangiSense 1", in addition to a graphical display screen [START_REF] Valls | Documentation TangiSense -Matériel et montages[END_REF] instead of LEDs. The sensing matrix is composed of 24 panels (6×4), each panel contains 16 square RFID antennas (4 × 4) of 3.75cm wide each, making it a total sensing surface of 90cm × 60cm. It is possible make other different sizes of this same tabletop by adding or removing panels. Note that the smallest possible sensing surface area of the tabletop is one panel itself, equivalent to 4 × 4 antennas or 15cm × 15cm area.

Each panel contains its Digital Signal Processor (DSP) to process RFID antennas readings, its antenna multiplexer and its communication processor [START_REF] Valls | Documentation TangiSense -Matériel et montages[END_REF]. Reading strategies are hierarchical and the code is distributed between the antenna reading processor, the multiplexing processor and the host computer. The panels are linked together by a control interface connected to the host computer via Ethernet. The response time obtained through Ethernet communication and RFID reading offers very promising speed performance. The "TangiSense 2" tabletop, has a 47" LCD screen allowing a very high quality display of virtual objects as well as a much higher visual accuracy than with version 1 of the tabletop, which uses LEDs. This surface incorporates on right and left sides of the tabletop a non-interactive and sensor-less area of about 7cm wide, allowing objects to be placed on its surface without being detected (Figure 3.2 (b)). Furthermore, this version of the tabletop is based on an Multi-Agent System (MAS) to define the interaction rules between tangible objects and virtual objects. Further details could be found in [START_REF] Lebrun | A multi-agent system approach for interactive table using rfid[END_REF] and in [START_REF] Lebrun | Interaction Between Tangible and Virtual Agents on Interactive Tables: Principles and Case Study[END_REF].

Interaction with tangible objects

There are two types of objects that can interact with the "TangiSense 2" tabletop: virtual objects and tangible objects. The first type is visual and graphical objects projected or displayed on the tabletop surface/screen. For this purpose, two technologies are possible on the TangiSense tabletop: the use of a set of LEDs placed on the surface of the table and/or the use of a video projector placed above it. These virtual objects can be manipulated by the user using a glove equipped with RFID chips to track the movements of the user's hand [START_REF] Kubicki | New human-computer interactions using tangible objects: Application on a digital tabletop with RFID technology[END_REF][START_REF] Merrad | Tangible tabletops and dual reality for crisis management: case study with mobile robots and dynamic tangible objects[END_REF][START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF] or with a tangible object as a mean of interaction (see Figure 3.3 (a)). As the table is equipped with RFID antennas, it can detect physical objects initially equipped with RFID tags (see Figure 3.3 (b)). With this technology, it even becomes possible to recognize objects superimposed on each other (provided that the tags are placed at a maximum height of 3cm from the RFID antennas). It is therefore possible to detect that an object is hidden under another larger one, determine its position and/or content if necessary, by reading information stored in its RFID tag(s) (which was not possible with a vision-based system even if the detection of stacked objects is conceivable [START_REF] Fujii | Tangible search for stacked objects[END_REF]). It is therefore possible to store information in these tags, such as tag movements history or the authentication information of the associated person or object in a unique and secure manner. This way, physical objects can be identified separately and used on the tabletop surface, including the possibility of several persons interacting simultaneously.

Software architecture of the tabletop

From a software perspective, the adopted architecture is comprised of different layers [START_REF] Kubicki | TangiSense: présentation d'une table interactive avec technologie RFID permettant la manipulation d'objets tangibles et traçables[END_REF][START_REF] Kubicki | New human-computer interactions using tangible objects: Application on a digital tabletop with RFID technology[END_REF], Figure 3.4 shows a global view of these layers. Here after we describe them in details, cited from lowest to highest level:

• The Capture and Interface layer: detects tangible objects each with one or more tags and feeds the information back to the traceability layer.

• The Traceability layer: handles events associated with objects and communicates the objects' positions changes to the application layer.

• The Application layer: manages the specificities of the applications associated with the tabletop. It serves as an interface with the user.

The application layer is divided into two parts:

• The part integrating the MAS which provides computing and reasoning power.

The MAS of the TangiSense interactive tabletop was developed using Jade platform [START_REF] Bellifemine | Jade: a white paper[END_REF][START_REF] Bellifemine | Developing multi-agent systems with a FIPA-compliant agent framework[END_REF]; it manages the behavior of tangible objects manipulated by users Figure 3.4: TangiSense tabletop software architecture [START_REF] Kubicki | RFID-driven situation awareness on TangiSense, a table interacting with tangible objects[END_REF] as well as virtual objects (video-projected or graphically displayed) that could be used within applications. Each tangible and/or virtual object is associated to an agent [START_REF] Lebrun | A multi-agent system approach for interactive table using rfid[END_REF]. Information from RFID readers is transmitted by the middleware (Capture & Interface and Traceability layers). This information is used by agents to build a vision of their environment. The hierarchical organization of agents allows an intelligent management of objects, but also to assign roles to each one of them [START_REF] Adam | Flexible roles in a holonic multi-agent system[END_REF]. In order to design a MAS that controls the behavior of tangible or virtual objects for a given application, it is necessary to define the relationships between the different types of agents as well as the functional roles they will play depending on the application.

• The Human-Machine Interaction (HMI) part, in charge of communication with users, it allows to transmit user interactions information to the other -lower-layers (e.g., creating and moving a virtual object under a tangible object manipulated by a user). This part contains all the necessary methods to make the tabletop usable by users (humans). It contains the tabletop's interface (the visual) and is in charge of displays (with LEDs, video-projection or LCD screen). It also manages various sound interactions that can be adapted to applications on the tabletop.

Customizing the software library

The TangiSense tabletop has its own SDK that is provided by RFIdées company. It comes as a ".jar" file that needs to be included in the application project, using Java programming language.

The customization made to this library was a sort of optimizing some methods content and changing some passing arguments for more flexibility and easiness of use. For instance in the original version of the library, when an event happens on the tabletop surface (entrance, movement or departure of an object), the responsible method for reporting this event uses the associated RFID tag ID number (string class) as an argument, we changed this argument to the RFID tag itself (TagRfid class) which includes the tag ID and many other attributes. This change allowed a better flexibility, better access and more availability of information.

Tactile feature implementation on "TangiSense 2"

Throughout some of our works, we have implemented a simulated tactile feature on the tangible tabletop "TangiSense 2". This feature uses gloves and RFID tags, glued at the end of each finger. Users have to wear these gloves in order to use the tactile feature on the tabletop (see Figure 3.5). Technically, the tabletop does not detect the user's fingers on its surface but the associated RFID tags at the end of the fingers; nonetheless, the user perceived interaction technique is tactile and not tangible. If a user wearing the glove(s) put his/her finger(s) on the tabletop surface, the tag(s) is(are) detected and consequently the event is triggered. This developed tactile interaction is mainly for selection tasks and pointing on the tabletop surface. The drag and drop of virtual objects may not work perfectly, because of the RFID sensors network precision (See Figure 3.1 and Figure 3.2): a matrix of 24 × 16 sensors of 3.75cm × 3.75cm each.

3.3 Determining optimal objects' size for performing elementary tasks on tangible tabletop surface

Study design

We conducted an experiment to compare users' perceived attention demand and performance between touch and tangible interaction when moving a set of objects. In particular, we are interested in understanding if the object size has an effect on the users' perceived attention and performance or not. We describe our study in the following sections.

Apparatus

The study was conducted on the TangiSense 2 tabletop, set on a display resolution of 1920 × 1080. The tabletop screen is a 47", and has a display surface of 90cm × 60cm.

It uses RFID technology to detect tangible objects on its surface and it offers a sensing capacity of 16 × 24 objects on its surface at the same time, corresponding to 16 × 24 -square-RFID antennas of 3.75cm long each. The tabletop is connected to a computer running Windows 10. Our software application is developed with Java. In the case of tangible technique, we used cylindrical white objects of 3 different sizes, described as follows:

1. Small size: 1.5cm of radius, 6.5cm of height and weighing approximately 10g. We refer to this size in this work by "S".

2. Medium size: 2.25cm of radius, 10.3cm of height and weighing approximately 20g. We refer to this size in this work by "M".

3. Big size: 3.5cm of radius, 14.5cm of height and weighing approximately 35g. We refer to this size in this work by "B".

Each cylindrical object is equipped with a RFID tag, which its size is smaller than the antenna size so it can be captured by only one antenna at once. We note that the RFID tag size is the same for the different objects' sizes. In the case of touch technique, we used two gloves: one for left hand and one for right hand, each of them equipped with a RFID tag in the index finger. They have the same characteristics as the cylindrical objects' ones. The gloves are used to simulate a touch with the tangible tabletop.

Our implementation of the application and this preliminary study are considered to take part inside the red dotted area of our model, shown in Figure 3.6. It deals with user performances and concentration using tangible and touch objects, hence respectively Tan-gibleElement and VirtualElement as represented in our model, on the tabletop's surface.

Tasks, procedure and design

Participants have to move a set of objects from the area A to the area B. Like in [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF], the areas positions are on the top (area A) and bottom (area B) of the surface. Examples of the graphical display in the touch and tangible techniques are shown in Figure 3.7. The trial ends when all objects inside the area A are entirely moved to the area B. The object displacement is ended when the object tag is entirely within a free target's boundary in the area B. To notify the participants that the -graphical-object is well selected, the graphical circle representing the object on the table becomes green around. To notify the participants that the object displacement is successful, the color of the target changes to green and cannot be used again, and the object representation in area A disappears. The next object displacement begins only when the current object displacement is successful. Similarly, the next trial starts only when the current trial is successful. At the end of each trial, the button "next" is displayed at the center of area A to ensure that the participant's hand is at the same position at the beginning of each trial. The participants were instructed to use only their dominant hands during the whole experiment, and to press on the "next" button to begin the next trial.

Dependent measures are analyzed using a 2 × 3 × 6 repeated measures within-subjects analysis of variance for the factors: interaction modality (touch and tangible), objects' sizes (S, M, and B); and cardinality. Cardinality can take the following values:

• (4,4): area A contains four objects and area B containing exactly four targets,

• (4,8): area A contains four objects and area B containing eight targets,

• (6,6): area A contains six objects and area B containing exactly six targets,

• (6,8): area A contains six objects and area B containing twelve targets,

• (8,8): area A contains eight objects and area B containing exactly eight targets,

• (8,16): area A contains eight objects and area B containing sixteen targets, Cardinality corresponds to the (numberOfObjects, numberOfTargets) used. The num-berOfObjects is the number of objects to move from area A and numberOfTargets corresponds to the number of available targets in the area B).

In the experiment phase, each trial began by presenting participants the set of objects to move from area A to area B and their sizes. Participants were instructed to use only their dominant hands to move all objects, regardless of their size and cardinality.

The experimenter is in charge of installing (respectively removing) tangible objects on the tabletop's surface when it is a tangible (respectively touch) interaction trial, as indicated by the application. For a given trial, it can only use one size of objects (S, M, or B) during the entire performance, see Figure 3.8 that compares the three different sizes of objects used in this study. When the participant selects/grabs an object, the application does not allow him or her to select another one until the selection of a target. In the case of tangible modality, participants were instructed to grab the object from area A, to move their -dominant-hand in the air and then to put it on the desired target. To have the same task condition, in the touch condition, participants were instructed to select the desired object, move the hand in the air and then to select the desired target. Participants were instructed to move the objects at normal speed. We do not give participants an order to select objects and targets, participants could select the object and the target they want in the order they prefer. In addition, for the touch condition, participants were instructed to use their index fingers, as the RFID tag is placed on this finger of the glove (see Figure 3.5).

We follow [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF] in setting up the objects in their starting areas, and we defined for each set of objects two different starting areas, each one moved with two repetitions, with a total of 2 techniques × 3 objects' sizes × 6 cardinality conditions × 4 repetitions (2 staring areas × 2 repetitions) = 144 trials produced by each participant. Our software application randomly presented the 144 sets to move to our participants. As trials are entirely randomized i.e., two successive trials can use different modalities, our participants were instructed to wear the -dominant hand-glove over all the experiment. When the modality changes between two successive trials or the two successive trials deal with tangible modality, the experimenter puts/removes the tangible objects on/from the tabletop surface. The experiment took in average 55 minutes to complete. Figure 3.9 summarizes the scenario and the sequence of the experiment; tasks highlighted in green are done by the experimenter in charge. Attention demand was collected at the end of the experimental task using absolute rating and relative ranking measurements, similar to the methodology of [START_REF] Rekik | Understanding users' perceived difficulty of multi-touch gesture articulation[END_REF][START_REF] Vatavu | Estimating the perceived difficulty of pen gestures[END_REF]. Rating was collected using a 5-point Likert scale (see Table 3.1), which was presented to participants as a table with five columns, one column for each rating value. Participants were asked to put each set of objects displacement condition (36 conditions to rate = 2 techniques × 3 objects' sizes × 6 cardinalities) in the appropriate rating column, af-ter having moved it one more time on the surface in order to re-enact the experience and, consequently, perceived attention demand. Participants were allowed to change the ratings of previously rated conditions at any time as they moved along with the rating process until they were confident of their final classification. After rating all conditions, participants were asked to provide an ordered list of the different experimental conditions in increasing order of attention demand, which represents our ranking measurement. We also asked participants to explain their assessment of attention demand: what they found demanding less attention for each task condition.

Rating

Explanation provided to participants 1. Not at all concentrated I moved the set of objects immediately with absolutely no need to pay attention. 2. Slightly concentrated I moved the set of objects, almost without paying attention.

3.

Somewhat concentrated I occasionally paid attention during execution.

Moderately concentrated

I paid special attention (i.e., I had to be concentrated) with each execution. 5. Very concentrated I paid very special attention (i.e., I had to be very concentrated) with each execution. Table 3.1: Likert questions employed to elicit attention demand rating scores. Inspired from [START_REF] Vatavu | Estimating the perceived difficulty of pen gestures[END_REF].

Results

Our results include the level of agreement between participants in terms of their perceived attention demand, performance measures, and qualitative observations.

Effect of Object Size on Attention Demand

We are interested in this section in the level of agreement between participants in terms of their perceived attention demand of objects displacements. To this end, we report and analyze 396 (result of 11 × 36) individual ratings of absolute attention demand and 11 rankings of relative attention demand collected from 11 participants.

Figure 3.10 illustrates participants' responses of the perceived attention demand of objects displacements when the object size is Small (S), Medium (M), and Big (B). Overall, we found a high degree of consensus between participants' responses. For instance, Kendall's coefficients3 stayed above 0.64.(W = 0.68, χ 2 (35)=264, p < 0.0001 for rating and W = 0.64, χ 2 (35) = 253, p < 0.0001 for ranking). These findings indicate that the objects displacement attention demand is perceived by people in a consistent manner, which justifies further investigation of the self-reported attention demand assessments.

We found a higher degree of consensus among participants when rating tasks done with touch technique than with tangible technique (W = 0.77 versus 0.74 for rating, and W = 0.77 versus 0.75 for ranking). However, there was non-significant effect of technique condition on self-perceived attention demand for either rating (z (N =11) = 0.60, n.s.) or ranking (z (N =18) = 0.41, n.s.).

We found a low degree of consensus among participants when rating tasks done with either big objects (W = 0.52, χ 2 (11) = 68.1, p < 0.0001 for rating and W = 0.44, χ 2 (11) = 57.9, p < 0.0001 for ranking), medium objects (W = 0.44, χ 2 (11) = 57.8, p < 0.0001 for rating and W = 0.44, χ 2 (11) = 58.8, p < 0.0001 for ranking) or small ones (W = 0.42, χ 2 (11) = 54.5, p < 0.0001 for rating and W = 0.48, χ 2 (17) = 152, p < 0.0001 for ranking). Friedman tests revealed that there were non-significant difference between the three object sizes for both rating (χ 2 (11) = 0.42, n.s) and ranking (χ 2 (11) = 4.53, n.s).

For touch condition, we found high degree of consensus among participants when rating tasks done big objects (W = 0.52, χ 2 (5) = 48.2, p < 0.0001 for rating and W = 0.68, χ 2 (5) = 44.6, p < 0.0001 for ranking), medium objects (W = 0.58, χ 2 (5) = 38.2, p < 0.0001 for rating and W = 0.66, χ 2 (5) = 43.8, p < 0.0001 for ranking) and small ones (W = 0.59, χ 2 (5) = 39.2, p < 0.0001 for rating and W = 0.60, χ 2 (5) = 39.3, p < 0.0001 for ranking).

For tangible condition, we found high degree of consensus among participants when rating tasks done big objects (W = 0.68, χ 2 (5) = 44.6, p < 0.0001 for rating and W = 0.7, χ 2 (5) = 46.2, p < 0.0001 for ranking), medium objects (W = 0.62, χ 2 (5) = 40.8, p < 0.0001 for rating and W = 0.70, χ 2 (5) = 46.1, p < 0.0001 for ranking) and small ones (W = 0.59, χ 2 (5) = 39, p < 0.0001 for rating and W = 0.70, χ 2 (5) = 46.2, p < 0.0001 for ranking).

User performance

The dependent measures are trial time, movement time, and distance. All analyses are multi-way ANOVA. Tukey tests are used post-hoc when significant effects are found.

Movement time. It is measured as the sum of the movement time of a each object inside the set of objects to move. The the movement time of a single object is measured from the first object movement, to target successfully selected.

Repeated-measures ANOVA revealed a significant effect of (numberOfObjects, num-berOfTargets) (F 5,50 =7.03, p < 0.0001) on movement time. Interestingly, post-hoc tests showed that moving a (8,8) cardinality (mean = 4968ms, s.d = 1175ms) is significantly slower than moving a (6,12) cardinality (mean = 3523ms, s.d = 1023ms), or a (4,4) cardinality (mean = 3187ms, s.d = 1039ms), or a (4,8) cardinality (mean = 2245ms, s.d = 996ms). We also found that (4,8) cardinality (mean = 2245ms, s.d = 996ms) is significantly faster than (8,16) cardinality (mean = 4509ms, s.d = 1175). There were no more main effect or interaction (p > 0.11) suggesting that the drawbacks of (8,8) cardinality and (8,8) cardinality are consistent across techniques and object sizes. Table 3 Similarly to movement time, there were significant main effect of cardinality (F 5,50 = 28.41, p < 0.0001) on trial time. Post-hoc tests revealed that the trial time decreased significantly as the number of objects to move decreased (p < 0.05).

Distance. It is defined as the sum of distance of each object of the set to move. The distance of a single object displacement is the distance between the object's placement (initial position) and its chosen target (final destination). We measured the distance in millimeters (mm).

There were significant main effects of object size (F 2,20 = 4.62, p = 0.0224) and cardinality (F 5,50 = 378.35, p < 0.0001) and a significant object size × cardinality interaction (F 10,100 = 2.49, p = 0.010196) on distance. Post-hoc tests revealed that, for each object size (S, M, and B), the distance increased significantly as the number of objects to move and their corresponding number of targets increased (p < 0.05). With (8,16) cardinality, the distance is significantly shorter when using big sized object (mean = 2405mm, s.d = 428mm) (respectively medium sized object (mean = 2620mm, s.d = 435mm)) than when using medium object size (mean = 2620mm, s.d = 435mm) (respectively small sized object (mean = 2756mm, s.d = 429mm)) (p < 0.05).

Mental model and qualitative results

We report in this section the different strategies used by our participants for objects and targets selection. These notes come mainly from our observations and our users comments during or after the experiment.

• One participant out of all had a tendency to start grabbing object from the center of the tabletop then finish on a side. Meanwhile, all the others had a tendency to start from their dominant hand side; i.e., right handed (respectively left-handed) participants start from right side (respectively left side) and finish on left side (respectively right side). We note that Yves Guiard has used the terminology of dominant and nondominant uni-manual; dominant and nondominant bi-manual, whether symmetrical or asymmetrical [START_REF] Guiard | Asymmetry in bimanual skills[END_REF].

• All participants who preferred tangible to tactile, or find it less concentrating, said that it was due to the physicality and the 3D nature of the tangible object. This physicality concerns the shape(s) of object(s) and not the data physical visualization(s) (representation) that can be achieved using a collection of self-propelled objects, as illustrated in [START_REF] Le Goc | Dynamic Composite Data Physicalization Using Wheeled Micro-Robots[END_REF].

• Participants who found that cardinalities (4,8), [START_REF] Arfib | Prototyping musical experiments for tangisense, a tangible and traceable table[END_REF][START_REF] Back | The virtual chocolate factory: mixed reality industrial collaboration and control[END_REF] and [START_REF] Art+com Studios | The Formation of Hamburg[END_REF][START_REF] Beckhaus | Back to the sandbox: playful interaction with granules landscapes[END_REF] are more attention demanding than respectively (4,4), [START_REF] Arfib | Prototyping musical experiments for tangisense, a tangible and traceable table[END_REF][START_REF] Arfib | Prototyping musical experiments for tangisense, a tangible and traceable table[END_REF], and (8,8) said that the reason was because they had to think and choose where to place the objects, as they have more than one option of targets. Whilst for the others (those who found the opposite), they said that it is for the same reason, i.e., because they had more options and liberty where to place the objects, hence less attention demand, thinking, and focus.

• Five participants had a tendency to stack objects in their targets from left to right, even when the cardinalities were (4,8), [START_REF] Arfib | Prototyping musical experiments for tangisense, a tangible and traceable table[END_REF][START_REF] Back | The virtual chocolate factory: mixed reality industrial collaboration and control[END_REF] and [START_REF] Art+com Studios | The Formation of Hamburg[END_REF][START_REF] Beckhaus | Back to the sandbox: playful interaction with granules landscapes[END_REF], i.e., even when they had plenty of options of free targets.

• One participant out of all said that, and particularly with small objects, having few objects -tangible or tactile-(i.e., (4,4) and (4,8)) is more demanding than having a lot of objects. This is because these latter offer more options from where to take and where to put or place them, whilst with few objects it requires to pay more attention when selecting objects and when choosing targets.

• Eight participants felt that moving small -tangible or touch-objects requires more time than medium and/or big objects. They said that it is because small objects require more precision when manipulating them.

Conclusion

We have presented in this chapter the TangiSense 2 tabletop, our research support that we use in all of our studies. We have presented its interaction principles, its technical details and how we could implement a touch interaction on its surface, thanks to RFID tag implanted in gloves' fingers. We have also briefly presented the hardware and software architecture of this tabletop, and its interaction layers. After having presented our interactive tabletop, we exposed our first preliminary study, which consisted of studying the user's attention and user performances with three different objects sizes, S, M, and B, of the same shape. Our findings indicate that there is non-significant effect of interaction technique for the three used sizes. Taking into consideration the low degrees of consensus between users, for ratings and rankings, we notice that users were mitigated when it come to which object's size demands the most attention. Therefore, we decided to proceed with our researches using the medium sized object, as this one is between both small and big sized objects. This is the size of object we use in our next experiment.

At the end, we finished by exposing users' mental models and our study qualitative outcomes. All of our findings can be used in harmony with the dual reality principles, as we will see in Chapter 5 and Chapter 6. In the next chapter, we present another preliminary study, that is based on results and findings of this first one, and which consists of understanding the user performance and attention demand using both hands on a tabletop.

Chapter 4

Preliminary study 2: studying user performance and attention demand using both hands on a tabletop

Introduction

In recent years, Tangible User Interfaces (TUIs) and touch on interactive surfaces have become ubiquitous [START_REF] Rekimoto | Augmented surfaces: a spatially continuous work space for hybrid computing environments[END_REF][START_REF] Wilson | PlayAnywhere: a compact interactive tabletop projection-vision system[END_REF] with interactive tabletops, offering the possibility to use both of them simultaneously, e.g., [START_REF] Tuddenham | Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks[END_REF][START_REF] Wobbrock | User-defined gestures for surface computing[END_REF]. Recent developments in multi-touch systems have paved the way to a world in which touch systems offer similar qualities generally reputed to tangible systems, such as two-handed input and collaborative use, while being easy to use and learn [START_REF] Rekik | Towards many gestures to one command: A user study for tabletops[END_REF][START_REF] Rekik | Match-up & conquer: A two-step technique for recognizing unconstrained bimanual and multi-finger touch input[END_REF][START_REF] Rekik | Understanding users' perceived difficulty of multi-touch gesture articulation[END_REF][START_REF] Wobbrock | User-defined gestures for surface computing[END_REF].

If designers would choose between using tangible objects and touch on interactive surfaces, they should know when it is better to use tangible objects instead of touch and inversely. Towards responding this question, researchers carried out several studies to determine the pros and cons of each one, e.g., [START_REF] Back | Listen reader: an electronically augmented paper-based book[END_REF][START_REF] Löchtefeld | Analysing the effect of tangible user interfaces on spatial memory[END_REF][START_REF] Piovesana | Multi-display map touring with tangible widget[END_REF][START_REF] Schneider | Benefits of a Tangible Interface for Collaborative Learning and Interaction[END_REF]. A set of guidelines have been then outlined to assist practitioners in this regard, according to which interaction modality performs better [START_REF] Lucchi | An empirical evaluation of touch and tangible interfaces for tabletop displays[END_REF][START_REF] North | Understanding multi-touch manipulation for surface computing[END_REF][START_REF] Terrenghi | Affordances for manipulation of physical versus digital media on interactive surfaces[END_REF][START_REF] Tuddenham | Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks[END_REF], ergonomically easier to use [START_REF] Piovesana | Multi-display map touring with tangible widget[END_REF][START_REF] Sapounidis | Tangible versus graphical user interfaces for robot programming: exploring cross-age children?s preferences[END_REF], easier to learn and recall [START_REF] Piovesana | Multi-display map touring with tangible widget[END_REF], and more interactive and enjoyable [START_REF] Sapounidis | Tangible versus graphical user interfaces for robot programming: exploring cross-age children?s preferences[END_REF]. However, these guidelines are not straightforward to apply because of the little understanding of the factors acting behind them.

Tangible and touch modalities allow to perform composite tasks [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF][START_REF] Kabbash | Two-handed input in a compound task[END_REF], where users can use two-handed interaction in addition to one-handed interaction. One benefit of using both hands is the improvement of users' performances [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF][START_REF] Kabbash | Two-handed input in a compound task[END_REF][START_REF] Rekik | Towards many gestures to one command: A user study for tabletops[END_REF][START_REF] Rekik | Match-up & conquer: A two-step technique for recognizing unconstrained bimanual and multi-finger touch input[END_REF][START_REF] Rekik | Understanding users' perceived difficulty of multi-touch gesture articulation[END_REF]. However, adding a second hand does not only improve users' performance. Two-handed interaction engages a complex behavior, because of the partition of the work between the dominant and non-dominant hand [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF][START_REF] Kabbash | Two-handed input in a compound task[END_REF]. For instance, for a manipulation task where users have to move a set of objects from a starting area to an arrival area, attention, decision making and fine motor control have to be phased with the coordination of the two hands [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF][START_REF] Roda | Human attention and its implications for human-computer interaction[END_REF]. However, attention demand is an important factor to design easy to learn and recall interaction techniques. In particular, the more the task requires attention and so concentration, the more the task is considered as difficult to do [START_REF] Rekik | Understanding users' perceived difficulty of multi-touch gesture articulation[END_REF][START_REF] Vatavu | Estimating the perceived difficulty of pen gestures[END_REF] and so more difficult to learn, recall and perform. It is therefore timely to understand what modality demands more attention to accomplish a composite task. In this paper, attention demand refers to the behavioral and cognitive process of selectively concentrating a discrete aspect of information, which is deemed subjective or objective, while ignoring other perceivable information. The need for such understanding is important in order to design effective tangible and touch interaction techniques. In this chapter, we examine the difference between tangible and touch modalities in term of the attention demand and the performance, through a composite task that consists of moving a set of objects. We note that this chapter extends our study published in [START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF].

In this chapter, first we present the study design detailed and explained in different sections: participants, apparatus, implementation, and finally tasks, procedure and design. Then we expose and explain our findings in terms of consensus between users, user performances, and methodologies of moving objects. Next, we discuss our results and their implications on design; and we highlight several ergonomics and design guidelines. Finally we end this chapter with a conclusion.

Study design

This preliminary study is similar in its setup to the previous one (Chapter 3): the first one deals with the size of different objects to study the user attention demand and user performances, while this second preliminary study deals with different hands synchronicity instead. We conducted an experiment to compare the user attention demand and user performances between touch and tangible interaction modalities in a composite task that consists of moving a set of objects from area A to area B. We also studied the effect of hands synchronicity and the population, that is the number of objects to move and the number of available targets. We choose to use this task as this type of task is often used on both tangible and touch interaction (e.g., [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF]). We then decided to use a composite task and not an elementary task where a single object is moved, as we think that moving a set of objects demands more attention, compared with an elementary task, as it requires sorting decisions and planning in relation with the coordination of the hands, the number of objects to move and the number of available targets. The rationale was also that if no effect was found with these settings, it would be likely that no such effect exists. We then employ the methodology of Vatavu et al. [START_REF] Rekik | Understanding users' perceived difficulty of multi-touch gesture articulation[END_REF][START_REF] Vatavu | Estimating the perceived difficulty of pen gestures[END_REF] to collect users' self-reported attention demand.

Participants

12 participants (3 females) volunteered to take part into our experiment. Aged 23 to 37 (mean = 28.92, s.d = 4.4). One participant was left handed. All had automation and computer science background; they were undergrads, PhD students and post-docs with little knowledge of tabletops.

Apparatus

The study was conducted on the TangiSense 2 tabletop, set on a display resolution of 1920 × 1080. The tabletop screen is a 47", and has a display surface of 90cm × 60cm. The tabletop uses RFID technology to detect tangible objects on its surface and it offers a sensing capacity of 16 × 24 objects on its surface at the same time, corresponding to 16 × 24 -square-RFID antennas of 3.75cm long each. The tabletop is connected to a computer running Windows 10. Our software application is developed with Java1 . In the case of tangible technique, we used cylindrical white objects, sized 2.25cm of radius, 10.3cm of height and weighing approximately 20g. Each cylindrical object is equipped with a RFID tag, which its size is smaller than the antenna size so it can be captured by only one antenna at once. In the case of touch technique, we used two gloves: one for left hand and one for right hand, each of them equipped with a RFID tag in the index finger (same RFID tag characteristics as the cylindrical objects' ones). The gloves are used to simulate a touch with the tangible tabletop.

As for the first preliminary study, this second preliminary study is positioned inside the red dotted area of our model, shown in Figure 4.1. This study is about understanding the user performance and attention demand using tangible and touch objects, hence respectively TangibleElement and VirtualElement as represented in our model, on the tabletop's surface. As described previously, we used Java to implement this application in Eclipse IDE. The display is transmitted to the TangiSense 2 tabletop via HDMI cable, from the computer to the tabletop; while events on the tabletop surface (tag entered to, tag moved on, and tag left the tabletop surface) are captured by the RFID antennas and transmitted to the computer (or software) via Ethernet cable (RJ45).

In order to connect the software application with the tabletop, the tabletop's software library (named "ttt-ci_library.jar") needs to be loaded into the software application. This software library came with the tabletop and is developed by the manufacturer (RFIdées company). As described previously, we have customized this software library; our version is named "lamih_ttt-ci_library.jar". Furthermore, a "properties" file that describes the tabletop has to be loaded into the software application. This file describes the tabletop in terms of IP address, communication port, RFID scan refresh time and other parameters.

Tasks, procedure and design

Participants have to move a set of objects from the area A to the area B. Like in [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF], the areas positions are on the top (area A) and bottom (area B) of the surface. Examples of the graphical display in the touch and tangible techniques are shown in Figure 4.2. The trial ends when all objects inside the area A are entirely moved to the area B. The object displacement ends when the object tag is entirely within a free target's boundary in the area B. To notify the participants that the object is well selected, the graphical circle representing the object on the table becomes green around. To notify the participants that the object displacement is successful, the color of the target changes to green and cannot be used again, and the object representation in area A disappears. The next object(s) displacement begins only when the current object(s) displacement is successful. Similarly, the next trial starts only when the current trial is successful. At the end of each trial, the button "next" is displayed at the center of area A to ensure that the participants' hands are at the same position at the beginning of each trial. The participants were instructed to press on the 'next' button to begin the next trial. In the experiment phase, each trial began by presenting participants the set of objects to move from area A to area B and the number of hands to use and their synchronicity. In the 1H-Seq condition, all objects had to be moved one after the other using the dominant hand. For the 2H-Alt condition, participants were instructed to move objects sequentially while alternating hands. Once the current object is successfully moved to the area B using the first hand, participant can start to move the next object using the second hand. For the 2H-Syn condition, participants were instructed to move two objects synchronously, each one by a different hand. See The number of hands and their synchronicity were fixed constraints that have to be fulfilled during the entire trial. In the cases of 1H-Seq and 2H-Alt, only one single object can be moved at a time, hence only one hand can be used at a time (see Figure 4.3). If the participant has selected/grabbed an object, the application does not allow him or her to select another one until the selection of a target, while in the 2H-Syn, the application allowed the selection of two objects. In the case of tangible modality, participants were instructed to grab the object(s) from area A, to move their hand(s) in the air and then to put it (them) on the desired target(s). To have the same task condition, in the touch condition, participants were instructed to select the desired object(s), move the hand(s) in the air and then to select the desired target(s). Participants were instructed to move the objects at normal speed. We do not give participants an order to select objects and targets, participants could select the object and the target they want in the order they prefer. In addition, for the touch condition, participants were instructed to use their index fingers (see Figure 4.3 (c) and (d)).

We follow [START_REF] Cattan | Effect of Touch Latency on Elementary vs. Bimanual Composite Tasks[END_REF] and we defined for each set of objects, two different starting areas, each one moved with two repetitions, with a total of 2 techniques × 3 hands synchronicity × 6 population conditions × 4 repetitions (2 staring areas × 2 repetitions) = 144 trials produced by each participant. Our software application randomly presented the 144 sets to move to our participants. As trials are entirely randomized i.e., two successive trials can use different modalities, our participants were instructed to wear the two gloves over all the experiment. In the cases, the modality changes between two successive trials or the two successive trials deal with tangible modality, the experimenter puts/removes the tangible objects on/from the tabletop surface. The experiment took in average 45 minutes to complete. Figure 4.4 summarizes the scenario and the sequence of the experiment; tasks highlighted in green are done by the experimenter in charge. Attention demand was collected at the end of the experimental task using absolute rating and relative ranking measurements, similar to the methodology of [START_REF] Rekik | Understanding users' perceived difficulty of multi-touch gesture articulation[END_REF][START_REF] Vatavu | Estimating the perceived difficulty of pen gestures[END_REF]. Rating was collected using a 5-point Likert scale (see Table 4.1), which was presented to participants as a table with five columns, one column for each rating value. Participants were asked to put each set of objects displacement condition (36 conditions to rate = 2 techniques × 3 hands synchronicity × 6 populations) in the appropriate rating column, after having moved it one more time on the surface in order to re-enact the experience and, consequently, perceived attention demand. Participants were allowed to change the ratings of previously rated conditions at any time as they moved along with the rating process until they were confident of their final classification. After rating all conditions, participants were asked to provide an ordered list of the different experimental conditions in increasing order of attention demand, which represents our ranking measurement. We also asked participants to explain their assessment of attention demand: what they found demanding less attention for each task condition.

Rating Explanation provided

Results

Our results include the level of agreement between participants in terms of user attention demand, user performance, and qualitative observations. These results have already been published in [START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF].

Consensus between users on the attention demand

Effect of hands synchronicity on attention demand

When calculating agreement for each hand synchronicity condition, we found a lower degree of consensus, as opposed to the modality variable. Kendall's W coefficients stayed above 0.44 for rating and above 0.49 for ranking (p < .001). The lower degree of consensus, compared to the previous modality condition, suggests that hand synchronicity is a factor with a stronger influence on the attention demand.

Although agreement was low overall, we nevertheless found that tasks performed with one handed condition led more agreement (up to 46.23% for rating and up to 64.21% for ranking) than the two handed conditions; i.e., 2H-Alt and 2H-Syn. Friedman tests revealed a statistically significant effect of hands synchronicity on both rating (χ 2 (2) = 15.6, p < 0.0001), and ranking (χ 2 (2) = 16.16, p < 0.0001). Post-hoc Wilcoxon signed-rank tests (Bonferroni corrected at p = 0.001) confirmed a significant difference between 1H-Seq and both two handed conditions and a non significant difference between the two handed conditions. We correlate these findings with participants comments. Our participants found that using both hands requires more concentration than using only their dominant hand. For instance, for 2H-Alt our participants felt that it requires more mental demand because of the workload related to which hand to use for each movement synchronization. Some participants said: "when using both hands alternatively there is an extra workload that concerns alternating the hands, but then after it has no effect on the demanding attention of the object movement", or "it is difficult to maintain a perfect alternation between both hands. I have to be fully focused with myself first, and then with the objects".

For 2H-Syn, our participants felt that it needs more attention than the 1H-Seq condition as they need to concentrate on the selection of two objects or targets simultaneously. Also, all participants felt that there is no difference between the dominant and the nondominant hand when using them both, whether synchronously or alternatively.

Effect of population on attention demand

Interestingly, we found a high degree of consensus, W > 0.7 for rating and W > 0.69 for ranking, among participants when calculating agreement for each population condition. We found a significant effect of the population on the attention demand measured as both rating (χ 2 (5) = 35.42, p < 0.0001) and ranking (χ 2 (5) = 32.62, p < 0.0001), with bigger population size causing an increase in the attention demand (see Figure 4.5). Post-hoc Wilcoxon signed-rank tests (Bonferroni corrected at p = 0.05) confirmed a significant difference between the smallest population (small-dense and small-sparse) and the remainder populations and a non significant difference between the medium populations and the big ones. For the same population size, we did not find a significant difference between the dense population and the sparse one (p > 0.05). We correlate these findings with users comments. Our participants feelings are mitigated : four participants out of twelve found that for the same population size, the sparse population is more attention demanding than the dense one as the area B is offering more options and liberty, they had to think more and choose where to place the objects. Meanwhile, for this exact same reason (more options and liberty where to place the objects), the other eight participants found that for the same population size, the sparse population is less attention demanding than the dense one, and they commented on this by "more choices and options implies less constraints on placing objects on targets", or "I do not care where to place an object since I have plenty of options", or "having less options on where to place objects makes me concentrate more than having more options when looking for the target(s)".

Given these first results, we decided to further investigate the attention demand by comparing the agreement between the two modality conditions when using the same hands synchronicity or when moving the same population.

Inter-dependency of technique and hand synchronicity on attention demand

For each modality condition, we found a high degree of consensus among participants when rating and ranking tasks done with each of the three hand conditions (0.60 < W < 0.79). Pair-wise comparison between touch and tangible modalities for the three hands synchronicity using Wilcoxon-Signed-Rank tests showed that there were no significant difference between the two modalities for the three hands synchronicity when rating them. In contrast, tangible modality is ranked as demanding significantly less attention comparing to touch modality when using 2H-Syn (p = 0.05) with no significant difference when using either 1H-Seq or 2H-Alt (p > 0.14). This result is particularly interesting as while our participants can give the same rate to two different task conditions, they were forced to give a different rank. To facilitate their choice, eight participants out of the twelve argue that the physicality nature of tangible objects makes manipulating tangible objects easier and demanding less attention than selecting virtual circles through touch, in particular when using both hands at the same time. Some quotes: "it is easier to grab physical, and 3 dimensioned, objects than graphical objects", or "virtual objects require a full vision attention whilst the real objects offer the possibility of haptic feedback which leads to less vision attention", or "it is the physicality and the 3D of the tangible objects that makes the difference, it offers some kind of haptic feedback of selection", or "I have to look for the 2-dimensional object and check whether it is selected or not, I do not necessarily do that with 3-dimensional objects since I can feel them with my hands". In addition, three participants felt that touch modality requires to pay attention twice: when selecting the object and when selecting its target. In contrast, when using tangible modality, participants felt that it requires to be attentive just once, when placing the object in its target. Some quotes: "picking up a physical object is much easier than selecting a graphical object, but for placing them in targets it is almost the same", and "grabbing objects is less concentrating than selecting objects through touch", and "the difference in the demanding attention comes mainly from the selection phase".

For both touch and tangible modalities, Friedman tests revealed that there were significant effect of hands synchronicity on attention for both rating and ranking scores. Post-hoc tests using Bonferroni correction showed that the 1H-Seq is significantly less attention demanding than both two-handed conditions (p < 0.05).

Inter-dependency of technique and population on attention demand

We found a very high degree of consensus among participants when rating and ranking tasks done with each of the six population conditions (0.80 < W < 0.89), using either touch or tangible modalities. Pair-wise comparison between touch and tangible modalities for the six populations using Wilcoxon-Signed-Rank tests showed that there were no significant difference between the two modalities for the six populations when rating them (p > 0.05). In contrast, tangible modality was ranked as demanding significantly less attention comparing to touch modality when moving the sparse populations by respectively 32.91% for the small-sparse population, 15.72% for the medium-sparse population, and 18.98% for the big-sparse population (p < 0.04). Again we correlate this finding with the physicality nature of tangible objects as compared to graphical objects. We also found that for each modality condition, the demanding attention increased as the population size increases or the population density increases (p < 0.05).

User performance

This section reports statistical tests for the dependent measures distance, movement time and trial time. All analyses are multi-way ANOVA. Tukey tests are used post-hoc when significant effects are found.

Distance

Distance is the sum of distances traveled by the different objects moved from area A to area B. The distance traveled by a single object is measured as the distance between the object's initial position and the position of its chosen target. Repeated-measures ANOVA revealed a significant effect of population (F 5,55 = 468.44, p < 0.0001) on distance. Post-hoc tests showed that the distance increases significantly as the population size or the population density increases (p < 0.05) except between bigdense and big-sparse. It is unsurprising that the distance increases as the population size increases. However, it is interesting to observe that for the same population size, the distance increases as the population density increases i.e., dense population implies larger distance than sparse population.

We correlate these findings with participants behaviour and comments. Actually, as sparse populations offer additional targets compared by dense populations and so potentially shorter distances between objects and targets, our participants felt that with more targets options they have more choice but also the possibility to reduce the physical effort by choosing closer targets. Three participants said: "having more targets than the number of objects offers more flexibility and more options and some targets become closer to objects, comparing with when having the same number of objects and targets". One other participant said: "I always try to shorten the distance between the object and its target, especially when having more choices. This allows me to reduce my physical and mental effort and finish the task more proficiently". There were no more significant main effect or interaction (p > 0.05), suggesting that the benefits of small and sparse populations are independent of the modality and the hands synchronicity conditions.

Movement time

It is the main dependent measure and is defined as the sum of the movement times of the different objects (couple of objects in the case of 2H-Syn). The movement time of one object is measured as the time between the selection of the object and the selection of its target.

There were significant main effects of modality (F 1,11 = 5.28, p = 0.0421) and hands synchronicity (F 2,22 = 36.59, p < 0.0001) and population (F 5,55 = 36.73, p < 0.0001). Interestingly, post-hoc tests showed that tangible modality (mean = 2505ms, s.d = 145ms) was significantly faster than touch modality (mean = 2923ms, s.d = 204ms) (p < 0.05). Without surprise, we found that 2H-Syn is significantly faster than 1H-Seq and 2H-Alt by respectively 30.41% and 34.83% (p < 0.05) with no significant difference between 1H-Seq (mean = 2946ms, s.d = 172) and 2H-Alt (mean = 3146ms, s.d = 243). In addition, we found that movement time increases significantly as the population size or density increases (p < 0.05) with no significant difference between big-sparse and bigdense populations. The increase of the movement time between two populations having the same size (e.g., small size) but a different density can be explained by the fact that the distance increases as the density increases and so more time is needed to move the object. There were no significant interaction (p > 0.08) suggesting that the benefits of tangible over touch are consistent across different hands synchronizations and populations.

Trial time

The trial time is measured from the first object selection to the last target selection. Repeated-measures ANOVA revealed a significant effect of hands (F 2,22 = 49.86, p < 0.0001), population (F 5,55 = 95.65, p < 0.0001) and a modality × hands × population interaction (F 10,110 = 1.95, p = 0.0448) on trial time. Post-hoc tests showed that for all hands conditions, for both tangible and touch modalities, the trial time increased significantly as the population size increased (p < 0.05). For tangible modality, we found that using 1H-Seq (respectively, 2H-Alt) to move a big-sparse (respectively, big-dense) population is slower than when using 2H-Alt (respectively, 2H-Syn) (p < 0.05). For touch modality, we found that using 1H-Seq to move a big-dense (respectively, big-sparse) population is slower than when using 2H-Syn (respectively, 2H-Alt and 2H-Syn) (p < 0.05).

Methodology for Moving the set of the objects

To better understand how participants were performing, we report here-after the different strategies elaborated by the participants in order to select the object(s) to move and its/their target(s); which is the by-product of the users comments during the experiment and also our observations.

Objects selection strategies

Different objects selection strategies were used depending on the hands synchronicity. For 1H-Seq, eleven of the 12 participants started by selecting objects placed on the same side of their dominant hand, i.e., right (respectively, left) side of the table for right-handed (respectively, left-handed) participants and so on until moving all the objects. This may be, at least in part, due to the shortest distance between their dominant hand and the objects placed on that side of the table. The last participant started selecting objects from the center of the table and continues randomly between right and left side towards the edges, he finally finishes by selecting the last object in one side (sometimes right and sometimes left) of the table.

For 2H-Alt, all participants selected objects, from area A, placed on the right side of the table with their right hands and objects placed on the left side of the table with their left hands. They finished the task with objects placed in the center, in area A, of the table. Finally, for 2H-Syn, all participants selected objects, using the two hands simultaneously, from the same side (right or left) and kept their hands close to each other, because this requires less attention. For instance, when the hands are distantly separated, our participants felt that their attention is divided and scattered into two separate areas, which makes it harder to select or grab two objects simultaneously. Some participants said: "having two objects close to each other means that they are both in my field of view, I don't need to look for objects in two different places", while some others said: "when the two objects are far from each other, I have to look in two different distant spots at once, which makes my attention go high".

Moreover, participants said that it was harder with touch modality than with tangible modality.

Targets selection strategies

Three main strategies are used here and contrarily to objects selection, they do not depend on the hands synchronicity. The first strategy consists of selecting the closest target. For instance, seven participants placed the selected object in one of the three closest targets to its starting location. Three of them clearly said: "I prefer to take an object and put it in the closest available target, preferably in the top line first and if not available I would place it in the bottom line of targets".

The second strategy consists of placing the objects on the bottom line first and then on the top line. This strategy is used by two participants. One of them said: "I prefer to start placing objects on the bottom line of targets so I don't have to raise my hand to overpass them otherwise, in addition of being ill-at-ease or constrained ".

The last strategy consists of placing the objects in targets in a regular manner from left to right, the same manner as if it was a stack, even when the population density is sparse i.e., offering more options and more liberty in target choice. This strategy is used by the five remainder participants. Participants who did this replied when asked for the reason as: "I am keeping an organized space, which makes the targets easier and faster to spot".

Interestingly, for both objects and targets selections, the modality condition does not effect the used strategy.

Discussion and design implications

We found that participants are moderately consistent when assessing the attention demand of moving a set of objects under various modalities and various populations, as indicated by Kendall's W coefficients of concordance between 0.61 and 0.76. At the same time, we found less consensus between participants' ratings and rankings for hands synchronicity (0.44 < W < 0.52), which suggests that people develop different preferences with practice. Overall, we found that using two hands, whether synchronously or alternatively, were significantly related to an increase of attention demand than using the dominant hand. Additionally, 2H-Syn was significantly faster than both 1H-Seq and 2H-Alt.

We also, found that the smallest population reduced significantly the attention demand compared to bigger ones. Interestingly, here, we found that the more the population size or density increases the more the traveled distance and the movement time increase. In addition, for each hand (respectively, each population) condition we found that participants are highly consistent when rating and ranking the attention demand of each modality condition (W between 0.60 and 0.79 (respectively between 0.80 and 0.89)). Finally and interestingly, our findings indicate that tangible modality decreases significantly the attention demand over touch modality when using 2H-Syn or when moving the sparse populations, while being faster, without compromising the distance.

Informed by our findings, we are able to outline 11 guidelines for designing touch and tangible techniques on interactive surfaces that address ergonomics and tabletops design. We have already published these guidelines in [START_REF] Rekik | Understanding the attention demand of touch and tangible interaction on a composite task[END_REF]. Note that some of these guidelines are common-sense, however we deliberately chose to state them explicitly, because they followed naturally from our study. Finally, our set of guidelines are in agreement with other recommendations available in the TUI and touch literature, e.g., [START_REF] Bakker | Design for the periphery[END_REF][START_REF] Hornecker | Physicality in tangible interaction: bodies and the world[END_REF][START_REF] Terrenghi | Affordances for manipulation of physical versus digital media on interactive surfaces[END_REF][START_REF] Tuddenham | Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks[END_REF], while they also open new opportunities for user interface practitioners to design touch and tangible interaction from the users' perspective:

Ergonomics guidelines

(a) Single hand sequential movements should be preferred over both two hands alternating movements and two hands synchronous movements, as they are globally perceived less attention demanding. (b) Two hands alternating movements and two hands synchronous movements should be equally exploited, as they were perceived as demanding similar attention. (c) Two hands alternating movements should be used with precaution. Our participants felt that using this hands synchronicity demands an extra workload that concerns alternating the hands. We also recommend to use two hands alternating movements to select distant objects placed on the right and left side of the table. (d) Two hands synchronous movements should be used to select objects close to each other, to not increase the attention demand. Our participants felt that if the objects to move are distant, they have to look in two different distant spots at once, which increases their attention. (e) Take into consideration the user position relatively to the tabletop. Our participants felt that the further the object is from them the harder the task is.

Tabletop design guidelines

(f) When possible, privilege tangible modality over touch modality as our participants felt that with tangible objects they can count on their peripheral vision to grab the physical and 3D object (see [START_REF] Bakker | Design for the periphery[END_REF]). This makes the task less attention demanding than when using touch, particularly when using two hands synchronous movements.

In addition, our participants felt that using tangible objects gives the impression to be more engaged with the task than when using virtual objects. Tangible interaction is also faster than touch interaction (see [START_REF] Hornecker | Physicality in tangible interaction: bodies and the world[END_REF][START_REF] Terrenghi | Affordances for manipulation of physical versus digital media on interactive surfaces[END_REF][START_REF] Tuddenham | Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks[END_REF]). (g) Commodity tabletops cannot move the tangible objects (e.g., resetting their positions) which most likely requires users to manually place them to the desired positions. Furthermore, tangible objects occlude the content below or in some cases also behind, which either requires more focus to infer the occluded content, or requires lifting up the tangible objects. In these cases, touch modality should be preferred to tangible one. (h) Design interaction technique that foster multimodal input. Combining touch and tangible inputs may lead to better user performance and satisfaction. This guideline is suggested by our participants. One participant felt that it can be interesting to select the object using one hand through touch and simultaneously select a target by placing a tangible object on it using the other hand. (i) Where possible, add to the visual feedback an haptic feedback when selecting objects/targets through touch to notify the users about the success of their action(s). This guideline was also suggested by our participants. (j) Consider tangible designs that exploit the surface of tangible object and/or its form, color, texture and its rigidity. Our participants felt that the features of the tangible object can enhance engagement and make the interaction experience more expressive and enjoyable [START_REF] Hornecker | Physicality in tangible interaction: bodies and the world[END_REF]. (k) Design for flexible input by giving users more target possibilities. Our findings indicate that sparser populations are faster and imply shorter traveled distance than denser ones without increasing the attention demand.

Conclusion

We presented results from this second preliminary study conducted to understand the attention demand and performance of touch and tangible modalities on a composite task.

Our key findings indicate that tangible modality requires less attention when using two hands synchronous movements or when moving the sparse populations, while decreasing the movement time over touch modality, without compromising the traveled distance.

From our experience we derived 11 guidelines for touch and tangible interaction that address ergonomics and tabletop designs. We use our findings in the next chapters to design interactive applications, which are based on our model proposed in Chapter 2. Our ergonomics and design implications prove useful for tabletops application design as we describe next.

Chapter 5

First study: a generic task for remote mobile-robot control on tabletop in a dual reality setup

Introduction

Recently robots are increasingly helping humans achieve and complete difficult tasks, across a wide range of usages and particularly in hostile environments [START_REF] Boden | Principles of robotics: regulating robots in the real world[END_REF][START_REF] Dunbabin | Robots for environmental monitoring: Significant advancements and applications[END_REF][START_REF] Manti | Stiffening in soft robotics: A review of the state of the art[END_REF][START_REF] Marconi | The sherpa project: Smart collaboration between humans and ground-aerial robots for improving rescuing activities in alpine environments[END_REF]. They assist firefighters and rescuers in their duties, as current technologies allow us to interact with them and to visualize distantly their surroundings. Current technologies also allow to perform actions which may affect their surroundings, thanks to the different sensors and actuators that can be implanted in them as needed. Crisis management might require to explore an uncertain or dangerous environment such as a nuclear disaster site or a collapsed building. In such environment, it is inconvenient for humans to interfere and robots can prove useful. Therefore, researchers have developed several applications and methods to remotely control -mobile-robots, using joysticks like in [START_REF] Chen | Effects of tactile cueing on concurrent performance of military and robotics tasks in a simulated multitasking environment[END_REF] or using computer mouse like in [START_REF] Habib | Towards adaptability of levels of automation with Human-machine cooperation approach[END_REF][START_REF] Habib | Adaptation of the level of automation according to the type of cooperative partner[END_REF][START_REF] Habib | Human-Robots Team Cooperation in Crisis Management Mission[END_REF] for instance. The intervention area's plan/map is often known and available for persons working on the crisis, i.e. firefighters or rescuers, and is used to better locate the robots while moving in and exploring the area.

In this context, using a tabletop offers a small-scaled bird's-eye view on the intervention area; mini graphical or physical robots can be placed on the tabletop surface to represent the real robots, and hence establish a virtual counterpart of the intervention area. A network mapping is needed for communication, which could be via Wi-Fi, xBee like in [START_REF] Habib | Towards adaptability of levels of automation with Human-machine cooperation approach[END_REF][START_REF] Habib | Adaptation of the level of automation according to the type of cooperative partner[END_REF][START_REF] Habib | Human-Robots Team Cooperation in Crisis Management Mission[END_REF] or other wireless communications. This chapter consists of comparing tangible and touch interaction techniques on tabletop, in a dual reality setup as defined in [START_REF] Lifton | Dual reality: Merging the real and virtual[END_REF][START_REF] Lifton | Dual reality: an emerging medium[END_REF], for tasks of remotely moving robots and exploring a disaster zone. First we present a study to measure users' performances. Then we provide results in terms of effectiveness, efficiency and usability. We also highlight the differences between these two interaction techniques and we compare them on metrics of the ISO 9241-210 standard.

In this chapter we present a study about investigating and understanding the benefits of tangible interaction, in a dual reality setup, when interacting with the virtual side of a dual reality to affect the real side. This study is extended from our previous work [START_REF] Merrad | Tangible tabletops and dual reality for crisis management: case study with mobile robots and dynamic tangible objects[END_REF], and it principally focuses on evaluating the users' workload and the usability of the sys-tem. We present our study design, apparatus, software application, tasks, and scenarios along with participants' details. We also show how the design of the application follows our proposed model in Chapter 2. Next, we present our findings, conclude this work and finally highlight possible improvements and perspectives.

Study design

Apparatus and software application

We have developed an application, using Java on the TangiSense tabletop described previously in Chapter 3, for crisis management which offers two different interaction modalities: tangible interaction and touch interaction. The application has the same functionalities, same design and same physical support (tabletop), the only difference is within the interaction modality. Both of tangible and touch versions of the application operate in a dual reality setup, where the virtual side is composed of the tabletop -hence the intervention map-and objects (tangible and graphical) on its surface, while the real side is the intervention field, the real robots on the ground and the supposed victims.

The TangiSense 2 tabletop used in this study is set on a display resolution of 1920 × 1080. It is equipped with a 47" screen of 90cm × 60cm display surface. The tabletop capture technology is based on RFID sensors to detect tangible objects on its surface (see 3 for more details); its sensing capacity is measured by the number of sensors on its surface, it is of 16 × 24 objects at the same time, corresponding to 16 × 24 -square-RFID antennas of 3.75cm long each. Finally, the tabletop is connected to a computer running Windows 10 (see Figure 5.6).

Each physical (dynamic or static) object used on the surface of the tabletop is equipped with a RFID tag (see Figure 5.1 (b)). To ensure that a tangible object is captured by only one antenna at once, the RFID tag size is smaller than the antenna's size. For the touch feature on this tabletop, we use a glove for the right hand and another glove for the left hand, each one of them is equipped with a RFID tag in the index finger (same RFID tag characteristics as the tangible objects' ones). Thus, the gloves are used to simulate a touch interaction feature with the tangible tabletop surface (see Figure 5.2 (b) and Figure 5.3). Furthermore, participants were required to wear the gloves during the whole experiment to guarantee the same conditions in both systems, even if the gloves were deactivated in the tangible version. To command the real robots on the ground and to interact with the system in the tangible version, participants use the tangible mini-robots and displace them on the surface of the tabletop. Whilst in the touch version, graphical representations are used as counterpart of the real robots on the ground; and to interact with the system participants tap on the screen to select objects and to point destinations. We also suppose that we previously know the intervention field map and we represent the supposed disaster area on a small scale on the tabletop (see Figure 5.2). Possible situations and types of disasters where our application could be used and helpful are shown in Figure 5.4, all of these types have a common principle which is the potential usage of robots to explore the area during the intervention. We use Lego Mindstorms NXT1 (Figure 5.5 (a)) as a base platform for designing and building mobile robots on the ground. They are low cost and small robots enabling fast prototyping of Human-Computer Interactions. Several programming languages can be used on these robots; we used RobotC programming language. The embedded program enables them to navigate autonomously from their current locations to given destination(s) based on a Model Predictive Control (MPC) framework, which is a model for the limited capacity of this type of robots. More details about the MPC algorithm can be found in the works of Habib et al. [START_REF] Habib | Adaptation of the level of automation according to the type of cooperative partner[END_REF] and Marzat et al. [START_REF] Marzat | Cooperative guidance of lego mindstorms nxt mobile robots[END_REF]. We use XBee wireless communication2 to send and receive data and desired/actual positions of robots from and to a dedicated computer (PC2 in Figure 5.6), i.e. only in the real side of dual reality. As the locations of the tabletop and the robots are geographically distanced, we ensure the communication between them using a WLAN (Wi-Fi). Plus, each robot is equipped with a smartphone camera to remotely visualize the robots surroundings. Figure 5.6 shows the global system architecture, the communications and how we collect data for the experiment, where the green flows express data from cameras installed on the experiment areas, blue flows express user answers to questionnaires and red flow express the experimenter remarks and notes. More details about the data and their origins can be found in Table 5.1.

Data Description Origins

Data_1

-The beginning of a task (timing).

-Errors and their categories.

Camera 1 and camera 2.

Data_2

The end of tasks (Timing). Camera 3.

Data_3

-Completion of tasks. -Errors and their categories.

-Potential remarks.

The observer's (experimenter) notes and evaluation.

NASA-TLX questionnaires

Participants' answers to questions related to workload.

NASA-TLX questionnaires, after finishing each task.

SUS questionnaires

Participants' answers to questions related to system usability. SUS questionnaires, after finishing performances on each system. Participant personal questionnaire -Participants' personal data.

-Previous knowledge on tabletops and robots control.

Questionnaires

given to participants in the beginning of the experiment. Table 5.1: Data descriptions and their origins.

The context of this application is similar to the work of [START_REF] Habib | Human-Robots Team Cooperation in Crisis Management Mission[END_REF], which is in the domain of crisis management with firefighters, where a human operator is in a command post (in our case it is represented by the virtual world of dual reality and the tabletop) and robots operate in a hostile environment (in our case it is the intervention area/real world). The hypothesis behind this study were the followings: H1. The TUI has a higher usability score than the touch user interface. H2. Users workload using the TUI is lower than when using the touch user interface.

H3. Users make less errors during their trials using the TUI than using the touch user interface.

Instantiating the proposed model

In this section we describe our application according to what we have proposed in Chapter 2. In this instantiated model, the general structure is preserved as in the model described in Figure 2.16. In this application, mobile robots are real components from the real side of dual reality. Therefore, in this instantiated class diagram, we notice that in the RealSide Package, MobileRobot class represents and replaces the RealElement class from the proposed model. Likewise, as the real side of dual reality is known to the designer (in our context, it is the intervention field, area, location, etc.), RealEnvironmentProperties class is represented and replaced by InterventionFieldProperties class, which describes the intervention field/area -of the crisis-properties.

The User Package and its contained classes are the same as in the model illustrated in Figure 2.16. Meanwhile in the VirtualSide Package, tangible and virtual components are described by classes specialized from VirtualObject class. We notice that Graphi-calMiniRobot class replaces VirtualElement class, which represents the graphical robots displayed on the tabletop surface that the user can interact with. TangibleMiniRobot class also replaces TangibleElement and which represents the tangible robot toys, used to interact with the real mobile robots (MobileRobot) on the other side. Each tangible object used on the tabletop surface is equipped with a RFID tag, hence the dependency with RFIDTag class, which replaces Tag class from our proposed model.

Eventually, other tangible objects might be used, depending on the context of the application; they must be equipped with RFID tags as well. In this application, users can take pictures of supposed victims and their surroundings using either a digital button or a tangible object; hence the need for CameraTangibleObject class, that is also associated with RFIDTag.

The CrisisManagementApplication class replaces the SoftwareApplication class as it is the class managing the whole dual reality application. It is linked to VirtualInterven-tionFieldProperties class which is the equivalent of VirtualEnvironmentProperties class in our proposed model. The application runs on the TangiSense 2 tabletop; therefore, the Platform class is represented and replaced by TangiSense 2 class. This latter has a LCD display represented by LCD class and supports two types of interaction: tangible interaction and multitouch interaction. They are illustrated respectively by Tangible and MultiTouch classes. Finally, PresentElementsOnTabletop andPresentElementsOnField classes represent PresentElementsOnVirtualSide and PresentElementsOnRealSide classes respectively.

Tasks

In this study, participants were asked to perform two main tasks, both of them consist of controlling robots remotely using the tabletop. We designed these two tasks according to generic tasks in HCI and according to our context of use [START_REF] Merrad | Reformulating Clancey's generic tasks for bridging both sides of dual reality[END_REF] (remote robot control in dual reality). See Appendix A for more details about the generic tasks. Our tasks in this study are the followings:

• The first one consists of controlling remotely one robot, moving it from point A (its current location) to point B (a predefined destination shown on the tabletop surface). • The second one consists of remotely controlling two robots at the same time, taking them from point A 1 and A 2 (their current locations) to point B 1 and B 2 respec-tively (their predefined destinations shown on the tabletop surface). As a secondary task and as we believe this will increase the participants workload, participants were required to explore the intervention field, by the mean of a live video feedback using the cameras installed on each mobile robot, and to capture pictures of the situation around supposed victims on the ground, all this while the robot (respectively robots) is (respectively are) moving to its (respectively their) destination(s). Figure 5.9 shows an example of a user manipulating a -second-robot (through a tangible toy) while exploring the disaster area and taking pictures of supposed victims. When using the tangible interface, participants control robots by manipulating the mini-robots (Figure 5.1) with their hands and place them on the desired or predefined destinations on the tabletop surface (on the map of the intervention field). In order to take photos of supposed victims, participants use a camera tangible object and place it on the corresponding video frame of the desired robot (See Figure 5.5 (b)). The picture will be taken at that instant of the video. Meanwhile, in the touch interface version of the application, participants use their -index-fingers on the tabletop surface to select the -graphical-robot they want to interact with, then point out the destination location. To take a picture of a supposed victim in this version, participants select (directly, with no need to previously select the robot) the video frame of the desired robot and tap on the camera icon. The picture will be taken at that instant of the video.

To make all of these tasks and the scenario more demanding and more stressful while using two robots, participants were instructed to manipulate the second robot (thus consequently explore the field and picture supposed victims' surroundings) simultaneously to when exploring the field and taking photos using the first robot, hence using the two hands in parallel. The instant to start displacing a robot was given by a colorful signal, flashing for few seconds on the left side of the tabletop, where each colour of a signal refers to the same colour of robot to move; i.e. green signal flashing means move the green robot and orange signal flashing means move the orange robot.

Scenarios

When participants came for the experiment and first of all, they were asked to fill in a pre-experiment questionnaire, consisting of their personal information and previous knowledge on tabletops and tangible interaction (see Appendix D.1). Then, participants were explained the functioning of the whole system, the tabletop application interfaces (tangible and touch), the usage of tangible objects and the principle behind the RFID technology. Participant were asked to try both versions of the application and get familiar with them; questions were encouraged in this phase in order to make everything clear about the experiment. After this phase and to avoid any misunderstanding of using the system in its two versions, we briefly tested our participants by asking them to perform some elementary tasks.

Next, we explained the experimentation context and the sequence of tasks. We also explained each task separately before it starts, along with the concerned user interface, tangible objects usage and the potential of using bi-manual interactions on the tabletop surface. Participants were required to complete -the same-two tasks on both system interfaces: one task using one robot and another task using two robots. At the end of the experiment, every participant performed a total of four tasks, whose sequence is counterbalanced between the two interfaces and the two tasks. We highlight that the tasks' order is the same in the two conditions (interfaces), i.e. a participant who started performing on one interface -whether tangible or touch-with one robot (respectively two robots), will start performing with one robot (respectively two robots) on the other interface. See Figure 5.10 which summarizes the whole scenario and sequences.

Participants filled a NASA-TLX questionnaire assessing their interaction [START_REF] Hart | NASA Task load Index (TLX)[END_REF] after finishing each task on each interface. The questionnaire assesses each participant's mental demand, physical demand, temporal demand, performance, effort and frustration towards the executed task. To evaluate the usability of the two applications/systems interfaces, participants answer a System Usability Scale (SUS) questionnaire, containing 10 standard questions [START_REF] Brooke | Sus-a quick and dirty usability scale[END_REF], after finishing performing on each application interface. The global score of usability is calculated, for each system separately, using the participants' answers to Figure 5.10: Tasks and scenario progress. "T1" refers to task one, "T2" refers to task two, "S1" refers to system one (tangible version of the application) and "S2" refers to system two (touch version of the application).

the SUS questionnaire as follows:

1. For each of the odd numbered questions, we subtract 1 from the scores.

2. For each of the even numbered questions, we subtract the given score from 5.

3. We sum together the new values (scores) then we multiply the result by 2.5.

For more details about the SUS questionnaire template, the scales and the score calculation method, the reader is referred to [START_REF] Brooke | Sus-a quick and dirty usability scale[END_REF]. During the performing, the experimenter observes the participant performance and notes whether the task has been successfully completed or not (1: successfully completed, 0: unsuccessfully completed or uncompleted at all), in addition to manipulation errors and their classifications. At the end of the experiment, participants were asked to complete a post-experiment questionnaire (see Appendix D.2), discuss their experience with the experimenter, share their remarks and make suggestions if they had any.

Participants

32 participants (9 female, 23 male) have been recruited in this study, mostly Ph.D. students in our lab and undergraduates with different scientific majors. Their ages ranged from 22 to 39 years old, with an average age of mean = 27.97 and s.d. = 4.28. All participants were right-handed, all with normal or corrected to normal vision and all having normal arm mobility; none of them had any kind of disability. Participants were instructed as follows: (1) Preferably use their dominant hands during the whole experiment trials. If a participant is more comfortable using his/her non-dominant hand he/she can use it. [START_REF] Adam | Flexible roles in a holonic multi-agent system[END_REF] The same hand must be used while performing on TUI and on touch interface, whether it is the dominant hand or the non-dominant one. (3) Every participant must go through both versions of the application (tangible and touch) in a given order by the experimenter, as the study was design as a repeated measure.

Results

The outcomes of this study have been classified into two main sections. The first one is in accordance with ISO 9241-210 standard [START_REF]Ergonomics of human-system interaction -Part 210: Humancentred design for interactive systems[END_REF]; while the second one is post experiment interviews and users' self-reported evaluations to several questionnaires. We provide details on each of them as follows.

Criteria from ISO 9241-210 standard

This standard recommends to include Effectiveness, Efficiency and Satisfaction to evaluate the usability of a system. We describe and present them as follows.

Effectiveness

This is the accuracy and completeness with which users achieve specified goals. It can be calculated through the following two methods:

• Completion rate: calculated by assigning a binary value of '1' if the participant manages to complete a task and '0' if he/she does not. It can be represented as a percentage using the following equation:

Completion rate = number of tasks completed successf ully total number of tasks undertaken × 100 %

• Number of errors: errors can be unintended actions, slips, mistakes or omissions that a user makes while attempting a task. For each task, an average number of errors is calculated as follows:

Average number of errors = number of users making an error total number of users

We start with the number of errors as some errors may lead to uncompleted tasks. Errors made by participants during their trials are classified into categories along with their description as shown in Table 5

Not detected

A tangible object is put on the tabletop surface but not detected and not known as present on its surface. Missed photo When participant does not take a photo of a victim on the ground that has appeared on the video feedback. Table 5.2: Errors categories, their description and frequencies (1R: one robot; 2R: two robots).

One remarkable thing about the result shown in Table 5.2 is that participants always made more errors using touch interface than using tangible interface, given the same number of robots used. Moreover, we notice that the sum of errors made when using one robot [START_REF] Bencina | Improved topological fiducial tracking in the reactivision system[END_REF] is less than half the sum of errors made when using two robots [START_REF] Detken | The search wall: tangible information searching for children in public libraries[END_REF]. This is coherent with the NASA-TLX outcomes that we describe next, however it requires further investigations to correlate these two metrics together.

Table 5.3 shows further errors analysis; here also tangible interaction using one robot (respectively two robots) outperforms the touch interaction when using one robot (respectively two robots). For the tasks performed without any mistake, we notice that the difference is bigger when using one robot than when using two robots, the same applies for the average number of errors per task. We believe that this is also related to the users' workloads; we investigate this purpose later in another section. The completion rates of tasks given in this study are shown in Table 5.4. We found these results based on the errors made by participants while attempting to perform the tasks (Table 5.2). As mentioned, we assigned '1' for a completed task and '0' for else. Our results show that the tangible interface outperforms the touch interface only when using two robots, with respectively 78.13% to 68.75%. Meanwhile, both interfaces have the same completion rate of 84.38% when using one robot. Furthermore, we believe that the difference of tasks' completion rates between one robot and two robots, no matter what user interface is used, is due to the user attention and focus, which are better when performing with only one robot. This is also coherent with the workloads outcomes shown previously (significant differences between tasks using one robot and tasks using two robots in the same interface). 5.4: Tasks completion rates by number of robots and by interface type.

Interface

Number of robots

Efficiency

Defined as the resources expended in relation to the accuracy and completeness with which users achieve goals. It can be calculated in one of the following two ways:

• Time-based efficiency: measured by "goals/unit of time". It is defined by the following equation:

T ime based ef f iciency = R j=1 N i=1 n ij t ij N R
where N is the total number of tasks (goals); R is the number of users; n ij is the result of task i by user j (if the user successfully completes the task, then n ij = 1, if not then n ij = 0); t ij is the time spent by user j to complete task i, if the task is not successfully completed, then the time is measured till the moment the user quits the task.

• Overall relative efficiency: it uses the ratio of the time taken by the users who successfully completed the task in relation to the total time taken by all users. It can be represented by the following equation:

Overall relative ef f iciency = R j=1 N i=1 n ij × t ij R j=1 N i=1 t ij × 100 %
where N is the total number of tasks (goals); R is the number of users; n ij is the result of task i by user j (if the user successfully completes the task, then n ij = 1, if not then n ij = 0); t ij is the time spent by user j to complete task i, if the task is not successfully completed, then the time is measured till the moment the user quits the task. The time-based efficiency is calculated in terms of goals/second. As the average time spent on each task is higher than 24 seconds, the efficiency results would be less than 0.05 goal/second, with small differences and thus hard to compare them. Therefore, we use the overall relative efficiency which is expressed as a percentage, it is easier to understand, compare and spot the differences.

We applied the formula on two different perspectives. The first one to compare tangible to touch interaction techniques, regardless of the task, i.e. number of robots. In this case, the number of tasks is N = 2 and it represents the tasks with one and with two robots. The second one as a detailed comparison and taking into consideration the number of robots, here we have N = 1 and we apply it to each interface× number of robots separately; R is always set to 32. Our findings are illustrated in Table 5 Results in Table 5.6 indicate that the tangible interaction technique outperforms the touch one, with a score of 79.7% vs 73.02%. This is highly correlated with the completion rates shown previously as this latter uses the tasks' completion scores. When it comes to comparing interaction techniques by number of robots used, we find that for one robot the efficiency is basically the same, 84.39% to tangible vs 84.25% to touch. Contrarily to when using two robots, we notice a considerable difference between tangible and touch interaction techniques (76.59% to tangible vs 66.06% to touch). This may be due to the difference in workloads that we investigate in the next section.

Furthermore, we conducted a paired t-test on the tasks' completion times to compare one robot tasks and two robots tasks in both user interfaces. Our findings indicate that there is non-significant difference between tasks using one robot, i.e. p >> 0.05, with (M = 24.26, SE = 0.36) to tangible interface vs (M = 25.3, SE = 0.69) to touch interface.

Satisfaction

It is about the comfort and acceptability of use of the system. Standardized satisfaction questionnaires can be used to measure it after each task and/or after the usage of each system. It is measured in two parts [START_REF] Mifsud | Usability Metrics -A Guide To Quantify The Usability Of Any System[END_REF]:

• Task level satisfaction: this is to measure how difficult is the task that has just been taken. The most popular post-task questionnaires are After Scenario Questionnaire (ASQ), The NASA Task Load Index (NASA-TLX) (measure of mental effort), Subjective Mental Effort Questionnaire (SMEQ), Usability Magnitude Estimation (UME) and Single Ease Question (SEQ) [START_REF] Mifsud | Usability Metrics -A Guide To Quantify The Usability Of Any System[END_REF]. We use the NASA-TLX questionnaire -for each task-as it is articulated through several sub-scales.

• Test level satisfaction: this is to measure the users' impression of the overall ease of use of the two systems. The following questionnaire are widely used in this matter: System Usability Scale (SUS), Standardized User Experience Percentile Rank Questionnaire (SUPR-Q), Computer System Usability Questionnaire (CSUQ), Questionnaire For User Interaction Satisfaction (QUIS) and Software Usability Measurement Inventory (SUMI) [START_REF] Assila | Standardized usability questionnaires: Features and quality focus[END_REF]. We use SUS questionnaires -for each system-for this purpose.

As we previously mentioned, we evaluate the participants' workload of each task using the NASA-TLX questionnaire. The evaluation was done separately on each subscale of this questionnaire. Figure 5.11 illustrates a workload summary of the four tasks performed, using one and two robots on tangible and touch interfaces, and through the six NASA-TLX sub-scales. Comparing the tasks with one robot (in tangible and touch), slight differences are found between the sub-scales means (respectively in tangible and touch), in favor of touch interaction only in mental demand and effort. Meanwhile it is in favor of tangible interaction in physical demand, temporal demand and frustration. The performance subscale shows an equal outcome between the two interaction techniques. When it comes to tasks with two robots, the gap between the means of each sub-scale is bigger than that of tasks with one robot, all in favor of tangibles except for the performance. Plus, the scores of all sub-scales are largely higher for tasks with two robots than the scores of tasks with one robot. However, if we notice the standard error bars we see that they overlap, which means that it is unlikely to find a statistically significant difference. Therefore, on each of the NASA-TLX sub-scales a paired (dependant) t-test has been conducted with the following alternate hypothesis: "participants' workload using the TUI is lower than when using the touch user interface, with the same number of robots".

The t-test results show that, in a confidence interval of 95%, all p-values corresponding to all sub-scales are largely greater than 0.05. Nothing about comparing workloads in tangible and touch user interfaces, in this context of remotely controlling robots using a tabletop, can be concluded on based on these results. Unlikely to other results which are quite significant, we present them in the following subsections.

Another one-sided paired t-test, in a confidence interval of 95%, has been conducted on the data with the following alternate hypothesis: "participants' workload when performing with one robot is lower than when performing with two robots in the same user interface". The results indicate that all p-values are less than 0.05, which means that performing with two robots is significantly more demanding than performing with one robot.

Our results of measuring usability show a significant difference of scores (calculated as a percentage) in favor of the tangible version, indicating that participants have expe-rienced better usability in TUI than in touch user interface. As shown in Figure 5.12, the Tangible User Interface had a higher mean score (M = 86.02, SE = 2) than the touch user interface (M = 81.17, SE = 2.39), t(31) = 1.99, p < 0.05, r = 0.34. These results are obtained from a one-sided paired t-test in a 95% confidence interval, it is a one-sided t-test because we were expecting a difference between the scores of tangible and touch user interface. As the Pearson's correlation coefficient (r) is between 0.3 and 0.5, we can say that the effect size is from medium to large. Although the standard error bars overlap, we can conclude that in this context of remotely controlling robots using a tabletop, the Tangible User Interface has a better usability than the touch interface.

Post-experiment evaluations

At the end of the experiment, participants took a brief interview and discussion. The interview questions are shown in Appendix D.2. We asked our participants to evaluate the following statements on a Likert scale from 1 (strongly disagree) to 5 (strongly agree):

• Statement 1: The "robot" tangible object is easy to manipulate.

• Statement 2: The "robot" tangible object seems significant (meaningful) to you in relation to its role in the application.

• Statement 3: I had a full control on the "robot tangible object" while using it (not the robot).

• Statement 4: I had a full control on the "graphical robot object" while using it (not the robot).

• Statement 5: The tangible object "take picture" is easy to manipulate.

• Statement 6: The "take picture" tangible object seems significant (meaningful) to you in relation to its role in the application.

Participants evaluations are illustrated in Figure 5.13, it shows the means along with standard error bars. As we notice, participants were very satisfied with the usage of tangible objects, particularly since the standard deviations are relatively small. Table 5 Some participants commented on the tangibles as "very useful and straight forward, as they constitute the object and the whole process of interaction". For instance, they said "when you place an object on the tabletop surface it detects it and triggers the final action, there is no need for intermediary actions, this can avoid us to choose among menu options". Furthermore, more than half of participants said that the experience with tangibles were more realistic and enjoyable compared to graphics (touch user interface), this is because of the haptic and the feeling of holding the object(s) in one's hand(s).

Conclusion

We have presented in this work a study that compares users performances on touch and Tangible User Interfaces, using an application for crisis management on tabletop. The application serves to remotely control robots using a tabletop and in a dual reality setup. It is implemented on tangible and touch user interfaces and it helped us with evaluating tangible and touch interaction techniques. In this context, we have seen how tangible interaction outperforms touch interaction in effectiveness and efficiency, for remote robots control only when using two robots. Tangible interaction also performed better in usability, which is part of satisfaction assessment, whereas for the workload, it is unclear which interaction technique is better as our results show non-significant differences.

We have also exposed how this dual reality application can be modeled according to our proposed model in Chapter 2. We designed this application following our model, and we presented a complete class diagram, as a proof of concept.

Our recommendations are that tangible interaction is better than touch interaction, in terms of rapidity, usability, effectiveness and efficiency, when interacting with both hands simultaneously. However, applying our findings to other touch and tangible user interfaces has to be done with particular attention to the application context, further experimentation will be needed before making strong conclusions about TUI versus touch user interfaces.

One perspective of this work is to reinvestigate the users' workload with different configuration(s) of tasks (eventually their levels of complexity) and/or put more robots at once. Another perspective is to analyze more user data collected from the eye tracker TobiiPro, that users were wearing during the experiment, and correlate it with our findings that we presented in this work. We also aim to do a similar experiment with multi-users, with more demanding and more stressful scenarios for stakeholders of crisis management and in other domains such as healthcare and education.

In the next chapter, we present another study based on our proposal made in Chapter 2, as a second proof of concept.

Chapter 6

Second study: monitoring patients' with Alzheimer's disease activity using distributed tangible tabletops and dual reality

Introduction

Healthcare services encompass a large number of different practices. They include internal medicine, pediatric, women's care, dental care, optometry, laboratory testing, and much more. Among other practices, we can mention monitoring patients health and following their evolution(s). Another type of healthcare practices is to monitor patients activities at home, particularly patients' with Alzheimer's disease who require a regular check on their elementary activities and Activities of daily living (ADLs) [START_REF] Galasko | An inventory to assess activities of daily living for clinical trials in Alzheimer's disease[END_REF], such as drinking water and taking daily medicines in time [START_REF] Dunne | Visual contrast enhances food and liquid intake in advanced alzheimer's disease[END_REF].

Different technologies can support the monitoring of patients' with Alzheimer's disease, like phone calls and video chats between the patients and their nurses or caregivers, for instance. Tabletops may also be used in this context as they become more and more ubiquitous. Smart homes might be equipped with smart tables capable of detecting, recognizing, and tracking physical objects on their surface. It is, therefore, possible to monitor some activities of patients' with Alzheimer's disease, using a tabletop installed inside the patients' home. In a dual reality setup, another tabletop can be installed in the healthcare center, allowing a complete reflection of -tangible-items on the first tabletop into a virtual symmetrical counterpart on the second tabletop. Furthermore, using a distributed architecture permits the healthcare organization to maintain several patients' tabletops connected to one healthcare center.

In this chapter, we extend our work [START_REF] Merrad | Towards monitoring patients with alzheimer's disease activity using distributed tangible tabletops and dual reality[END_REF] and we present the first prototype of a software application implemented on two distributed TangiSense 2 tabletops. The application aims to help healthcare centers and families to monitor patients' with Alzheimer's disease activities at home remotely, using tangible objects, tagged with RFID tags, on the tabletop surface. First, we present the software application prototype, its implementation and functionalities. Then, we expose its distributed aspect and general architecture. Next, we validate the design of the application according to our model proposed in Chap-ter 2, and we show how the distribution of such application can be modeled. We present afterwards the perspectives of this work and potential evaluations. Finally, we end this chapter with a conclusion.

First prototype

We have developed a distributed application prototype, using Java and Jade multi-agent platform, on two TangiSense tabletops (the same tabletop previously described in Chapter 3), for monitoring certain activities of patients' with Alzheimer's disease. The application prototype runs on two tabletops: one tabletop is installed within the patient's house or living facility, which consists of the real side of dual reality; and another tabletop is installed on the healthcare or patients monitoring center, which is considered to be the virtual side of the dual reality setup. The application prototype may help to remotely monitor patients' with Alzheimer's disease daily vital and essential activities, like eating (lunch and dinner), drinking (water), and taking medicines. We also suppose that patients wear a device around their neck, allowing us to capture swallowing. We use this information to synchronize it with events and activities on the tabletop surface.

The application operates in a dual reality setup, meaning that components on one side are consistently reflected on the other side. The real side of the application contains food recipients, cups, and pill dispensers arranged on the tabletop surface (see an example in Figure 6.1). Every item is equipped with a RFID tag, allowing it to be detected and identified by the tabletop, then consequently be duplicated into the virtual counterpart of the dual reality; see Figure 6 The TangiSense 2 tabletops used in this design and implementation are both set on a display resolution of 1920 × 1080. They are equipped with a 47" screen of 90cm × 60cm display surface. The tabletops capture technology is based on RFID sensors to detect tangible objects on its surface (see Chapter 3 for more details); its sensing capacity is measured by the number of sensors on its surface, it is of 16 × 24 objects at the same time, corresponding to 16 × 24 -square-RFID antennas of 3.75cm long each. Finally, each tabletop is run using a computer running Windows 7.

Each physical object used on the surface of the tabletop is equipped with a RFID tag (see Figure 6.3). To ensure that a tangible object is captured by only one antenna at a time, the RFID tag size is smaller than the antenna's size.

We note that our software application (and its prototype) do not use any camera or other tracking technologies/devices. Therefore, it respects the privacy of patients as it does not record any other events excepts those on the tabletop surface. Furthermore, it is feasible and extensible with a reasonable cost, due to the use of passive RFID tags to be glued on everyday's life objects. In return, we cannot get all the details of the activities; e.g., if the patient grabbed the glass and although s/he wears the device around her/his neck, nothing guarantees that s/he drank it completely. Therefore, we just need to have traces of key activities, to react at a first level of the incentive, before asking people (family members, caregivers, or rescuers) to come by the house or taking other actions, such as sending alerts and warnings to concerned people. We note that for this prototype of the software application, the RFID tag is glued to the plate and is visible from below; it would be possible to have items such plates, cups, etc., with completely build-in, invisible and integrated tags.

The distributed aspect

The design of this application takes into consideration the distributed aspect. The two tabletops are therefore connected through a computer network using a switch; the UI is managed by a Multi-Agent System using Jade platform and XML [START_REF] Lebrun | A multi-agent system approach for interactive table using rfid[END_REF]. Each object (whether tangible or digital) is then associated with an agent. Objects might be tangible on one side (in this case in the real side of dual reality) and digital on the other side (virtual side of dual reality in our case). In fact, designing a distributed TUI application with tangible objects on both sides is a challenge; this is mainly due to the fact that static tangible objects (that cannot move by themselves) on one tabletop do not have the same representation(s) and/or behavior(s) on the other tabletop. Instead, having dynamic tangible objects (hence can move by themselves) on both sides makes it easier to maintain the symmetry and the consistency between the two sides of dual reality. Figure 6.4 shows the global architecture of this application. Each tabletop and computer is connected to a central switch, that allows the communication between all nodes of the network. The network is of type star as shown in Figure 2.3, where each node can communicate with all other nodes. In our case, patients' side tabletops can only communicate with the healthcare center tabletop. Therefore, tabletops (additional real sides of dual reality, meaning additional patients) can be added without disturbing the network setup and/or the functioning of other tabletops. This network architecture and configuration serves only as proof of concept in our application; eventually, other network types and settings can be used in real life and large-scale implementations, such as Wireless Local Area Network (WLAN) or Virtual Private Network (VPN).

From a technical point of view, the healthcare center tabletop is implemented as a server instance; all the other tabletops (i.e., patients' tabletops) are implemented as clients instances in the same network. Therefore, the main application runs on the server (healthcare center tabletop) to reflect a selected client (among many patients, if available) and monitor his/her activity. Figure 6.5 describes this architecture with several patients and their tabletops. We note that in such configuration, only one patient is monitored at a time, hence the need to select a patient from the healthcare centre tabletop (illustrated by *Select Patient* in Figure 6.5). Figure 6.5: Architecture of our system with several patients connected to one healthcare center.

Validating the design through our model

The design of this software application prototype is based on our model proposed in Chapter 2. In the VirtualSide Package, several -tangible-objects inherit from the VirtualObject class: GraphicalPill, GraphicalPlate, and GraphicalCup; representing respectively a pill, a plate, and a cup. The counterpart of these elements in the RealSide Package are Pill, Plate, and Cup classes respectively. In order to keep the symmetry between the two sides, each object is characterized by its position on the tabletop, modeled by the Position class and associated with VirtualObject class.

As this application runs on a Multi-Agent System using Jade platform, each virtual and/or physical object used on the tabletop is associated with one agent at most, depicted by the Agent class in the VirtualSide Package. The dependency depicts the association, and its multiplicities, with VirtualObject class.

In the class diagram shown in Figure 6 In this application there are two kinds of users: patients, acting from the real side of dual reality; and nurses (or other employees in the healthcare center in charge of monitoring patients' activities), acting from the virtual side of dual reality. Therefore, in the User Package, this aspect is modeled by the two classes Patient and Monitor, that inherit from the User class. The Monitor class can represent nurses, caregivers, and Alzheimer's disease experts; potentially in the future, it may represent a software program or a robot with an Artificial Intelligence that monitors the patients' activities. The other classes in this package are derived from our proposed model.

In the VirtualSide Package, PatientHouseProperties class represents properties and characteristics of the environment (usually a house or a dedicated facility) where the patient and the corresponding tabletop are located; it replaces the RealEnvironment-Properties class in our model.

The Cup, Plate, and Pill classes are all specialized from RealObject class, they represent respectively a real cup, plate, and pills of everyday life. To track these objects and maintain the symmetry and consistency between the two sides of dual reality, each object has a position on the tabletop. This is modeled by the Position class associated to RealObject class. Some objects used by the patient might be equipped with sensors, like fullness sensors, they can be therefore linked to Sensor class as shown in Figure 6.6.

Perspectives of this chapter

This work is focused primarily on the design of the distributed system, from a dual reality perspective. Nonetheless, there are many perspectives to this work that we present in this section.

First, we envision to evaluate the application with healthcare practitioners, in order to receive a feedback from people emerged in this domain of healthcare. Nurses, caregivers and Alzheimer's disease experts for instance are practitioners who deal the most with elderly and with patients' with Alzheimer's disease; their experiences with everyday life patients can improve our tasks design and enrich our application, with other fundamental functionalities and activities.

Second, we plan to run a lab experiment with two kind of users: users playing the role of nurses (eventually caregivers and/or care assistants) and others playing the role of patients with Alzheimer's disease; they will be given well defined tasks to perform on each tabletop. This lab experiment will aim to evaluate the application and the monitoring task with only one patient, then compare it to actual monitoring methods. We plan to ask practitioners about their preferences and their appreciation of this application. Finally, we plan to extend this lab experiment with several patients connected to one healthcare center.

Conclusion

We have presented in this chapter a software application implemented in dual reality setup, and using two distributed tabletops. The purpose of such application would be to help healthcare centres to remotely monitor patients' with Alzheimer's disease ADLs, while these latter are at home; these activities include among others taking medicines, eating food at the right time, and drinking water. We also exposed in this chapter how the design of this distributed application is based on our model, proposed in Chapter 2; we instantiated our model of class diagram and explained every class related to this application and its dependencies.

Next, we detailed the distribution aspect and illustrated the communication between the two sides of dual reality, and also between potentially additional real sides, connected to one virtual side (equivalent to one healthcare centre monitoring several patients remotely). Finally, we presented our research perspectives for this work, highlighted possible improvements and potential evaluations.

General conclusion

The purpose of this thesis was to propose a novel theoretical framework explaining interactions and representations operating in dual reality spaces and on tabletops. We recall in this conclusion our contributions and experimental results of our work, we highlight its novelty and we expose our perspectives.

Our contributions

First of all, we have presented various researches carried out in the field of tangible interaction, tabletops, and dual reality. The presented work illustrates the background of this thesis, and shows the potential and usefulness of tabletops and dual reality in several domains, for different users categories. We have synthesised and classified 112 tabletops, through characterisation according to several criteria such as their capture technologies, interaction mode, display and communications. We have then highlighted the differences between Augmented Reality, Mixed Reality, Virtual Reality, and dual reality. This synthesis may prove helpful for designers of applications on tabletops surfaces and in dual reality setups.

Next, we have focused our work on proposing a framework, using UML structural and behavioral diagrams models; it takes into consideration design aspects of applications in dual reality, that uses tabletops as interaction platforms and that focuses on tangible interaction. This means that our framework keeps the two sides of dual reality separated, while bridging them at the same time, and offering mutual influence between them. Furthermore, the designed framework models different platform characteristics like its display technology, interaction modalities, and its Artificial Intelligence if it is equipped with.

We have then provided two proofs of concept using our proposed framework, after conducting two preliminary studies, in laboratory; these studies consist on better understand the user experience on tabletops. We have validated our framework through instantiating our UML design models. First, we designed an application for crisis management on tabletop, based on the two preliminary studies outcomes and operating in a dual reality setup. The application helps to distantly control mobile robots from a tabletop platform, by displacing graphical and tangible mini-robots on its surface; while the mobile robots and the tabletop are completely in two separate environments, yet these latter are symmetrical, interconnected, and can mutually influence each other. we have run a study in this context and reported our findings in Chapter 5.

Second, we designed an application prototype, in dual reality and as a proof of concept, that may help with monitoring Alzheimer's patients' activities remotely. Using distributed tabletops (a tabletop for each patient and one tabletop in the healthcare center), caregivers and nurses can monitor patients' Activities of daily living, such as feeding and drinking. The design of this application is based on the two preliminary studies outcomes and uses our framework proposed in Chapter 2; it instantiates model of class diagrams. It proves the technical feasibility and adaptability of our modeling in different situations, in a dual reality setup and using distributed tangible tabletops on both sides of the dual reality. There are several research perspectives for this application prototype, in particular starting with evaluations with two types of users (patients and caregivers/nurses) as we explain later in our research perspectives.

The novelty of our work

Our work is in line with current research on new interactions, new platforms, and/or new virtual environments. In fact, the novelty of our work lies in the mix and use of interactive tabletops, tangible interaction, and dual reality. Indeed, we believe that interactive tabletops could become more omnipresent in the future, in both work and entertainment spaces for a daily usage. Trends in current researches and orientations support our idea.

Recent tabletops permit multimodal interactions, such as multi-touch and tangible interactions with physical objects on their surfaces. We think of using tabletops as new medium of interaction in dual reality, to mutually reflect and influence both of the digital and physical side of a dual reality environment, rather than using tabletops in their own local environments. This context of usage can be in a distributed mode (with several interconnected tabletops) and with multi-user configuration on each tabletop.

As interactive tabletops are multi-user platforms, the work environment can for example become collective. This is an important aspect in our proposed framework, which allows to model different user-environment situations. In fact, by taking into consideration the possibility to have several users in only one side and/or in both sides of dual reality, we believe that it is relevant to adapt the distributed User Interface according to the situations of use, in the same way as the numerous researches already carried out and aiming at adapting the User Interface of computers or mobile devices according to their context of use.

Lastly, another strength of our work lies in the creativity and innovation of the studies we conducted and their evaluations. The first laboratory study exposed demonstrated the importance of considering objects' size on tangible tabletop surface, for performing elementary tasks. Meanwhile, the second laboratory study demonstrated the interest of studying user performance and attention demand, when using one and two hands, with tangible and tactile elements on the tabletop's surface. The third laboratory study consisted of studying the user experience and performances, in a task of remotely controlling mobile robots, using a -tangible and tactile-tabletop platform. Finally, the last study consists of an application (prototype) intended for healthcare centers and caregivers; it runs on at least two distributed tabletops, and it helps to monitor patients with Alzheimer's disease ADLs. Nonetheless, our work and contributions still can get some improvements that we describe next in the perspectives section.

Research Perspectives

Although we have proposed and evaluated our proposal, many perspectives could have been suggested in the conclusions of Chapter 2, Chapter 5, and Chapter 6. We have identified several research perspectives based on our contributions and their limitations.

• Enrich our framework with other interaction models, such as activity diagrams and state transition diagrams using UML. This perspective will offer more modelling possibilities for designers, for various applications contexts. We now need to focus on designing more interaction models to achieve a complete framework, that can be used in different application types and domains in dual reality, to show the full potential of our design.

We can add several structural and, particularly, behavioral diagram models to our framework. This includes component diagram(s), composite structure diagram(s), and deployment diagram(s) as structural diagrams. While for behavioral diagrams, we can envision activity diagram(s) and state machine diagram(s). The use of a diagram type instead of another depends on the system to be modelled. Some systems might be better explained with one or many structural diagrams, while other systems might be better explained with one or many behavioral diagrams. This does not exclude the usage of both structural and behavioral diagrams to model a given system.

• Carry on with our researches on the patients with Alzheimer's disease ADLs study, by running a lab experiment using two distributed tabletops. The experiment setup simulates two distant and connected rooms (both are equipped with -tangibletabletops); where the first one simulates the healthcare center, with users acting as nurses and/or caregivers; while the second one simulates the patients' with Alzheimer's disease homes or facilities where they reside, with users performing Activities of daily living on the tabletop's surface.

The purpose of this study is to evaluate the application prototype and to ensure a distributed user-centered design of its functionalities. We plan to involve, in a second time, real health monitors and caregivers, in order to collect their feedback in terms of preferences, usability, and helpfulness. We suggest next to improve the application prototype following the results of our experiments and the participants suggestions. We also plan to enrich the application prototype with other monitoring functionalities, particularly on the healthcare center side (virtual side of dual reality) with a dashboard and more indicators.

• Explore the potential of proposing an interaction technique in dual reality spaces, particularly for interacting with dynamic objects using tabletops. This interaction technique should offer real time interactions in dual reality setups easiness in use, flexibility while manipulating objects, should fit into the dual reality paradigm, and preserve its characteristics. We believe that working on such a research perspective may improve the user experience in general in dual reality spaces. In fact, current interaction techniques were not designed around dual reality paradigm, and a new interaction technique that takes into consideration this aspect would go beyond the current limitations.

Clancey has also worked on generic tasks and operations, beside knowledge engineering. We find in his famous paper [START_REF] Clancey | Heuristic classification[END_REF] a generic model for operations (tasks) that we can do to or with a system. Figure A.1 summarizes hierarchically these generic operations. Operations are grouped in terms of those that construct a system and those that interpret a system, corresponding to what is generally called synthesis and analysis.

Clancey describes that the terms between brackets are common synonyms of the generic operations (in capital letters). He also explains in [START_REF] Clancey | Heuristic classification[END_REF] that INTERPRET operations concern a working system in some environment. In particular, IDENTIFY is different from DESIGN in that it requires taking I/O behavior and mapping it onto a system. Whilst PREDICT is the inverse, taking a known system and describing output behavior for given inputs. Moreover, Simulate is a specific method for making predictions, suggesting that there is a computational model of the system, complete at some level of detail. For the CONTROL, not often associated with heuristic programs, takes a known system and determines inputs to generate prescribed outputs [START_REF] Vemuri | Modeling of complex systems: an introduction[END_REF]. Thus, these three operations, IDENTIFY, PREDICT and CONTROL, logically cover the possibilities of problems in which one factor of the set input, output, system is unknown.

Further explanations are given in [START_REF] Clancey | Heuristic classification[END_REF], when the author notes that MONITOR and DIAGNOSE presuppose a pre-existing system design against which the behavior of an actual, running system is compared. Thus, one identifies the system with respect to its deviation from a standard. In the case of MONITOR, one detects discrepancies in behavior (or simply characterizes the current state of the system). In the case of DIAGNOSE, one explains monitored behavior in terms of discrepancies between the actual (inferred) design and the standard system.

The Design is taken to be the general operation that embraces both a characterization of structure (CONFIGURATION) and process (PLANNING). DESIGN is conceptual, it describes a system in terms of spatial and temporal interactions of components. Therefore, The idea of "executing a plan" is moved to the more general term ASSEMBLE, meaning the physical construction of a system [START_REF] Clancey | Heuristic classification[END_REF]. Also from the same reference, SPEC-IFY refers to the separable operation of constraining a system description, generally in terms of interactions with other systems and actual realization in the world (resources affecting components). Of course, in practice design difficulties may require modifying the specification, just as assembly may constrain design (commonly called "design for manufacturing").

A.2 Generic tasks in HCI

The researches on Human-Computer Interaction and cognitive sciences keep rising year after another. Meanwhile, the importance of task modeling and analysis has also become more important than ever, such as in computer science and in human automation.

The interest of doing tasks analysis can be seen in three main points: update an exiting system to get a new one, create a new system from many existing systems and create a new system from scratch [START_REF] Stammers | Task analysis[END_REF]. Kolski in his book [START_REF] Kolski | Interfaces Homme-machine: application aux systèmes industriels complexes[END_REF] and Diaper et al. in [START_REF] Diaper | The handbook of task analysis for human-computer interaction[END_REF] described several human tasks modelling and analysis methods. Some of them are based on software engineering approaches, while some others are based on teamwork collaboration and ergonomics G.A. Boy in his book [START_REF] Boy | Tangible Interactive Systems[END_REF] dedicated a chapter for concepts and tools for designers, where he discussed the Human Centred Design (HCD) and the task/activity distinction towards a system. He proposes to combine HCD and technology-centered engineering to make human-systems integration. He says that we should focus more on human tasks while designing a system since the beginning of the process, and also to distinguish between a task (what is prescribed on user requirements) and an activity (what is effectively performed). This difference has place often between the practice (physical world) and the theory (virtual ideal world). Therefore, to fill this gap between these two worlds, using generic tasks can guarantee a standardization of user tasks or interactions towards the system, resulting in rationalization of different kind of interactions and more stability of the system. It would also allow to avoid any unexpected behaviour from the user that may question the efficiency of the system. 2 shows a cognitive function as a transformation of a task into an activity, influenced by human attributes (e.g., fatigue, motivation, competence), the current situation (i.e., the state of the world around), the organization (i.e., the various actors involved in the execution of the task), and the system itself [START_REF] Boy | Tangible Interactive Systems[END_REF].

A.3 Linking two interinfluenced environments

Our work is based on the fusion of the dual reality paradigm and the generic tasks concepts of Clancey [START_REF] Clancey | Heuristic classification[END_REF] which we exposed in the previous section. We propose this generic model [START_REF] Merrad | Reformulating Clancey's generic tasks for bridging both sides of dual reality[END_REF].

Our proposal allows to generically model the user activities, as well as to bridge the both sides of dual reality. We note that the interaction is bidirectional (as defined in the dual reality paradigm) and each element from one world, whether it is a common component or not, can influence and be influenced by the same world or the other world components, be they common or not. The standard components of the two worlds are elements that we find in both of them, with the same properties. For instance, object states, machine properties, and user inputs are all common between the two worlds. While for components of one world, they are elements that can be found in one world but not in the other. However, they can be represented by similar elements. For example, a person is a real physical component of the real world that can be represented by an avatar in the virtual world, which cannot exist in the real world (see figure A.3).

Moreover, the bidirectional arrows show that an object may interact with another object from the same world (and actually the same family), using one or many generic tasks. In this case, the interaction will be reproduced in the other world between the corresponding elements of the corresponding generic task(s). The curved zigzags between the three most important parts of the model Figure A.3) indicate that there are no clear limits. Thus, merging or bridging both sides of the dual reality is similar to making these boundaries transparent for the users and providing a human-computer interface that ensures the same environment, in the real or virtual world, while using a system.

The sensors and actuators networks will ensure the communication: sensing, sending and actioning, regardless of the used protocols, between the two worlds. Information (instructions, user data, etc.) can circulate and be exchanged based on the model for any possible combination of components as described previously. The exchange is done according to one or many generic tasks and cannot take any form else.

Any interaction can be constructed and described using the generic tasks listed in the middle part of Figure A.3. In fact, according to Clancey [START_REF] Clancey | Heuristic classification[END_REF] we can combine some of those generic elementary tasks to get a sequence of operations to solve a given problem. The two commonly occurring sequences are:

• The construction cycle: SPECIFY + DESIGN {+ASSEMBLE}.

• The maintenance cycle: {MONITOR + PREDICT + } DIAGNOSE + MODIFY. Clancey notes in [START_REF] Clancey | Heuristic classification[END_REF] that "heuristic classification is well-suited for problems of interpretation involving a system that is known to the problem solver", and this includes intelligent interactive systems. For example, the Kinect of Microsoft enables a broad range of interaction with old and new applications [START_REF] Zhang | Microsoft kinect sensor and its effect[END_REF]. For instance, using body gestures (whole body, hands, head, etc.) to calibrate the Kinect for a game, the user has to go through multiple primitive and combined generic tasks, we mention for example and not limited to:

• CONTROL: moving the cursor to select an item in the screen,

• {MONITOR + PREDICT}: also known as test, user checks each time he or she makes an interaction if the result is convenient or not,

• MODIFY: when the user moves an object on the screen using the cursor (and his/her hand),

• IDENTIFY: the user can identify from a set of known device positions which one suits him/her well for a given game.

Also, the construction cycle can be approached while setting up a game (via Kinect or usual joystick). The user has to SPECIFY some criteria, such as how many players, DESIGN the game like choosing places, environments and players characteristics. Finally, the user ASSEMBLE the system (the game) after validation.

According to Clancey, the maintenance cycle is the familiar pattern of medical programs, such as MYCIN [START_REF] Shortliffe | Computer-based medical consultations: MYCIN[END_REF]. The test consists of repeatedly observing system behavior on the input selected to verify output predictions.

  (a) Serious game for waste sorting on the TangiSense 2 tabletop [148]. (b) Performing on TangiMusic application (on TangiSense 1 tabletop) [6]. . . 1.2 Interactive wall prototype [45]. . . . . . . . . . . . . . . . . . . . . . . . . 1.3 (a) The eLabBench in the laboratory. (b) A biologist doing a lab experiment on the eLabBench [245]. . . . . . . . . . . . . . . . . . . . . . . . . 1.4 A warehouse built by an apprentice with augmented information (accessibility information displayed on the top of the shelves and forklift simulation) [232]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Manipulating and simulating landscapes with flexible, elastic and viscous rendered material properties [193]. . . . . . . . . . . . . . . . . . . . . . . 1.6 Conjure interface [150]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Tangible CityScape interface incarnating buildings [122]. . . . . . . . . . 1.8 Device and application examples of the ChainFORM hardware system. a:

  . (b) Microsoft Surface Studio 2 [182]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.14 (a) RFID technique of recognizing objects [128]. (b) RFID tags of different sizes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.15 Rear DI technology principle, adapted from [230]. . . . . . . . . . . . . . 1.16 Illustration in general of a camera capture system (here, below the table)

  1.18 (a) Children part of the application. (b) The supervisor part of the application [25]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3. 1

 1 Overview of RFID technology (here by a matrix of antennas) [128] . . . . 3.2 (a) TangiSense v1 panel with RFID tags. (b) External aspect of the TangiSense v2 [24]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 (a) A user manipulating virtual objects on TangiSense 2. (b) A user manipulating tangible objects on TangiSense 2. . . . . . . . . . . . . . . . . 3.4 TangiSense tabletop software architecture [130] . . . . . . . . . . . . . . 3.5 (a) Left hand glove tagged with a RFID tag. (b) Right hand glove tagged with a RFID tag. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Positioning of the elements concerned by this preliminary study in relation to our model described in the previous chapter. . . . . . . . . . . . . . . 3.7 Display at the beginning of tasks with different objects' sizes and different interaction techniques (tangible on top and touch in bottom). . . . . . . 3.8 different objects' sizes compared to a user hand. . . . . . . . . . . . . . . 3.9 Tasks and scenario progress. . . . . . . . . . . . . . . . . . . . . . . . . .

4. 1

 1 Positioning of this preliminary study in relation to our proposed model (in Chapter 2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Display at the beginning of a touch task (left) and at the beginning of a tangible task (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 A user performing tasks with three different hands synchronicity. . . . . . 4.4 Tasks and scenario progress of this study. . . . . . . . . . . . . . . . . . . 4.5 Median values for the attention demand of displacement of set of objects measured as absolute rating (top) and relative ranking (bottom) in terms of the modality used, the hands synchronicity and the population. Note: In both graphs, task conditions are ordered by ascending rating values; higher numerical values show larger demanding attention. . . . . . . . . . 5.1 (a) Mini-robot toy used on the tabletop surface, equipped with RFID tags. (b) The RFID tag stuck to the mini-robot from below, used on the tabletop surface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 (a) The tangible version of the application the Tangisense Tabletop. (b)

13 Acronyms

 13 . 5.5 (a) Lego Mindstorms NXT robot equipped with a camera. (b) The camera tangible object used to take photos in the tangible interface version of the application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Global architecture of the system and data flow. . . . . . . . . . . . . . . 5.7 Instantiating our proposed model of class diagram to describe our application.114 5.8 (a) Task one illustrated (using one robot). (b) Task two illustrated (using two robots). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9 A participant performing with two tangible robots, exploring the disaster area and taking pictures of supposed victims. . . . . . . . . . . . . . . . . 5.10 Tasks and scenario progress. "T1" refers to task one, "T2" refers to task two, "S1" refers to system one (tangible version of the application) and "S2" refers to system two (touch version of the application). . . . . . . . . 5.11 NASA-TLX sub-scales means for one and two robots and in tangible and touch interaction technique. . . . . . . . . . . . . . . . . . . . . . . . . . 5.12 SUS global scores means with standard error bars, for tangible and touch interaction technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.13 Participants post experiment evaluations. . . . . . . . . . . . . . . . . . . 6.1 Tangible items on the tabletop (real side of dual reality) . . . . . . . . . 6.2 Digital items on the tabletop (virtual side of dual reality -virtual counterpart of Figure 6.1). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 A food plate used in this application equipped with a RFID tag. . . . . . 6.4 Scheme of the application showing the two tabletops, computers and switch.132 6.5 Architecture of our system with several patients connected to one healthcare center. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Instantiating our proposed model of class diagram, describing our application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1 (a) Generic operations for synthesizing a system. (b) Generic operations for analyzing a system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Transformation of a task into an activity [27]. . . . . . . . . . . . . . . . A.3 Generic bridging model between the real and virtual worlds [177]. . . . .List of Tables1.1 Census of tabletops and their technical specifications. . . . . . . . . . . . 1.2 The difference between Virtual reality, Augmented reality, Mixed reality and Dual reality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Likert questions employed to elicit attention demand rating scores. Inspired from [268]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Movement times by cardinality. . . . . . . . . . . . . . . . . . . . . . . . 4.1 Likert questions employed to elicit attention demand rating scores. Inspired from [268]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Data descriptions and their origins. . . . . . . . . . . . . . . . . . . . . . 5.2 Errors categories, their description and frequencies (1R: one robot; 2R: two robots). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Tasks' performed without any mistakes and average number or errors per tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Tasks completion rates by number of robots and by interface type. . . . . 5.5 Average tasks' completion times and their standard deviations. . . . . . . 5.6 Overall relative efficiency results. . . . . . . . . . . . . . . . . . . . . . . 5.7 Post-experiment interview results. . . . . . . . . . . . . . . . . . . . . . . 1D one-dimensional. 43 1H-Seq one-handed sequential movement. 95, 96, 100-105 2D two-dimensional. 43 2H-Alt two-handed, alternating movements. 95, 96, 100, 101, 103-105 2H-Syn two-handed synchronous movements. 95, 96, 100, 101, 103-105 3D three-dimensional.

Figure 1 . 1 :

 11 Figure 1.1: (a) Serious game for waste sorting on the TangiSense 2 tabletop [148]. (b) Performing on TangiMusic application (on TangiSense 1 tabletop) [6].

Figure 1 . 2 :Figure 1 . 3 :

 1213 Figure 1.2: Interactive wall prototype [45].

Figure 1 . 4 :

 14 Figure 1.4: A warehouse built by an apprentice with augmented information (accessibility information displayed on the top of the shelves and forklift simulation) [232].

Figure 1 . 6 :

 16 Figure 1.6: Conjure interface [150].

Figure 1 . 7 :

 17 Figure 1.7: Tangible CityScape interface incarnating buildings [122].

  .8).

Figure 1 . 8 :

 18 Figure 1.8: Device and application examples of the ChainFORM hardware system. a: ChainFORM hardware configurations and modules, b: reconfigurable display, c: shape changing stylus, d: animated character, e: haptic glove [189].

  .10).

Figure 1 .

 1 Figure 1.10: (a) Teegi [63]. (b) A user observing and analyzing his own brain activity in real-time while manipulating Teegi [64].

Figure 1 .

 1 [START_REF] Back | Listen reader: an electronically augmented paper-based book[END_REF] shows examples of multi-touch user interfaces tabletops.

Figure 1 .

 1 Figure 1.11: (a) BendDesk tabletop [277]. (b). ELC2 tabletop [54].

Figure 1 .

 1 [START_REF] Back | The virtual chocolate factory: mixed reality industrial collaboration and control[END_REF] shows examples of tabletops based on tangible user interfaces.

Figure 1 .

 1 Figure 1.12: (a) Cooking ideas on TangiSense tabletop (version one) [142]. (b). A musician using mixiTUI tabletop [206].

Figure 1 .

 1 Figure 1.13: (a) Users collaborating around The reacTable tabletop [117]. (b) Microsoft Surface Studio 2 [182].

Figure 1 .

 1 14 (a) shows this principle of identifying objects on an interactive surface made of a matrix of RFID antennas, while Figure1.14 (b) shows different sizes and shapes of RFID tags.

Figure 1 .•

 1 Figure 1.14: (a) RFID technique of recognizing objects [128]. (b) RFID tags of different sizes.

Figure 1 . 15 :

 115 Figure 1.15: Rear DI technology principle, adapted from [230].

Figure 1 . 17 :

 117 Figure 1.17: An example of distributed application architecture (Cooking Ideas [142]).

Figure 1 .

 1 Figure 1.18: (a) Children part of the application. (b) The supervisor part of the application [25].

Figure 1 . 19 :

 119 Figure 1.19: Fundamental mappings between the real (left) and the virtual (right)[START_REF] Lifton | Dual reality: an emerging medium[END_REF].

Figure 1 . 20 :

 120 Figure 1.20: Representation of a virtuality continuum, adapted from [184].

1. 4 . 2 . 1

 421 Example one: The chocolate factory Back et al. have designed a virtual factory[START_REF] Back | The virtual chocolate factory: mixed reality industrial collaboration and control[END_REF] that reflects a real world chocolate factory[START_REF] Ventures | [END_REF] located in San Francisco, USA. Data is collected and imported, by means of sensors implanted in the real factory, to the virtual environment where several users can do simulations, visualizations and collaborate using a set of interlinked, real-time layers of information. Figure1.21 shows the virtual (left) and the real (right) environments.

Figure 1 .

 1 Figure 1.21: (a) An avatar in the Multiverse Virtual Factory. (b) The Tcho factory floor under construction. Adapted from [12].

Figure 1 . 22 :

 122 Figure 1.22: Side view of the final implementation of Shadow Lab, which includes data ponds [154].

Figure 1 . 23 :

 123 Figure 1.23: Environment in the real, virtual and Dual Reality condition [215].

Figure 1 .

 1 [START_REF] Bouabid | Contribution à la conception d'interfaces homme-machine distribuées sur tables interactives avec objets tangibles[END_REF] shows an example of a Pokémon in the AR environment.

Figure 1 .

 1 Figure 1.24: A Pokémon in Augmented Reality environment [234].

Figure 1 .

 1 [START_REF] Bouabid | Design and evaluation of distributed user interfaces between tangible tabletops[END_REF] shows a person completely immersed in a VR environment and performing actions.

Figure 1 .

 1 Figure 1.25: A user performing actions in Virtual Reality [57].

Figure 1 .

 1 Figure 1.26: A man wearing Microsoft HoloLens smartglasses [179].

Figure 1 . 27 :

 127 Figure 1.27: Comparison of different aspects and emergence to and from real and virtual environments. (a) Reality. (b) Virtual Reality. (c) Augmented Reality. (d) Augmented Virtuality.

Figure 2 . 1 :

 21 Figure 2.1: Our basic modeling using package diagram.

  .3.

Figure 2 . 2 :

 22 Figure 2.2: Topology of master-slave computer network communication.

Figure 2 . 3 :

 23 Figure 2.3: Topology of star network communication.

  e. each component on one side has its own counterpart on the other side. Each CoreApplication Package can communicate with the Communication Package of its side, in bidirectional way, and with the Platform (respectively Environment) Package in the VirtualSide (respectively RealSide) Package (see Figure 2.4).

Figure 2 . 4 :

 24 Figure 2.4: Dependencies between our basic modeling packages.

  Figure 2.5, Figure 2.6 and Figure 2.7. The dependencies between each package classes, along with dependencies linking classes from different packages, will be provided with further details in Section 2.2.2.

Figure 2 . 5 :

 25 Figure 2.5: The VirtualSide Package classes.

Figure 2 . 6 :

 26 Figure 2.6: The RealSide Package classes.

Figure 2 . 7 :

 27 Figure 2.7: The User Package classes.

Figure 2 . 8 :

 28 Figure 2.8: The VirtualSidePackage class diagram.

Figure 2 . 9 :

 29 Figure 2.9: The RealSidePackage class diagram.

Figure 2 .

 2 Figure 2.10: The UserPackage class diagram.

Figure 2 .

 2 [START_REF] Back | Listen reader: an electronically augmented paper-based book[END_REF] shows a class diagram of how the tangible object can be integrated and in the VirtualSide Package, while Figure2.12 shows a class diagram of how the tangible interaction can be taken into consideration materially.

Figure 2 . 11 :

 211 Figure 2.11: Class diagram for integrating tangible object into a dual reality system.

Figure 2 .

 2 Figure 2.12: Class diagram of the virtual side integrating tangible interaction.

Figure 2 . 13 :

 213 Figure 2.13: An example of a tangible object used on a tabletop surface [227].

  Figure 2.14 shows -non exhaustive-examples of interaction techniques inheriting from Interaction class.

Figure 2 . 14 :

 214 Figure 2.14: Interaction techniques example inheriting from Interaction class.

Figure 2 . 15 :

 215 Figure 2.15: Taking into account the MAS interaction.

Figure 2 .

 2 [START_REF] Beckhaus | Back to the sandbox: playful interaction with granules landscapes[END_REF] shows this class diagram along with packages of both dual reality sides and user; the figure is rotated 90 • anticlockwise.

Figure 2 . 16 :

 216 Figure 2.16: General dual reality class diagram. 67

Figure 2 . 17 :

 217 Figure 2.17: The VirtualSide Package after taking into account the tangibility.

Figure 2 . 18 :

 218 Figure 2.18: General dual reality package diagram.

Figure 2 . 19 :

 219 Figure 2.19: Sequence diagram for interacting with a real object through a virtual object.

Figure 2 . 20 :

 220 Figure 2.20: Sequence diagram for making changes in the real side by using a virtual object.

Figure 2 .

 2 21 describes this interaction.

Figure 2 . 21 :

 221 Figure 2.21: Sequence diagram for interacting with a virtual object through a real object.

Figure 2 .

 2 22 and Figure 2.23.

Figure 2 . 22 :

 222 Figure 2.22: Sequence diagram for a disruptive element affecting a RealObject.

Figure 2 . 23 :

 223 Figure 2.23: Sequence diagram for a disruptive element affecting the RealEnvironment-Properties.

Figure 2 . 24 :

 224 Figure 2.24: Sequence diagram of a disruptive element influencing a real object and its environment.

Figure 3 .

 3 2 shows a panel of the first version of TangiSense tabletop (panels of this version were made of 8 × 8 RFID antennas) and the second version of the TangiSense tabletop. In this Figure 3.2 (b), in red is an RFID antenna, in green is a panel composed of 4 × 4 RFID antennas and in blue is non-interactive area (still can display graphical components in it).

Figure 3 . 2 :

 32 Figure 3.2: (a) TangiSense v1 panel with RFID tags. (b) External aspect of the TangiSense v2 [24].

Figure 3 . 3 :

 33 Figure 3.3: (a) A user manipulating virtual objects on TangiSense 2. (b) A user manipulating tangible objects on TangiSense 2.

Figure 3 . 5 :

 35 Figure 3.5: (a) Left hand glove tagged with a RFID tag. (b) Right hand glove tagged with a RFID tag.

3. 3 . 1 . 1

 311 Participants 11 participants (2 females) volunteered to take part into our experiment. Their age was between 25 and 35 years (mean = 29.36, s.d = 4.27). 10 participants were right handed. All had automation and computer science background; they were undergrads, PhD students and post-docs with little knowledge of tabletops and surface computing.

Figure 3 . 6 :

 36 Figure 3.6: Positioning of the elements concerned by this preliminary study in relation to our model described in the previous chapter.

Figure 3 . 7 :

 37 Figure 3.7: Display at the beginning of tasks with different objects' sizes and different interaction techniques (tangible on top and touch in bottom).

Figure 3 . 8 :

 38 Figure 3.8: different objects' sizes compared to a user hand.

Figure 3 . 9 :

 39 Figure 3.9: Tasks and scenario progress.

Figure 3 . 10 :

 310 Figure 3.10: Median values for the user-perceived attention demand of displacement of set of objects measured as absolute rating (top) and relative ranking (bottom), highlighting the used interaction technique, the size of object to move and the the cardinality. Note: In both graphs, displacement conditions are ordered by ascending rating values; higher numerical values show larger attention demand.

Figure 4 . 1 :

 41 Figure 4.1: Positioning of this preliminary study in relation to our proposed model (in Chapter 2).

Figure 4 . 2 :

 42 Figure 4.2: Display at the beginning of a touch task (left) and at the beginning of a tangible task (right).

Figure 4 .

 4 3 which illustrates the three different hands synchronicity.

  (a) 1H-Seq tangible interaction (b) 2H-Syn tangible interaction (c) 2H-Alt touch interaction (right hand) (d) 2H-Alt touch interaction (left hand)

Figure 4 . 3 :

 43 Figure 4.3: A user performing tasks with three different hands synchronicity.

Figure 4 . 4 :

 44 Figure 4.4: Tasks and scenario progress of this study.

Figure 4 .

 4 Figure 4.5 illustrates participants' responses of the attention demand when performing a composite task with touch and tangible modalities. Overall, we found a moderate degree of agreement between participants' responses, as reflected by Kendall's coefficient of concordance 2 (W = 0.53, χ 2 (35) = 230, p < 0.0001 for rating and W = 0.56, χ 2 (35) = 241, p < 0.0001 for ranking). These findings indicate that the level of the attention demand is perceived by people in a consistent manner, which justifies further investigation.

Figure 4 . 5 :

 45 Figure 4.5: Median values for the attention demand of displacement of set of objects measured as absolute rating (top) and relative ranking (bottom) in terms of the modality used, the hands synchronicity and the population. Note: In both graphs, task conditions are ordered by ascending rating values; higher numerical values show larger demanding attention.

Figure 5 . 1 :

 51 Figure 5.1: (a) Mini-robot toy used on the tabletop surface, equipped with RFID tags. (b) The RFID tag stuck to the mini-robot from below, used on the tabletop surface.

Figure 5 . 2 :

 52 Figure 5.2: (a) The tangible version of the application the Tangisense Tabletop. (b) The touch version of the application the Tangisense Tabletop.

Figure 5 . 3 :

 53 Figure 5.3: The gloves that simulate the touch feature on the tabletop, with the RFID tags in the index fingers. (a) The left hand glove. (b) The right hand glove.

Figure 5 . 4 :

 54 Figure 5.4: Possible types of disaster where our system could be used. (a) Contaminated zones[START_REF] Christensen | 30 years after the chernobyl meltdown, why is the ukrainian government pushing nuclear energy[END_REF]. (b) Fire zones[START_REF] Pedneault | Fire inside an abandoned convent in massueville, quebec, canada[END_REF]. (c) Battlefields; adapted from[START_REF] Cnn | The battle of aleppo in 20 photos[END_REF].

Figure 5 . 5 :

 55 Figure 5.5: (a) Lego Mindstorms NXT robot equipped with a camera. (b) The camera tangible object used to take photos in the tangible interface version of the application.

Figure 5 . 6 :

 56 Figure 5.6: Global architecture of the system and data flow.

Figure 5 .

 5 [START_REF]Floating.numbers[END_REF] shows the class diagram of this application; the figure is rotated 90 • anticlockwise.

Figure 5 . 7 :

 57 Figure 5.7: Instantiating our proposed model of class diagram to describe our application.

Figure 5 .

 5 8 (a) illustrates this first task setup on the tabletop (tangible version).

Figure 5 .

 5 8 (b) illustrates this second task setup on the tabletop (also tangible version).

Figure 5 . 8 :

 58 Figure 5.8: (a) Task one illustrated (using one robot). (b) Task two illustrated (using two robots).

Figure 5 . 9 :

 59 Figure 5.9: A participant performing with two tangible robots, exploring the disaster area and taking pictures of supposed victims.

  use the right object for a given task.

  Missed signalWhen participant does not see the flashing signal to start moving a robot.

Figure 5 .

 5 Figure 5.11: NASA-TLX sub-scales means for one and two robots and in tangible and touch interaction technique.

Figure 5 .

 5 Figure 5.12: SUS global scores means with standard error bars, for tangible and touch interaction technique.

Figure 5 . 13 :

 513 Figure 5.13: Participants post experiment evaluations.

  .

  2.

Figure 6 . 1 :

 61 Figure 6.1: Tangible items on the tabletop (real side of dual reality)

Figure 6 . 2 :

 62 Figure 6.2: Digital items on the tabletop (virtual side of dual reality -virtual counterpart of Figure 6.1).

Figure 6 . 3 :

 63 Figure 6.3: A food plate used in this application equipped with a RFID tag.

Figure 6 . 4 :

 64 Figure 6.4: Scheme of the application showing the two tabletops, computers and switch.

  Figure 6.6 shows the class diagram of this application prototype (Figure rotated 90 • anticlockwise), inspired from the model illustrated in Figure 2.16.

Figure 6 . 6 :

 66 Figure 6.6: Instantiating our proposed model of class diagram, describing our application.

. 6 ,

 6 VirtualPatientHouseProperties class replaces VirtualEnvironmentProperties class; while PresentElementsPatientTabletop class replaces the PresentElementsOnRealSide class, and PresentElementsOnTabletop class replaces PresentElementsOnVirtualSide class. All of these classes are associated with AlzheimerMonitoringApplication class, which represents the core of the software and replaces SoftwareApplication class in our proposed model. Multiplicities of the dependencies are all 1 on the edge of AlzheimerMonitoringApplication class, and are all of 1..* for all other classes, i.e., VirtualPatientHouseProperties, PresentElementsOnTabletop, and PresentElementsPatientTabletop.

Figure A. 1 :

 1 Figure A.1: (a) Generic operations for synthesizing a system. (b) Generic operations for analyzing a system.

Figure A. 2 :

 2 Figure A.2: Transformation of a task into an activity [27].

Figure A.

  Figure A.2 shows a cognitive function as a transformation of a task into an activity, influenced by human attributes (e.g., fatigue, motivation, competence), the current situation (i.e., the state of the world around), the organization (i.e., the various actors involved in the execution of the task), and the system itself[START_REF] Boy | Tangible Interactive Systems[END_REF].

  shown inFigure A.3 for mapping between the two worlds and ensure a generic manner of interaction from and towards real and virtual worlds.

Figure A. 3 :

 3 Figure A.3: Generic bridging model between the real and virtual worlds [177].

2

  Design principles for dual reality applications supported by tangible tabletops 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Taking into consideration the duality . . . . . . . . . . . . . . . . . . . . 2.2.1 Package diagrams for dual reality applications modeling . . . . . . 2.2.2 Class diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Integrating tangibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Modelling the interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Interacting with virtual object . . . . . . . . . . . . . . . . . . . . 2.4.2 Interacting with real object . . . . . . . . . . . . . . . . . . . . . 2.4.3 Disruptive element affecting the system . . . . . . . . . . . . . . . 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1.3 Tasks, procedure and design . . . . . . . . . . . . . . . . 3.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2.1 Effect of Object Size on Attention Demand . . . . . . . 3.3.2.2 User performance . . . . . . . . . . . . . . . . . . . . . . 3.3.2.3 Mental model and qualitative results . . . . . . . . . . . 3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . General conclusion Our contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The novelty of our work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Generic tasks in HCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 Linking two interinfluenced environments . . . . . . . . . . . . . . . . . .

	Appendices
	A Generic tasks
	A.1 B Objects size experiment instructions
	C Hands synchronicity experiment instructions
	4 Preliminary study 2: studying user performance and attention demand
	using both hands on a tabletop

3 Preliminary study 1: determining optimal objects' size for performing elementary tasks on tangible tabletop surface 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Presentation of the "TangiSense 2" tabletop . . . . . . . . . . . . . . . . 3.2.1 Description of the tabletop . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Interaction with tangible objects . . . . . . . . . . . . . . . . . . 3.2.3 Software architecture of the tabletop . . . . . . . . . . . . . . . . 3.2.4 Customizing the software library . . . . . . . . . . . . . . . . . . 3.2.5 Tactile feature implementation on "TangiSense 2" . . . . . . . . . 3.3 Determining optimal objects' size for performing elementary tasks on tangible tabletop surface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Study design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1.2 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Study design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D Mobile-robots study questionnaires D.1 pre-experiment questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . D.2 post-experiment questionnaire . . . . . . . . . . . . . . . . . . . . . . . . Bibliography List of Figures 1.1

  . . . . . . . 1.21 (a) An avatar in the Multiverse Virtual Factory. (b) The Tcho factory floor under construction. Adapted from [12]. . . . . . . . . . . . . . . . . 1.22 Side view of the final implementation of Shadow Lab, which includes data .23 Environment in the real, virtual and Dual Reality condition [215]. . . . . 1.24 A Pokémon in Augmented Reality environment [234]. . . . . . . . . . . . 1.25 A user performing actions in Virtual Reality [57]. . . . . . . . . . . . . . 1.26 A man wearing Microsoft HoloLens smartglasses [179]. . . . . . . . . . . 1.27 Comparison of different aspects and emergence to and from real and virtual environments. (a) Reality. (b) Virtual Reality. (c) Augmented Reality. (d) Augmented Virtuality. . . . . . . . . . . . . . . . . . . . . . . . . . . 1.28 STOP metaphor in both environments and interconnection between real and virtual side of dual reality. . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Our basic modeling using package diagram. . . . . . . . . . . . . . . . . . 2.2 Topology of master-slave computer network communication. . . . . . . . 2.3 Topology of star network communication. . . . . . . . . . . . . . . . . . . 2.4 Dependencies between our basic modeling packages. . . . . . . . . . . . . 2.5 The VirtualSide Package classes. . . . . . . . . . . . . . . . . . . . . . . . 2.6 The RealSide Package classes. . . . . . . . . . . . . . . . . . . . . . . . . 2.7 The User Package classes. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 The VirtualSidePackage class diagram. . . . . . . . . . . . . . . . . . . . 2.9 The RealSidePackage class diagram. . . . . . . . . . . . . . . . . . . . . . 2.10 The UserPackage class diagram. . . . . . . . . . . . . . . . . . . . . . . . 2.11 Class diagram for integrating tangible object into a dual reality system. . 2.12 Class diagram of the virtual side integrating tangible interaction. . . . . . 2.13 An example of a tangible object used on a tabletop surface [227]. . . . . .21 Sequence diagram for interacting with a virtual object through a real object. 2.22 Sequence diagram for a disruptive element affecting a RealObject. . . . . 2.23 Sequence diagram for a disruptive element affecting the RealEnvironment-

ponds

[START_REF] Lifton | Dual reality: Merging the real and virtual[END_REF]

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.14 Interaction techniques example inheriting from Interaction class. . . . . . 2.15 Taking into account the MAS interaction. . . . . . . . . . . . . . . . . . 2.16 General dual reality class diagram. . . . . . . . . . . . . . . . . . . . . . 2.17 The VirtualSide Package after taking into account the tangibility. . . . . 2.18 General dual reality package diagram. . . . . . . . . . . . . . . . . . . . . 2.19 Sequence diagram for interacting with a real object through a virtual object. 2.20 Sequence diagram for making changes in the real side by using a virtual object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.24 Sequence diagram of a disruptive element influencing a real object and its environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  [START_REF] Couture | Geotui: a tangible user interface for geoscience[END_REF][START_REF] Detken | The search wall: tangible information searching for children in public libraries[END_REF][START_REF] Diaper | The handbook of task analysis for human-computer interaction[END_REF][START_REF] Fitzmaurice | Bricks: laying the foundations for graspable user interfaces[END_REF][START_REF] Guiard | Asymmetry in bimanual skills[END_REF][START_REF] Holzmayr | Looplex-interaktives sound interface[END_REF][START_REF] Ideum | Pano Multitouch Table[END_REF][START_REF]Scape Lab[END_REF]

	HCI Human-Computer Interaction. 19, 20, 22, 23, 27, 41, 111, 115 SEQ Single Ease Question. 123
	HDMI High-Definition Multimedia Interface. 94 SMEQ Subjective Mental Effort Questionnaire. 123
	HMI Human-Machine Interaction. 79 SUMI Software Usability Measurement Inventory. 123
	ID Identification. 39, 79 SUPR-Q Standardized User Experience Percentile Rank Questionnaire. 123
	IDE Integrated Development Environment. 94 SUS System Usability Scale. 11, 113, 117, 119, 123, 125
	IoT Internet of Things. 23 TUI Tangible User Interface. 39, 41, 106, 113, 119, 124, 125, 127, 131
	IP Internet Protocol. 94 TUIs Tangible User Interfaces. 2, 23, 38, 91, 127
	IPS In-Plane Switching. 32 UI User Interface. 43, 131, 138
	IR Infrared. 34, 36, 38, 40, 64 UME Usability Magnitude Estimation. 123
	ADLs Activities of daily living. 128, 136-139
	AI Artificial Intelligence. 55, 57, 135, 137
	ANOVA Analysis of variance. 88, 102-104
	AR Augmented Reality. 9, 10, 19-22, 47-50, 137
	ASQ After Scenario Questionnaire. 123
	AUTOS Artifact, User, Task, Organization, Situation. 57
	AV Augmented Virtuality. 10, 49
	B Big. 81, 83, 84, 86, 90
	CSUQ Computer System Usability Questionnaire. 123
	DI Diffused Illumination. 9, 31, 33-40
	DSI Diffused Surface Illumination. 32, 34
	DSP Digital Signal Processor. 76
	FIFO First In First Out. 55, 57, 59 QUIS Questionnaire For User Interaction Satisfaction. 123
	FTIR Frustrated Total Internal Reflection. 31, 32, 35 RFID Radio-frequency identification. 2, 3, 9-12, 23, 32-39, 63, 64, 75-81, 84, 90, 92-94,
	109, 110, 115, 117, 128-131
	GPS Global Positioning System. 47
	GUIs Graphical User Interfaces. 23, 43 S Small. 81, 83, 84, 86, 90
	SDK Software Development Kit. 79
	HCD Human Centred Design. 57, 144

LAMIH Laboratory of Industrial and Human Automation control, Mechanical engineering and Computer Science. 4, 111 LCD Liquid-Crystal Display. 30, 32-38, 43, 77, 79, 115 LED Light-Emitting Diode. 30, 32-38, 43 LEDs Light-Emitting Diodes. 76, 77, 79 LLP Laser Light Plane. 32, 34, 40 M Medium. 81, 83, 84, 86, 90 MAS Multi-Agent System. 10, 55, 64, 66, 68, 77-79, 131, 135 MPC Model Predictive Control. 111 MR Mixed Reality. 20-22, 48, 50, 137 NASA-TLX The NASA Task Load Index. 11, 113, 117, 120, 123, 124 NFC Near-field communication. 33, 36 PCT Screen Projected Capacitive Touch Screen. 37 QR code Quick Response code. 63 UML Universal Modeling Language. 20, 21, 51, 70, 74, 137, 139 VPN Virtual Private Network. 132 VR Virtual Reality. 9, 10, 19-22, 47-50, 137 WIMP Windows, Icons, Menus, Pointing device. 20, 38 WLAN Wireless Local Area Network. 112, 132 XML Extensible Markup Language. 131

table .

 .

	Thus, because
	of the capture technology used or the general characteristics of the table concerned,
	a table can eventually manage the touch, without the authors having chosen to use
	it. In this case, we consider that the table is not tactile since the technology is not
	used. If the information is not mentioned by the authors/constructors, we indicate
	the value (Not Communicated). The possible values are then Yes; No and N.C.
	• Multi-touch: it indicates whether the tabletop allows multi-touch, i.e. capable of
	detecting multiple and simultaneous contact points. As with tactile characteristic,
	we rely on the usages made and proposed by the authors/constructors (the table
	can be tactile but the authors do not work with multitouch). The possible values
	are therefore Yes; No; N.C and N.A.
	• Tangible: it indicates if the tabletop allows interactions with tangible or physical
	objects on its surface, regardless of the allowed number of objects. Possible values
	are Yes or No.

Table 1 .

 1

	Characteristics	Virtual	Augmented	Mixed	Dual
		reality	reality	reality	reality
		(VR)	(AR)	(MR)	(DR)
	Is the user aware about the	No	Yes	Yes	Yes
	real world?
	Can user interact with the	No	Yes	Yes	Yes
	real and virtual worlds in
	the real-time?
	Can real and virtual	No	No	Yes	Yes
	contents interact with each
	other in the real-time?
	Can the real and the virtual	Yes	No	No	Yes
	worlds be geographically
	separated?
	Can actions made in one	No	Yes	Yes	Yes
	world be reflected in the
	other world?

Table summarizes the main differences between AR, VR, MR and dual reality. 2: The difference between Virtual reality, Augmented reality, Mixed reality and Dual reality.

Table 3 .

 3

	.2

2: Movement times by cardinality.

Trial time. It is measured from the first object movement (first object selected or taken off the table) to the last object successfully moved to a valid target.

  .2.

				Number of errors
	Error category Description	Tangible		Touch
			1R 2R 1R 2R
	Incorrectly	An object is not placed by the partici-	0	3	4	2
	placed	pant on the right position on the table-
		top.

Table 5 .

 5 3: Tasks' performed without any mistakes and average number or errors per tasks.

			Tangible	graphical
	Number of robots	One	Two	One	Two
		robot	robots	robot	robots
	Tasks performed without	84.38%	59.38%	78.13%	56.25%
	any mistake
	Average number of errors	0.19	0.66	0.38	0.72
	per task

Table 5 .

 5 5 describes the participants timings of each task on each interface, it shows the means and the standard deviations in seconds.

	Number of robots	One robot	Two robots
	Interface	Tangible	Graphical Tangible	Graphical
	Means (in seconds)	24.26	25.3	36.45	40.78
	S.D. (in seconds)	2.06	3.89	6.2	6.86

Table 5 .

 5 5: Average tasks' completion times and their standard deviations.

  .6.

	Interface		Tangible	graphical
	Number of robots	One	Two	One	Two
		robot	robots	robot	robots
	Detailed	84.39%	76.59%	84.25%	66.06%
	General		79.7%		73.02%
	Table 5.6: Overall relative efficiency results.

Table 5 . 7

 57 .7 describe this appreciation.

	Statements	Mean	Standard er-	Standard de-
			ror	of	the	viation
			means
	Statement 1	4.59	0.15		0.84
	Statement 2	4.13	0.18		1.01
	Statement 3	4.25	0.17		0.95
	Statement 4	4.03	0.18		1.06
	Statement 5	4.34	0.14		0.79
	Statement 6	3.78	0.21		1.18

: Post-experiment interview results.

With a focus on usability.

Other tabletops of this manufacturer with the same configurations are available at https://www. table-tactile.fr/produits-table-tactile.html

German Research Center for Artificial Intelligence

JADE is a software framework for the development of intelligent agent, implemented in Java. https://jade.tilab.com/

The two preliminary studies have been conducted in collaboration with Yosra Rekik for their design and their results analysis.

Website: www.rfidees.com. Accessed on 2019-12-01.

Kendall's coefficient of concordance is a normalization of the statistic of the Friedman test used to asses continuity of judgment among multiple individuals. W takes values in [0, 1], where 0 denotes no agreement at all and 1 perfect agreement[START_REF] Kendall | The problem of m rankings[END_REF].

Java is a software platform and specifications developed by Sun Microsystems (later acquired by the Oracle Corporation), it provides a system for developing software applications and deploying them in a cross-platform computing environment.

Kendall's coefficient of concordance is a normalization of the statistic of the Friedman test used to asses continuity of judgment among multiple individuals. W takes values in [0, 1], where 0 denotes no agreement at all and 1 perfect agreement[START_REF] Kendall | The problem of m rankings[END_REF].

These robots were used in SUCRé project within the automation department of LAMIH

XBee devices communicate with each other over the air, sending and receiving wireless messages. The devices only transfer those wireless messages; they cannot manage the received or sent data. However, they can communicate with intelligent devices via the serial interface. Source: https://www.digi.com/ resources/documentation/Digidocs/90001456-13/concepts/c_how_xbees_communicate.htm

Acknowledgements

Appendices Appendix A

Generic tasks

A.1 Definition

The need for generic tasks evolves from the fact that the level of abstraction of much work in Knowledge-Based Systems (e.g. rules, frames, logic) is too low to provide a rich vocabulary for knowledge and control. Chandrasekaran [START_REF] Chandrasekaran | Towards a functional architecture for intelligence based on generic information processing tasks[END_REF] provided an overview of a framework called the Generic Task approach that proposes that knowledge systems should be built out of building blocks, each of which is appropriate for a basic type of problem solving. Each generic task uses forms of knowledge and control strategies that are characteristic to it, and are generally conceptually closer to domain knowledge. He follows next in the same paper [START_REF] Chandrasekaran | Towards a functional architecture for intelligence based on generic information processing tasks[END_REF] that the abstract specification of a generic task is:

• The function of the task. What type of problem does it solve? What is the nature of the information that it takes as input, and produces as output?

• The representation and organization of knowledge. What are the primitive terms in which the forms of knowledge needed for the task can be represented? How should knowledge be organized and structured for that task?

• The control strategy. What control strategy (inference strategy) can be applied to the knowledge to accomplish the function of the generic task?

In another paper of Chandrasekaran [START_REF] Chandrasekaran | Generic tasks as building blocks for knowledge-based systems: the diagnosis and routine design examples[END_REF], it is stated that each generic task relies on forms of knowledge and control strategies that are characteristic to it, and are in general conceptually closer to domain knowledge. In [START_REF] Brown | Design problem solving: knowledge structures and control strategies[END_REF] authors mention that each generic task is characterized by information about the following:

• The type of problem (the type of input and the type of output). What is the generic task used for?

• The representation of knowledge. How should knowledge be organized and structured to accomplish the function of the generic task? In particular, what are the types of concepts that are involved in the generic task? What concepts are the input and output about? How is knowledge organized in term of concepts?

• The inference strategy (process, problem solving, control regime). What inference strategy can be applied to the knowledge to accomplish the function of the generic task? How does the inference strategy operates on concepts?

Appendix B

Objects size experiment instructions

This appendix concerns the study presented in Chapter 3. The experiment consists of moving a set of objects on the tabletop from a departing zone (area A) to an arriving zone (area B). Following the displayed interface type -or interaction technique-and objects' size (small, medium and large), the user needs to move the objects from their initial locations (in red) to their destinations/targets (in blue).

If there are more destinations/targets than the number of objects to move, the participant is free to choose any destinations/targets among them where to place the objects. A trial will end when all objects are moved to the targets zone (area B). Please note that there is no order of moving or placing the objects on their targets.

At the end of the experiment, participants are asked to evaluate (rate) each trial and then classify (rank) them increasingly according to their degree of paid attention while performing.

Every participant must respect the following:

• Use only the dominant hand to move the objects.

• If a target is already occupied, it is not possible to put another object on it.

• If an object is selected it must be moved to a destination/target before proceeding to the next object.

• Move the objects at normal speed.

• When "Tangible" is displayed, tangible objects must be used, depending also on their sizes. It is the experimenter who places the tangible objects in their starting positions (in area A) and collect them from area B after finishing a trial. Participants must not do this or help the experimenter with the preparation as it is not part of their tasks. This is also in order to save their energy and focus for the experiment task.

• When "Touch" is displayed, participants have to use their index finger(s) to select and point, respectively, graphical objects and targets. Therefore, they need to wear the tactile gloves during the whole experiment. Tactile gloves must be wore also when performing with tangible objects, in order to guarantee the same experiment conditions in tactile and tangible interaction.

Every participant can take a break if needed or if s/he feels tired at any time. The break has to be between trials (cannot take a break while the running trial is not finished yet, i.e. all the objects have to be whether in area A or all in area B).

Appendix C Hands synchronicity experiment instructions

This appendix concerns the study presented in Chapter 4. The experiment consists of moving a set of objects on the tabletop from a departing zone (area A) to an arriving zone (area B). Following the displayed interface type and hands synchronization (1dominant-hand, 2 hands alternatively or 2 hands in parallel/simultaneously), the user needs to move the objects from their initial locations (in red) to their destinations/targets (in blue).

If there are more destinations/targets than the number of objects to move, the participant is free to choose any destinations/targets among them where to place the objects. A trial will end when all objects are moved to the targets zone (area B). Please note that there is no order of moving or placing the objects on their targets.

At the end of the experiment, participants are asked to evaluate (rate) each trial and then classify (rank) them increasingly according to their degree of paid attention while performing.

Every participant must respect the following:

• Use only the dominant hand to move the objects when it is shown "one hand" on the tabletop surface.

• If a target is already occupied, it is not possible to put another object on it.

• Move the objects at normal speed.

• Using a given hand, If an object is selected it must be moved to a destination/target before proceeding to the next object using the same hand.

• When "2 hands parallel" is displayed, the participant is required to use both hands at the same time: selecting or grabbing objects at the same time, moving the objects simultaneously, then placing them in targets at the same time.

• When "2 hands alternate" is displayed, the participant is required to use both hands but one after another: select, move and place a first object with the first hand then select, move and place another object with the other hand. The participant is free to start with right or left hand in this case.

• When "Tangible" is displayed, tangible objects must be used. It is the experimenter who places the tangible objects in their starting positions (in area A) and collect them from area B after finishing a trial. Participants must not do this or help the experimenter with the preparation as it is not part of their tasks. This is also in order to save their energy and focus for the experiment task.

• When "Touch" is displayed, participants have to use their index finger(s) to select and point, respectively, graphical objects and targets. Therefore, they need to wear the tactile gloves during the whole experiment. Tactile gloves must be wore also when performing with tangible objects, in order to guarantee the same experiment conditions in tactile and tangible interaction.

Every participant can take a break if needed or if s/he feels tired at any time. The break has to be between trials (cannot take a break while the running trial is not finished yet, i.e. all the objects have to be whether in area A or all in area B).

Appendix D Mobile-robots study questionnaires

This study has other questionnaires than those presented in Chapter 5: pre-experiment questionnaire and post-experiment questionnaire.

D.2 post-experiment questionnaire

Questionnaire about the tangible objects and the tabletop general usage feedback: For each of the following statements, circle one answer that best describes your reactions to the objects.

• The "robot" tangible object is easy to manipulate. Strongly disagree 1 2 3 4 5 strongly agree Justification (optional):

• The "robot" tangible object seems significant (meaningful) to you in relation to its role in the application. Strongly disagree 1 2 3 4 5 strongly agree Justification (optional):

• I had a full control on the "robot tangible object" while using it (not the robot). Strongly disagree 1 2 3 4 5 strongly agree Justification (optional):

• I had a full control on the "graphical robot object" while using it (not the robot). Strongly disagree 1 2 3 4 5 strongly agree Justification (optional):

• The tangible object "take picture" is easy to manipulate. Strongly disagree 1 2 3 4 5 strongly agree Justification (optional):

• The "take picture" tangible object seems significant (meaningful) to you in relation to its role in the application. Strongly disagree 1 2 3 4 5 strongly agree Justification (optional):

Comments and suggestion about the experimentation (optional):

151