Keywords: Human-computer interaction, Complex Data, Large displays, Multi-display environments, Immersive environments, Multi-degrees of freedom devices

I would like to express my gratitude to Emmanuel Dubois and Marcos Serrano, my supervisors, without whom this work would not have been possible. You welcomed me, guided and supported me during my whole time at the elipse team, on both a professional and personal level. I would like to thank you for all the time you gave me, for all the meetings that we had, for your patience and continuous encouragement. You thought me everything I know today in research. This work is a testament of all the guidance and support I have received from you.

I would also like to thank my reviewers, Nadine Couture and Sté phane huot, and my examiner, Marie-Pierre Gleizes, for the valuable feedback they gave me, and for accepting the invitation to participate in the jury.

I am also thankful to Pr. Pourang Irani, for allowing me to experience research abroad, by inviting me to work for a few months in his lab. I enjoyed working with you and learned a lot from you. This thesis was funded by the neOCampus project and would not have been possible without financial support. For that, I would like to thank Marie-Pierre Gleizes for accepting to provide the necessary funds for it.

I would also like to thank all the members of the Elipse team, for the good times we had together, the endless discussions and the feedback they gave me while I was there. A big thanks goes to Elio, Julie, Matthieu and Sandra.

A special thanks goes to my friends, Gary, Joca, Lamine and Manel for the good times we had together.

And last, but not least, I would like to thank my family, first, my mom, because you were always there, because you always pushed me to aim for more, because of all the sacrifices you made for us, because all I have achieved in my life is down to you. Thank you. I love you and I always will; I would also like to thank my second mother, my grandma, because you always took care of me, because your education is what made me the man I am today; I would like to thank my father, for all he has given me, my sisters and brothers, in this life; Finally, a big thank you goes to my sisters and brothers, for their love and unconditional support. I will end this acknowledgement section with you, Meryem, you may not have been there from the begining, you may not have had a big impact on the directions I took during my thesis, but you are shaped my future and quickly became the next chapter of my life. I love you beautiful. Dedicated to my mother.

Abstract

Today's ever-growing data is becoming increasingly complex due to its large volume and high dimensionality: it thus becomes crucial to explore interactive visualization environments that go beyond the traditional desktop in order to provide a larger display area and offer more efficient interaction techniques to manipulate the data. The main environments fitting the aforementioned description are: large displays, i.e. an assembly of displays amounting to a single space; Multi-display Environments (MDEs), i.e. a combination of heterogeneous displays (monitors, smartphones/tablets/wearables, interactive tabletops…) spatially distributed in the environment; and immersive environments, i.e. systems where everything can be used as a display surface, without imposing any bound between displays and immersing the user within the environment.

The objective of our work is to design and experiment original and efficient interaction techniques well suited for each of the previously described environments.

First, we focused on the interaction with large datasets on large displays. We specifically studied simultaneous interaction with multiple regions of interest of the displayed visualization. We implemented and evaluated an extension of the traditional overview+detail interface to tackle this problem: it consists of an overview+detail interface where the overview is displayed on a large screen and multiple detailed views are displayed on a tactile tablet. The interface allows the user to have up to four detailed views of the visualization at the same time. We studied its usefulness as well as the optimal number of detailed views that can be used efficiently.

Second, we designed a novel touch-enabled device, TDome, to facilitate interactions in Multi-display environments. The device is composed of a dome-like base and provides up to 6 degrees of freedom, a touchscreen and a camera that can sense the environment.

Having a unique device for interaction in these environments limits the homing effect when switching from one device to another and leads to a coherent set of interactions with the MDE, contributing to a more fluid task flow, a key element in such environments.

Finally, we introduced a new approach to interact in immersive environments with complex data. It is based on the use of the forearm as a physical support to assist tangible interactions with a multi-degrees of freedom device. We proposed a design space for this approach and we validated its feasibility through an experiment aimed at establishing the range, stability and comfort of gestures performed in this new paradigm.

All along this research work, resulting interaction techniques and environments have been concretely illustrated for exploring energy consumption data in the context of neOCampus, a project of the University of Toulouse 3 that aims at exploring the Campus of the Future, i.e. a smart, innovative and sustainable campus. surface. (A digital rendering is superimposed on the photograph to demonstrate the effect.) A cutting plane through the volume data is projected (like a shadow) onto the table below, where multitouch gestures are used to navigate and interrogate the data.

List of tables

List of figures

After navigating to a useful view of these imaging data of a heart, the user is now defining a smooth 3D curve (e.g., the shape of a catheter delivery system) relative to the anatomical data set

Context

Our ever-growing computing capabilities have led to a dramatic increase in data collection in the last twenty years. In 1992, Huber [START_REF] Huber | Issues in computational data analysis[END_REF] defined a taxonomy of large data sets and used the term "huge" to describe a 10 10 Bytes volume of data. At that time, multiple hard disks were necessary to hold that amount of data. Today, 26 years later, we have memory cards 1 that can hold 50 times the data sets described as "huge" in Huber's terms.

However, exploring this large collection of data, which is particularly important in scientific fields, is by no means an easy task, not only due to their volume but also to their heterogeneity. As early as 1988, Wegman [192] argued that computing resources were altering the character of some classes of data sets, making them not only much larger, but also high dimensional and less homogeneous. Today, Wegman's assumption still holds true. Making sense of such complex data, its high dimensions and large volumes, becomes even more difficult.

This thesis is part of the neOCampus operation, a project that relies heavily on collecting and exploiting data. This multidisciplinary project, launched in June 2013 at the University of Toulouse, involves 11 research laboratories from different fields. The goal of the project is to improve the confort of everyday life for the university users (students, professors and staff), while decreasing its ecological footprint by reducing the functioning ressources (water, electricity and so on). The project promotes research work, offering in the campus of the university, a platform for innovative experiments, done at a large scale and in vivo, with real users and real situations. The project aims to achieve its goals by taking advantage of the proliferation of inexpensive connected devices. The approach consists in creating a "smart campus" that would connect not only sensors and smart devices at fixed positions in the university, but also the personal mobile devices of their users (smartphones, tablets, wearables…). The data produced by such an approach is complex and time-dependent, and its exploration requires adequate analysis tools. The broad objective of the work presented in this manuscript is to design and evaluate interaction solutions to facilitate exploration of such data sets.Today, there is no settled interface or interaction solution to visualize and manipulate such complex data (volume, dimensionality, heterogeneity). However, a first general solution to improve visualization and manipulation of data consists in augmenting the display and visualization surface.

Several environments have been explored for this specific purpose. Some of the early ones involved the use of large displays, usually video projectors, with very low resolutions.

While their size allows them to easily scale up the data displayed, their resolutions limited the volume of data that can be displayed [START_REF] Andrews | Information visualization on large, high-resolution displays: issues, challenges, and opportunities[END_REF]. Eventually, large displays comprised of multiple tiled screens emerged, their combination offered high resolutions and large congruous display areas [START_REF] Andrews | Information visualization on large, high-resolution displays: issues, challenges, and opportunities[END_REF]. Their high resolutions allow them to display large volumes of data, which in turn, facilitate data visualization by allowing the user to have an overview of data when visualizing it from afar, and a more detailed view when getting closer to it.

A second approach consists in distributing the displays in space, to compose what we will refer to in this manuscript as multi-display environments. Such environments are efficient for visualizing multidimensional data: distributing the data among displays helps organizing it and facilitates interaction with the assortment of dimensions composing it.

The rapid evolution of mobile technology (smartphones, tablets and wearables in general)

widened the definition of multi-display environments (MDE). They introduced the notion of personal displays in addition to new input possibilities (small touchscreens and sensors).

A third way of augmenting display surfaces involves the use of immersive technologies.

The last decade saw the democratization of immersive systems, their rapid development contributes to their affordability which has a direct effect on the extent of research exploring their capabilities. Like multi-display environments, they allow the user to distribute data among the display area. However, the notion of display in immersive environment is broader, it refers to an area where data can be attached, rather than the digital technology showing the data. Immersive systems' stereoscopic and tracking capabilities allow the use of the natural spatial perception of the user in understanding the different dimensions as well as the spatial relationships in data.

Chapter 1 -Introduction 22

Interaction challenges

While there is no debate over the benefits of each environment for data exploration, it comes at the cost of several interaction challenges tightly related to their inherent characteristics. In the following, we provide a broad overview of those challenges for each one of the environments described above:

Large displays

The considerable display area of large screens introduce scalability issues in that, data visualizations previously designed for traditional monitors need to be redesigned when transitioning to large high resolution displays. To fully capitalize on the high count of pixels they afford and have a full view of the displayed data sets, interaction in those environments must be performed from a distance allowing the user to have the full visualization in his field of view. It becomes then important to be able to access and interact with unreachable content.

Multi-display environments

While the heterogeneity of displays composing an MDEs makes them a compelling solution for data visualization, they introduce their fair share of challenges in terms of interaction. The different sizes, resolutions of displays and the distributed aspect of the visualized data require a suitable interaction technique for content transfer between displays. The different input modalities of each display require a unified input technique that can be redirected from one display to the other. The large displays and tabletops in MDEs introduce a privacy problem in that users should be able to see private information if needed.

In addition to their physical characteristics, the displays composing MDEs have different input capabilities. The touch input offered by smartphones or tablets is not suitable for large displays; pointing interaction techniques used to reach distant objects in large displays are not suitable for the accessible display of an interactive tabletop; the traditional mouse and keyboard used for desktop monitors are not suitable for smartphones, tablets or large displays.

Immersive environments

A large interaction vocabulary is required to tackle the diverse tasks involved in data exploration. In immersive environments, these tasks are not entirely covered by the existing solutions. Approaches based on mouse, touch and mid-air interaction fail to offer enough degrees of freedom (DoF); other solutions are often ambiguous and tiring (especially mid-air gestures); and many restrict the user's interaction to a defined place, usually a desktop, to use the input device. The challenge then is to provide interactive solutions for immersive visualizations that preserves the freedom of movement of mid-air interaction and the DoFs of tangible interactions.

Contributions

The objective of our work is to improve interaction in each one of the previously described environments. Our thesis was driven by the following research questions:

-How to explore and manipulate physically unreachable content in multiple zone of interest on a large display?

-How to cope with heterogeneous input and display surfaces in MDEs ?

-How on-body interaction (i.e. gestures performed with the user's body as support)

can reduce fatigue while preserving the degrees of freedom required for interacting in an immersive environment ?

We address these questions through the following contributions:

-Designing and evaluating a multi-view overview + detail interface to interact with large data sets in large displays [157].

-Exploring the use of everyday objects for interaction in public multi-display environments [159].

-Designing and evaluating a touch enabled 6DOF interactive device for multidisplay environments [158].

-Using the body as a support for tangible interactions to explore data in immersive environments [160].

Contribution 1: Interaction with large datasets in large displays

We took interest in interaction with large datasets on large displays. We specifically focused on simultaneous interaction with multiple regions of interest of the displayed visualization. We implemented and evaluated an extension of the traditional overview+detail interface to tackle this problem: an overview + detail interface where the overview is displayed on a large screen and multiple detailed views are displayed on a tactile tablet. The interface allows the user to have up to four detailed views of the visualization at the same time. While the multi-view approach in itself is not new, the optimal number of detailed views has not been investigated. Using a single detailed view offers a larger display size but only allows a sequential exploration of the overview; using several detailed views reduces the size of each view but allows a parallel exploration of the overview. We experimentally evaluated the effect of the number of detailed views in a task related to interaction with large data sets.

Contribution 2 & 3: Interaction with multi-display environments

Our second contribution attempts to improve interaction with multi-display environments in two different contexts: a public context and a more usual office/work context.

A) The first contribution is based on the observation that public multi-display environments are as yet mainly used to display information due to the limited interaction possibilities they offer. Using this as our departure point, we identified the unique requirements of such environments: their public aspect limits the use of expensive devices as the risk of it being stolen or damaged is significant; the casual and quick nature of interactions performed in a public context requires easy to discover, easy to perform and opportunistic interactions; the interaction proposed must respect the personal space of its users. We proposed to explore the use of everyday objects as tools to perform tangible interactions to interact with these environments. They are always available, they offer easy to perform interactions and their shapes may help suggest their potential use.

B) The second contribution is TDome, a novel touch-enabled 6DOF input and output device to facilitate interactions in MDEs. TDome offers a private display as output, and multiple degrees of freedom as input by combining touch gestures on the display with physical rotation, roll and translation manipulations of the device. TDome allows versatile interactions that address major MDE tasks, which we illustrate through various proof-ofconcept implementations: detect surrounding displays, select one display, transfer data across displays, reach distant displays and perform private interactions. Having a unique device for interaction in these environments limits the homing effect when switching from one device to another and leads to a coherent set of interactions with the MDE, contributing to a more fluid task flow, a key element in such environments. We explore TDome's usability and suitability for MDEs through three user studies.

Contribution 4: Interaction with immersive environments

We introduced a new paradigm to interact in immersive environments with complex data requiring multiple degrees of freedom. It is based on the use of the forearm as a physical support to assist tangible interactions with a multi-degrees of freedom device.

The use of the body as a support for the interaction allows the user to move in his environment and avoids the inherent fatigue of this mid-air interactions-popular in immersive environments-. We proposed a design space for this approach describing the main characteristics on the interaction support as well as the interaction performed with the tangible device. We validated the adequacy of such an approach for immersive environments through an experiment aimed at establishing the range, stability and comfort of gestures performed in this new paradigm.

Outline of the dissertation

This manuscript is composed of seven chapters (including the present one). Each major contribution is described in a separate chapter. The manuscript begins by an introduction, followed by a review of the existing work for each one of the three environments introduced beforehand. It ends up with perspectives for the future and a conclusion.

Chapter 2 -Related work

Chapter 2 details the existing work in each of the previously described environments.

It is composed of three main sections: Large display, Multi-display environments and immersive environments. Each section introduces the environments by defining them and describing their major characteristics. The core of each section describes their interaction challenges and the main solutions proposed for to address them. Finally, a summary of the main solutions proposed for each environment is provided at the end of each section.

Chapter 3 -Investigating the effect of splitting the detailed view in an Overview + detail multi-display interface

In chapter 3, we present split-focus, the multi-display overview + detail visualization interface we designed and developed to improve work on multiple regions of the data Chapter 1 -Introduction space simultaneously. First, we lay down the benefits of using multiple-views to tackle the aforementioned problem. Then, we review Baldonado's [START_REF] Baldonado | Guidelines for using multiple views in information visualization[END_REF] eight rules to design and use multiple views in information visualizations. Next, we state the rationale behind the design of the visualization interface and describe each view composing it. Finally, we report on the results of the study conducted to evaluate the visualization interface as well as the optimal number of views to use in such a system.

Chapter 4 -Interacting with multi-display environments (MDE)

In the first part of this chapter, we focus on interaction with multi-display environments in a public context. We introduce the proposed approach: using everyday objects for interaction with these environments. Next, we describe a creativity session, conducted to study the use of predefined everyday objects to perform specific tasks related to MDEs in a public context. We detail the taxonomy we used to classify the proposed interaction techniques and recap the lessons learned from the results.

In the second part of the chapter, we investigate the use of a multi-degrees of freedom mouse (TDome) to interact with multi-display environments in a work context. In a first step, we identify the requirements of interaction in such environments. Next, an overview of the device is provided, it details the degrees of freedom allowed by TDome, it describes a scenario illustrating the potential use of such a device in an MDE. Then, we detail the core elements used in its implementation and follow up by discussing its suitability for interaction with MDEs. In a first study, we explore the usability and confort of performing physical and touch gestures with the device. We experimentally validate their feasibility and identify a set of gestures that can be easily performed and efficiently detected. Finally, we collect user feedback to identify natural mappings between gestures and MDE interactions.

Chapter 5 -Interaction in immersive environments Chapter 5 introduces a new paradigm for interaction with immersive environments: the use of the forearm as a support for tangible interactions to explore complex data.

After introducing the concept, we identify the main interaction requirement for immersive visualizations. We then proceed by describing the multi-dof device used for interaction as well as the forearm as a support. We conclude the first part of this work by providing a design space for tangible interactions supported by the forearm. Next, we report on the results of an experiment aimed at establishing the range, stability and comfort of gestures Chapter 1 -Introduction 27 performed with a multiple degrees of freedom mouse on the forearm. We conclude the chapter by discussing the stability of the device as well as possible mappings between gestures and the most common tasks performed in data visualizations.

Chapter 6 -The neoCampus project Chapter 6 presents the neOCampus project in more details. First, we highlight the objectives of this thesis in relation to the project. Next, we introduce a description space built to identify and organize the relevant characteristics to consider when designing interactive solutions to fulfill those objectives. Finally, we discuss our contributions in relation to the description space and give a concrete example of how each solution can be applied to the neOCampus project.

Chapter 7-Perspective and conclusion

This chapter concludes the manuscript. It summarizes the work presented in this thesis and presents medium-term and long-term perspectives related to the work conducted in this thesis.

Related work 2 Related Work

Introduction

Over the past decades, HCI researchers have developed a wide range of visualization interfaces and interaction techniques to tackle the unceasing challenges posed by the complexification of data. The scope of these solutions ranges from interaction techniques as simple as using a mouse to filter data on a regular 2D screen [START_REF] Hurter | FromDaDy: Spreading Aircraft Trajectories Across Views to Support Iterative Queries[END_REF], to a more advanced tangible interaction in an immersive environment to explore large data sets [162].

This chapter provides an overview of the main approaches designed to support data visualization tasks on the following environments: Screens combined to form a large, highresolution display (Section 2.2); spatially distributed displays (Section 2.3); immersive displays (Section 2.4). As we wanted to focus on interaction techniques that can be used with different types of data (text wall for instant data, graphical for historic, 3D for building application), we did not focus on specific sets of data and preferred instead to focus on interaction techniques that can be used with any set of data.

Interaction in large displays

Before discussing the various solutions proposed for interaction with large displays, it is important to define large displays and discuss their characteristics and the opportunities they offer (sub-section 2.2.1). Then, we introduce the underlying challenges such an environment poses for researchers and the main solutions proposed to address them (subsection 2.2.2). A summary of the main interaction techniques as well as the challenges they address are given at the end of the section (sub-section 2.2.3).

Large displays

As per Andrews's [START_REF] Andrews | Information visualization on large, high-resolution displays: issues, challenges, and opportunities[END_REF] work on information visualization on large, high-resolution displays, the term "display" refers to the combined visual output that serves as a single contiguous space, whether it is composed of a single large display unit or multiple tiled units. One of the most important aspects of a display for designers is the resolution it offers. The resolution of a display (rather than its size) is the criteria that will define the quantity of data it can visually represent. However, acknowledging the importance of the resolution of a display without mentioning its size would be misleading and would offer an incomplete view of the importance of each attribute. Indeed, it is important to take into consideration the size of the display and its resolution as a combination. This is the concept behind the "dot per inch" (DPI) metric, which defines the amount of pixels available per inch. Andrews et al. [4] argue that a higher DPI means that visualizations can be shown with more details, prompting users to get closer to the display to access the details, or move away from it for an overview. While this assessment holds true for low DPI displays, accessing the overview in those displays require the user to move at a greater distance. This greatly influences the approach one would use for interaction: for instance a selection technique based on raycasting rather than tactile input would be more appropriate in this case.

Beyond the advantage of displaying more information, large displays also facilitate the physical exploration of data [START_REF] Bi | Comparing usage of a large high-resolution display to single or dual desktop displays for daily work[END_REF][START_REF] Czerwinski | Women take a wider view[END_REF]148,172,173,179,199]. Andrews et al. [3] explain how the physical environment created by large displays supports the use of a wide range of humans' physical embodied resources. Ball et al. [START_REF] Ball | Move to improve: promoting physical navigation to increase user performance with large displays[END_REF] argue that these human abilities (motor memory, peripheral vision, focal attention, spatial memory…), promoted by physical navigation (walking, head rotation and every motion that changes how we view the information space) enhance the experience of the user, his understanding, as well as his performance when interacting with data visualization.

Large displays change our approach to working with data, since interaction with data is more centered around the user than around the display itself. Andrews et al. [START_REF] Andrews | Space to think: large high-resolution displays for sensemaking[END_REF] argue that the human-centric perspective on large displays introduces new design guidelines for visualizations. In the following, we discuss the main interaction challenges posed by large displays and review the existing solutions proposed to address them.

Several solutions have been designed to support visualization and interaction with large datasets on large displays. These solutions can be classified into two categories [START_REF] Andrews | Information visualization on large, high-resolution displays: issues, challenges, and opportunities[END_REF]:

1) visualization interfaces, which comprise the design of data visualization interfaces to answer the scalability issue inherent to the transition to large high dimensional displays;

and 2) interaction with data visualization, which involves interaction techniques covering some of the main tasks in these visualizations (selection, reaching distant objects, navigation and alternative input devices) [135]. In the following subsections, we will review and discuss some of the more prominent work in each category.

Visualization interfaces

Thanks to their high resolutions, large displays can show a great amount of data, while keeping a great depth in scale [START_REF] Furnas | Space-scale diagrams: Understanding multiscale interfaces[END_REF]. This allows the user to have an overview of the displayed data by moving away from it, and a more detailed view by getting closer to the display. However, the user may need to work on the detailed view of a particular region of the visualization while simultaneously having access to the overview. Three main visualization paradigms have been proposed to satisfy this requirement:

Focus+context, Overview + detail and Zooming (Figure 2.1). These paradigms have been extensively reviewed and their benefits and drawbacks discussed in several works [START_REF] Cockburn | A Review of Overview+Detail, Zooming, and Focus+Context Interfaces[END_REF][START_REF] Furnas | A fisheye follow-up: further reflections on focus + context[END_REF][START_REF] Herman | Graph visualization and navigation in information visualization: A survey[END_REF][START_REF] Hornbaek | Reading of Electronic Documents: The Usability of Linear, Fisheye, and Overview + Detail Interfaces[END_REF]106,114]. prompting the user to interact with the views separately which require him to assimilate the relationship between the views [START_REF] Hornbaek | Navigation patterns and usability of zoomable user interfaces with and without an overview[END_REF].

A focus + context interface does not suffer from this problem as the focused view is integrated directly into the context. An example of such interface is the fisheye view [START_REF] Furnas | Generalized fisheye views[END_REF]161] which uses non-linear scaling to allow the user to see a selected region in full details. Baudisch et al. [13], argued that fisheye views are a good alternative to overview+detail interfaces, as they allow the user to keep adjacent information together, thereby avoiding the need for explicit switching between multiple views, as is the case in overview + detail interfaces. At the same time, they introduce distortion, which makes them unsuitable for tasks where proportions and distances matter. They followed on by proposing a focus + context interface that combines the best of both approaches in their "focus + context screens" interface. It combines a projected view, serving as a contextual view and showing an overview of a scalable area of the visualization, with a highresolution display serving as a focus, displaying a more detailed view of a particular region from the visualization (Figure 2.2). the display for either an overview or a detailed view, Cockburn et al. [START_REF] Cockburn | Comparing speed-dependent automatic zooming with traditional scroll, pan and zoom methods[END_REF] showed that the transitions between the zoomed in view (detail) and the zoomed out view (overview) disoriented the users of such systems.

While they have benefits and drawbacks, these interfaces improve the visual exploration of data. When applied to human scale displays where the display's size and resolution are closely matched to the sphere of perception and influence of the human body [START_REF] Andrews | Information visualization on large, high-resolution displays: issues, challenges, and opportunities[END_REF], all regions of the visualization are accessible to the user. However, in larger displays, when the region of interest is outside of the user's field of view and reach (the top left corner of the display for example), having more details of that region becomes difficult.

Enhanced versions of overview+detail interfaces have been proposed following the democratization of mobile devices. Some of these approaches are based on a combination of large displays with smartphones or tablets in what we call multi-display systems [START_REF] Shakeri | Multi Surface Interactions with Geospatial Data: A Systematic Review[END_REF][START_REF] Cauchard | Visual separation in mobile multi-display environments[END_REF][START_REF] Cheng | Tablet interaction techniques for viewport navigation on large displays[END_REF]149]. These environments usually combine tablets, large displays and tabletops, to extend the overall interaction space. They have been proven to be useful to interact with large contexts such as geographical data [START_REF] Shakeri | Multi Surface Interactions with Geospatial Data: A Systematic Review[END_REF]. Multi-display systems have been used

in overview+detail configurations [13,[START_REF] Cockburn | A Review of Overview+Detail, Zooming, and Focus+Context Interfaces[END_REF]. Rashid et al. [149] found that for searching on large maps, a multi-device approach was better than a simple mobile one as it allows the user to access different regions of the maps relatively easily. Cheng et al. [START_REF] Cheng | Tablet interaction techniques for viewport navigation on large displays[END_REF] showed that, in an overview+detail multi-display technique, moving the position of the detail in a miniaturized view was preferred over other techniques. Overview + detail configurations, combining a large display for the overview and a tablet for the detailed view, allow the user to explore the overview without influencing the detailed view. The detailed view does not clutter the large display as opposed to a traditional implementation of an O+D interface where the detailed view occupies a part of the large display (Figure 2.3).

Chapter 2 -Related work However, while the interaction solutions described above facilitates access to distant regions of the data visualization, they do not allow the user to work on different regions of the data visualization simultaneously without having to constantly switch the detailed view between the regions of interest.

The use of multiple focused views has been proposed to allow working simultaneously on multiple regions of large contexts [START_REF] Butscher | SpaceFold and PhysicLenses: simultaneous multifocus navigation on touch surfaces[END_REF][START_REF] Elmqvist | Melange: space folding for multi-focus interaction[END_REF][START_REF] Javed | Polyzoom: multiscale and multifocus exploration in 2d visual spaces[END_REF]. Polyzoom [START_REF] Javed | Polyzoom: multiscale and multifocus exploration in 2d visual spaces[END_REF] allows multi-scale and multi-focus exploration in 2D visual spaces by offering the user the possibility to create several hierarchies of zoomed views (Figure 2.4).

Interaction with large displays

Interaction with visualizations in large displays have been extensively researched. The proposed solutions are diverse and cover most of the tasks frequently performed in data visualization. In this section we will focus on the main tasks: selection, reach distant objects, navigation and interaction with separate control and widgets [135].

Pointing and reaching distant objects

Several approaches have been proposed to reach and interact with distant objects in large displays, most notably: Raycasting, mid-air gestures and the use of smartphones and wearables.

Raycasting is a popular approach when it comes to pointing in large displays.

Raycasting based pointing techniques were proposed as early as the nineties. Kirstein et al. [109] proposed a simple interaction technique, using a laser pointer detected by a video camera, as a pointing device to move and hide/show the mouse cursor. Chen et al. [START_REF] Chen | Lumipoint: Multi-user laser-based interaction on large tiled displays[END_REF],

adopting a similar approach, used multiple cameras to detect multiple pointers while Ji-Yong et al.

[136] based their system on the blink patterns of the pointers to distinguish them. Bi et al. [28] proposed an approach that supports collaboration.

More recent approaches [119,[START_REF] Shizuki | Laser pointer interaction techniques using peripheral areas of screens[END_REF]103,131,[START_REF] Gallo | Design and comparative evaluation of smoothed pointing: A velocity-oriented remote pointing enhancement technique[END_REF]122] exploited the advances technology made, to offer more compact and efficient solutions. Nancel et al. [131] investigated pointing on large displays from a distance, they explored the limit of existing remote pointing techniques, and they investigated dual-precision techniques combining coarse and precise pointing. Mid-air gestures have been extensively used for interaction with large displays since they do not hinder the movement of the user which is critical in physical exploration and navigation. Walter et al. [189] used mid-air gestures inspired by commercial solutions and enhanced them for better usability to select items on an interactive public display. Bailly et al. [START_REF] Bailly | Comparing free hand menu techniques for distant displays using linear, marking and finger-count menus[END_REF] proposed a mid-air selection technique based on extending a certain number of fingers towards the display to activate a menu command. Vogel et al. [182] proposed two interaction techniques to perform selection tasks (AirTap, ThumbTrigger) and demonstrated that absolute pointing is more efficient when interacting from great distances. Cockburn et al. [51], Nancel et al. [134] and Haque et al. [START_REF] Haque | Myopoint: Pointing and Clicking Using Forearm Mounted Electromyography and Inertial Motion Sensors[END_REF] proposed mid-air pointing techniques where users point at targets using their arms and fingers. Raycasting technique and mid-air interactions are two popular approaches for objects selection. They offer a quick and easy way to point and select objects. However, their accuracy is limited when selecting small or distant objects. They also suffer from occlusion as the raycast and the tracking device detecting mid-air interaction should not be obstructed. Finally, an inherent problem to physical interaction is fatigue, the physical nature of both approaches render them prone to tiredness. While they are not as natural as the Raycasting and mid-air techniques, interaction techniques based on smartphones and tablets offer better accuracy through the virtual interactions they propose.

Navigation and interaction with controls

Over the years, researchers tried to improve navigation either by implementing interactions techniques based on the traditional pan and zoom, or by designing new approaches [132,149,[START_REF] Cheng | Tablet interaction techniques for viewport navigation on large displays[END_REF][START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF]175,[START_REF] Bergé | Design and Evaluation of an "Around the SmartPhone" Technique for 3D Manipulations on Distant Display[END_REF]173]. Rashid et al. [149] showed that controlling large displays using smartphones is the best approach for tasks involving map, text and photo exploration. Many interaction techniques for navigation and content exploration use this modality on large displays.

Cheng et al. [START_REF] Cheng | Tablet interaction techniques for viewport navigation on large displays[END_REF] proposed four interaction techniques to explore an overview (DualTap, DirectTap, TabTilt and TapPoint). Dual tap allowed the user to use multi-touch interactions to change the position and size of a rectangle representing the detailed region.

DirectTap is a version of DualTap offering absolute placement. TabTilt uses the sensors of the tablet to detect tilting and position the region of interest in the overview accordingly. TabPoint uses a WiiMote 2 to point towards the region to select. Cheng reported that DualTap is the preferred interaction technique. Nancel et al. [132] used a handheld device and free space gestures to perform pan and zoom on wall-sized displays.

They identified several factors for the design of similar interaction techniques. They reported on fatigue and lack of efficiency emanating from gestures performed in free space and on the benefits of guidance for input gestures. They also found that linear gestures allowing clutching were more efficient than circular, clutch-free gestures. Sollich et al.

[173] explored the use of spatially aware smartphones to make sense of data changing over time with developmental biologists. Bergé et al. [START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF] proposed a multi-device overview+detail interface facilitating personal 3D exploration in public displays. They designed and evaluated three types of interaction techniques to translate the detailed view: two of them were based on mid-air interactions (with the mobile device, around the mobile device) and the third used the device's touchscreen. They followed this work by evaluating around the smartphones techniques with tactile and tangible techniques for 3D manipulation [START_REF] Bergé | Design and Evaluation of an "Around the SmartPhone" Technique for 3D Manipulations on Distant Display[END_REF]. Their results

show that around the smartphone interaction techniques are better than tactile and at least on par with tangible techniques. Touchscreen [START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF] Chapuis et al. [START_REF] Chapuis | Smarties: an input system for wall display development[END_REF] proposed the customization of mobile interfaces programmatically, to support virtual widgets and gestures and to create and interact with content on a large display (Figure 2.9).

Figure 2.9: Object selection and grouping with three users. The interface running on a phone (left, middle user) and on a tablet (right, right user). Six state button widgets have been added by the application. The" Cursor Inside" action (device on the right) is attached to the active green puck, and acts as a mouse cursor confined inside a window [START_REF] Chapuis | Smarties: an input system for wall display development[END_REF].

Jansen et al. [START_REF] Jansen | Tangible remote controllers for wall-size displays[END_REF] used tangible widgets attached to a tablet to manipulate remote content on a distant display. Their approach support locomotion and allow for rich interaction from a distance. They also showed that a tangible approach allows for more accurate manipulation (Figure 2.10).

Figure 2.10: Two users performing dynamic queries on a scatter plot using tangible remote controllers [START_REF] Jansen | Tangible remote controllers for wall-size displays[END_REF] Sousa's DETI-Interac [175] is a system that allows interaction with public displays using gestures detected by a Microsoft kinect 3 . They argue that their approach allows for more natural interactions than existing approaches.

Baudisch's drag-and-pop [START_REF] Baudisch | Drag-and-Pop and Dragand-Pick: Techniques for Accessing Remote Screen Content on Touch-and Pen-Operated Systems[END_REF] allows interaction with distant objects using proxy objects, a rubber band graphic is kept during the interaction to provide feedback related to the connection between the objets. Arguing that the current text entry methods are not adapted to large display, where users need to move freely, Markussen et al. (2013) explored the use of selection-based text entry methods for text input mid-air. Walter et al. [188] proposed a design space for mid-air gestures to interact with large public displays, they evaluated them and found that dwell was efficient for items selection, confirming Hespanhol et al. [81] findings. In addition to that, they provided recommendations for designers of such interaction solutions.

Summary

In this section, we highlighted the advantages large displays offer for the exploration of large volume of data, namely: their high resolutions, which allow them to display large amount of data in greater details; the physical exploration they support, which affords the use of a wide range of the users physical embodied resources, enhancing his experience, his understanding as well as his performance when interacting with data visualizations.

We identified the main challenges they generate, specifically: the scalability issue related to adapting visualizations previously designed for smaller displays; The reachability issue designers need to take into consideration when designing interaction technique to select and manipulate distant objects, navigate in the large display and interact with multiple regions of interest simultaneously.

We reviewed the existing solutions proposed to address the challenges described above, notably: the three main visualization paradigms proposed to allow the user to work on a specific region of the visualization while keeping a contextual view: Overview + Detail, Focus + context and zooming; Raycasting technique, mid-air interactions and mobile devices based interaction to perform the most common tasks in large displays; the multi-view approach to interact with several regions of interest of large datasets simultaneously.

While several aspects of the solutions proposed above have been researched

exentensively, we identified a missing piece in interaction with several regions of the same dataset simulateously. Although several techniques based their solution on multi-view to address this last challenge, the optimal number of detailed views has not been investigated. The first work of this thesis, Split-focus, aims to address this point by investigating the effects of splitting detailed views in Overview+Detail interfaces.

Interaction in multi-display environments

Multi-display environments (MDEs) combine several displays, usually smartphones, tablets, large displays and tabletops, to extend the overall interaction space. MDEs have been used extensively in multiple contexts: medical field [START_REF] Bardram | [END_REF][START_REF] Bardram | CLINICAL SURFACES-Activity-Based Computing for Distributed Multi-Display Environments in Hospitals[END_REF]169]; meeting rooms [147, [START_REF] Cook | Project Nick: meetings augmentation and analysis[END_REF]176]; traffic management [144]; home automation [START_REF] Jin | A Multi-Display System for Deploying and Controlling Home Automation[END_REF]; exploration of large datasets [START_REF] Michel Beaudouin-Lafon | Multisurface Interaction in the WILD Room[END_REF]120]. The heterogeneity of displays (mobility, orientation, position, resolution, size) composing such environments offers new opportunities for data visualization as well as

Chapter 2 -Related work 43 new challenges when designing interaction solutions. Each one of their inherent characteristics influences the type of data visualized and changes the way users interact with such displays. In the following subsection 2.3.1, we will discuss those characteristics, their benefits and the interaction challenges they entail.

Multi-display environments

MDEs usually contain at last one large display. They offer many benefits in visualizing large quantities of data (as described in the previous section-Large displays). In addition to those benefits, their size and high resolutions make them primary candidates to serve as an overview in MDEs. Having an overview display in an MDE has been proven to improve collaboration. Brudy et al. [38] conducted an empirical study that explores the use of an overview device in a collaborative trip-planning task performed in an MDE.

Figure 2.12: Left: P8 (WO2) points toward overview device, other members shifted their attention to it. In NO groups pointing rarely led to shared attention (right): P29 (INT1) points toward her device; other members' focus stays on own devices [START_REF] Brudy | Investigating the Role of an Overview Device in Multi-Device Collaboration[END_REF].

They found that having an overview display in such environments facilitates decisionmaking and sense making. Wallace et al. [187] investigated the use of personal and shared displays during group work, in an MDE composed of personal workspaces (laptops) and a shared virtual work space (wall projection). They found that using a shared display appeared to support synchronization of the group activity via body language and gaze.

MDEs containing large displays have also been proven to be useful to interact with large contexts such as large datasets [START_REF] Shakeri | Multi Surface Interactions with Geospatial Data: A Systematic Review[END_REF].

Horizontal displays-usually interactive tabletops in MDEs-favour collaboration. Rogers et al. [155] compared tabletops to wall displays in collaborative tasks. While they did not find a difference in terms of display orientation, they found that the side-by-side arrangement of users allowed by tabletops encourages more discussion.

Mobile displays-like smartphones and tablets-trade screen real estate for mobility and privacy. Their small size allows them to be used as private displays to view personal data [START_REF] Döring | Exploring gesture-based interaction techniques in multi-display environments with mobile phones and a multi-touch table[END_REF]. In addition to that, they offer a great range of input capabilities which, combined to their mobile nature, makes them an interesting controller for fixed displays.

Another characteristic that can impact interaction in MDEs is the spatial distribution of displays. As content is distributed among display in MDEs, a badly positioned display might affect the visibility of the content, an unreachable display may require an interaction solution for reaching distant objects. Su et al. [178] showed that display position impact performance and workload and proposed guidelines on how to position displays in an MDE. Fender et al. [63] developed a system that automatically suggests positions and sizes for MDEs' displays, based on user behaviour analysis.

MDEs where displays are aware of the users' position and/or other displays' positions are called spatially aware multi-display environments. Spatial awareness impact how an interaction technique is designed. As an example, Chuckling [START_REF] Nabeel Hassan | Chucking: A OneHanded Document Sharing Technique[END_REF] is a one-handed document sharing technique that lets users physically throw content to displays, in different locations, to share information. It uses a combination of touch interaction to select the content to share and the accelerometer of the mobile device to detect the direction of the toss, and consequently, the display that would receive the shared content.

In this instance, the mobile device is aware of the position of other displays.

In addition to their physical characteristics, the displays composing MDEs have different input capabilities. The touch input offered by smartphones or tablets is not suitable for large displays; pointing interaction techniques used to reach distant objects in large displays are not suitable for the accessible display of an interactive tabletop; the traditional mouse and keyboard used for desktop monitors are not suitable for smartphones, tablets or large displays.

While the heterogeneity of displays composing an MDEs make them a compelling solution for data visualization, they introduce their fair share of challenges in terms of interaction. The different sizes, resolutions of displays and the distributed nature of the visualized data require a suitable interaction technique for content transfer between displays. The different input modalities of each display require a unified input technique that can be redirected from one display to the other. The large displays and tabletops in MDEs introduce a privacy problem in that users should be able to see private information if needed. Large displays showing distant content require that the input modality used allow for interaction with unreachable content. In the following, we present a range of techniques designed to support interaction in MDEs.

Early multi-display interaction techniques

While mouse input is suited for interactions with multiple desktop monitors [START_REF] Benko | Multi-monitor mouse[END_REF],

such a device does not adapt well to multi-display environments (MDEs) where displays may be scattered within the physical space [185].

The need to design alternatives to the mouse and keyboard for such environments is consolidated by the current trend leaning towards the use of portable displays (Laptops, smartphones, tablets). One of the earliest solutions, Pick'n Drop [152], uses a stylus to transfer information from one device to another (Content redirection). The user touches an object with the stylus on a display to pick it up, and perform the same action on another display to drop it (Figure 2.13). and keyboard redirection system where each display in the MDE participates in the interaction either as a source or target of pointers events. Benko et al. [20] proposed a more basic approach in using hotkeys to redirect the cursor of a mouse from one display to another, they demonstrated that their approach was quicker than a mouse in a regular multi-monitor desktop. Nacenta et al. [129] used the stylus for pointing, a particularly difficult task when displays are large or far from the user.

In the early days of multi-display environments, input rediction and content transfer have been identified as the main challenges to address when interacting with multi-display environments. In the following, we will review and discuss the major solutions proposed to address those requirements.

Adapting mobile and wearable devices for MDEs

It is not surprising that mobile displays are an integral and almost unavoidable part of today's MDEs. The benefit of mobility has been highlighted in early multi-display environments when mobile phones and tablets were not as available as today [START_REF] Heath | Reconsidering the virtual workplace: flexible support for collaborative activity[END_REF]117].

The democratization of small mobile displays provides not only output capabilities in

MDEs, but also a large range of input capabilities. Performing physical gestures with mobile devices leverages significantly more DOF than those available with existing devices, such as mice. The main reason is that such devices combine a number of sensors that expand the input/output space (e.g. touch, tilt).

Content transfer is one of the tasks that benefitted the most from the arrival of mobile devices, researchers used them particularly as gesture mechanisms. Döring et al. [START_REF] Döring | Exploring gesture-based interaction techniques in multi-display environments with mobile phones and a multi-touch table[END_REF] proposed a set of usable motion gestures, suitable for being used in multi-display environments with smartphones and interactive tabletops ( Similarly, Dachselt et al. [START_REF] Dachselt | Natural throw and tilt interaction between mobile phones and distant displays[END_REF] proposed throwing and tilting to transfer content to large displays. Adalberto et al. [START_REF] Simeone | A cross-device drag-and-drop technique[END_REF] proposed an interaction techniques based on where the user can transfer data between a fixed display and a mobile device using a two-handed gesture: one hand is used to suitably align the mobile phone with the larger screen; the other hand is used to select and drag an object between the two devices and choose which application should receive the data (Figure 2.16). The camera on the mobile device has also been used to transfer data between MDE displays: Boring et al. [START_REF] Boring | Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays[END_REF] developed Touch Projector, an interaction technique using the After reaching the destination display (d), the item can be positioned precisely by moving the finger (e). When the finger is released, the item has been transferred successfully (f) [START_REF] Boring | Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays[END_REF] Chang et al. [START_REF] Chang | Deep shot: a framework for migrating tasks across devices using mobile phone cameras[END_REF] used the smartphone's camera to identify displays, capture the work state on the display, and transfer it to another display. Other approaches for content transfer include: using smartphones for copy-and-paste operations [163], using smartwatch-centric gestures in cross-device applications [START_REF] Houben | WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications[END_REF] and using the rotation/tilting of smartphones [START_REF] Bhandari | Exploring gestural mode of interaction with mobile phones[END_REF].

In addition to content transfer, mobile devices have been used to explore large datasets on large displays. Bergé et al. [25] used mobile devices for interacting with distant 3D content. Nancel et al. [133] used them for pointing on a large display. Other researchers used them for continuous map navigation [START_REF] Boring | Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays[END_REF]132]. Another common approach is to use mobile devices for multi-display overview+detail tasks [START_REF] Baur | Virtual projection: exploring optical projection as a metaphor for multi-device interaction[END_REF][START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF].

Most of the interaction solutions described above require the mobile device to be held mid-air, which can be tiring [START_REF] David Hincapié-Ramos | Consumed endurance: a metric to quantify arm fatigue of mid-air interactions[END_REF]. It can also affect the precision of the interaction. To overcome these problems, mobile devices can be actuated. Mobile devices have also been used in complete MDE interfaces. Serrano et al. [168] developed Gluey: a user interface based on the combination of a head-worn-display with a camera, which facilitates seamless input transitions and data movement across displays. To enable mobile users to associate personal displays with other displays in an MDE, and facilitate selection and discovery of displays, Gostner et al. [START_REF] Gleizes | neocampus: A demonstrator of connected, innovative, intelligent and sustainable campus[END_REF] proposed two spatial interfaces: the first one is a list ordered by distance, describing the displays in the MDE;

the second one is a miniature map of the MDE. They evaluated the two spatial interfaces in comparison to a simple alphabetical list of displays (baseline) and their results provided clear evidence of users preferring the miniature map of display over the other two options.

Augmenting regular mice for MDEs and multi-dof devices

In the early days of multi-display environments and upon identifying the inadequacy of traditional mice and keyboards for the aforementioned environments, researchers proposed several solutions based on augmenting regular mice. Booth et al. [START_REF] Bimber | Spatial Augmented Reality Merging Real and Virtual Worlds[END_REF] proposed the Mighty Mouse, a remote control technique that allows the user to choose the display he wants to control from a list of all available displays in the MDE. The system then uses the VNC5 protocol to redirect mouse input to the chosen display. Baudisch et al. [START_REF] Baudisch | Mouse ether: accelerating the acquisition of targets across multi-monitor displays[END_REF] developed the mouse ether, an interaction technique that facilitates the movements of the cursor from one display to another in a multi-display environment where the displays have different resolutions and/or orientation. Benko et al. [20] augmented the mouse with hotkeys, allowing the user to redirect it between displays. The Perspective Cursor technique [130] exploit the user's perspective of the room, to map the cursor to the display space that appears the more natural and logical from the user's position. Nacenta et al.

[130] evaluated their interaction technique and found that it performs significantly faster for targeting tasks compared to the traditional techniques. They also showed that Perspective Cursor is effective for systems that require time-efficient interactions, and that it was strongly preferred by users. Kobayashi et al. [111] proposed the 'ninja cursors' technique that replicates the mouse cursor as much as necessary, to improve pointing LensMouse [197] uses a touchscreen coupled with an input device. Other mice have also proposed the use of multi-touch [START_REF] Benko | Design and evaluation of interaction models for multi-touch mice[END_REF].

Summary

In this section, we introduced multi-display environments: a combination of several displays, usually mobile devices, large displays and tabletops, to extend the overall interaction space.

We described the inherent characteristics related to the heterogeneity of the displays composing them: mobility, orientation, position, resolution, size.

We discussed the advantages resulting from those characteristics as well as the interaction requirements that stem from them: input redirection (i.e. redirect input channels to different displays), output redirection (i.e. move content between displays), physical relationship (i.e. possess high-level information on the spatial layout of the displays), reachability (i.e. interact with a distant display) and personal data management (i.e. personal input and output interaction).

We reviewed a range of techniques proposed to fulfill the requirements described above, notably: using tracking solutions to detect the displays of the environments and create a continuous relation between them; adapting the mouse to a multi-display setup to redirect input from display to another; using pen based interaction and adapting mobile devices to facilitate content transfer between displays or reach distant objects.

However, while several interaction techniques have been proposed to improve interaction in MDEs, they usually address one requirement at a time. To our knowledge no device has been specifically implemented to address this full set of requirements. The second contribution of this thesis lies on the design and evaluation of a novel touchenabled device, TDome, designed to facilitate interactions and address a range of tasks in MDEs through its multiple degrees of freedom.

Chapter 2 -Related work 54

Interaction in immersive environments

Immersive technologies went from expensive, heavy VR headsets a few years ago to lightweight and as affordable as sub-50$ VR headsets that can be used with most smartphones today. Extremely performant headsets like the Hololens6 , the Oculus rift7 , MetaVision8 or Moverio9 can be bought commercially for a few thousands of dollars. This opened the door to new research fields like immersive analytics, reflecting the potential of such environments.

One of the main advantages of immersive environments is their spatial capacities that support human cognitive abilities and allow for a spatial comprehension of data [START_REF] Bach | The Hologram in My Hand: How Effective is Interactive Exploration of 3D Visualizations in Immersive Tangible Augmented Reality[END_REF]124].

In immersive environments, the information is spatially displayed around the user supporting physical exploration of data. Their performant tracking systems allow them to offer a natural way of interaction.

Beyond the challenges of interaction in direct relation to the characteristics of immersive systems, challenges of interaction with complex data in immersive environments can be task-dependent. We will focus on the challenges related to the most common data visualization tasks, which include: selection, navigation, filtering and manipulation of objects in 3D environments.

As we will see in the following section, most of the early interaction techniques proposed for these environments were inspired from desktop interfaces. The advent of affordable immersive headset combined to that of smartphones, tablets and wearables in general changed the direction of research to more creative solutions.

The rest of this section will review the interaction solutions proposed for immersive environments according to the modality of interaction involved.

Tactile interactions

In today's world, tactile is the preferred interaction technique for a myriad of tasks However the tasks/actions covered were limited and involved only panning and zooming. technique [START_REF] Kim | Finger Walking in Place (FWIP): A Traveling Technique in Virtual Environments[END_REF] allows its user to navigate a virtual world by sliding his fingers on a multitouch sensitive surface. 10 Head-Worn Displays A miniature version of the 3D data appears to float in the air above the table surface. (A digital rendering is superimposed on the photograph to demonstrate the effect.) A cutting plane through the volume data is projected (like a shadow) onto the table below, where multitouch gestures are used to navigate and interrogate the data. After navigating to a useful view of these imaging data of a heart, the user is now defining a smooth 3D curve (e.g., the shape of a catheter delivery system) relative to the anatomical data set [118].

Manipulating multidimensional data is improved with interaction techniques or devices supporting 6DOF (Translation, rotation and tilting). Researchers tried to augment the number of degrees of freedom offered by tactile interaction through multitouch interactions. Hancock et al. [75] proposed a 5DOF movement with one-touch interactions (2DOF input), up to 6 DOF using two-touch interactions (4DOF input) and a direct mapping of 6 DOF to three-touch interactions (6DOF input). Some of the takeaways from their work include that a higher number of touches allows more natural and flexible interaction and that the users are able to perform separable simultaneous control of rotation, tilting and translation. Jingbo et al.

[100] limit the number of fingers needed for 6DOF manipulations to two by using a learning-based approach. However, these approaches are not natural. Martinet et al. [121] studied the integration and separation of degrees of freedom and found that, separating the control of translation and rotation significantly affects performance for 3D manipulation. Besançon et al. [26] compared tactile interaction to other modalities for 3D data manipulation and demonstrated that tactile interaction is not the most suitable to interact with multi-dimensional data.

Chapter 2 -Related work 57Bergé's [START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF] work confirmed that tactile interactions were neither the most efficient, nor the most preferred. It requires a dedicated surface to perform interaction [START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF] and in an immersive environment, this can divert the attention of the user from the task to perform to the interaction tool, when the dedicated surface is a tablet or a smartphone. It can also constrain his movements in the case of fixed tactile display or an interactive tabletop

[110].

Mid-air interactions

From the Microsoft Kinect 12 to the Optitrack system 13 , a multitude of efficient tracking solutions are available to researchers today. They are largely available, affordable (Microsoft Kinect), and accurate (Optitrack system). The impact of these solutions on the HCI field is palpable. It is even more obvious in environments where the focus is on physical exploration of data. As we saw in (Subsection 2.2, Large displays), mid-air interactions with all the advantages they offer (unconstrained mobility, light and easy to perform) [151,[START_REF] Clarke | Naturalmotion: Exploring gesture controls for visualizing time-evolving graphs[END_REF]174] are noticeably used to interact with immersive environments. The Hololens 14 propose a set of hand gestures to allow users to take action in augmented reality: its two core gestures are Air tap and Bloom 15 . Air Tap is "a tapping gesture with the handheld upright, similar to a mouse click or select. This is used in most HoloLens experiences for the equivalent of a 'click' on a UI element after targeting it with Gaze".

The Bloom gesture is 'the "home" gesture and is reserved for that alone. It is a special system action that is used to go back to the Start Menu. It is equivalent to pressing the

Windows key on a keyboard or the Xbox button on an Xbox controller. The user can use either hand. Microsoft argues that those gestures were designed with simplicity in mind, rather than precision.

The Hololens has an efficient tracking system which allows users to design their own gestures. It has been used in combination with a Microsoft Kinect in Yim's [198] Radkowski et al.

[145] used the Microsoft Kinect to track hand movement and recognize hand gestures. They proposed an augmented reality system to perform assembly of 3D models of technical systems and designed mid-air interaction techniques to select, manipulate, and assemble 3D models of that system. To select an item, users had to move a yellow sphere representing a 3D virtual cursor towards the object of interest by moving their hand, a collision with the object highlights it, and a fist gesture selects it.

Manipulating an object (translation, rotation and scaling) is performed by selecting a function, a coordinates system or a 3D cursor appears to assist the task. The interaction techniques were evaluated and found easy to perform but at the same time, some of them were not understood as the authors intended.

Benko's [START_REF] Benko | Multi-point interactions with immersive omnidirectional visualizations in a dome[END_REF] Pinch-the-Sky dome interface is an interactive immersive experience where users can use mid-air gestures to interact with an augmented dome (Figure 2.27).

The system answers to speech commands with free hand gestures. The gestures include:

hand pinch, two hand circle, one hand clasp, speech recognition and interactions with an Chapter 2 -Related work 59 IR laser pointer. They performed a study in the form of public demonstrations (1000 participants) and reported the following: the proposed mid-air interaction were simple to perform, but understanding how to perform them was not self-evident. The need to give further explanation was highlighted. This is an important drawback of mid-air interactions.

Figure 2.27: Performing a pinching gesture pans the night sky imagery in World Wide Telescope [START_REF] Benko | Multi-point interactions with immersive omnidirectional visualizations in a dome[END_REF] Other well-known problems with mid-air interaction are fatigue [START_REF] Burgess | Selection of largescale 3d point cloud data using gesture recognition[END_REF] and without an appropriate feedback, ambiguity. Moreover, they are not easily discoverable and need to be memorized first, before being used [START_REF] Hinckley | The VideoMouse: a camera-based multi-degree-of-freedom input device[END_REF]126]. Finally, finding the right mid-air interaction for a given task/action is challenging. Even if designers use the most natural real life gestures as the interaction technique for a specific task, it may not be the same from one user to another. One of the most used approaches to design mid-air interaction is elicitation studies [196]. Piumsomboon et al. [142] compiled a set of gestures to guide designers to achieve consistent user-centered gestures in AR. They conducted a guessability study focused on hand gestures, they elicited 800 gestures for 40 selected tasks from 20 participants. They used the results of the study to create a user-defined gesture set for augmented reality interaction. Other work that used elicitation studies to design mid-air interaction for immersive environments include [START_REF] Lee | User Defined Gestures for Augmented Virtual Mirrors: A Guessability Study[END_REF]113].

Tangible interactions

Tangible interaction is a good alternative to tactile and mid-air interactions, as it covers their lack of degrees of freedom (necessary to interact with immersive visualization) [START_REF] Bach | The Hologram in My Hand: How Effective is Interactive Exploration of 3D Visualizations in Immersive Tangible Augmented Reality[END_REF][START_REF] Bergé | Design and Evaluation of an "Around the SmartPhone" Technique for 3D Manipulations on Distant Display[END_REF]200] More recent approaches include: Jackson's [START_REF] Jackson | A Lightweight Tangible 3D Interface for Interactive Visualization of Thin Fiber Structures[END_REF] lightweight tangible 3D interface for interactive visualization of thin fiber structures; Issartel's [START_REF] Issartel | A Portable Interface for Tangible Exploration of Volumetric Data[END_REF] portable interface for tangible exploration of volumetric data;

Besançon et al. [START_REF] Besançon | Mouse, Tactile, and Tangible Input for 3D Manipulation[END_REF] compared tangible interactions to tactile and the mouse for multidimensional data related tasks. They found that tangible interactions perform better than its mouse and tactile counterpart, that tangible's affordance removes the need for a learning phase. However, they point out that the mouse was more precise overall. The use of tangibles in mid-air without support may have played a role in that last result.

While their mid-air usage allows them-like mid-air interactions-to support physical exploration of data by not constraining the movement of the user, a prolonged usage in that condition would incur fatigue. Using the dwell time approach to validate requires the user to keep his head static for a certain amount of time, which makes selecting small target extremely difficult in addition to causing fatigue of the neck muscle. It may also result in a midas touch effect [START_REF] Jacob | What You Look at is What You Get: Eye Movement-Based Interaction Techniques[END_REF] where targets are selected unintentionally. The button approach may cause the Heisenberg effect [START_REF] Bowman | Using pinch gloves (tm) for both natural and abstract interaction techniques in virtual environments[END_REF], where the click on the headset button for validation moves the They investigated the number of necessary DOF to navigate in a virtual environment and found that, the rotate by roll technique, offering 4 DOF provides good performance.

Chapter 2 -Related work 65

They also found that the usage of the roll in smartphones to control the heading in virtual environment was preferred and seems to be the appropriate approach for such a task. Wang et al. [190] presented Object Impersonation, a new HMD metaphor that allows the user to manipulate a virtual object from the outside or the inside, by becoming the object. The metaphor is based on the use of a tablet in combination with an HMD.

Smartphones and wearables are a good option when it comes to interaction with multi-dimensional data, they offer a large number of degrees of freedom as well as the necessary sensor to exploit them, touch capabilities and they can be freely moved in space, which does not hinder the movement of the user. However, this mid-air usage can induce fatigue. Their potential use in immersive environments may be limited in that, their display is not usable in a virtual reality context where the user's sight is obstructed by the HMD. In an augmented reality context, their mid-air usage also incurs fatigue.

The sensors may produce noise which can impact the precision of handheld devices. Hürst et al. [START_REF] Hürst | Mobile 3D Graphics and Virtual Reality Interaction[END_REF] evaluated smartphones and tablets based interaction techniques to interact with virtual reality and highlighted the unreliability of the sensors equipping them.

Chapter 2 -Related work 66

On-body interactions

As opposed to the previous modalities described in this section, on-body interactions have been scarcely used for interaction in immersive environments. Serrano et al. [167] explored the use of hand to face gestures arguing that it is well suited for HDWs (Figure 2.34). They performed a guessability study that showed that participants preferred hand-to-face gestures to interact with the HWD. Their findings include: participants agreed on similar hand-to-face gestures for panning and zooming;

the cheek was the most promising area of the face for zooming and panning due to its large interaction surface and lack of fatigue; hand-to-face gestures were as acceptable socially as the HWD ones. they conducted a study to evaluate their approach. They found that users considered most of the area on the belt appropriate for short interactions, and only the front area, above the trouser pockets as acceptable for long interactions.

Chapter 2 -Related work 67 Wang et al. [191] focused on text input for smart-glasses: they proposed PalmType, an interaction technique that enables users to type with their fingers on the palm.

Other on-body interaction techniques that were designed for other environment and could be integrated into immersive environments include the following:

Skinput [START_REF] Harrison | Skinput: appropriating the body as an input surface[END_REF], which is a technology that allows the skin to be used as an input device, it provides an always available and naturally portable on-body input system. This approach could easily be adapted to immersive environments.

Belly gestures [181], which is an interaction technique using the belly as support for interactions. The authors argue that the belly's large surface which is easily reachable by two hands in any circumstances (standing, walking, running…) is an appropriate surface for interaction.

On-body interaction techniques are an interesting approach that needs to be further explored for immersive environments. They allow eyes-free interactions by exploiting the proprioception 16 capabilities of users. They do not divert the attention of users from the task at hand. However, without being augmented by a complimentary device, these approaches offer a limited set of possible gestures, which impacts the number of tasks that can be performed. Furthermore, on-body interaction do not offer a high degree of precision, which can make them unsuitable for tasks like data visualization in immersive environments.

Summary

In this section, we introduced immersive environments as well as the central benefits they offer for data visualization: Their spatial abilities allowing a spatial comprehension of data; their support of physical exploration which leads to a more natural interaction with data; their performant tracking systems.

We identified the challenges of interaction with such environments, often task-related, and reviewed a range of interaction solutions designed to improve interaction in these environments by modality of interaction: Tactile interactions; mid-air interactions;

tangible interactions; smartphones and wearables based interactions; on-body interaction.

We highlighted the limitations of these solutions: inadequate degrees of freedom for the multidimensional tasks performed in these environments; hindering the physical exploration they allow; visual occultation; lack of accuracy; fatigue.

The last contribution of this work aims to improve interaction with immersive environments through a new paradigm: on-body tangible interaction.

Tangible interactions offer several degrees of freedom. When used mid-air, the tangible object does not hinder the movement of the user when exploring data. When used on an always-available body support, fatigue is minimized and the accuracy of interaction is improved.

Chapter 2 -Related work 69

Conclusion

In this chapter, we introduced large displays, multi-display environments and immersive environments. We identified the challenges in designing solutions to improve interaction with these environments and reviewed the state-of-the-art of existing solutions.

In the following chapters, we report on our efforts to improve interaction in each one of the environments discussed above: first, through split-focus, an overview+detail multidisplay interaction interface addressing the challenge of interaction with multiple regions of interest of the same overview in large displays (Chapter 3); second, through the exploration of everyday objects to design quick and opportunistic interaction techniques for MDEs in a public context and TDome, a multi-degrees of freedom device to fluidify interaction with MDEs in a work context; finally, through the exploration of a new interaction paradigm, on-body tangible interactions, to improve interaction with immersive visualizations.

Interaction with Large Displays

Investigating the effects of splitting detailed views in overview+detail interfaces to interact with large data spaces Chapter 3 -Interaction with Large Displays 723 Interaction with Large Displays

Introduction

The work presented in this chapter was originally inspired by our collaboration with biologists carrying research on cancer. They archive knowledge in graphs called molecular interaction maps (MIM) [112]. These MIM graphs contain several types of nodes (molecules, protein, etc.) and connections. There is no limit to the number of nodes that can be connected by one connection and each connection can also be connected to other connections, e.g. genes playing the role of catalysts of this connection. As research on cancer progresses, results are added to existing MIM maps, which grow extremely largethe Alzheimer MIM map contains 1347 nodes-[128] making them difficult to read and edit using the traditional panning+zooming [START_REF] Furnas | Generalized fisheye views[END_REF] interactions (See Figure 3.1). Moreover, connected nodes can be located far apart from each other, thus, requiring even larger surfaces to visualize the data. The context described above is not specific to the biology field as data lies at the heart of many other scientific fields. A great deal of efforts have been devoted to simplify working with these ever-growing data (cf chapter 2). Overview+Detail (O+D) interfaces are one of these efforts and a well-known approach for data visualization and manipulation [START_REF] Elmqvist | Melange: space folding for multi-focus interaction[END_REF]. They allow the user to have an overview of the data, and a more detailed view that allows him to finely explore the data while keeping the overview.

Chapter 3 -Interaction with Large Displays

73

In this work, we will specifically focus on these visualization interfaces. Despite the advantages the offer when working on large datasets (like graphs), these interfaces reach their limits when it comes to work on multiple regions of the overview simultaneously.

An example from the context of this work would be connecting distant nodes of very large graphs for example. Moving the detailed view repeatedly from one region to another is tedious and interaction complexity increases with the number of regions to work on [START_REF] Furnas | Generalized fisheye views[END_REF][START_REF] Furnas | A fisheye follow-up: further reflections on focus + context[END_REF].

To address this situation, several techniques have been designed in single or multidisplay configurations to support the use of more than one detailed view simultaneously [START_REF] Elmqvist | Melange: space folding for multi-focus interaction[END_REF]13,[START_REF] Butscher | SpaceFold and PhysicLenses: simultaneous multifocus navigation on touch surfaces[END_REF][START_REF] Javed | Polyzoom: multiscale and multifocus exploration in 2d visual spaces[END_REF].

Multi-display systems have been used in an overview+context configuration [START_REF] Balakrishnan | Symmetric bimanual interaction[END_REF][START_REF] Cockburn | A Review of Overview+Detail, Zooming, and Focus+Context Interfaces[END_REF]. Rashid et al. [149] found that for searching on large maps, a multi-device approach was better than a simple mobile one. Cheng et al. [START_REF] Cheng | Tablet interaction techniques for viewport navigation on large displays[END_REF] showed that, in a focus+context multisurface technique, moving the position of the focus in a miniaturized view was preferred over other techniques. In our work we apply this approach to multi-detail interaction.

The use of multiple detailed views has been proposed to allow working simultaneously on multiple regions of large contexts [START_REF] Elmqvist | Melange: space folding for multi-focus interaction[END_REF]13,[START_REF] Javed | Polyzoom: multiscale and multifocus exploration in 2d visual spaces[END_REF]. Polyzoom [START_REF] Javed | Polyzoom: multiscale and multifocus exploration in 2d visual spaces[END_REF] allows multi-scale and multi-focus exploration in 2D visual spaces by offering the user the possibility to create several hierarchies of zoomed views. Melange [START_REF] Elmqvist | Melange: space folding for multi-focus interaction[END_REF] uses a distortion-based technique that offers the possibility to bring together two regions of a large space by folding them.

SpaceFold [START_REF] Butscher | SpaceFold and PhysicLenses: simultaneous multifocus navigation on touch surfaces[END_REF], inspired by Melange, introduces a multi-touch interaction technique to improve the manipulation of the folds.

Several works have focused on the design of a set of rules for working with multiple views [START_REF] Baldonado | Guidelines for using multiple views in information visualization[END_REF]: the "rule of diversity" recommends the use of one view per information type and the "rule of parsimony" suggests using multiple views minimally. However, none of these works has investigated the optimal number of detailed views to use. The optimal number of detailed views that will benefit complex tasks is thus still an open question.

In this chapter, we compare the use of different number of detailed views to interact with very large graphs, such as the aforementioned MIM maps.

Chapter 3 -Interaction with Large Displays 74

Our work aims at answering two questions: 1) are multiple detailed views better than one to interact with large graphs? And 2) what is the optimal number of detailed views needed to perform tasks with multiple graph nodes? Answering these questions is not obvious: using a single detailed view constrains the user to translate the view sequentially to each interesting region of the graph whereas using several detailed views allows parallel access to different locations of the graphs but limit the size of each detailed view to avoid the need for a larger screen real estate to display them.

To answer these questions, we implemented an interface based on the O+D scheme.

Our interface supports the simultaneous use of up to 4 detailed views independent from each other. The overview (the overall graph) is displayed on a large screen while the detailed views are displayed on a single tablet: we hereafter refer to them as the split views. Deploying O+D interfaces on multiple displays has been shown to improve data visualization and manipulation [START_REF] Cheng | Tablet interaction techniques for viewport navigation on large displays[END_REF]149].

We experimentally compared three values for the number of split views (1, 2 or 4) in a node connection task, where the user is asked to create a link between 2, 3 or 4 nodes.

These types of multi-node links are usual in large graphs such as MIMs [112].

Using an overview + multi-detail interface to interact with large surfaces

To contribute to the previously identified challenges of overview + detail and better understand the potential advantages of using multiple detailed views, we first focused on the design of such a solution.

Rules for multiple views in information visualization

Baldonado et al. [START_REF] Baldonado | Guidelines for using multiple views in information visualization[END_REF] defined several rules for multiple views interfaces. These rules are categorized in two groups:

-Rules to help designers and users assess the suitability of multiple view systems calls for designers to examine the user's learning costs and the computational and display costs of additional views by applying the 3 rules described above. Indeed, in addition to the cost of context switching, the use of multiple views introduce system complexity.

Designers must take the cost of such a system in consideration when deciding if a multiple-view system is adequate for their application.

The second category of Baldonado's [START_REF] Baldonado | Guidelines for using multiple views in information visualization[END_REF] guidelines concerns the use of multiple views.

The Space/Time resource optimization rule (R5) indicates that the display space as well as the computational time to present multiple views side-by-side are two important aspects of designing such systems. Thus, they encourage designers to balance the spatial and temporal costs of presenting multiple views with the spatial and temporal benefits of using the views. The Self-Evidence rule (R6) focuses on the use of adequate feedback. It recommends the use of perceptual cues to make relationships between multiple view more visible to the user. The Consistency rule (R7) indicates that in addition to feedback, consistency in designing the interface of multiple views helps the user learn to use the system more quickly. Through the last rule, Attention management (R8), the authors point out that having multiple views requires the system to direct the user's attention to Chapter 3 -Interaction with Large Displays 76 the right view at the right time, this would prevent the user from continuously monitoring the system for events that demand his attention.

Table 3.1 below, presents a summary of these rules as described in [START_REF] Baldonado | Guidelines for using multiple views in information visualization[END_REF]:

ID

Interface Design

Based on these recommendations, we designed and implemented an O+D visualization interface that consists of a large screen to display the contextual information and a tablet to show a magnified version of selected region(s) of the large space in addition to additional information about the selected region. We describe the three main views of Chapter 3 -Interaction with Large Displays 77 our interface (overview, split views and translation view) as well as a set of interaction techniques that allow the user to move the split-views.

Split views

Our technique allows the user to have up to four independent split views at the same time (Figure 3.2), offering a detailed view on a graph region. We implemented three configurations for the multiple views on the tablet: 1-view, 2-views and 4-views. Using split views allows to decompose (R3) the complex graph rendering. A swipe gesture inside one of the split views moves the underlying graph in the same direction: this behavior is consistent (R7) with regular map interactions on mobile devices.

Finally, when the user selects a node in one of the split views, appropriate feedback is provided so that users' attention (R8) is focused on the appropriate view.

Overview

The overview displays the entire graph on a large display. The ratio between the overview size and the split views size is 9 for the 1-view configuration (overview is 9 times bigger), 18 for 2-views and 36 for 4-views. These ratios were chosen to explore the effect of a zoom factor bigger than 30 (threshold identified in [165]). A contour color is applied Chapter 3 -Interaction with Large Displays 78 to the split views on the tablet and to its representation on the overview to help the user establish the relationship between the points of view (R6) (Figure 3.3).

Translation view

We call a translation technique the interaction allowing the user to explore his data by moving the detailed views to the region of interest. In our interface, positioning the split views can be achieved using two translation techniques: 1) a regular pan on the splitviews; 2) A translation interface called translation view. The translation view is activated when the user presses the black button "switch" displayed on the tablet (Figure 3.3). The translation view provides a representation of the position of the 1, 2 or 4 split views on the overview. In the translation view, each split view position is represented using a view icon. Given the density of the graphs, displaying a miniature of it on the tablet would be useless. Therefore, the view icons are displayed on a void background.

By looking at the overview, the user can use multiple (R1) view icons in complementarity (R2) for selecting multiple nodes. The user can adjust the position of one or several view icons simultaneously by direct touch manipulation as recommended in [START_REF] Cheng | Tablet interaction techniques for viewport navigation on large displays[END_REF]. Using two hands and the multi-touch screen, the user can theoretically translate 4 view icons at the same time. Closing the translation view restores the split views. In our configuration, no zoom is allowed: this ensures a higher consistency over the split views (R7).

User Study

Using our multi-view interface, we conducted a controlled experiment to evaluate the effect of using multiple split views (1, 2 or 4) when connecting various number of nodes (2, 3 or 4) situated on different areas of large graphs.

Task

Participants were asked to create a connection between 2, 3 or 4 nodes. The overview displayed only the nodes to connect on a white background. To connect several nodes, participants had to select them by touching each node in the split views displayed on the tablet. Selecting one node required translating one of the split views displayed on the tablet so that the node becomes visible. On each trial, participants could translate each of the split views with swipe gestures directly in the split view or through the validated with a single tap on the node, which was then highlighted in blue. Before each task, the position of the split-views were reset to a default position.

Node positions

To define the position of the 2, 3 and 4 nodes to connect, we decided to fix their distance from the center of the overview and change their relative distance as well as their distribution. We used eight absolute positions corresponding to the intersection of an ellipse positioned at the center of the overview with horizontal, vertical and diagonal axes (Figure 3.5). The ellipse shape is used so that the positions of the nodes are spread across the width and height of the tablet. We selected 10 combinations of these positions for each number of nodes, equilibrating the number of neighbor nodes (i.e. on consecutive positions) and the cases where all nodes were far from each other with the cases where nodes were close to each other.

Apparatus

The experimental apparatus consisted of a multi-device setting involving one PC and (2560x1600px). Nodes on the split views (i.e. the targets to touch) on the tablet measured 40x157px. On the translation view, each view icon measured 826x526px for 1-view configuration, 413x526px for 2-view configuration and 413x263px for 4-view configuration. A Dlink DIR-615 18 router was used to establish a wireless connection between the workstation and the tablet. We placed the tablet on a desk and allowed users to interact with both hands, a usual configuration in multi-display settings to avoid fatigue during long interactions and to benefit from multi-touch input [154]. The tablet rested on its cover at a 60° angle and in the same field of view than the large display, which has been shown to be paramount in multi-display environments [START_REF] Cauchard | Visual separation in mobile multi-display environments[END_REF]. Participants sat at 1m from the display and we ensured that there were no light reflections on the tablet.

Experimental Design

The experiment followed a 3x3 within-subject design with number of split views

Procedure and instructions

To begin a trial, the participant pressed a "start" button displayed in the center of the tablet. Between each block, the user was informed via an information screen that he was about to start another condition. Participants were asked to finish each trial as quickly as possible using any number of hands or fingers. They were told they could take

Collected Data

We logged all touch events from the screen tablet. We measured trial completion time from stimulus onset to screen release, the number of actions to complete each trial and the number of switches between overview and split views on the tablet. We also logged the number of view icons translated simultaneously, i.e. the number of fingers performing a view icon translation at the same time.

Results

We used a Shapiro-Wilk test to determine the normality of collected data. Our data could not be normalized, so we used a non-parametric Friedman test to compare more than 2 conditions and Wilcoxon tests otherwise. When needed we used the Bonferroni correction. There is no significant difference between using 2V and 4V when connecting 2 nodes, but using 4V, participants required 15% less time than with 2V when connecting more than two nodes (3 nodes: Z= -3.06, p <.01, 4 nodes: Z=-3.06, p <.01). icons seemed affordable for most participants. We observed that most of these bi-touch gestures were done with one finger of each hand in a bimanual coordinated gesture.

Completion time

Switches between Translation and Detailed view

SUS Scores and User preference

SUS scores reveal that the 1V and 4V conditions were deemed good (75 and 80 respectively) while the 2V was deemed excellent [START_REF] Hornbaek | Navigation patterns and usability of zoomable user interfaces with and without an overview[END_REF]. Interestingly, when asked, users preferred the 4V condition for the tasks where they had to work on more than two nodes while opinions were mixed for the task with two nodes only: some participants liked having four views at hand, others disliked having smaller views than under the 2V condition.

3.5 Discussion and Perspectives

Possible ameliorations of the split-view interface

Below, we discuss two possible improvement of the split-view interface. The first concerns the translations techniques and the second the coherence of the split-views.

Translation technique

A possible improvement for the translation technique resides in the speed of translations. When interacting with a large information space, it is important that the position of the detailed view can be adjusted quickly and precisely. The translation view is quick and it can be precise if the information space is not too large. However, with a large information space combined to the small display on tablets, moving detailed view with precision becomes difficult. One possible solution to that would be to use the regular pan, directly on the detailed view while allowing the user to control the speed of translations. We can exploit the number of fingers used to perform the translation to achieve that. The result of a one finger translation would be a regular translation, the result of a two finger translation would be a translation twice the speed of the regular one, and so on. One of the problems resulting from a multi-view approach is the coherence of the detailed-views positions. As the user are able to translate the views freely on the overview, a situation where the disposition of the detailed-views on the tablet may not be coherent with their icons on the overview can arise as in (Figure 3.9-right).

We developed an improved version of the split-views interface that would prevent an The same principle is applied to the three other detailed views.

The user has the possibility to release the locks between 2 subregions allowing the split views to be translated in the newly created subregion (opening the lock between the green and magenta detailed views in (Figure 3.10, B). Openning the 4 locks would make all the overview available to all the split views.

This approach is similarly applicable to the 2-views version of the split-view interface.

Chapter 3 -Interaction with Large Displays 87

Perspectives

While previous work on symmetric bimanual interaction (where each hand is assigned an identical role) has already highlighted its benefit in some settings [START_REF] Balakrishnan | Symmetric bimanual interaction[END_REF]127], we are only aware of one work [START_REF] Geyer | Harnessing the benefits of bimanual and multi-finger input for supporting grouping tasks on interactive tabletops[END_REF] exploring symmetric bimanual multitouch interaction (each finger performs a pointing gesture on a different target). In this previous work, up to 47% of the trials for some tasks were performed using multiple fingers in a bimanual setting. In contrast, our results indicate that symmetric bimanual multi-touch input is hard to perform. We believe these results are highly dependent on the task. Therefore, there is a Chapter 3 -Interaction with Large Displays 88 need to further explore the factors influencing symmetric bimanual multi-touch interaction.

Given our findings, a perspective to our work could concern three design questions.

First, It would be interesting to explore the limits of the number of views. In this work, to respect the guidelines regarding the zoom ratio between a detailed view and an overview, we limited the number of detailed views to 4. Generalizing the results to configurations of more than 4 detailed views is not feasible without altering the design of the interface, whether it relates to the size of the screen displaying the detailed views or a mechanism to display 4 detailed views at a time and switch between them. In both cases, it is necessary to conduct a further experiment to evaluate the new design . Second, it would be interesting to explore how to improve bimanual multitouch interactions to facilitate the translation of several split views at the same time. One idea could be to study combinations of fingers that can be moved synchronously and to help the user in employing these fingers. Third, as most participants used only one finger of each hand, it would be interesting to consider other potential uses of the remaining fingers: for example additional fingers might act as modifiers to bring split views together, or to move views to specific positions such as corners, or to dynamically release locks.

Conclusion

In this work, we studied the effects of splitting the detailed view in an overview+detail interface to work on large graphs. We implemented an O+D multi-displays interface where the overview is displayed on a large screen while 1, 2 or 4 split views are displayed on a tactile tablet. We experimentally evaluated the effect of the number of split views according to the number of nodes to connect. We evaluated three multi-view configurations: one detailed view (1V), two split views (2V) and four split views (4V).

Overall, results show that using two or more split views is significantly faster than using only one detailed view. Results reveal that using 4 split views is only better than 2 split views for working on more than 2 regions of the graph.

An interesting finding of our experiment is that, when using 4 split views, users did not take full benefit of bimanual multitouch interaction to translate several view icons at This type of environments have shown significant value for interacting with heterogeneous data sources and in multiple contexts such as 3D exploration [START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF],

collaborative scenarios [START_REF] Bortolaso | OrMiS: a tabletop interface for simulation-based training[END_REF], crisis management [START_REF] Chapuis | Smarties: an input system for wall display development[END_REF] and scientific data visualization [168].

They offer numerous advantages for organizing information across displays, for enhancing individual and group work, for providing support to peripheral information and for extending the interaction space.

Interaction with each one of these individual displays has been substantially explored: tactile interactions for smartphones, mouse and keyboard for a PC, touchpads for laptops to cite the most common interaction techniques. But these interaction techniques are not as efficient when used in a multi-display context in which the various displays do not necessarily share the same characteristics (display size, form, orientation) and input capabilities: for instance, tactile interactions are not appropriate for distant displays, and the mouse is not efficient for covering large display areas. For these reasons, designing interaction techniques for MDEs is a complex task.

Chapter 4 -Interaction with Multi-Display Environments 93

In addition, the context in which the MDE is deployed is also of importance for the design process. MDEs deployed in a public context require interaction techniques that respect the personal space of the user, are easy to learn and well integrated in the environment. In a work environment, the user needs an efficient interaction technique, allowing him to switch between displays and redirect content without interrupting his task flow.

In section 4.2, we present an approach based on the use of everyday objects as an interaction medium in MDEs in a public context. We detail the results of a creativity study conducted to generate ideas on how to use physical objects to interact with public MDEs and/or the content displayed on them. In section 4.3, we propose a novel device, TDome, partially based on the results of the creativity study (section 4.2), for interaction with MDEs in work contexts. We present the characteristics of the device and its suitability to MDEs. Then, we focus on the usability and comfort of the device through a set of studies. Finally, we explore the mappings between the feasible gestures with the device and the main MDEs tasks.

Interaction with MDEs in a public context

MDEs in public environments (such as train stations or airports) offer little or no means of interaction with the displayed information [104,137]. One reason is the absence of adequate interaction techniques. We argue that this is in part due to the difficulty of designing such interaction techniques. A difficulty that stems from the necessity of fulfilling several interaction requirements specific to the these environments.

Substantial amount of research focused on the use of smartphones and smartwatches as interaction tools [START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF][START_REF] Boring | Touch projector: mobile interaction through video[END_REF]133,164]. This approach fail to respect the personal space of the user through the installation of third-party applications, necessary for the interaction with the MDE, on his personal device. Other approaches use tactile gestures [188,139]: while this is one of the most widespread and accessible interaction techniques for public MDEs, it limits the interaction space to the accessible parts of the displays. Other researches proposed the use of gestural interaction techniques [189]. However, this approach is ambiguous and does not offer enough visibility of the possible gestures without a learning phase, one that the passing-by users of these MDEs do not necessarily have. Tangible interactions are a good alternative to the aforementioned approaches.

Chapter 4 -Interaction with Multi-Display Environments 94 They rely on natural gestures. These gestures are often suggested by the physical characteristics of the objects used [171]. However, this approach is dependent on the availability of objects around the MDE. We propose to overcome this limitation through the use of everyday objects, which can be found on or around the user. This approach offer a quick, natural and opportunistic way of interacting with public MDEs. In this section, we explore the use of everyday objects to perform common tasks in a public MDEs. We present a study that was conducted in the form of a creative session and which aim was to identify the way to use objects of different shapes and materials to perform common tasks in MDEs. We amend an existing taxonomy to classify the proposed interaction techniques. Finally, we discuss the results and the main lessons learned from this creativity session.

The Creativity Session

We conducted a creativity session focusing on the use of everyday objects to interact with public MDEs. During this session, we asked participants to come up with ideas on how to use predefined objects to interact with public MDEs and/or the content displayed on them. In this section, we present the list of objects used, the proposed tasks and a detailed description of the creativity session.

Objects used during the session

We based our objects list on the work done previously by Pohl et al. [143], in which the authors identify the most common objects around us through a participative production service. In their work, they extracted the most common objects around the users' smartphones and classified them according to their forms in 5 categories: spherical, semi-spherical, cylindrical, rectangular and complex. We have adopted this categorization with some modifications (tFigure 4.1): we combined the spherical and semi-spherical categories due to their similar physical characteristics; we redefined the complex category as a composite category so it includes not only the object which form is not Spherical, Cylindrical or Rectangular but also objects made up of at least two objects belonging to the other 3 categories. As we suspected that the material of the object may play a role in how the object is used, we proposed for each category two objects made of different materials: Soft, Rigid. The multi-display environment used for this creativity session consisted of an interactive table, two video projectors and a monitor (Figure 4.2). The interactive table was put in the center of the system with two video projectors at its sides. The monitor was placed between the two projections ( In this setup;

 The interactive tabletop displayed a map of the campus (Figure 4.2, Thumbnail 1). The user could interact with the map using pans and zooms to choose the building (s) he wanted to simulate the energy consumption for.

 The chosen building was then displayed on one of the "floor" displays (numbered 2 in Figure 2, Thumbnail MDE). These two displays allowed the user to choose a floor of the building and visualize its energy consumption (

Procedure

The objective of the creativity session was to find interaction techniques for the 7 predefined tasks described in (4.2.1.5) and for each object of the list described in (4.2.1.1) (tFigure 4.1). To limit the session to a reasonable duration and avoid fatigue, we separated the 8 participants into 4 groups of 2. For each task, each group was responsible for a category of objects (including a soft and a rigid version) and had to propose interaction techniques for the objects belonging to it. The objects were randomly redistributed to the 4 groups at the beginning of each task. We made sure that each group, at the end of the study, had used each category of objects at least once. The creativity session was conducted as follows:

 2 minutes of introduction: each task began with a 2 minutes introduction where the task to perform was detailed to the participants and the objects were distributed to each group. An example of interaction with each object for the current task was given during the distribution process to stimulate the participants' creativity.

 5 minutes of thinking and discussion: each group had to think of ways to use the provided objects to perform the task at hand. The interaction techniques proposed had to be noted on post-it notes and classified from the most promising to the least promising.

 2 minutes of presentation: each group had to present their ideas, from the most promising to the least promising. At the end of the restitution of each task, all the participants had to vote to choose their preferred idea and object for the task at hand.

Collected data

During the restitution part of the session, we noted the ideas proposed by the participants. At the end of each task, participants were asked to provide the Post-it notes on which they wrote their ideas. In addition to the written notes, we also filmed and recorded the creativity session so that we could go back to the videos to annotate the exact gestures made by the participants.

Classification of the ideas -Taxonomy-

Participants produced a large variety of gestures with the objects at their disposal during the creativity session. We analyzed and classified these gestures using a taxonomy we inferred from the results of the session. The taxonomy is composed of 3 dimensions that we considered relevant to our study and that we describe below: Nature of the gesture, Gesture basis, Human effectors. We manually processed and analyzed the proposed ideas according to this taxonomy. The three dimensions are detailed below.

Nature of the gesture

The nature of the gesture dimension contains 3 values: gestures made With the object, On the object or Around the object. The idea behind this dimension is to determine if the interaction technique proposed is a tangible interaction (With the object), a tactile interaction (On the object) or mid-air interaction relative to the object (Around the object). The user had necessarily the object in his hand for the With the object gestures but not systematically with On and Around the object gestures.

Gesture basis

The gesture basis dimension is composed of 5 values: Form, Material, Analogy, Function and Other. It explicits the origin of the gesture.

 A gesture is based on the Form of an object if it uses the physical shape of the object. An example would be to roll a ball: the gesture "roll a ball" is based on the round shape of the ball.

 A gesture is based on the Material of the object if it uses the physical characteristics of the object's material. For example, the gesture "fold a sheet of Chapter 4 -Interaction with Multi-Display Environments 100 paper" is based on the foldable property of the material. If the sheet of paper was rigid, performing the gesture would be impossible.

 A gesture is performed by Analogy when there is a functional coherence with a common gesture performed on another device: the badge and the smartphone having approximately the same shape, making tactile gestures on a badge, for example, is an analogy to making tactile gestures on a smartphone.

 The Function criteria represents the reuse of classical gestures made with the object for other purposes. An example would be to write with the pen the number of the screen to select.

 Each gesture that can not be classified in one of the previous categories is labeled as an Other gesture.

Human effectors

We hypothesized that gestures requiring the use of one hand would be easier to perform especially in a context where the users of the multi-display environment are going to be passers-by whose hands are not necessarily free. Therefore, it was necessary to be able to classify the interaction techniques according to two additional criterias: Unimanual or Bi-manual. For Bi-Manuel interactions, we distinguished between two-handed interaction techniques in which the hands are used one after the other (sequentially) and where they are used at the same time (parallel).

Summary

The table below summarizes the three dimensions.

Results

During a creativity session that lasted 2 hours 40 minutes, the participants produced 194 ideas for the 7 tasks with the 8 objects at their disposal. The number of ideas provided per category was balanced (Spherical-Semi-spherical: 27.32%, Rectangular: 24.74%, Cylindrical: 25.77%, Composite: 22.16%) as well as the number of ideas per object which ranged from 10.31% for the Post-it note object to 13.92% for the anti-stress ball. However, the ideas proposed per tasks varied from 10.82% for the task of selecting multiple screens to 18.04% for the validation task. Below, these results are presented with more details according to the following axis:

-The objects (4x2) described in 4.

Gesture basis

When looking at the results regarding the gesture basis (Figure 4.4), the participants were mostly inspired by the form of the object (39.2%) and the materials (28.4%). A large part of the interaction techniques proposed for rigid objects is based on form (54.1%). This is especially true for gestures made with the objects: Bowl (53.9%), Bottle (70.9%) Chapter 4 -Interaction with Multi-Display Environments 103 and Pen (65.2%). Unsurprisingly, the interaction techniques proposed for soft objects use its material as a main base for the interaction (56.3%): Cable (84.6%), Post-it notes (55.0%) and Paper (52.2%). There are two exceptions to the trend described previously:

1) The majority of the ideas proposed for the stress ball-soft spherical object-are based on its shape (48.2%); 2) the majority of the ideas proposed for the badge-rigid rectangular object-are based on analogies (use of a smartphone, a remote control, …).

Human effectors

The vast majority of the proposed interactions techniques required one hand to be performed (Uni-Manual) regardless of the task and the type of material of the object (Figure 4.5): Badge (88%), Stress Ball (85%), Bowl (96%), Bottle (92%), Paper (65%), Post-it (75%), Pen (96%). However, for the cable object, most of the gestures proposed were Bi-Manual (77%).

User preference

Each group of participants was asked to choose the most adequate interaction techniques for the current task from the set of ideas they proposed. At the end of each task, the participants voted for their preferred idea (all objects included). We report on this in the following table: The gestures proposed for the stress ball were essentially physical manipulations (throw the ball: 30%, tilt / rotate the ball: 30%, squeeze the ball: 19%). These gestures were preceded or followed by gestures of initiation or validation of the interaction.

Example: to make a pan on the map, one of the participants proposed to squeeze the ball to set the speed of the pan (initiation gesture) and then to tilt the ball to activate the pan towards the direction of inclination.

Bowl:

The most frequent interaction performed with the bowl was tactile (46%). The place of the gesture varied between the bowl's edge and its bottom (after turning the bowl upside down). Example: for the content transfer task, one of the proposed ideas was to turn the bowl over. Make a swipe gesture on the bottom of the bowl in the direction of the destination display.

Bottle:

Inclinations and rotations (54%) were the most often suggested gestures to perform the different tasks of the study with this object. Example: to make a pan, one of the ideas proposed was to use the bottle as a joystick and performing the pan by tilting it.

Cable:

The soft nature of the material composing the cable was extensively used in the proposed interaction techniques using this object (50%: join the two edges 38%, roll up 12%). The cable was also used to perform point gestures (23%). Example: for the display selection task, one of the participants proposed to join both ends of the cable to make a viewfinder which he used to aim at the display to select.

Badge:

Participants favored doing tactile gestures on the badge (44%). Example: in a content transfer task, one of the participants proposed to make a swipe gesture on the surface of the badge towards the destination display.

Sheet of paper:

As was the case with the cable object, The soft nature of the sheet of paper was the most exploited characteristic of the object. The most frequent interaction technique Chapter 4 -Interaction with Multi-Display Environments 108 proposed using it was: folding the sheet of paper (26%). Example: in order to complete the validation task, one of the proposed ideas was to fold the paper in half.

Pen:

Pointing: 30%, Tilting / Rotating: 22%, Touch on the pen: 17% were the most frequent interaction techniques proposed for this object. Example: To select one of the controls to interact with, a participant suggested pointing towards the control with the pen.

Post-it note:

There was no particular gesture made with the Post-it note that stood out. The trend of exploiting the soft material of the object of interaction continued (40%-Fold: 10%, Crumple up: 10%, Scroll: 10%, Roll up: 5%, Cut: 5%). Example: to select a display, one of the proposed ideas was to remove a sheet of the Post-it notes, crumple it up and shoot it towards the display to select.

Discussion

Limitations

The study was carried out in a single configuration of fixed displays (3.2.2), we think that it would be interesting to evaluate the most promising ideas proposed during the creativity study in differents display configurations. Moreover, it would be interesting to explore a more extensive list of tasks, such as 3D manipulation or text input. Indeed, while the tasks identified for the study were the most common in an MDE, they do not cover all the possible tasks.

Lessons learned from the creativity session

Even though the interaction techniques proposed during the creativity session still need to be implemented and evaluated experimentally to validate them, we think that the results of our study represent a trail of natural and intuitive solutions. We suggest the following guidelines to design interaction techniques for tangible objects to interact with MDEs according to their shape and/or material:

Chapter 4 -Interaction with Multi-Display Environments 109  Interaction techniques With the object are preferred for spherical / semi-spherical, cylindrical, composite and rectangular categories if the object is made of a soft/deformable material.

 For rigid rectangular objects, whose shape resembles that of a common device (smartphone, remote control, music player, etc.), we suggest interaction techniques On the object and specifically, by analogy to those that already exist for the device resembling the object.

 When the object is made of soft / deformable material, we suggest focusing on techniques based on the material of the object. Conversely, if the object is rigid, we advise to favor its shape.

 Regarding spherical objects, we recommend the design of interaction techniques based on their shape, whether the object is deformable or rigid.

Summary

Most of the gestures (80%) proposed by the participants were tangible gestures (With the object). We believe that this is due to the fact that gestures Around the object are not yet common in real life, thus, limiting the ideas proposed by the participants to With the object and On the object interactions. On the object were proposed mainly for the badge object (52% of the interaction for this object). The vast majority of the On the object interaction proposed for the badge were made by analogy. We concluded that the shape of the badge and its size-that resemble those of a smartphone or a remote control-encourages users to make gestures similar to those usually made on devices of the same shape and size (smartphones, music players, remote controls ….).

Participants preferred overall using the material of the object when it was a soft/flexible one instead of the shape of the object. The only exception to that is the stress ball for which, despite its soft nature, the interaction proposed were mainly inspired by its form. We believe that the rounded shape of the ball is more compelling to the users than its material in this case. The intrinsic degrees of liberty it offers allow for different ways of using it in a tangible interaction. Based on those findings, we decided to explore the rounded shape of the object in the work described in the second part of this work

Chapter 4 -Interaction with Multi-Display Environments 110 which consists in the design, usage and evaluation of a touch-enabled 6DOF interactive device for multi-display environments.

Interaction with MDEs in a professional environment

As opposed to MDEs in public contexts, which suffer from a lack of adequate interaction techniques due to their specific requirements, several interaction techniques have been proposed for MDEs in work environments.

Researchers have mainly proposed adapting existing devices to tackle individual MDE tasks, such as the mouse for multi-monitor pointing [START_REF] Benko | Multi-monitor mouse[END_REF], or smartphones for cross-display data-transfer or distant pointing [START_REF] Boring | Touch projector: mobile interaction through video[END_REF]133]. However such adaptations can result in undesirable side effects: mice are not appropriate when the user is standing [133] and smartphones held in mid-air can be tiring and cumbersome for long interactions [START_REF] Hinckley | The VideoMouse: a camera-based multi-degree-of-freedom input device[END_REF].

Recent research has demonstrated the use of wearable devices to perform cross-device interactions [START_REF] Houben | WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications[END_REF]168]. However, current wearables lack proper input mechanisms and mainly serve private purposes. If MDEs are to become the office of the future, as envisioned by many [150,153], can we design a device specifically tuned for such an environment? Adopting a unique device would indeed avoid the homing effect when switching from one device to another, enhance privacy in such environments through personal data control and visualization, lead to a coherent set of interactions with the varied MDE applications, and ultimately contribute to a more fluid task flow, a key element in MDEs [START_REF] Benjamin | Interfaces for staying in the flow[END_REF].

To this end, we designed a novel touch-enabled device, TDome, to facilitate interactions and address a range of tasks in MDEs [START_REF] Boring | Interacting in Multi-Display Environments[END_REF]168]. TDome is the combination of a touchscreen, a dome-like Mouse [140] providing 6 DOF, and a camera that can sense the environment. TDome thus inherits properties of other existing mouse-like devices but includes many novel features to tackle the needs of common MDE tasks [START_REF] Boring | Interacting in Multi-Display Environments[END_REF]168]: TDome identifies the spatial layout of displays; facilitates distant interaction and data transfer across displays; and enables personal interactions by using the touchscreen as a private output medium. To do this, we designed and implemented different techniques employing two versions of TDome (small and large touchscreen) to address these MDE tasks.

Chapter 4 -Interaction with Multi-Display Environments 111

In this section, we address two major challenges for applying TDome in MDEs: first, the device's usability, which demands the user to coordinate a physical manipulation with a touch gesture (we refer to as combined gestures-see Figure 4.7-c); second, the mapping between TDome gestures and MDE tasks. To validate TDome's usability and suitability for MDEs, we conducted three user studies. We first carried out a formative study to discard gestures deemed too uncomfortable. We followed this with a controlled system validation in which we identified the success rate and performance of combined gestures.

Finally, using the resulting set of gestures, we collected user feedback on the best mappings from TDome gestures to common MDE tasks.

TDome overview

TDome is a touch-enabled multi-DOF input device that embodies features and a form factor that facilitate MDE interactive tasks. This unique device results from the composition of a touchscreen with a dome-like mouse RPM [140], providing rotation, Roll, Translation and Lift-Up motions (6 DOF). The device also includes a camera that can sense the environment. As a result, TDome support the control of multiple commands, which is required in MDE to control the applications and their content but also managing the MDE. We present an illustrative usage scenario with TDome prior to presenting its features. Harry is an engineer working on a smart campus project that monitors data collected by multiple sensors on the university. To visualize and interact with the large datasets of energy consumption, the university has set up a multi-display environment composed of several displays, a large projection wall and two TDome devices.

As Harry enters the room to start his daily supervision of the energy data, he grabs one TDome and uses it to initialize the multi-display environment by simply pointing at each active display. He then selects the wall projection by rolling the device toward the wall. Harry decides to spread the data visualization across two displays: he selects both displays with TDome and transfers the visualizations from one to the other with a TDome gesture. As he wants to look closer at information on the second display, he grabs TDome and walks towards the display, using the device in mid-air to perform a zoom on the data for a closer look.

Later that day, Mary enters the room and grabs the second TDome. They have a meeting to explore the university map to mark points of interest. Harry and Mary take their personal smartphones and bind them with each TDome to benefit from personal interactions. Each smartphone shows a personal view of the projected map, which allows them to add and access personal annotations. Before ending, Harry wants to log onto the campus website and upload his annotations: he rolls TDome towards himself to display a virtual keyboard on the device's touchscreen and enter his personal password discreetly on the login page, displayed on the tabletop.

This scenario illustrates how TDome allows users to detect surroundings displays arrangement, select one display, move content between displays, reach content at distant displays and perform personal interactions on TDome.

Device Manipulation

Interacting with TDome requires the explicit combination of a physical manipulation with a tactile gesture on the touchscreen. The sequential combination of both actions acts as a delimiter whose accidental activation is unlikely, as demonstrated in our controlled evaluation (4.3.6). This approach reduces the risk of issuing a command after performing a physical manipulation inadvertently and improves the robustness of the device. As Initially, we favored a one-handed interaction where the dominant hand was used to perform the physical manipulation on the device and the touch gestures on the display.

But our preliminary tests revealed that some gestures were easier to perform in a bimanual mode, thus extending the touch vocabulary.

TDome versions: small and large touchscreen

We implemented two design variations of TDome resulting from different device composition alternatives [141]: one with a small touchscreen inserted into the spherical shell (Small version) and one with a larger touchscreen laid on top of the spherical shell (Large version). As these two versions were meant to be complimentary, we favored the possibility of rapidly switching them as opposed to having two separate devices. This opens interesting possibilities, such as switching to the large touchscreen when a larger display is needed.

Implementation

TDome core elements

We used the Roly-Poly mouse (RPM) [140] design guidelines to define the dimension of our device: a diameter of 8 cm (~ 3.15 in) was the easiest to handle and manipulate.

As with the original RPM, we weighed the device with putty so that the device returns to its initial upright position when released (roly-poly toy principle).

Regarding the touchscreens, we implemented both the Small and Large versions. To restrict our device to the selected size, we had to limit the small screen size to less than 8 cm. To create the Small version, we removed the bracelet from an Android smartwatch SimValley AW-414 19 (63 g, 45x44x14mm, 28x28mm touchscreen) and enclosed the smartwatch into TDome. To implement the Large version, we used a Galaxy S4 20smartphone (5 in, 134 g, 137x70x8mm). We used the smart-watch camera, which is situated on the edge of the watch, to provide TDome with a horizontal camera view. The camera has a 3 MP sensor and a resolution of 1728x1728 pixels. The position of the smartphones camera does not offer the possibility of having a similar view on the smartphone version.

To support device modularity, the interchange of both touchscreens had to be easy and quick. We thus 3D printed two plastic adaptors that can be adjusted on a 3D printed base: the first one holds the watch while the second one fixes the phone using a magnet (Figure 4.9). The two plastic adaptors are very rapidly interchangeable. Altogether, the Small version, involving a smartwatch, weighted 207 g in total and the Large version, involving a smartphone, weighted 297 g. We used TCP sockets over a local Wi-Fi network to connect the watch to the main computer.

Physical manipulation detection

The spherical shell holds an x-IMU of x-io Technologies 21 (48 g, 57 mm × 38 mm × 21 mm) to detect the Roll and Rotation of the device in 3D. The IMU is composed of a triple-axis gyroscope, accelerometer and magnetometer. The refresh rate of the sensors goes up to 512 Hz and we used Bluetooth to connect the IMU with the computer. The IMU offered an angular precision of 1°. We 3D printed a holder to fit the IMU in a horizontal position inside TDome (Figure 4.9).

To detect the displacement of the device, we used an infrared bezel (Zaagtech 22 , 42" ) that generated TUIO 23 events. We implemented a filtering process to discard touch events that were detected when fingers touched the surface around the device. Thresholds were also empirically defined to avoid the detection of unwanted Translations, Rolls or Rotations: user's physical manipulations must reach a minimum amplitude to be detected

(5 cm for Translation, 30° for Roll, 45° for Rotation). Lift-Up was detected as soon as

TDome was no longer in contact with the table.

Suitability of TDOME for MDEs

In this subsection, we discuss how TDome properties suit the interaction requirements specified in Section 2.

Spatial sensing

TDome physical manipulations allow performing 3D pointing in the surrounding space. Combined with the on-board camera, it allows sensing the environment. This can be used to detect and locate nearby displays, creating a spatial layout of the MDE displays represented through a radar-view (physical relationship).

21 http://x-io.co.uk/x-imu/ 22 http://www.zaagtech.com/X-Series-Features.html

Input interaction

TDome allows up to 3 types of 2D pointing: by moving the device, by rolling it or by interacting with the touchscreen. These ranges of positioning facilitate input redirection.

This also offers input that best suits a given display, such as a cursor for precise tasks, or touch input for coarser input.

Output redirection

The touchscreen display can be used as a visual buffer to move data among displays in MDEs (output redirection). It may also be useful to display a zoomed-in version of a selected area on a distant display (reachability). The built-in vibratory capabilities are an alternative to discretely provide the user with private information (personal data management).

Through the easy interchange of the Small and Large TDome versions, the user can adopt the most appropriate display for each task; e.g., to visualize large graphs, the user can choose the Large version, but to display the compact radar-view (i.e. a view of the MDE spatial layout), a smaller display is more appropriate (output redirection).

Mid-air interaction

Two of TDome's physical manipulations (Roll and Rotate) can be used in mid-air, thus facilitating physical displacements to interact with distant displays (reachability). It also offers more flexibility to the user to ensure the privacy for some of its tasks (personal data management).

Form factor

TDome's tilting capabilities facilitate orienting the device towards oneself for private input and output interaction (personal data management); and attaching their personal smartphone to TDome's base allows users to access their personal applications and data (personal data management).

Chapter 4 -Interaction with Multi-Display Environments 117

TDome MDE interaction techniques

We now introduce a set of proof-of-concept prototypes illustrating how the previous properties contribute to facilitate interaction in MDEs.

. Input redirection

One recurrent need in MDEs is to manage input redirection. In addition to changing focus from one display to another, TDome offers an input interaction that matches the input possibility to the display it is connected to. TDome can be used as a touch input device through its embedded touchscreen, as a mouse with its translation capability and as a 3D mouse with its rotation and tilting capabilities depending on the input capability of the display it is redirected to.

For instance, to interact with a map on a distant touchscreen, the user can perform

Output redirection

We developed two interaction techniques to move content from one display to another. The Translation + Pinch/Spread technique combines a physical manipulation of TDome to select a display and a gesture on the touchscreen to grab or place some content on the selected display (Figure 4.11). In our implementation, Translation + Pinch grabs the application of the screen selected by the translation's direction, and displays it on the tabletop; while Translation + Spread sends the tabletop application to the screen situated in the translation's direction. We implemented a second technique using the radar view on TDome to create a virtual information tunnel between two displays ( maintaining the user in the flow of his activity [START_REF] Benjamin | Interfaces for staying in the flow[END_REF].

Informal tests also provided some early feedback on the importance of precision and on the required number of available gestures: a precise control of the device is important to perform spatial interactions, such as rolling to select a display; and the user requires a wide set of gestures to cover the multiple set of controls and interactions across displays.

Therefore, conferring the highest usability level to TDome is essential to ensure MDEs will take full advantage of the device properties.

For these reasons, we first focused on exploring the usability of the device itself. To this end we performed a user experiment dedicated to identifying the set of most precise and robust TDome gestures.

Exploratory study: TDome gestures and users comfort

The goal of this exploratory study was to inform the implementation of input gestures combining physical manipulations with touch input, by studying only their comfort and collecting initial user feedback. Ultimately we wanted to discard gestures that would be deemed too uncomfortable. While literature on physiology could be anticipatory, it would not help in identifying all the appropriate combinations of wrist gestures and multi-touch finger input. For this reason, we did not want to discard any gesture immediately and ran this exploratory study to reduce the initial gesture design space.

Protocol

We carried this exploratory study with 4 participants (all right-handed) from the local university. We instructed participants to manipulate the TDome with their dominant hand. During the experiment they were confronted with the two different versions of TDome (Small and Large). In both settings, they tested three physical manipulations Participants performed 2 TDome versions × 19 physical manipulations × 6 touch gestures = 228 combined gestures per participant. We asked participants to repeat each combined gesture 3 times, i.e. each participant performed 684 trials. We asked them to rate each gesture combination from 1 (comfortable) to 5 (uncomfortable) to help them verbalize their opinion and comment on their ratings. We report on their qualitative comments.

Results

Participants were very positive about performing the following gestures both with the Small and Large versions of TDome:

 Tap and Drag combined with any physical manipulation (Translation, Rotation, Roll or Lift-Up).

 Pinch and Spread in a bi-manual setting (one hand manipulates the rolling device while the other touches the display) when combined with a Translation, Rotation or Lift-Up.

However some other gestures seemed too uncomfortable to be performed:

Performing Pinch and Spread with a single hand was always deemed very uncomfortable when combined with any physical gestures and for both TDome versions (Small and Large).

Performing Pinch and Spread in a bi-manual setting in combination with a Roll gesture was perceived to be very uncomfortable.

We decided to remove these uncomfortable gestures (Pinch and Spread with a single hand or in combination with Roll) from our subsequent work. The goal of this controlled experiment was to validate the feasibility of combined gestures, i.e. physical manipulation followed by a touch input. We hypothesize that certain touch gestures could be difficult to perform on the Small version, on which certain combinations could lead to errors.

Combined Gestures

From the previous exploratory study, we decided to use two touch gestures with one hand: Tap and Drag. Gestures using two fingers, i.e. Spread and Pinch, were performed with two hands: one hand held the device while the other performed the touch gesture.

These touch gestures were used in combination with a Translation, a Roll, a Rotation and a Lift-Up of TDome.

Task

Participants were requested to perform each gesture, according to visual indication displayed on a tabletop display (Figure 4.15). TDome was placed in an initial position at the center of the tabletop display, indicated by a visual feedback. We let users hold the device as they pleased. We asked participants to perform the gestures as fast as possible with high accuracy. We provided continuous visual feedback indicating the state of the device (position, Roll and Rotation) as well as touch gestures on the display. We provided them with knowledge of result and in case of error we indicated which gesture (physical manipulation and/or touch gesture) had been erroneously performed. Each trial started when the user pressed a button on the tabletop, which displayed the instructions, and ended when a combined gesture had been recognized.

Apparatus

We used the TDome implementation described earlier (Section 4.3.2). The device was used on a tabletop display (96 cm × 72 cm) of 102 cm high thus requiring the user to stand during the experiment. We used the display in an area limited to the size of the infrared bezel (42 inches, 1920×1080px).

Design and protocol

The experiment followed a 2x4x4 within-subjects design, with Display (Small, Large), Physical manipulation (Translate, Roll, Rotate and Lift-Up) and Touch gesture (Tap, Drag, Pinch and Spread) as factors. We did not test the condition combining Roll with Pinch/Spread, as this combination appeared to be highly uncomfortable in our pre-study.

We also decided to study one random translation direction to limit the experiment length:

previous studies on RPM [START_REF] Bortolaso | OrMiS: a tabletop interface for simulation-based training[END_REF] showed that all translation directions were as easy to

Chapter 4 -Interaction with Multi-Display Environments 125 perform. For the other physical manipulations, participants performed eight Roll directions and two Rotations (left/right).

Our pre-study also showed that Pinch and Spread gestures seemed more difficult than Tap and Drag. Therefore, we paired Tap with Pinch, and Drag with Spread to balance the different blocks length and difficulty. Trials were grouped in four blocks: one block corresponded to one Display and two touch gestures (Tap/Pinch or Drag/Spread).

The four blocks were counterbalanced across participants using a 4x4 Latin Square.

For each block, we ordered touch by difficulty: first Tap or Drag, then Pinch or Spread.

For each set of trials corresponding to one touch gesture, the physical manipulations were We collected 192 × 12 participants = 2304 trials in total, which took approximately 45 minutes for each participant.

Collected Data

We logged all gestures from start to finish. We calculated success rates, completion time from instruction onset to validation, unintended touch gestures on the Display and amplitude of the physical manipulations. We classified errors in three categories according to the gesture that had been erroneously performed: physical, touch or both. Finally, we asked participants to rate each condition on a 1-5 Likert scale on perceived difficulty.

translation (8 directions) or a Lift-Up with a Tap gesture (with a success rate of 95.83%, 91.67% and 92.22% respectively).

With the Large version, the experiment reveals that 54 combined gestures can be The findings of this controlled experiment established the usability of TDome. Now, how should these possible inputs be mapped to MDEs' most common tasks?

Mapping tdome gestures to mde tasks

We elicited user input through a user study to explore how the selected set of gestures from our previous experiment can be mapped to MDE interactive tasks.

Overview and rationale

We asked users to choose, for each TDome task, one gesture from the set of gestures selected in the controlled experiment.

Chapter 4 -Interaction with Multi-Display Environments 130

MDE tasks considered

From our scenarios, we considered the 7 tasks, each task represents one of the requirements described in section 2: pointing on a distant display (reachability); zooming on a distant display (navigation); displacing a window from one display to another (horizontal tunnel, vertical tunnel) (Output redirection); sending a window from Tabletop/TDome (user position) to a distant display and vice-versa (output redirection);

selecting an icon on the radar view (interaction with UI controls); panning and zooming a focused view of a distant context (navigation); and typing on a private keyboard (personal data management).

Participants

12 (1 female) students and researchers from the local university volunteered for this study. They were aged 31.9 years on average (SD=9). Five of them took part in the previous studies.

Procedure

Participants were given the two TDome versions (Small and Large) and were situated in an MDE environment comprised of a tabletop, 4 displays and 1 video-projection. We familiarized our participants with TDome capabilities by showing them a video illustrating the combined gestures (without showing any interactive task). For each combined gesture, we asked participants to perform it themselves with both versions of TDome. Then, we asked participants to select and justify, for each task and each TDome version, which gesture they preferred. The session took about 15 minutes.

Collected data

Every user generated one sheet with a summary of the gestures chosen for each task and TDome version. We recorded users' verbal comments.

Results

Amongst all available combined gestures, only one was never used in our study (Lift-Up + Drag). Overall, participants took advantage of the gestures diversity to match the different tasks. The agreement scores [195,197] of the combined gestures (Physical Chapter 4 -Interaction with Multi-Display Environments 131 manipulation + touch gestures) range between 0.3 and 0.6. These scores are in line with previous studies [197]. To find more consensus between participants and to complete the agreement score analysis, we will detail the choice of physical manipulations and touch input separately.

Physical manipulations

Our results were similar for the Small and Large versions concerning which physical gesture to use. Thus we report both results together (i.e. 24 gestures per task).

Two physical gestures were used more often: Translate and Roll (Figure 4.19). For some tasks, one was preferred over the other: Translation for panning (17/24), or for moving a focus (15/24); Roll for private pincode input (19/24). For other tasks, such as redirecting data using the tunnel, output redirection or display selection, there was no clear preference for one of these two gestures.

The Lift-Up gesture was used for zooming 13 times (i.e. lifting up the device activates zoom mode). Rotation was used only once for each of our zooming tasks. The embedded display can be used as a personal display area to augment output visualization, such as in overview + detail techniques;

The embedded display can also be used to show feedback of the TDome interactions, such as displaying a copied object for the copy/paste technique.

Lessons learned for mapping TDome gestures to MDE tasks

We propose a set of guidelines to map TDome gestures to MDE tasks based on our mapping study as well as on our experience developing TDome interaction techniques:

 TDome offers a diversity and large number of possible gestures of which users can take advantage as illustrated in our mapping study. Some of these gestures have natural mappings with MDE tasks, such as Rolling towards oneself to display private information, Pinch and Spread for zooming or dragging for sending data to another display. Appropriately combined with a physical manipulation or a touch gesture, these should become the "default" combined gestures with TDome on MDEs.

 While some mappings are obvious and have a large consensus, others are sometimes split between two types of gestures (usually Roll or Translation): this suggests that using TDome in an MDE could benefit from a device personalization step wherein the user defines his preferred mapping, especially for output redirection.

 Interacting in an MDE involves system tasks (i.e. tasks related to the environment, such as display selection) and application tasks. Since these tasks could be assigned to the same TDome gestures, there is a need for a mode switching gesture. The Lift-Up manipulation combined with touch input, is a good candidate as it was considered for switching between pan and zoom tasks in our mapping study.

Memorability of a large number of gestures

TDome offers a large set of usable combined gestures. This diversity of available controls is particularly relevant to tackle tasks in MDEs. However, increasing the number of controls might make them hard to memorize. The physical nature of these combined 4.3.8.4. Collaboration MDEs are naturally designed to support collaboration. We can envision multiple TDome like devices, each controlled by the MDE's users. However, in such cases input and output redirection mechanisms would need to be effectively controlled. Control mechanisms have been proposed by others [START_REF] Chapuis | Smarties: an input system for wall display development[END_REF] to handle synchronization, locking and input conflicts, and future iterations of TDome will adapt or build on such proposals.

Discussion

In this work, we focused on the suitability of TDome capabilities for MDE tasks and the feasibility of its combined gestures. TDome interaction techniques still need to be fine-tuned and future work should compare their performance with a baseline for each MDE task. Theoretically, since TDome integrates the same capabilities as existing MDE devices, we hypothesize that it can perform similarly for each individual MDE task. For instance, TDome can perform translations like a mouse, and has the same touch and midair capabilities as a smartphone. Moreover, since TDome is a unique device that supports a range of core MDE tasks, it should improve the overall performance by reducing homing transition times and promoting the interaction flow. Therefore, beyond individual controlled comparisons, it would be interesting to carry a longitudinal study. We leave these studies for future work.

Beyond these aspects, we plan to focus on user expertise of TDome techniques: most menus or command techniques consider novice and expert modes as well as the transition from novice to expert [START_REF] Cockburn | Supporting Novice to Expert Transitions in User Interfaces[END_REF]. In our work we focused on how the combined gestures are performed. It will be interesting to design techniques that efficiently support both novice and expert users and the transition from one group to the other, as done with the Marking Menus [201].

We also plan to investigate the extension of our physical manipulation gestures by adding thresholds. For instance, each Roll gesture could launch two different commands according to the Roll amplitude (under or over 42° according to our study results).

Chapter 4 -Interaction with Multi-Display Environments 135 Technical alternatives also need to be investigated to replace the infrared bezel used to detect the TDome translations. We are currently exploring the application of conductive paint on the external surface of TDome, which will allow using the device on any capacitive surface.

Finally, TDome has been proposed for MDEs in a professional setup as using it in a public environment or an unsafe one where it might be damaged or stolen is its current limit. Exporting a similar device to public spaces remains a possibility for future work.

Conclusion

This chapter describes the work carried to improve interaction with multi-display environments in two contexts: public and professional environments.

In the first part of this chapter, we studied the use of everyday objects to interact with public MDEs. To this end, we carried out a study, through a creativity session, whose purpose was to identify the possible usages of objects of different shapes and materials to achieve a set of tasks representative of the most common tasks in MDEs.

We defined a taxonomy to classify the ideas collected from the study. Our results showed that users prefer to rely on the materials of the objects at their disposal to perform their gestures when they are soft (flexible). Conversely, users rely on the shapes of objects when they are rigid. However, we noted some exceptions: gestures based on analogies were preferred for the badge and gestures based on the shape were preferred for the stress ball, despite its soft material. Indeed, the round shape of the stress ball was its most influencing criteria. Based on those findings, we decided to further explore the rounded shape of tangible objects to interact with professional MDEs. We focused on a rigid material as it is most probable in a work context. We also reconsidered the interaction metaphor as it is not suitable for work MDE.

In the second part osf this chapter, we presented TDome, a dome-shaped device designed for interactions in MDEs. We designed two TDome prototypes: a Small version with an integrated touchscreen and a Large version based on attaching a smartphone.

We discussed how TDome properties suit the interaction requirements of MDEs and introduced a set of proof-of-concept prototypes illustrating how its properties contribute Chapter 4 -Interaction with Multi-Display Environments 136 to facilitate interaction in those environments. We explored combined gestures involving a physical manipulation (Translation, Roll, Rotation or Lift-Up) followed by a touch gesture (Tap, Drag, Pinch or Spread) through a 3-step process. First, an exploratory study focusing on comfort established that 60 combined gestures could be comfortably performed. Second, a controlled experiment evaluated the user's performance as well as the subjective perceived difficulty. Results revealed that the number of gestures that can be precisely and easily performed is 17 with the Small version, and 54 with the Large version. Finally, a user survey explored the mappings between these gestures and MDE tasks. Results show that some combined gestures are more prone to be used in specific tasks than others. In general, we find participants are able to match TDome features to MDE tasks.

Interaction with Immersive Environments

On-body tangible interactions for immersive data visualizations Immersive systems such as the Hololens 24 , MetaVision25 ou Moverio26 allow the user to display numerical data and visualizations directly on the physical word by attaching them to a fixed physical anchor; we hereafter refer to these as immersive visualizations These technologies offer new interaction opportunities that are to this day insufficiently explored. As such, we do not have implicit design rules to guide the developer when designing solutions for these environments. This results in a compilation of partially satisfactory solutions for interaction. Indeed, while the numerous advantages of immersive systems make them a compelling alternative to visualizing multidimensional data on 2D displays, existing interaction techniques for exploring and manipulating this type of data is unsuitable for immersive systems. These existing solutions do not have enough degrees of freedom [START_REF] Dobbelstein | Belt: An Unobtrusive Touch Input Device for Head-worn Displays[END_REF]126,156] and are often ambiguous and tiring (especially mid-air gestures [START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF]126,138]). Moreover, some of them constrain the mobility of the user to a defined place where the device (3D mouse or other) can be used [110], usually a desktop.

The challenge is to maintain the freedom of movement of mid-air interactions, the degrees of freedom of tangible interactions and the accuracy of the mouse to provide a flexible and precise solution for interaction with immersive visualizations.

In this work, we propose to study on-body tangible interactions, i.e. using the body as a physical support for interaction with an input device. We thus present a new approach that combines 1) the use of a multi-DOF mouse-like wireless device, combining Chapter 5 -Interaction with Immersive Environments 139 the precision of a mouse, tangibles' multiple degrees of freedom, and 2) the use of the body to guide the physical manipulations of the device and exploit users' proprioception 27(i.e. sensing its own body parts) while limiting muscle fatigue inherent to mid-air interactions.

To explore this new interaction approach, we define a new design space that encompasses the physical properties of the body (support) and the interactions that can be performed on the body. To evaluate the feasibility of such an approach, we conducted an experiment investigating the amplitude and comfort of on-body tangible gestures.

Our contribution is both conceptual and experimental. First, we detail our design space for tangible interactions on the body. Then we evaluate them through an experiment. Finally, we discuss the advantages and disadvantages of these tangible interactions before illustrating them through two concrete scenarios.

On-Body tangible interactions

In this section, we present a new interaction approach for immersive visualizations based on the use of the body as a support for tangible interactions. We detail the main requirements to interact with immersive visualizations, our choices of body parts and tangible objects to use, before presenting the design space.

Interaction requirements for immersive visualizations

There are different types of immersive visualizations. They range from a simple interactive visualization of a 3D object to complex multidimensional data. These immersive visualizations all share a set of basic requirements: Unconstrained mobility (R1): the main advantage of immersive systems for data visualization is that they offer physical exploration capabilities. It has been demonstrated that the physical exploration of data, as opposed to the virtual one, allows for a better spatial understanding of the visualization. The user can have an overview of the visualization from afar, or a more detailed view by getting closer. He can also analyze the data from different angles [START_REF] Jansen | Tangible remote controllers for wall-size displays[END_REF]. It is thus important that the interaction techniques do not constrain the mobility of the user.

Multiple degrees of freedom (R2): the multidimensional nature of the data visualized in this type of systems requires enough degrees of freedom to tackle the tasks related to their manipulation [START_REF] Beaudouin-Lafon | Instrumental interaction: an interaction model for designing post-WIMP user interfaces[END_REF].

Limited visual clutter (R3): the interaction techniques should not occult the data visualization. They should also allow the user to interact with data without having to divert his attention from the visualization [START_REF] Jansen | Tangible remote controllers for wall-size displays[END_REF].

-Precision (R4): the interaction techniques should offer enough precision to tackle the type of tasks performed in immersive systems, such as filtering.

The on-body tangible interaction approach can satisfy the requirements mentioned above. Indeed, the body is an always available physical support that favours physical exploration of data. It does not constrain the movement of the user (R1). Thanks to the body's natural capacity to sense its own body parts (proprioception), the user can perform tangible interactions on the body without having to switch his attention from the data visualization to the interaction tool (R3).

Tangible device

Regarding the tangible device, we decided to explore the use of the Roly-Poly Mouse (RPM) [140], an input device with a semi-spherical shape that offers up to six degrees of freedom. This device is particularly suitable for manipulating multidimensional data (R2, R4) [140]. Moreover, the device can be manipulated mid-air and therefore does not constrain the user's movement (R1).

RPM allows 3 types of physical manipulations (Figure 5. We decided to focus on the forearm of the non-dominant arm as a support for the interaction for several reasons: it offers a large surface on which the tangible interaction can be performed, and it is effortlessly accessible by the dominant hand as opposed to the arm which needs a consequent effort to be touched by the dominant hand. Moreover, several poses can be adopted with the forearm (Figure 5.3) increasing the possible interaction vocabulary.

Referential

As interaction with data is performed in a spatial context in immersive systems, it is important to choose the right frame of reference for the interaction. The frame of reference can be allocentric (external: it can be world-centered, data centered…) or egocentric (relative to the body). In an egocentric frame of reference, the output of a given manipulation is determined by how it is performed with regards to the body. A translation parallel to the body for example will have the same effect regardless of the body's position and orientation in the world. In our approach, we adopt an egocentric frame of reference to allow the user to interact from anywhere with geographically-anchored data in the physical world [124].

Design space for tangible interaction supported by the forearm

As a result of the previously identified characteristics, we propose a design space that describes the properties of the physical interaction support. It is composed of 3 dimensions: the Pose, the Place of motion and the Range of motion.

Pose (POS)

We identified three main poses for the forearm: Vertical, Parallel (to the body) and Forward (Figure 5.3). The three poses embody the 3 axes of a three-dimensional cartesian coordinate system.

In the Vertical pose, the forearm is vertical, the hand points upwards. In the Forward pose, the forearm is perpendicular to the shoulders. In the Parallel pose, the forearm is parallel to the shoulders. We explore these three dimensions in a controlled experiment, detailed in the next section.

Experiment: Tangible Interactions On The Forearm

The aim of the experiment described in this section is to study the characteristics of the implementation of an on-body tangible interaction solution, using RPM as a tangible object for interaction, and the non-dominant forearm as the support.

Goals

The main objective of the experiment is to study the movement of the device on the forearm and specifically, its translations. Indeed, we hypothesized that performing translations on the forearm could lead to systematic rolls. We decided in this first study to limit our evaluation to the translations. The experiment includes measuring the possible range of motion as well as identifying the areas of the forearm (width) on which translations can be performed distinctly. The second objective is to study the stability of the forearm as a support for the interaction in addition to the stability of the device (RPM) during its use. It has been observed before [140,158] that the device suffers from involuntary rolls (up to 12°) when used on a flat surface. This aspect of the device can potentially have an impact on its usability.

Chapter 5 -Interaction with Immersive Environments 145

Task

During the study, we asked participants to perform translations on different places of motion: the length of the forearm (length POM) and the width of the forearm (Elbow POM, Middle POM, Wrist POM). A trial is defined as a back-and-forth translation on the forearm (Figure 5.6). The starting points of the translations were chosen by the participants at the beginning of each group of 10 trials: the possible starting points of the length POM are the elbow or wrist. The possible starting points for all the width POM are the inside or the outside of the forearm. It was not necessary to control the starting points for gestures as the participants did the same gesture 10 consecutive times in each group. The device had to be manipulated with the dominant hand while the forearm of the non-dominant hand acted as physical support. The participants had to perform translations on each of the four Places of motion (Length POM, Elbow POM, Middle POM, Wrist POM), and for each Pose (Vertical, Parallel, Forward). The poses and places of motion were explained and illustrated to the participants at the beginning of the experiment. Participants were free to grasp the device as they wished. Since the purpose of the experiment was to study the use of the device on the forearm, no feedback was provided to the participants. To locate the position of the device and the body parts, an OptiTrack system composed of 12 cameras was used. The cameras track infrared reflective markers to detect objects with a precision of 1 mm. The markers were carefully placed on the device so that they do not influence the participant's grasp (Figure 5.7).

In order to detect the position of the forearm and identify the different poses described previously, additional infrared reflective markers were placed on the main joints of the arm / forearm (Figure 5.7). The wrist and the elbow of the non-dominant arm were tracked as well as the shoulders of the user.

Participants

We recruited 12 participants (5 females), 10 from the local university, aged 26 years on average (SD=5,4). 4 of the participants were PHD students, 5 were MSc students, 1 was a research engineer and 2 were external to the university. All the participants were right-handed.

Collected Data

We measured the circumference of the forearm near the elbow and the wrist for each participant as well as the inner and outer length of the forearm (Figure 5.8). We collected the position of the device, the wrist, the elbow, the shoulders using the infrared reflective markers and the optitrack system. We also collected the rotations and rolls of the RPM device using the IMU. To evaluate the physical fatigue associated with the use of the RPM device on the forearm, we asked participants to fill out a Borg scale [START_REF] Borg | Borg's Perceived Exertion and Pain Scales[END_REF] for each (Pose, Place of motion) couple.

Results

In this section, we report on the results of the study. First, we focus on the physical support (forearm): we present the results concerning its stability as well as the exploited surfaces for each Place of motion, in each Pose. Then, we detail the results related to the Chapter 5 -Interaction with Immersive Environments 148 usability of the device. Finally, we present the results on users fatigue. All error bars in the following results represent 95% confidence intervals.

Forearm stability: elbow and wrist movements

Ideally, for the forearm to be a support for interaction, it is important that it remains stable. We therefore logged the movements of the forearm during interactions: we measured the positions of the elbow and the wrist every 10 milliseconds. The movements of each of the two joints were computed with regards to their starting position, collected at the beginning of each group of 10 trials. We report the average movement of the elbow and the wrist during these 10 trials using the axes described in Figure 5.9. The elbow was relatively stable (Figure 5.10): the maximum movement (all directions included) did not exceed 1,7 cm on average. The biggest movement of the elbow was lateral (on the X axis) and it ranged from -0,86 cm to 0,84 cm for a total of 1,7 cm. The smallest movement was vertical, ranging from -0,65 cm to 0,5 cm for a total of 1,15 cm.

The results observed for the wrist are similar to the elbow, i.e. generally stable, with a maximum movement of 1.57 cm on average. The biggest movement of the wrist was sagittal (on the Z axis) and it ranged from -1 cm to 0,57 cm for a total of 1,57 cm. The smallest movement was lateral, ranging from -0,59 cm to 0,42 cm for a total of 1,01 cm. These results remain valid when we consider each Pose independently. Taking into consideration these findings, we can say that the forearm is sufficiently stable to be used as a support for tangible interactions in immersive systems.

Range of motion

Range of motion along the forearm

Participants exploited at least 93,6% of the forearm when performing translation on the Length POM. Overall, the translation distance ranged from 93,6% to 101,7% of the forearm (green bar Figure 5.12). However there were no significant differences between the poses. We observed that the translations performed in the Parallel pose extended to the hand, thus surpassing the wrist (explaining the values going above the 100%-wristmark on Figure 5.12).

Range of motion around the forearm

We also calculated the range of motion for translations performed around the forearm (width POM). We observed that for each width POM, the exploited surface of the forearm equaled 15,6% of the total length of the forearm on average. This value ranged from 13,8% for the wrist POM in the Forward pose to 21% for the elbow POM in the same pose. The largest exploited surface in the width POM is smaller than a third of the forearm. Theoretically, it is possible to consider using the three width POM. It should be noted, however, that these three regions of the forearm should be distinct, in other words, Finally, despite the clear instruction that required participants to perform translation from the elbow to the wrist, we can observe that a fair number of translations were performed beyond the wrist position (i.e. on the hand, above the 100% mark).

Fatigue

Fatigue was measured using a 6-20 Borg scale [START_REF] Borg | Borg's Perceived Exertion and Pain Scales[END_REF] (Table 5.1). The average Borg score obtained ranged from 'extremely light' for the Forward [START_REF] Balakrishnan | The Rockin'Mouse: integral 3D manipulation on a plane[END_REF][START_REF] Fender | HeatSpace: Automatic Placement of Displays by Empirical Analysis of User Behavior[END_REF] and Parallel [START_REF] Balakrishnan | The Rockin'Mouse: integral 3D manipulation on a plane[END_REF][START_REF] Heath | Reconsidering the virtual workplace: flexible support for collaborative activity[END_REF] pose to 'very light' for the Vertical pose (9,58) (Figure 5.16). The pose does not appear to affect the fatigue scores. Overall, participants did not consider the interaction with the device tiring, despite using the device for at least 25 minutes. It should also be noted that while participants had the opportunity to take breaks during the experiment between each group of 10 trials, only one participant asked for a break.

Chapter

Discussion

The findings presented above consolidate our hypotheses that on-body tangible interaction is a promising approach for use in immersive visualizations.

Support stability

Elbow and wrist movements are minimal, making the forearm a steady and reliable support for interaction. For this reason, we believe that tangible interactions on the forearm could be performed without locating the position of the arm in real time: the pose could be deduced from the movement of RPM relative to the user.

Places of motion

The surfaces covered while performing translations around the forearm were small enough to consider three distinct regions of the forearm for interaction. However, the dispersion of these surfaces showed that they overlap, making it difficult to employ more than 2 distinct regions of the forearm in practice. However, we believe that with a visual feedback showing the position of each region on the forearm in the immersive environment, the three regions would be easily distinguishable. This hypothesis should be studied in a complementary study.

Other physical manipulations of RPM

The involuntary rolls observed during the translation of RPM in a Length POM were minimal and in line with the previous findings of Perelman et al. [140]. The rolls observed when translating RPM around the forearm seemed voluntary, since participants systematically tilted the device. These rolls go up to 78° regardless of the Pose or the POM on which the interaction was performed. It thus appears impossible to distinguish a translation around the forearm, from a roll or rotation. Consequently, these two physical manipulations cannot be performed for different tasks around the forearm.

Mapping between gestures and tasks

Overall, results show that the translations performed on the length POM were the most stable in terms of involuntary rotations and rolls in addition to offering the largest exploitable interaction surface. Therefore, these gestures can potentially be used to control a large set of values, to have a substantial precision or a greater data coverage. These gestures could be used for instance to manipulate slider type controllers that require a certain degree of precision depending on the manipulated data. The translations performed in the Width POM seem to be better adapted to controllers that do not require a large amplitude given the limited interaction surface they allow. We believe that they could be mapped to " Rate control" type controllers for example. They could also be used to control a menu with a limited number of sub-items or to activate toggle menus

Perspectives

The next step in this work will consist in conducting studies to validate the accuracy and usability of the physical manipulations for basic tasks, like controlling a slider. Due to its complexity, the study described in this chapter was limited to translations. A short-term perspective would be to study the other physical manipulations offered by the device. A first conclusion regarding rolls and rotations can be deduced from our experiment: it is impossible to distinguish translations from rolls when performing translation around the forearm (Width POM). This also closes the door to the exploitation of combined physical manipulation (simultaneous rolls and translations) in the Width POM (i.e. translations around the forearm). However, this is not the case for translations performed on the length of the forearm thanks to the minimal involuntary rolls observed in this POM.

Therefore, we can explore potential usages of the approach that would exploit the combined physical manipulations of the device.

Conclusion

In this chapter, we proposed, described and studied a new paradigm for interaction with immersive visualization: On-Body tangible interactions. This approach is based on the use of the forearm as a physical support for tangible interactions using a device with multiple degrees of freedom. It takes advantage of the body's natural capacity to sense its own body parts (proprioception) to allow the user to perform tangible interactions on the body, without having to switch his attention from the data and minimizing the fatigue.

We proposed a design space for the support of interaction. It describes the Pose (Forward, Parallel, Vertical) in which the interaction is performed, the Place of Motion (Length POM, Width POM: Elbow POM, Middle POM, Wrist POM) of the interaction and the range of motion of the interaction. To explore the feasibility of such an approach, we conducted a study with the following objectives: studying the stability of the forearm as a support for tangible interaction; studying the stability of the RPM mouse; measuring the Range of motion of translation in each (Pose, Place of Motion) couple; identifying the regions of the forearm on which translations can be performed distinctly; measuring the fatigue relative to this type of manipulations.

Chapter 5 -Interaction with Immersive Environments 159

The results showed that on-body tangible interactions are a promising approach to interact with immersive visualizations since the interaction support (forearm) is stable and can support a tangible interactions appropriately. The device we used (RPM) offers enough degrees of freedom, precision and is stable enough to be used in an immersive context. With an adequate visual feedback, the user could benefit from 3 regions for interaction around the forearm. The study also showed that users found the approach comfortable.

Finally, we illustrated the possible usages of this approach through two concrete usage scenarios: the first scenario describes interaction with fairly simple 3D objects, while the second explores a more elaborated interaction with multidimensional data.

The neOCampus Project

A smart, innovative and durable campus [START_REF] Bach | The Hologram in My Hand: How Effective is Interactive Exploration of 3D Visualizations in Immersive Tangible Augmented Reality[END_REF] The neOCampus project

Introduction

This thesis is part of the neOCampus project, an initiative of the University of Toulouse, launched in June 2013 in a bid to create an innovative and smart campus. The objectives of the project are three-fold: 1) to improve the daily comfort of the university students and personnel; 2) to decrease the ecological footprint of its buildings; 3) to reduce its operating costs (fluid, water, electricity…). To attain its objectives, the neOCampus project relies on repurposing the large number of connected devices available on the university campus and completing this net of connected devices with eco-friendly connected sensors, to better gather and exploit data.Similar to a small city with its 407 000m 2 of built-up areas, 70 research structures, several solutions of mobility and in excess of 39000 employees and students, improving the quality of life inside the University of Toulouse's campus can be equally challenging [START_REF] Gleizes | neocampus: A demonstrator of connected, innovative, intelligent and sustainable campus[END_REF]:

-The heterogenous devices and sensors composing the campus and designed to observe specific features result in large volumes of heterogenous data, that require the creation of new tools and norms to explore and manage them.

-The non-linearity of the campus where a small change in the input may result in big output changes make them difficult to control and predict.

-The openness of such systems, where sensors and devices can be easily added or removed is a key component in making the system sustainable and needs to be facilitated.

-The spatial distribution of the campus's entities may require new types of communication technologies and infrastructure and may even change the way systems are developed for this platform.

-The large-scale collection of data in such a large campus may introduce privacy issues and require the design of new development methodologies taking into consideration privacy when designing IT applications.

Chapter 6the neOCampus project 163

The challenges are numerous and require a wide range of skills to be addressed. To reach its objectives and address those challenges, the neOCampus project favoured a multidisciplinary approach which comes from the 11 laboratories participating to the project and the different fields that they cover (a detailed listing of the participating laboratories can be found here 28 ). Each laboratory brings its own scientific expertise, thus, transforming the university into a platform for innovative experiments performed at large scale and in vivo (with real end users, in real situations).

HCI axis of the neOCampus project

As one of the participating partners and an HCI oriented research team, the Elipse research group 29 focuses on the challenges related to the exploration of the complex data provided by the numerous sensors and devices distributed over the campus. It aims to design and evaluate novel interaction solutions to visualize and interact with these data.

The possible usages of these solutions include, but is not limited to:

-To review or monitor energy consumption data in real time or deferred at various scales (building, room, sensor…).

-To pilot a simulator at campus scale that would include: energy consumption data, weather data, crowd behaviour, etc.

-To offer intelligent solutions to remotely control heating systems, sunblinds, lights, etc.

However, designing interactive solutions for such a diverse context is not straightforward. The campus provides different types of data that can be exploited by several profiles of users and in different ways, some may just want to visualize data while others would want to extract meaning from it. An important part of designing these interactive solutions consists in identifying the potential users of the solution, their needs as well as the manipulated data.

In this chapter, we will detail a description space built to illustrate the different aspects of data exploration in the neOCampus context, we demonstrate its use through a set of interactive situations and we discuss our contributions in relation to it.

Description space

To go beyond the simple design of ad hoc interactive solutions on the campus, we built a description space that identifies and organizes the relevant characteristics to consider when designing these solutions.

In this section, we will present the description space's dimensions and illustrate them through a set of interactive situations related to the neOCampus context.

This categorization is the result of a collaboration with the 11 laboratories participating to the project and several in-situ observations. It has been validated by the steering committee of the project and was one of the deliverables of an ANR30 project 31 (project link).

Dimensions

Users

The user dimension is a classic HCI criteria, when designing an HCI system, it is important to focus on the potential users of the system and the way each category of users is going to use it. We classify the users in three categories:

Casual

The Casual category includes all users that will use the system occasionally. In the context of the neOCampus project, it could translate into visiting researchers, decision makers, visiting elected officials, etc.

Regular

The Regular category includes users that will use the system in a regular fashion without it being their daily work tool. In the context of the neOCampus project, it could translate to students, university staff, faculty members, etc.

Frequent

The Frequent category includes users for whom the system is a daily working tool. In the context of the neOCampus project, it could translate to the local maintenance staff like plumbers, heating specialists, electricians, etc.

Services provided

We identified four types of services the system can provide:

Visualization A Visualization service offers the user the necessary tools to view data (energy consumption, affluence, temperature, etc).

Comprehension

A Comprehension service enables the users to understand and analyze data. An example would be diagnosing an electricity overconsumption using energy consumption data.

Production

A Production service supports the user in producing something to enrich the data.

An example would be an electrician generating an intervention roadmap from building locations data and a list of interventions.

Collaboration

A Collaboration service allows a group of users to aggregate, inform and create knowledge from their collaboration.

Data exploited

We identified four types of data that can be manipulated:

Chapter 6the neOCampus project 166

Raw data

It represents the data as collected from the sources, unmodified. An example would be energy consumption data: water, electricity, gas, etc.

Activity data

It represents the data inherent to the activity. An example would be the postprocessed energy data used to diagnose an electricity overconsumption.

Incident data

It represents the data related to an improper execution of an activity which may generate an alert or a blockage that may use or generate specific data. An example would be the data related to a heating problem, a network issue or a power failure.

Ambient data

It represents the data characterizing the environment in which the activity takes place. It may refer to the data related to the interior and exterior environment of the campus: temperature, weather, affluence, CO2 consumption, confort level, diversity of flora and fauna of the local ecosystem, etc.

Deployment context

We identified two possible deployment contexts:

Open-access system

Open-access systems are usually available in public places. They are accessible to the general public. The interaction ressources in those systems are usually scalable. Their numbers and types vary depending on: the interaction devices used by the users interacting with the system (Smartphones, tablets, wearables, etc); the interaction space already available (large displays, interactive tabletops, etc).

Dedicated-access system

Dedicated access systems are usually available in a fixed context like an office or a control room. In those systems, the interaction ressources are stable, predefined and always available.

Chapter 6the neOCampus project 167

Diagram representation

The dimensions described above are summarized in the following diagram (Figure 6.1): As the sole aim of these scenarios is to illustrate the description space defined previously, no interactive solution is going to be proposed to address them in this section.

Chapter 6the neOCampus project 168 6.2.3.1. Scenario A: energy consumption visualization

Description

In the course of the promotion of the neOCampus project, its steering committee invites an elected official of the city to visit the campus of the University of Toulouse and to attend a demonstration of the project (Figure 6.2).

An interactive demonstration is prepared: it allows the visualization of different data provided by the numerous sensors installed on the campus (water, electricity, gas, temperature, weather, affluence, CO2 consumption…).

The setup of the demonstration contains two displays: a tabletop containing the 2D map of the university and a second display showing complementary information.

On the day of the demonstration, the elected official is received by a representative of the Department of Heritage and Logistics of the University of Toulouse.

Characterization

User(s): The elected official is a Casual type of users as he never used the system before.

Service(s) used: The system is used to visualize data and compare the data provided by several buildings. The service provided by the system is a Visualization service.

Exploited data: Two types of data are manipulated in this interactive situation: Raw in electricity, water and gas; Ambient in temperature, affluence, CO2 consumption.

Deployment context: As the demonstration and the data provided are public, the system used in this interactive situation is an Open-access one.

Description diagram: The characterization translates to the following description diagram (Figure 6.3): The logistics service (LS) uses the interactive system to find a conference room that is heated and contains the required equipment for the lecture to take place. To accomplish this task, the LS checks the teaching schedule, the available conference rooms and the equipment list of each conference room. The LS can guide the lecturer to his new conference room if necessary. Then, he generates an intervention roadmap. In scenario C21 and C22, the electrician sent by the external company uses the system from time to time making him a Casual user. While it is not his daily work tool, the CSD agent uses the system regularly. He is a Regular user of the system. This categorization could be used to identify recommendations for the design of interaction techniques for each value and/or dimension of the description space.

Moreover, it can be used to consider possible improvements to current interaction techniques over one of the axes like adding a new service, widening the type of data that can be manipulated through it or adapting it to a public usage.

Translating contributions into concrete usages

This section will discuss the contributions proposed in this thesis in relation to the description space introduced previously. Each subsection will focus on one interaction solution and will detail which values of each dimension the interaction solution could cover. That is to say, identify:

-The profile of users it could attend to.

-The types of data it could manipulate.

-The type of service it could provide.

-The type of system it would be suited to.

At the end of each discussion, a concrete example of each solution applied to the neOCampus project will be presented if applicable. Otherwise, a scenario highlighting its potential use will be described.

6.3.1 Split-focus: interaction in large displays 6.3.1.1. Discussion The split-focus visualization (Chapter 3) and interaction solution is easy to understand and to use. It does not require training nor an adaptation period which makes it suitable for all profiles of users.

In its current version, split-focus does not offer a solution to visualize raw data directly and is used instead to show graphical representations of data similar to the molecular interaction maps described in Chapter 3, 3.1.

Chapter 6the neOCampus project 178 Split-focus allows the user to visualize and interact with data to better understand it.

Split-focus was not evaluated nor designed to produce something or to support collaboration.

Although it helps with data visualization thanks to the overview + detail multidisplay paradigm it is based on, the interaction solution was designed to address a specific problem: interaction with multiple regions of interest of the same visualization simultaneously. Today, this problem is more pertinent for a professional exploration of data which takes place mainly in a dedicated-access system. Chapter 6the neOCampus project 179 While the interaction solution has been designed for an MDE composed of a large screen and a tablet, it can be adapted to any combination of displays provided that one of them offers enough screen real estate to display the large overview (map).

We implemented an application using the split-focus approach for a concrete usage in Figure 6.13. A video demonstrating it can be found here 32 . The implemented solution offers the following:

-The exploration of the map using up to 4 detailed views.

-A circular menu allowing the user to switch between the views and perform additional actions.

-A quick way to translate views from one region to the other: the translation view described in Chapter 3, 3.2.2.3.

32 https://www.youtube.com/watch?v=E1HxtnfzPB0

Chapter 6the neOCampus project 180 -The visualization of energy consumption data: the data is displayed in the colored part at the top of each detailed view. The displayed data is updated when the view is translated.

-The possibility to switch between numerous energy consumption data: water, electricity, gas, etc. Switching between the data displayed is done by performing swipe gestures on the colored part of the detailed view.

-The possibility to keep the views coherent, by locking them in a specific region of the overview.

-The possibility to show different types of data for the same building by linking the translations of two detailed views.

-The possibility to compare energy consumption data for up to 4 regions of the map.

In its actual version, the split-view interface supports visualization and comprehension of data. It can easily be adapted to support the user in producing something from the visualized data by introducing new functions through menus or multitouch gestures. The configuration of the working MDE is as follows (Figure 6.15):

-An interactive tabletop displaying a map highlighting the buildings, classrooms, conference rooms.

-A screen displaying the information related to each room's equipment and ambient data (temperature, affluence, etc).

-A third display showing the teaching/lectures schedule and the available rooms.

Chapter 6the neOCampus project 182 The steps required to find an available conference room, that is heated and contains the equipment required by the lecturer are as follows:

-Interaction with the interactive tabletop to have a broad idea of the available conference rooms around the position of the lecturer. This task includes:

o Navigating in a map (pan and zoom)

o Selecting a region and sending it to the calendar display.

-Redirecting the input to the calendar display and interacting with it.

-Selecting a conference room from the agenda display to visualize the available equipment and the temperature of the conference room.

The tasks described above are common tasks in MDEs. We showed in Chapter The proposed interaction solution aims to facilitate interaction with complex data visualized in immersive environments (Chapter 5). Due to the requirement of interaction with a multiple degrees of freedom device and the type of tasks performed on complex data, the profiles of users that would use such a system are regular or frequent users.

The display capabilities of immersive systems allow them to display all types of data.

The interaction technique is not designed for data in particular but to interact with the immersive system. Thus, it can cover the four types of data.

The interaction technique was designed to support several tasks related to data exploration (visualization, comprehension and production) and interaction with immersive system.

While it may be adapted to other configurations, the current design and study of the approach pertain to a professional, dedicated-access system.

Chapter 6the neOCampus project 184 This approach could be used in the activity described in scenario C albeit in a different setup. The local electrician has a list of failures on several sites of the university campus and he wants to visualize the energy consumption data to diagnose the failures. To conduct his task, he exploits the spatial capabilities of a hololens headset and the onbody tangible interaction approach.

The system allows the electrician to display a specific type of data (electricity consumption for example) at a specific point in time and in a specific location. The data is multidimensional. The electrician distributes it spatially over the 3 egocentric axes.

The system allows him to switch the dimension of data attached to each axis and offers him the possibility to anchor the volume of data on a specific physical position which helps him in his understanding of it. Indeed, the electrician can have an overview The on-body tangible interaction approach (Figure 6.17) allows him to interact with the three axes using a multi-degree of freedom device supported by the forearm with efficiency and without dividing his attention between the data and the interaction tool.

The electrician can use the system to select data, apply commands on it, scale it, rotate its visualization, etc. These tasks are mapped to different poses of the forearm, different regions of the forearm and different physical manipulations of the device (translations, tilting, rotations).

Although it has not been evaluated in a collaborative context, the interaction paradigm can support collaboration in several ways: The users can use one device for interaction, passing it from one user to the other or they could use several devices.

Chapter 6the neOCampus project 186

Conclusion

In this chapter, we described the neOCampus project and highlighted the related objectives falling in the scope of this thesis. They consist in addressing the challenges of exploration of the complex data provided by the numerous sensors and devices distributed on it.

To illustrate the different aspects of exploration of data in such a large context, we proposed what we called a "description space" which aims to identify and organize the relevant characteristics to consider when designing interactive solutions. In such contexts, the aspects covered by the description space includes the profiles of users the interactive solutions are designed for, the manipulated data, the type of services provided and its context of deployment.

We demonstrated the use of the description space through 3 scenarios representing situations of interest to us and covering the different aspects of interaction with complex data. We demonstrated the potential use of this thesis's contributions in this project by discussing how they could be applied to address the challenges of interaction in the 3 aforementioned scenarios. We detailed the profiles of users each solution targets, the type of data that can be explored through it and the services it provides.

In terms of perspectives, a potential use of the description space resides in identifying recommendations for the design of interaction techniques for each value and/or dimension of the description space. This chapter provides a summary of the main contributions of this thesis and discusses the associated medium-term and long-term perspectives.

Conclusion & Perspectives

Thesis summary

Today, several display spaces are available for data visualization. They offer numerous advantages and introduce new interaction challenges. In this manuscript, we discussed three display environments for data exploration: large displays, multi-display environments (MDEs) and immersive environments. We described them and detailed their characteristics, we presented their advantages and identified the interaction challenges they introduce. In this thesis, our goal was to improve interaction in each one of the previously cited environments. Below, we summarize our main contributions.Our work on large displays consisted in improving interaction with several regions of interest simultaneously. Our proposed approach is based on the use of a multi-view approach. We evaluated the influence of the number of detailed views on the user performance. To this end, we designed a visualization interface that offers multiple detailed views. The interface is based on an Overview+detail approach deployed on two displays: a large display showing the overview and a tablet displaying the detailed view (Figure 7.1). Our design follows Baldonado et al. [10] guidelines for multiple views. Baldonado [START_REF] Baldonado | Guidelines for using multiple views in information visualization[END_REF] proposed a set of rules that helps designers assess the adequacy of multi-view systems for their application and make design choices related to the use of the views. They also help usability experts evaluate such systems. Our interface offer two interaction techniques to navigate in the overview: a basic one that consists in regular pan directly on the detailed view; a more advanced one in the form of dedicated view we called the translation view, it allows users to translate the detailed views by manipulating their icons on a mini-map.

The translation view allows users to use multitouch to translate several detailed views at the same time. Using our interface, we experimentally evaluated the effect of the number of detailed views in a task related to the manipulation of large graphs in which users were asked to create a connection between 2, 3 or 4 nodes. We evaluated three multi-view configurations: one view (1V), two split views (2V) and four split views (4V). The results show that for tasks involving 4 regions of interest, 4 detailed views was the most efficient configuration. However, users performed similarly in 2V and 4V for tasks involving 2 regions of interest. The findings related to the use of the translation view, which allowed users to translate the detailed view in parallel through multi-touch interactions, showed that users did not take full benefit of that parallelism.

Next, we focused on multi-display environments (MDEs). We described their characteristics, highlighted their heterogeneous nature as well as the benefits they offer and the challenges they introduce. We identified two types of MDEs: MDEs deployed in a public context and MDEs deployed in a work context. While they share the same characteristics and interaction requirements (input redirection, output redirection, reachability, personal interaction), the profiles of their potential users and the type of tasks performed on them are different. Indeed, the users of public MDEs are usually passers-by, they may not engage in interaction with the MDEs if it's too difficult or takes too long. They may have never used the system before which makes them novice users requiring quick and easy to understand interactions. Moreover, they are mainly used to display data as they offer little or no means of interaction. To improve interaction in public MDEs, we explored the use of everyday objects as tools to perform tangible interactions to interact with these environments. They are always available, they offer easy to perform interactions and their shapes may help suggest their potential use which could translate into easy to understand interactions. We conducted a creativity study to identify the way to use objects of different shapes and materials to perform common tasks in MDE and amended an existing taxonomy to classify the proposed gestures. Among our findings was the fact that participants took benefit of the material of the object when it was soft, rather than its shape with the exception of the spherical object. Indeed, despite the soft nature of the ball, participants preferred exploiting its rounded shape for their proposed gestures.

Building on the findings of the creativity study, we proposed TDome, a rounded device designed to overcome the lack of a unified device or interaction technique dedicated to multi-display environments in a work context. TDome is an input and output device with a semi-spherical base offering multiple degrees of freedom (rolls, rotations and translations). The device is augmented with a touchscreen that allows it to display information and detect touch input. The combination of physical manipulations and touch gestures makes TDome a robust device and increases the number of available gestures. We discussed the suitability of TDome in addressing the main MDEs requirements, while avoiding the need to have more than one device in the workspace, and demonstrated its benefits through a set of interaction techniques (Figure 7.2). We evaluated the usability of the device through an experimental study. The results

show that up to 71 combined gestures can be comfortably performed with the device. We explored potential mappings of TDome gestures to MDE's tasks through a user survey.

Results show that some combined gestures are more prone to be used in specific tasks than others. In general, we find participants are able to match TDome features to MDE tasks.

Finally, we explored on-body tangible interactions in immersive environments with complex data requiring multiple degrees of freedom. The approach is based on the use of a multi-degrees of freedom device (RPM) supported by a body part (forearm). The use of RPM is motivated by its multiple degrees of freedom and its suitability for interaction We proposed a theoretical contribution in the form of a design space describing the main characteristics of the approach and a practical contribution through its evaluation.

To validate the adequacy of such an approach for immersive environments, we conducted an experiment aimed at establishing the range, stability and comfort of gestures performed with the device on the forearm.

The results show that: the forearm is a suitable support for tangible interactions (minimal movement during interaction); the forearm offers a large area where interaction is possible; the device was stable when used to perform physical manipulations on the forearm; no fatigue was reported by the participants when using the approach.

We discussed possible mappings of the tangible gestures on the body to different controls. We highlighted the potential of translations performed on the length of the forearm for controls requiring substantial precision or a great data coverage; we also discussed the possible use of translations performed on the width of the forearm for " Rate control" type controllers. In this section, we present and discuss research opportunities and ideas for future work.

Medium-term

Below, we present several medium-term perspectives related to the work conducted in this thesis. One of the findings of our evaluation was that users did not fully exploit the parallel exploration offered by our interface. They found symmetric bimanual multi-touch input difficult to perform. An alternative to the use of all fingers for parallel translations of detailed views is to repurpose the role of unused fingers. They can be used as modifiers for the interaction: to dynamically activate or release locks without using the menu designed for that on the detailed view; to perform quick translation movements like reverting to a previous position, translating to a corner, translating to the closest detailed view (forming a continuous view) …; to increase/decrease the size of the detailed views' icons while translating them; to give a temporary overview of a specific region i.e. while translating a detailed view (DV) with one finger, a touch on the overview (TO) with another finger of the same hand can display temporarily on DV the area pointed by TO.

Removing the finger from TO would revert to showing the area covered by the detailed view icon.

It would be interesting to explore these potential uses of unused fingers and evaluate them in further depth through the adequate experimental setup.

. Multi-display environments

In our implementation of TDome to interact with MDEs (Chapter 4), we used a semiautonomous version of RPM [140]. Indeed, while the rotations and rolls were detected by the inertial measurement unit equipping it, the translations (positions) of the device were always detected using an external sensor limiting the usage of the device. The TDome prototype used an infrared touch overlay which restricts the use of the device to the surface where it is installed and reduces the benefits of the device. The RPM mouse on which TDome is based on [140] used the mocap system 33 . it offers a more precise detection of translations, rotations and rolls as well as a larger area where it can be used. However, this solution is expensive, cumbersome to implement and difficult to move which hinders the portability of the approach. A next step in the improvement of RPM/TDome consists in studying the possible use of integrated and affordable sensors to detect translations accurately as it is the only barrier to have a fully self-contained device.

A more research-oriented perspective consists in comparing TDome to several baselines in interaction with MDEs. We demonstrated that the device offers a wider variety of interactions than existing solutions support which demonstrates that it has the potential to be suitable for MDEs. However, this potential is hard to validate without any baseline comparison of the device with existing interaction solutions that for example, based on studies reported in the literature, has been shown to provide the best support in MDEs. Our contribution has no such baseline comparison and instead focuses on studying the usability of the device in such context.

Finally, our work focused mainly on stationary MDEs, it would be interesting to study how TDome could be used in more mobile context where smartphones and wearable have been used to compose the MDE [START_REF] Xiang | Duet: exploring joint interactions on a smart phone and a smart watch[END_REF][START_REF] Grubert | Multifi: Multi fidelity interaction with displays on and around the body[END_REF].

Immersive Environments

Our work on-body tangible interactions was a first step in evaluating the proposed interaction approach. It focused on studying the stability of the forearm and the multidof device. This study was limited to the translations of the device. Indeed, we hypothesized that performing translations on the width of the forearm would lead to voluntary rolls due to the cylindrical shape of the forearm. This hypothesis, which was validated by the study results, prompted us to focus on translations first to evaluate the feasibility of the proposed interactions. The next step in this work is to evaluate to remaining physical manipulations allowed by the device in the same circumstances.

Another perspective is to widen the possible use of the forearm for interactions. In our design space, the forearm was restricted to the role of simple support. Another dimension could specify the role it plays in interaction. It could augment the interaction and act as an additional tool of interaction. For instance, the users could move the forearm to choose the granularity of control provided by the device. In this case, it acts as a modifier for the interaction. It would be interesting to explore the combination of support movement (forearm moving) and tangible interactions to construct a bigger interaction vocabulary and larger amplitudes. Similarly, the poses we proposed for the forearm embody the 3 axes of a three-dimensional cartesian coordinate system.

Alternative poses could be evaluated to perform interactions. We highlighted at several points in this manuscript the heterogenous nature of MDEs.

However, in proposing a multi-dof device to interact with these environments, we focused mainly on the device, its suitability for MDEs and its usability in a specific context in which the displays were at fixed positions. The displays composing MDEs can offer most varying characteristics. Some more than others can influence greatly the way we perform interactions with them. The positions and rotations of displays may change frequently in those environments: either involuntarily as displays like smartphones, tablets and wearables in general are meant to be mobile or voluntarily as users can exploit the spatial reconfigurability/flexibility of MDEs to create a work environment in which they feel comfortable. In both cases, the relationship between displays changes which could have an undesirable effect on the flow of the tasks usually performed on the MDE. For instance, the distribution of data between displays may be different in a newly arranged MDE which could have a direct effect on how a content transfer task is performed between displays. It would be interesting to explore how the topology of displays influences the flow of interaction and how TDome is used in different dispositions of displays.

Physical visualizations

A second long-term perspective consists in exploring interaction with spatial data displayed on three-dimensional physical models. This thesis has so far focused on interaction with display spaces where data is visualized virtually (large displays, MDEs, immersive environments). Recently, growing emphasis has been placed on the physicalisation of data i.e. displaying data in the spatial or physical context that generated it. Such visualizations are becoming possible through the use of spatial augmented reality (SAR) [START_REF] Bimber | Spatial Augmented Reality Merging Real and Virtual Worlds[END_REF]. Several works in HCI are highlighting the benefits of such environments and arguing that today's visualization systems will slowly get enriched with physicalized instruments [START_REF] Jansen | Physical and Tangible Information Visualization[END_REF]. This type of physical visualizations allow for in-place or insitu analytics [START_REF] Elmqvist | Ubiquitous analytics: Interacting with big data anywhere, anytime[END_REF], but also constitute effective communication tools when presenting information to non-expert users [START_REF] Jansen | Evaluating the efficiency of physical visualizations[END_REF]. They have been used in multiple contexts: to display air quality measures in the physical context of the sensors [193]; to present traffic flow on city models [START_REF] Hadhrawi | Illuminating LEGOs with Digital Information to Create Urban Data Observatory and Intervention Simulator[END_REF]; to embed data representations on buildings [START_REF] Hadhrawi | Illuminating LEGOs with Digital Information to Create Urban Data Observatory and Intervention Simulator[END_REF]194]. Research for this type of visualizations is still at an early stage and work is still to be done to improve interaction with these environments. Our work for immersive environments which focused on virtual 3d representation of data could be adapted to these environments. It would be interesting to explore it further and evaluate it in such contexts.

Mots-clés

6. 1

 1 Introduction ................................................................................................. 6.2 Description space .........................................................................................6.2.1 Dimensions ................................................................................................................. 6.2.2 Diagram representation .............................................................................................. 6.2.3 Illustration of the description space ........................................................................... 6.3 Translating contributions into concrete usages ........................................... 6.3.1 Split-focus: interaction in large displays .................................................................... 6.3.2 TDome: A multiple degrees of freedom device to interact with multi-display environments ...................................................................................................................................

Figure 2 . 1 : 32 Figure 2 . 2 : 33 Figure 2 . 3 : 35 Figure 2 . 4 : 35 Figure 2 . 5 : 36 Figure 2 . 6 : 37 Figure 2 . 7 : 37 Figure 2 . 8 : 39 Figure 2 . 9 : 40 Figure 2 . 10 : 40 Figure 2 . 11 : 42 Figure 2 . 12 : 43 Figure 2 . 45 Figure 2 . 14 : 46 Figure 2 . 15 : 47 Figure 2 . 16 : 48 Figure 2 . 17 : 48 Figure 2 . 18 : 49 Figure 2 . 19 : 50 Figure 2 . 50 Figure 2 . 21 : 51 Figure 2 . 22 : 52 Figure 2 . 23 : 52 Figure 2 . 55 Figure 2 .

 2132223323352435253626372737283929402104021142212432452144621547216482174821849219502502215122252223522552 Figure 2.1: Examples of Overview + Detail (a), Zooming (b) and Focus + context (c) interfaces [50] ................................................................................................................... 32 Figure 2.2: F+c prototype combining a monitor having a flat surface with a projection system. [13] ...................................................................................................................... 33 Figure 2.3: Example of a multi-display overview + detail interface [1] .......................... 35 Figure 2.4: The polyzoom technique................................................................................ 35

56 Figure 2 . 26 : 58 Figure 2 . 27 : 59 Figure 2 . 28 : 60 Figure 2 . 61 Figure 2 . 30 : 62 Figure 2 . 31 : 63 Figure 2 . 32 : 64 Figure 2 . 33 : 65 Figure 2 . 34 : 66 Figure 2 . 35 : 67 Figure 3 . 1 : 72 Figure 3 . 2 : 77 Figure 3 . 3 : 79 Figure 3 . 4 : 79 Figure 3 . 5 : 80 Figure 3 . 6 : 82 Figure 3 . 7 : 83 Figure 3 . 8 :Figure 3 . 9 : 86 Figure 3 . 87 tFigure 4 . 1 : 95 Figure 4 . 2 : 96 Figure 4 . 3 :Figure 4 . 4 :Figure 4 . 5 :Figure 4 . 6 :Figure 4 . 7 :Figure 4 . 8 :Figure 4 . 9 :Figure 4 . 10 :Figure 4 .Figure 4 . 12 :Figure 4 .Figure 4 . 14 :Figure 4 .Figure 4 . 16 :Figure 4 . 17 :Figure 4 . 18 :Figure 4 . 19 :Figure 4 . 20 :Figure 5 . 1 :Figure 5 . 2 :Figure 5 . 3 :Figure 5 . 4 :Figure 5 . 5 :Figure 5 . 6 :Figure 5 . 7 :Figure 5 . 8 :Figure 5 . 9 :Figure 5 . 10 :Figure 5 . 11 :Figure 5 . 12 :Figure 5 .Figure 5 . 14 :Figure 5 . 15 :Figure 5 . 16 :Figure 6 . 1 :Figure 6 . 2 :Figure 6 . 3 :Figure 6 . 4 :Figure 6 . 5 :Figure 6 . 6 :Figure 6 . 7 :Figure 6 . 8 :Figure 6 . 9 :Figure 6 . 10 :Figure 6 . 11 :Figure 6 . 12 :Figure 6 .Figure 6 . 14 :Figure 6 .Figure 6 . 16 :Figure 6 .Figure 7 . 1 :Figure 7 . 2 :Figure 7 . 3 :Figure 7 . 4 :

 5622658227592286026123062231632326423365234662356731723277337934793580368237833839863874195429643444546474849410441244144416417418419420515253545556575859510511512551451551661626364656667686961061161266146616671727374 Figure 2.26: The user is wearing the HoloLens and using Kinect gestures to change the rotation of the model [198] .............................................................................................. 58 Figure 2.27: Performing a pinching gesture pans the night sky imagery in World Wide Telescope [21] ................................................................................................................... 59 Figure 2.28: Variants of hand poses observed among gestures in [142] .......................... 60 Figure 2.29: A user manipulates the WIM using the physical clipboard and button-ball props [177] ........................................................................................................................ 61

Figure 2 . 1 :

 21 Figure 2.1: Examples of Overview + Detail (a), Zooming (b) and Focus + context (c)interfaces[START_REF] Cockburn | A Review of Overview+Detail, Zooming, and Focus+Context Interfaces[END_REF] An overview + detail interface allows the user to have an overview of the data and a detailed view of a region of interest. The two views are spatially separated, thereby

Figure 2 . 2 :

 22 Figure 2.2: F+c prototype combining a monitor having a flat surface with a projection system. [13] Zooming interfaces involve a temporal separation between views [50]: the user can view only one of the two views (detailed, overview) at one point in a time. The user has to zoom in the visualization to access a detailed view or zoom out if he requires an overview of the data. While zooming interfaces allow the user to exploit the entirety of

Figure 2 . 3 :

 23 Figure 2.3: Example of a multi-display overview + detail interface[START_REF] Shakeri | Multi Surface Interactions with Geospatial Data: A Systematic Review[END_REF]

Figure 2 . 4 :

 24 Figure 2.4: The polyzoom technique Melange [61] uses a distortion-based technique that offers the possibility to bring together two regions of a large space by folding them (Figure 2.5). SpaceFold [41], inspired

Figure 2 . 5 :

 25 Figure 2.5: Folding a 1D video editing timeline using the mélange technique

Chapter 2 -

 2 Related work 37More recently,Matulic et al. [122] proposed a multi-finger raycasting technique where each finger projects a ray onto the display. The technique allows its user to perform direct input using patterns of ray intersections created by his hand posture. This interaction technique can be used for tasks including object selection, object moving and zooming.

Figure 2 . 6 :

 26 Figure 2.6: Multi-finger raycasting for large displays [122]

Figure 2 . 7 :

 27 Figure 2.7: The ARC-Pad technique [123]Smartphones and tablets based interaction are more and more popular among researchers.Mccallum et al. [123] used a handheld device as a remote controller to move a distant cursor through its touchscreen (Figure2.7).Nancel et al. [133] worked on improving pointing accuracy in a similar approach. Boring et al.[START_REF] Boring | Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays[END_REF] used smartphones to control a pointer on a large display by scrolling, tilting or moving the smartphone to achieve the task at hand.

Figure 2 . 8 :

 28 Figure 2.8: Top: General setting of smartphone-based Overview+Detail interface on a 3D Public Display. Bottom: navigation techniques: a) Mid-Air Hand, b) Mid-Air Phone and c)Touchscreen[START_REF] Bergé | Exploring smartphone-based interaction with overview+detail interfaces on 3D public displays[END_REF]

Figure 2 . 11 :

 211 Figure 2.11: An example of a heterogeneous multi-display environment [170]

Figure 2 .

 2 Figure 2.13: Rekimoto's [152] Pick-and-drop technique They followed on two years later [153] by proposing an augmented multi-display environment (Figure 2.14) where the physical objects and displays are augmented digitally to create a spatially continuous shared workspace. This environment allows users to easily transfer content between displays (Content rediction). The system is augmented by video projectors and uses camera tracking to detect interactions.

Figure 2 . 14 :

 214 Figure 2.14: Rekimoto's [153] augmented surfaces PointRight [101] is an interaction solution designed to provide common keyboard and mouse control for heterogeneous MDEs (Input redirection). It uses a peer-to-peer 4 pointer

Figure 2 .

 2 [START_REF] Baudisch | Mouse ether: accelerating the acquisition of targets across multi-monitor displays[END_REF], across several application domains. Examples of proposed gestures included throwing from a mobile device to a digital tabletop to send data, as well as pulling from a digital tabletop to a mobile device to collect data.

Figure 2 . 15 :

 215 Figure 2.15: Natural gesture interactions with the mobile phone in a multi-display poker game: (a) look into cards, (b.1) fold with cards open, (b.2) fold with cards closed, and (c)check[START_REF] Döring | Exploring gesture-based interaction techniques in multi-display environments with mobile phones and a multi-touch table[END_REF]

Figure 2 . 16 :

 216 Figure 2.16: Drag-and-Drop concept: (a) a user holds the mobile phone next to the desktop screen and selects a data item. (b) The user drags it inside the screen. (c) In the other direction, a user selects data on the PC and (d) drops it on the phone [2] Jokela et al. [102] designed three interaction techniques using the smartphone's inherent characteristics to move content (visual objects) between smartphones. The first interaction technique (Tray) enables the user to move objects through a virtual tray shared between devices; the second interaction technique requires the users to perform a simple tap to move content between them; the last technique-called Device Touch-is based on the devices physically touching (Figure 2.17).

Figure 2 . 17 :

 217 Figure 2.17: Different cross-display object movement methods: a) Tray, b) Transfer Mode, and c) Device Touch [102]

Chapter 2 -

 2 Related work 49 smartphone's camera to manipulate and transfer content from one display to another (Figure 2.18).

Figure 2 . 18 :

 218 Figure 2.18: Walkthrough of the original metaphor: The user aims at a display (a) and touches the item of interest (b). When moving the device off-screen, a thumbnail of the dragged item is showing (c).After reaching the destination display (d), the item can be positioned precisely by moving the finger (e). When the finger is released, the item has been transferred successfully (f)[START_REF] Boring | Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays[END_REF]

  Pad [166] is an MDE composed of an interactive tabletop and a self-actuated display that can move autonomously in mid-air to navigate through three-dimensional space (Figure2.19).Kim's [108] G-Raff is an elevating tangible block equipped with a display that supports 3D interaction on tabletop displays(Figure 2.20).

Figure 2 . 19 :

 219 Figure 2.19: Overview and details of the Hover Pad hardware setup (a) with details regarding the sliding carriages for x, y-motion (b), the telescope bars for vertical motion (c), and the display's frame for rotation (d), comprising two motors (i, iv), a controller board (iii), a battery pack (v), and 16 capacitive buttons (ii)

  spatially-aware menus and modes changing the role of devices based on their orientation or distance; cross-device flicking to transfer data between displays (Figure2.21).

Figure 2 . 21 :

 221 Figure 2.21: The HuddleLamp detects and tracks mobile devices and users' hands for ad hoc multi-device collaboration on desks [146]

  performance. Other multi-DOF input devices have been proposed in the literature, although they were not specifically designed nor tested in the context of MDEs. Their capabilities and the degrees of freedom they offer make them interesting candidates to fulfill MDEs' requirements. The Rockin' Mouse[START_REF] Balakrishnan | The Rockin'Mouse: integral 3D manipulation on a plane[END_REF] and the VideoMouse [84] have rounded shapes that allow tilting the device and thus, offer additional DOF in comparison to a regular mouse that can be used to perform multi-display tasks(Figure 2.22).

Figure 2 . 22 :

 222 Figure 2.22: The Rockin'Mouse [8] (left) and the VideoMouse [84] (right) The Roly-Poly Mouse (RPM) [140] uses a completely rounded bottom to augment the mouse's DOFs (Figure 2.23). It has been shown to provide larger amplitude of movement than previous tilting devices, and it also enables compound gestures (see Table1in 140) for a summary on the differences between RPM and previous tilting and multi-DOF mice).

Figure 2 . 23 :

 223 Figure 2.23: The Roly-Poly mouse [140] While the rounded dome-like shape of RPM offers multiple degrees of freedom, it hinders the device's stability. Unintended physical manipulations (e.g. Roll during Translation) are common on devices with such a form factor (cf. study 1 in [140]). The

  and environments. Immersive environments are no exception. Whether the interaction is Chapter 2 -Related work 55 integrated directly in the immersive device (HWD 10 , Smart glasses) [156], or deported to an external supporting device (Smartphones, Tablets, Wearables, interactive tabletops) [116, 58], several research focused on the use of tactile interactions for immersive visualizations. Rudi et al. [156], explored the design space for map interaction techniques on HMDs: they proposed tactile interactions to navigate a large map (Figure 2.24).

Figure 2 .

 2 Figure 2.24: A depiction of how control inputs (e.g., moving the mouse along the x-axis to the left or right) correspond to map interactions (i.e., moving the map along the same axis/in the same direction) for: (a) mouse controls, (b) touch controls, (c) haptic controls on OHMD, (d) head controls on OHMD [156] Giannopoulos et al. [115] went further, they mapped the input function of a Samsung VR headset 11 (Touchpad and a programmable back button) to the core functions offered by digital maps to design interaction technique to perform pan gesture, zoom gesture and selection of a point of interest in a map. Dane et al. [58] approach was based on the use of a tabletop for interaction with a large stereoscopic display. They proposed a widget based interface controlled by a set of interaction techniques to navigate 3D visualizations. The tasks covered include: data selection, controlling slicing planes and writing annotations. In a similar fashion, Claes et al. [118] use a multi-touch interactive tabletop to explore medical visualizations (Figure 2.25). Ji Sun's finger walking in place (FWIP)

Figure 2 .

 2 Figure 2.25:A miniature version of the 3D data appears to float in the air above the table surface. (A digital rendering is superimposed on the photograph to demonstrate the effect.) A cutting plane through the volume data is projected (like a shadow) onto the table below, where multitouch gestures are used to navigate and interrogate the data. After navigating to a useful view of these imaging data of a heart, the user is now defining a smooth 3D curve (e.g., the shape of a catheter delivery system) relative to the anatomical data set[118].

Chapter 2 -

 2 Related work 58 the map as well as several gestures specific to the task at hand (lowering water levels, viewing graphs ….).

Figure 2 . 26 :

 226 Figure 2.26: The user is wearing the HoloLens and using Kinect gestures to change the rotation of the model [198]

Chapter 2 -work 60 Figure 2 . 28 :

 260228 Figure 2.28: Variants of hand poses observed among gestures in [142]

Figure 2 . 62 Figure 2 . 30 : 63 Figure 2 . 31 :

 26223063231 Figure 2.29: A user manipulates the WIM using the physical clipboard and button-ball props [177] Schkolne et al. [162] proposed an immersive interface for designing DNA components for applications in nanotechnology (Figure 2.30). Their system uses tangible 3D input devices: a raygun tool: a tangible handheld object in the form of a gun, used for picking points in space; tongs: doubly sensed tongs that can detect strong and weak grabs, usedto move molecules; multipurpose handle tool: a handle containing an action button, a menu button and an embedded magnetic motion sensor, used to activate functions like drawing. A user study performed with scientist shows that they find the immersive interface and the tangible approach more satisfying than a 2D interface due to the enhanced understanding gained by direct interaction within the 3D space.

Chapter 2 -

 2 Related work 64 2.4.4 Mobile and Wearables devices Kharlamov et al. [107] argue that the current techniques for 3D selection in VR environments are not adapted to the requirements of such a task. They usually use head rotation followed by a dwell time or a click of a button on the headset to validate the selection.

  cursor and results in a miss-selection. To solve those potential problems,Kharlamov et al. [107] proposes TickTockRay (Figure2.32), a smartwatch-based raycasting technique for smartphone-based head mounted displays. The technique implements fixed-origin raycasting using off-the-shelf smartwatch hardware to perform selection in the virtual world. They proposed several approaches to confirm the selection: tapping on the screen of the smartwatch, a grabbing gesture, a finger-snapping gesture, and a poking gesture.

Figure 2 . 32 :

 232 Figure 2.32: TickTockRay enables freehand pointing in mobile VR using an off-the-shelf smartwatch [107] Benzina et al. [23] used a combination of a smartphone's touch capabilities and his sensors to develop a one-handed navigation technique in virtual environments. They use the touch capability of the smartphone for translations, and the sensors for rotation control. They developed four interaction techniques based on their approach: rotate by roll, rotate by roll with fixed horizon, rotate by heading and merged rotation. The interaction technique maps a certain number of DOFs of the phone to the VR app (+4).

Figure 2 .

 2 Figure 2.33: Steer Based Rotation Control Technique[START_REF] Benzina | Empirical evaluation of mapping functions for navigation in virtual reality using phones with integrated sensors[END_REF]

Figure 2 . 34 :

 234 Figure 2.34: Proposed hand-To-Face input for navigation by [167] includes: a) Panning, b) Pinch zooming, c) Cyclo zooming, d) Rotation zooming Dobbelstein et al. [59] proposed the use of the belt as a tactile surface to interact with HWD. Encircling the user's hip, the belt offers a wide input space. They mapped quickly accessible information and applications on the belt. With social implications in mind,

Figure 2 . 35 :

 235 Figure 2.35: Dobbelstein et al. [59] belt technique

Figure 3 . 1 :

 31 Figure 3.1: MIM map (left) and detail (right) illustrating the density and complexity of such graphs

Chapter 3 - 75 -

 375 for their applications (R1: Diversity, R2: Complementarity, R3: Decomposition, R4: Parsimony) Interaction with Large Displays Rules to help designers and users make design choices related to their multiple view system as well as to help usability experts and system evaluators pinpoint trouble spots in an existing system (R5: Space/Time resource optimization, R6: Self-Evidence, R7: Consistency, R8: Attention management)The Diversity rule (R1) indicates that a single view containing a multitude of diverse data and requiring the user to simultaneously assimilate may create significant cognitive overhead. The diversity of the data to visualize is one of the principal reasons to consider multiple view systems. The Complementarity rule (R2) states that another reason to consider multiple view systems is the need to understand the relation (correlations and/or disparities) between two components. The authors argue that multiple views leverage perceptual capabilities to improve understanding of relations among views. The Decomposition rule (R3) stipulates that partitioning complex data into multiple views create manageable pieces of information and allow a better understanding of the different dimensions composing it. The last rule of the first category, the Parsimony rule (R4),

  when there is a diversity of attributes, models, user profiles, levels of abstraction, or genres. R2 Complementarity Use multiple views when different views bring out correlations and/or disparities. R3 Decomposition Partition complex data into multiple views to create manageable chunks and to provide insight into the interaction among different dimensions. and temporal costs of presenting multiple views with the spatial and temporal benefits of using the views. R6 Self-Evidence Use perceptual cues to make relationships among multiple views more apparent to the user. R7 Consistency Make the interfaces for multiple views consistent, and make the states of multiple views consistent. R8 Attention management Use perceptual techniques to focus the user's attention on the right view at the right time.

Figure 3 . 2 :

 32 Figure 3.2: The three explored versions of split-focus With the 1-view technique (Figure 3.2, A), the split view occupies the entire tablet display; with 2-views (Figure 3.2, B), each view occupies half; and with 4-views a quarter (Figure 3.2, C). This design conforms to the rule of consistency (R7) as the overall detailed area size is consistent over the 3 versions of our technique and when several focus are displayed their relative size is consistent as well. It also presents different conditions of space/time resource allocation (R5): sequential for 1-view, and side-by-side for 2-views and 4-views.

Figure 3 . 3 :

 33

Figure 3 . 4 : 3 -

 343 Figure 3.4: Experiment setup

Figure 3 . 5 :

 35 Figure 3.5: Nodes repartition

3 -

 3 one tablet. The PC had a 23 inches display, showing the overview (1920x1080px). Nodes Chapter Interaction with Large Displays 81 on the overview measured 15x37px. The tablet was a 10.5 inches Samsung galaxy tab S 17

(

  NViews factor: 1V, 2V or 4V) and number of nodes to connect (NNodes factor: 2N, 3N or 4N) as factors. The NViews factor was counterbalanced by means of a 3x3 Latin square: three blocks were run, one for each value of the NViews factor. Trials in a block were grouped by the NNodes factor. Each subject performed 3 NViews x 3 NNodes x 10 predefined Node Positions x 3 repetitions = 270 trials. The training consisted of one block for each value of the NViews factor (36 trials in total). The experiment lasted 60 minutes on average.

Figure 3 . 6 :Chapter 3 -

 363 Figure 3.6: Trial completion time per number of nodes and number of views (95% IC). Friedman tests reveal a significant effect of the NViews on completion time for each number of nodes (2N: χ2(2) = 34.58, 3N: χ2(2) = 6.61, 4N: χ2(2) = 20.30 with p <.01).A Wilcoxon test confirms a significant difference between 1V and 2V (Z= -2.98, p <.01),

A

  Figure 3.7).

Figure 3 . 7 : 3 -

 373 Figure 3.7: Number of switches between the translation and detailed views (95% IC)

Figure 3 . 8 : 3 -

 383 Figure 3.8: Nb. of icons moved at the same time (95% IC)We could expect users to move 3 or even 4 icons simultaneously by using a bimanual multi-touch gesture under the 4V condition. This actually happened, but in low proportion: over the 1080 trials done with 4V, 20% were performed moving only one view icon at the same time, 77% moving two icons at the same time, 2% (22 trials) moving three and 0.5% (6 trials) moving four icons (the rest 0.5% of trials did not involve moving any icon). The same user did 15 of these 22 trials (75%) performed with 3 fingers. Five participants did the other 7 trials: they tried the gesture once or twice but did not use it any longer. The analysis of the 6 trials done with four fingers raises similar results: one subject did it 2 times, and four users tried it once. Instead, moving simultaneously two

Chapter 3 -Figure 3 . 9 :

 339 Figure 3.9: Examples of coherent (left) and incoherent (right) configurations of the split views.

  incoherent configuration from happening, the new version uses what we call locks. The overview is divided into 4 subregions of equal size, the principle is to lock each split view in a sub-region of the overview so that the spatial configuration of the split views on the tablet is always coherent with the icons on the overview (Figure 3.10, A). The top-left detailed view, represents, and can move only in the top-left sub-region of the overview.

Figure 3 .

 3 Figure 3.10: A possible solution to the coherence problem

Chapter 3 -Chapter 3 -

 33 Interaction with Large Displays 89 the same time. Most of them (77%) used a sequential approach, first using one finger of each hand to move two icons, and then moving the two remaining view icons. Interaction in a professional environment Design, usage and evaluation of TDome, a touch-enabled 6DOF interactive device for multi-display environments Chapter 4 -Interaction with Multienvironments (MDEs) are more and more prevalent in our daily life.Nowadays, interaction with MDEs is not limited to work environments only: they are used at home, with TVs, monitors, laptops, tablets and smartphones; in public places, where many displays have been installed in recent times, either to display commercials (outdoor advertising), give us directions (malls) or information about the delayed departure time of a train or plane (Airports, Train stations, …). Today, whether in a professional context or in our personal lives, we are overequipped with devices containing displays. The environments created by their combination is what we call multi-display environments. They combine multiple displays to offer the user another way of visualizing and interacting with information.

Chapter 4 -

 4 Interaction with Multi-Display Environments 95 tFigure 4.1: Objects used in the creativity session 4.2.1.2. We did not limit our list of items to those that are more likely to be available in our daily lives in a public context. Instead, we preferred to chose objects that are representative of the different categories of objects present in our daily lives.Apparatus

Figure 4 . 2 )

 42 . The interactive table, made by the company Immersion was 42 inches wide and had a resolution of 1920x1080. The two projectors had resolutions of 1920x1080 (Sony) and 1600x1050 (Sanyo). The monitor had a diagonal of 26 inches and a resolution of 1920x1080. Chapter 4 -Interaction with Multi-Display Environments 96

Figure 4 . 2 :

 42 Figure 4.2: The MDE setup used in the creativity session and the content displayed on each screen

Figure 4  4 -

 44 The control display (numbered 3 in Figure 4.2, Thumbnail MDE) contained a set of tools to manage the simulation (Figure 4.2, Thumbnail 3). The controls were Chapter Interaction with Multi-Display Environments 97 arranged horizontally and vertically one after the other. To simulate the energy consumption of a building, the user had to select the display in which the building was shown and manipulate the controls. ) aged 27 on average (SD = 8.58) from our laboratory participated to the creativity session. The group of participants was composed of 3 doctoral students, 4 master students and an assistant engineer. Participants had varying experience with tangible interactions, 2 of the participants had expertise in tangible interactions. 4 were involved in the neOCampus project.4.2.1.5. MDE TasksThe set of tasks described in this paragraph represents the most common tasks done in MDEs. In the context of our study, they translate as follows:

Chapter 4 -

 4 Interaction the material of the object Analogy Gesture based on an analogy to the use of another object Function Gesture based on the primary function of the object Other Any gesture that can not be classified in the

2 . 1 . 1 Chapter 4 -- 5 - 2 -

 211452 Interaction with Multi-Display Environments 102 The tasks (7) described in 4.2.1.The dimensions of the taxonomy (3) described in 4.2.The produced gestures -Users preference 4.2.3.1. The nature of the gesture

Figure 4 . 3 :

 43 Figure 4.3: The nature of the proposed gestures (Overall A, Per object B) The majority of the proposed interaction techniques are performed With the object (80.4%) followed by interaction techniques On the object (17.5%). Interaction techniques Around the object have been performed only 2.1% of the time (Figure 4.3.A). Interaction techniques On the object were mainly proposed for the badge object (52.0% of the gestures) and the plastic bottle (38.5% of the gestures). This distribution remains the same regardless of the task for which the ideas had to be generated (Figure 4.3.B).

Figure 4 . 4 :

 44 Figure 4.4: Gesture basis (Type of object, Object)

Chapter 4 -

 4 Interaction with Multi-Display Environments 104

Figure 4 . 5 :

 45 Figure 4.5: Human effectors (Objects)

Chapter 4 - 3 :Chapter 4 -Figure 4 . 6 : 4 -

 434464 Figure 4.6: Frequently proposed gestures

Figure 4 . 7 :

 47 Figure 4.7: TDome combines a small (a) or large (b) touchscreen with a dome-like mouse. TDome supports performing combined gestures (c), i.e. a 6 DOF physical manipulation followed by a touch input.

Chapter 4 -

 4 Interaction with Multi-Display Environments 112 4.3.1.1. Usage scenario

Chapter 4 -

 4 Interaction with Multi-Display Environments 113 illustrated in Figure 4.8, four different physical manipulations (Translations, Roll, Rotation and Lift-Up) can be combined with four different touch gestures (Tap, Drag, Pinch, Spread) for using TDome.

Figure 4 . 8 :

 48 Figure 4.8: TDome allows performing combined gestures, i.e. a physical manipulation followed by a touch gesture

Figure 4 . 9 :

 49 Figure 4.9: Arrangement of TDome elements for the Small version (left). Both TDome versions are rapidly interchangeable (right).

4. 3 . 4 . 1 .

 341 Physical relationshipTo fulfill the physical relationship and arrangement requirement, we implemented a semi-automatic acquisition of the displays layout in the MDE. This technique allows detecting the displays and building a radar view interface of them, which can be later exploited to interact with the displays of the environment.During the detection phase, TDome detects a QR code ascribed to each display (better recognition algorithms may not necessitate codes for detection as demonstrated byHuddleLamp [146]. The user orients TDome toward each display successively, so that the device's on-board camera detects the QR codes. Once the QR code is recognized, the user taps the touchscreen to terminate the identification: the detected display is assigned a position in the environment thanks to the incorporated IMU.

Figure 4 .(

 4 Figure 4.10: TDome's on-board camera detects displays (left) and creates a radar-view of the spatial layout (center). Then the user can select a display by Rolling + Tapping towards it (right). The user progressively creates a radar view describing the relative position of all detected displays, with TDome in its center (Figure 4.10-center). The user can manually adjust the distance of each display to TDome on the radar view. Once created, the radar view can be used with a Roll + Tap on TDome to select a specific display, by rolling TDome in the direction of the display and tapping on the touchscreen to validate (Figure 4.10-right).

a

  Roll + Drag on TDome to pan, and a Lift-Up + Pinch on TDome to zoom. Both touchscreen gestures (Drag and Pinch) are the same as what would be used on the distant touch display. While using only TDome's touchscreen gestures would be possible, using them in combination with the physical gestures (Roll or Translate) ensures a high recognition rate, prevents false positives, as demonstrated by our controlled study presented below and offers additional controls: the Roll angle may impact the panning speed.

Figure 4 .

 4 Figure 4.11: A Translation + Spread gesture sends the tabletop content (left) to a secondary display (right).

Figure 4 .

 4 12-left). The user creates the tunnel by sequentially selecting two displays on the radar view. Once the tunnel is defined, the user can move content along the tunnel with a Roll + Tap on TDome: rolling is performed in the spatial direction of the second display (i.e. a Roll to the right if the display is on the right of the first one); a Tap gesture finalizes the transfer.

Figure 4 . 12 :Chapter 4 -Figure 4 .

 41244 Figure 4.12: Using the virtual tunnel technique to transfer information between displays.

Figure 4 .Chapter 4 -

 44 Figure 4.14: privacy conservation when typing a password

( 4 -

 4 Roll and Translate in 8 different directions, Rotate in two directions, Lift-Up) in combination with four touch gestures (Tap, Drag, Pinch, and Spread). Pinch and Spread gestures being more complex to perform, participants repeated these gestures twice: once Chapter Interaction with Multi-Display Environments 122 with the dominant hand and once with the non-dominant hand (e.g. in a bi-manual setting).

Chapter 4 - 4 . 3 . 6

 4436 Interaction with Multi-Display Environments 123 Controlled experiment: Feasibility of TDome's combined gestures

Chapter 4 -Figure 4 .

 44 Figure 4.15: experimental context

1 Translation + 2

 12 ordered in a predefined way (Lift-Up, Translation, Roll and Rotation) because a random sorting would have made the instructions difficult to follow. Each combined gesture was repeated three times. Completing the four blocks took approximately 25 minutes. The study started with a training set made of the same four blocks as in the experiment. The training consisted of 94 trials and took approximately 20 minutes. After the training, each participant performed 192 trials: 144 trials for the Tap and Drag: 2 Displays x 12 Physical Manipulations (1 Translation + 8 Rolls + 2 Rotations + 1 Lift-Up) × 2 Touch gestures × 3 repetitions. 48 trials for the Pinch and Spread: 2 Displays x 4 Physical Manipulations (Rotations + 1 Lift-Up) × 2 Touch gestures × 3 repetitions.

  comfortably and efficiently performed: 16 results from the combination of a Roll (95.49% success rate) with Tap or Drag gesture, 36 results from the combination of a Translation (91.67% success rate), or Lift-Up (95.83% success rate) with one of the four touch gestures (Tap, Drag, Pinch, Spread) and 2 results from the combination of a Rotation with a Drag (98.61% success rate).

Figure 4 . 18 :

 418 Figure 4.18: Summary of the 17 (Small) + 54 (Large) combined gestures which offer a good usability and performance.

Figure 4 . 19 : 4 -

 4194 Figure 4.19: Percentage of users that chose each physical gesture on both versions of TDome for MDE tasks. Touch gestures While only the Tap gesture is feasible on the Small version, users selected different gestures on the Large version according to the task (Figure 4.20). For instance, Pinch and Spread were preferred for zooming (10/12), and Drag was preferred for sending

Figure 4 . 20 :Chapter 4 -

 4204 Figure 4.20: Percentage of users that chose each touch gesture on the Large version for MDE tasks

Chapter 4 -

 4 Interaction with Multi-Display Environments 134 gestures can help cluster them according to the physical manipulation, as shown in our mapping study. Further experiments are required to identify how such clustering can improve gestures or command memorization.

Chapter 5 -

 5 Interaction

Figure 5 . 1 :

 51 Figure 5.1: RPM's degrees of freedom

Figure 5 . 2 :

 52 Figure 5.2: Body parts

Chapter 5 -Figure 5 . 3 :

 553 Figure 5.3: Poses

Figure 5 . 4 :

 54 Figure 5.4: Place of motion

Figure 5 . 5 :

 55 Figure 5.5: Range of motion

Figure 5 . 6 :

 56 Figure 5.6: Length and Width trial

Figure 5 . 7 :

 57 Figure 5.7: Placement of the infrared tracking markers

Figure 5 . 8 :

 58 Figure 5.8: Forearm measurement

Figure 5 . 9 :

 59 Figure 5.9: Mouvement -Axes

Chapter 5 -Figure 5 . 10 :

 5510 Figure 5.10: Elbow and Wrist movements

Figure 5 . 11 :Figure 5 . 12 :

 511512 Figure 5.11: An example of distance covered for a group of gestures The Range of Motion was measured by calculating the average distance covered by RPM for each group of 10 successive trials (Figure 5.11). It was computed for each pair of Pose and Place of motion. As the size of the forearm differs from one participant to another, we standardized the collected data.

Chapter 5 -

 5 Interaction with Immersive Environments 151 they should not overlap each other. Therefore we study the dispersion of these exploited surfaces in the next section.

5. 4 . 5

 45 Dispersion of the translations performed on the width POM on the forearm This measure describes the distribution of the points of contact of RPM on the forearm (between the elbow and the wrist) for the complete experiment (i.e. 360 trials).

Figure 5 .

 5 Figure 5.13: Dispersion of the translations performed on the width POM (0: Elbow position, 100: Wrist position) The results show that there's a fairly large dispersion of the exploited surfaces (Figure 5.13). The surfaces exploited in the elbow POM and Middle POM overlap on the forearm subpart ranging from 30% to 60% (Figure 5.13); similarly, the surfaces exploited in the Middle POM and Wrist POM overlap on the forearm subpart ranging from 70% to 90%. However, it appears clearly that the translations performed on the elbow POM and Wrist POM were always done in distinct regions of the forearm throughout the experiment.

Chapter 5 -

 5 Interaction with Immersive Environments 152 5.4.6 Width POM: Device Rolls As the RPM device offers multiple degrees of freedom, namely: Translations, Rotations and Rolls, it was important to study the separability of these physical manipulations when used on the forearm. In the following, we will report on the involuntary Rolls of the device for each POM in each Pose when performing translations of RPM. The results are presented as averages accompanied by 95% confidence intervals.

Figure 5 . 14 :Figure 5 .Chapter 5 - 5 . 4 . 7

 51455547 Figure 5.14: Device rolls when performing translation on the width POM (degrees) Figure 5.14 shows that the translations performed in each one of the Width POM (performed around the forearm on its width) and in each Pose are systematically accompanied by a pronounced roll. The device is not maintained horizontally during translations. The results show that the rolls were more conspicuous in the Vertical pose where the average roll was approximately ~ 78°. This number decreases to about ~ 62° on average fo the Parallel pose and ~ 58° for the Forward pose. The results were constant for all POM (Elbow POM, Middle POM, Wrist POM).

Figure 5 . 15 :

 515 Figure 5.15: Device rolls when performing translation on the length POM (degrees) Results show that on average, the involuntary roll did not exceed 13° for the Forward pose, 8,2° for the Parallel pose and 2,7° for the Vertical pose (Figure 5.15). The Vertical pose clearly triggers less involuntary rolls than the other poses. It also seems that the Forward pose is the most prone to unwanted rolls. These findings are in line with the results of the studies conducted on RPM: the involuntary rolls of RPM when performing translations were of 12° on average [140].

Figure 5 .

 5 Figure 5.16: Fatigue

Figure 6 . 1 :

 61 Figure 6.1: Description space

Figure 6

 6 Figure 6.3: Description diagram of scenario A

Figure 6 . 4 :

 64 Figure 6.4: Scenario B, distant collaboration

Figure 6 . 6 :

 66 Figure 6.6: the local electrician is diagnosing the failures and preparing his intervention plan Scenario C2: If the local maintenance group is unable to intervene, the Construction and Study Department is contacted (CSD). This service deals with requests related to mandating external companies for services on the university campus. In this case, the service relates to intervening on the electrical failures described previously. Two possible sub-scenarios arise: -Scenario C21: The external company is assisted by the CSD. The CSD completes the external company's electrical expertise with its campus expertise and helps it generate an intervention roadmap (Figure 6.7).

Figure 6 . 7 :

 67 Figure 6.7: the CSD and the external electrician collaborating to diagnose the failures and prepare an intervention plan -Scenario C22: The external company intervenes by itself, without the assistance of the CSD (Figure 6.8). In this case, the company's electrician uses the system to consult the history of failures, the location of the failures, the equipment installed in each one of these locations and the current energy consumption data.

Figure 6 . 8 :

 68 Figure 6.8: the external electrician is diagnosing the failures and preparing his intervention plan without the assistance of the CSD Characterization User(s): In scenario C1, in which the local maintenance group intervenes, the electrician uses the system as a daily work tool which makes him a Frequent user.

Chapter 6 -Figure 6 . 9 :Figure 6 . 10 :Figure 6 . 11 :

 669610611 Figure 6.9: Description diagram of scenario C1

Figure 6 . 12 :

 612 Figure 6.12: Description diagram of the values overed by split-focus

Figure 6 .

 6 Figure 6.13: A concrete implementation of the split-focus solution in the neOCampus project (2 detailed views configuration)

6. 3 . 2 Figure 6 . 14 :

 32614 Figure 6.14: Description diagram of the values overed by TDome

Figure 6 .

 6 Figure 6.15: A possible use of TDome in the neOCampus project

4 that

 4 TDome is suitable to perform such tasks. In this context, TDome's translations facilitate panning on the map and the zoom function can be mapped to rotations. Sending the selection from the interactive tabletop to the calendar display involves content transfer Chapter 6the neOCampus project 183 between two displays, interacting with the second display involves input redirection. With a good understanding of the displays composing the MDE, TDome's tilting can be used to select the display to interact with. Its tunnel implementation (Chapter 4) can be used to transfer content from one display to the other.With an affordable and autonomous version of TDome, its use could be extended to open-access systems.

Figure 6 . 16 :

 616 Figure 6.16: Description diagram of the values overed by our on-body tangible interations approach

Chapter 6 -

 6 the neOCampus project 185 of the data by moving away from it, or a more detailed view by getting closer. He can also observe data from different angles if necessary.

Figure 6 .

 6 Figure 6.17: A possible use of the On-body tangible interactions approach in the neOCampus project

Chapter 7 -

 7 Conclusion

Chapter 7 -

 7 Figure 7.1: the split-focus technique

Chapter 7 -

 7 Figure 7.2: a) display registration using its embedded camera, b) device selection and c) cross-display data transfer

Chapter 7 -

 7 Conclusion & Perspectives 192 with multidimensional data. The use of the forearm as a support is motivated by its social acceptability, its large surface of interaction and its accessibility. The combination of the multi-DOF mouse and the forearm allows the user to move in his environment to explore data, improve the accuracy and avoid the inherent fatigue of mid-air interactions (Figure7.3).

Figure 7 . 3 :

 73 Figure 7.3: On-body tangible interaction for immersive data visualization

Chapter 7 -

 7 Conclusion

7. 2 . 1 . 1 .

 211 Large displaysIn our work on improving interaction with multiple regions of an overview simultaneously, we proposed an overview + detail visualization interface where multiple detailed views are displayed on a tablet. The visualization technique allows the translation of the detailed views in the overview through a view we called "translation view". It allows the users to move up to 4 detailed views simultaneously using multi-touch interactions (Figure7.1).

Chapter 7 -

 7 Conclusion

Figure 7 . 4 :Chapter 7 - 1 .

 7471 Figure 7.4: Possible posesAs an example, a pose between the forward pose and the parallel pose (Figure7.4) could allow the interaction with the two controllers/axes mapped to them simultaneously.

7. 2 . 2 . 3 .

 223 Integration of the proposed solutions in the neOCampus project A final long-term perspective consists in integrating the developed interaction techniques on the campus of the University of Toulouse. Indeed, we briefly touched upon potential usages of our contributions on the campus through 3 usage scenarios in Chapter 6 (neOCampus) and described their potential use. However, the pertinence of the designed solutions to the campus's challenges have been discussed and evaluated in controlled laboratory conditions. Any adaptation of the interactive solutions should be evaluated in-situ and in collaboration with the end users. d'aujourd'hui deviennent de plus en plus complexes à cause de la forte croissance de leurs volumes ainsi que leur multidimensionnalité. Il devient donc nécessaire d'explorer des environnements d'affichage qui aillent au-delà du simple affichage de données offert par les moniteurs traditionnels et ce, afin de fournir une plus grande surface d'affichage ainsi que des techniques d'interaction plus performantes pour l'exploration de données. Les environnements correspondants à cette description sont les suivants : Les écrans large ; les environnements multi-écrans (EME) composés de plusieurs écrans hétérogènes spatialement distribués (moniteurs, smartphones, tablettes, table interactive …) ; les environnements immersifs. Dans ce contexte, l'objectif de ces travaux de thèse est de concevoir et d'évaluer des solutions d'interaction originales, efficaces et adaptées à chacun des trois environnements cités précédemment. Une première contribution de nos travaux consiste en Split-focus : une interface de visualisation et d'interaction qui exploite les facilités offertes par les environnements multi-écrans dans la visualisation de données multidimensionnelles au travers d'une interface overview + multi-detail multi-écrans. Bien que plusieurs techniques d'interaction offrent plus d'une vue détaillée en simultané, le nombre optimal de vues détaillées n'a pas été étudié. Dans ce type d'interface, le nombre de vues détaillées influe grandement sur l'interaction : avoir une seule vue détaillée offre un grand espace d'affichage mais ne permet qu'une exploration séquentielle de la vue d'ensemble ; avoir plusieurs vues détaillées réduit l'espace d'affichage dans chaque vue mais permet une exploration parallèle de la vue d'ensemble. Ce travail explore le bénéfice de diviser la vue détaillée d'une interface overview + detail pour manipuler de larges graphes à travers une étude expérimentale utilisant la technique Split-focus. Split-focus est une interface overview + multi-détails permettant d'avoir une vue d'ensemble sur un grand écran et plusieurs vues détaillées (1,2 ou 4) sur une tablette.Une seconde contribution de nos travaux consiste en TDome : un dispositif d'interaction en entrée et sortie conçu pour pallier le manque de techniques d'interaction dédiées aux environnements multiécrans. Sa base semi-sphérique lui permet d'offrir plusieurs degrés de liberté sous la forme de manipulations physiques(inclinaison, rotation, translation). Le dispositif est augmenté d'un écran tactile qui lui permet d'afficher de l'information et de détecter des entrées tactiles. La combinaison de manipulations physiques et de gestes tactiles fait de lui un dispositif robuste et augmente l'espace de gestes qu'il offre. Ceci lui permet de répondre aux multiples besoins des environnements multi-écrans tout en évitant la multiplication de dispositifs dans l'espace de travail et, notamment, la détection des écrans dans l'espace de travail, la sélection d'écrans, le transfert de données entre écrans et l'interaction avec des écrans distants.Enfin, une troisième contribution de ces travaux consiste en l'introduction d'une nouvelle approche d'interaction avec des visualisations immersives. L'approche combine 1) l'usage d'un dispositif sans-fils de type souris multi-DOF, alliant la précision inhérente à une souris, la souplesse d'utilisation du tangible et les larges capacités de contrôle propres aux dispositifs à multiples degrés de liberté et 2) l'utilisation du corps de l'utilisateur pour guider les déplacements du dispositif et exploiter ainsi la proprioception de l'utilisateur, tout en limitant la fatigue musculaire propre aux interactions en l'air.Tout au long de ce travail de recherche, les techniques d'interaction conçues ont été concrètement illustrées pour l'exploration de données de consommation énergétique dans le contexte de neOCampus, un projet de l'université Paul Sabatier qui a pour objectif d'améliorer le confort au quotidien pour la communauté universitaire tout en diminuant l'empreinte écologique des bâtiments et en réduisant les coûts de fonctionnement (fluide, eau, électricité…).

  Interaction Homme-Machine ; Données complexes ; Ecrans larges ; Environnements multi-surfaces ; Environnements immersifs ; Dispositif à plusieurs degrés de liberté. Discipline: Computer Science Institut de Recherche en Informatique de Toulouse -UMR 5505 Université Paul Sabatier, 118 route de Narbonne, 31 062 Toulouse Cedex 4, France

  6.3.3 On-body tangible interactions for immersive data visualization ................................ 6.4 Conclusion ................................................................................................... 7 Conclusion & Perspectives .............................................................188 7.1 Thesis summary ........................................................................................... 7.2 Future work .................................................................................................7.2.1 Medium-term ............................................................................................................. 7.2.2 Long-term ...................................................................................................................

Table 3 .

 3 1: Summary of Baldonado et al.'s [10] guidelines for using multiple views in information visualization .................................................................................................

Table 4 .

 4 1: Most common tasks in MDEs adapted to our creativity session's scenarios.

Table 4 .

 4 2: Taxonomy .....................................................................................................

Table 4 .

 4 3: Preferred interaction techniques and objects for each task ..........................

Table 5 .

 5 1: Borg Scale .....................................................................................................

Table 3 .

 3

1: Summary of Baldonado et al.'s [10] guidelines for using multiple views in information visualization

  Interaction with Large Displays 82 a break if required between trials. At the end of the experiment, participants were asked to fill a System Usability Scale questionnaire (SUS).

	17 https://www.samsung.com/fr/tablets/galaxy-tab-s/

18

https://eu.dlink.com/fr/fr/products/dir-615-wireless-n-300-router

Table 4 .

 4 1: Most common tasks in MDEs adapted to our creativity session's scenarios.

Table 4 . 2 :

 42 Taxonomy

Table 5 .

 5 5 -Interaction with Immersive Environments

	Rating	Perceived Exertion
	6	No exertion
	7	Extremely light
	8
	9
	10
	11
	12
	13
	14
	15	Hard
	16
	17	Very Hard
	18
	19	Extremely hard
	20	Maximal exertion
		154

1: Borg Scale

https://en.wikipedia.org/wiki/Peer-to-peer

Virtual Network Computing (VNC) is a graphical desktop sharing system that remotely controls another computer (https://en.wikipedia.org/wiki/Virtual_Network_Computing).

https://www.microsoft.com/en-us/hololens

https://www.oculus.com/rift/

http://www.metavision.com/

https://epson.com/moverio-augmented-reality

http://www.i-montres.net/simvalley-aw-414-go-un-smartphone-android-au-poignet/

https://www.samsung.com/uk/smartphones/galaxy-s4-i9505/GT-I9505ZKABTU/

An open framework that defines a common protocol and API for tangible multitouch surfaces (see https://www.tuio.org/).

https://www.microsoft.com/en-us/hololens

http://www.metavision.com/

https://epson.com/moverio-augmented-reality

The sense of the relative position of one's own parts of the body and strength of effort being employed in movement (see https://en.wikipedia.org/wiki/Proprioception).

https://www.irit.fr/neocampus/

Elipse is an interdisciplinary research group (computer scientist, neuroscientist, HCI specialist) focusing on Advanced forms of Interactive Techniques as research tool and research object. https://www.irit.fr/-Equipe-ELIPSE-

http://www.agence-nationale-recherche.fr/

Figure 6.2: Scenario A, energy consumption visualization.

Alemayehu Seyed (2013). Examining User Experience in Multi-Display Environments

Acknowledgments

Results

A Shapiro-Wilk test established that the data was not normal and we could not normalize it. Therefore we used a Friedman test (we report χ2 and p) to compare more than 2 conditions, and Wilcoxon tests otherwise (we report p value). Where appropriate, we used a Bonferroni correction.

We first discuss the success rate for the Small and Large versions separately as a Wilcoxon test showed a significant effect of Display on the success rate (p <.001).

Success rate: Large version

When using the Large version, we found no significant effect of Touch gestures (Friedman: χ2=3.87, p=0.2) or Physical manipulations (Friedman: χ2=4.1, p=0.2) on the success rate. Overall, success rate with the Large version was 94.44%. Errors were distributed among Physical Manipulations (2.52%) and Touch gestures (2.86%).

A Friedman test reveals a significant effect of Touch gesture on the success rate when performing a Rotation (χ2=13.32, p=.003): a Wilcoxon test reveals a significant difference between Tap and Drag (81.94% vs. 98.61%; p=.022) and between Tap and Pinch (81.94% vs. 97.22%; p=.045). A Friedman test reveals a significant effect of Touch gestures on the success rate when performing a Lift-Up (χ2=16, p=.001), a Translation (χ2=15.

Completion time

A Wilcoxon test did not show any difference between the Small and the Large versions (p=.08). Overall, it took participants 2.5 seconds to perform a combined gesture. While we found some differences across gesture combinations, all of them are compatible with the micro-interactions concept [START_REF] Daniel | enabling mobile microinteractions[END_REF], i.e. fast interactions that take less than 4s completion:

all times ranged between 2.1s and 2.7s.

Unintentional touches

We recorded unintended touches on the Small and Large versions. Overall, results

were similar for both versions: we detected unintentional touches in 2% of the trials.

These touches did not necessarily raise errors. The sequential use of a touch interaction after a physical gesture prevents from launching a command unintentionally.

Subjective feedback

When considering the physical manipulations, results show that with the Small version, more than 50% of the participants found easy or very easy (4 or 5 on Likert scale) to perform a combined gesture involving a Roll, Translation or Lift-Up. In the case of the Large version, more than 75% of participants rated these gestures as easy or very easy.

When considering the touch gestures, we observed that with the Small version more than 50% of participants found difficult or very difficult (1 or 2 on Likert scale) to perform a combined gesture involving a Spread or Pinch. With the Large version, 60% or more of the participants found easy or very easy to perform combined gestures involving any kind of touch gesture.

Summary

Results reveal differences between the Small and Large versions (Figure 4.18). With the Small version, the experiment reveals that 17 combined gestures can be comfortably and efficiently performed: those based on the combination of a Roll (8 directions), a (two modes, discrete two-state tasks). In the following section, we describe two usage scenarios that illustrate a possible and concrete use of on-body tangible interactions.

Usage Scenarios

Taking into consideration our findings, we will illustrate a detailed use of the Onbody tangible interaction approach presented in this chapter, through two concrete scenarios: 1) an interior design scenario where we show a possible use of the approach to manipulate simple 3D objects; 2) a more elaborated scenario detailing a possible use of on-body tangible interactions to interact with multidimensional data.

Interior design

Jeremy is an interior designer. He rethinks the interior of his clients' homes to make them more functional and pleasant by choosing the appropriate furniture. To this end, Jeremy visualizes the furniture in the final space using a mixed reality headset (Hololens).

He manipulates RPM on the body to move, rotate and scale the virtual furniture.

To move the furniture, Jeremy uses RPM on the Length POM on the forearm. He puts his forearm on the pose representing the movement to be made and adjusts the position of the piece of furniture by performing translations of RPM over the length of his forearm. For example, if he wants to bring the furniture closer to him, he places his forearm in the pose Forward and performs translations of RPM over the length of his forearm, from the wrist to his elbow, whereas the Parallel pose allows him to move the piece of furniture to the right/left. The Vertical pose allows him to adjust the height of a photo frame or a mirror for example.

Jeremy can also rotate furniture using the same principle as for translations. He puts his forearm in the pose representing the axis in which to make the rotations and perform translations around the forearm, on the Width POM, according to the direction of the rotation to make.

Data exploration

Emily is an air traffic controller. Part of her work consists of improving traffic management in the control tower [START_REF] Hurter | FromDaDy: Spreading Aircraft Trajectories Across Views to Support Iterative Queries[END_REF] (analyzing past conflicts, improving the ecological Emily uses a mixed reality headset (Hololens) to visualize the data and our device (RPM) to interact with it. The tasks Emily performs on the data are [START_REF] Cordeil | Immersive solutions for future Air Traffic Control and Management[END_REF][START_REF] Hurter | FromDaDy: Spreading Aircraft Trajectories Across Views to Support Iterative Queries[END_REF] selecting data using range-sliders; applying a command on the selected data (e.g. data subsampling); changing colors; scaling, etc. Emily has configured her system so that each pose of the forearm represents a coordinate in the immersive visualization: the latitude is represented by the Forward pose, the longitude by the Parallel pose and the altitude by the Vertical pose.

Range sliders are controllers that allow the user to select values included in a range (an interval). The range sliders are composed of two cursors, one defines the minimum value and the second defines the maximum value. To control the range slider and select data, Emily uses translation over the length of the forearm (on the Length POM). The cursor to manipulate is automatically selected according to the starting position of RPM on the forearm: if RPM is placed on the Wrist POM, the cursor defining the maximum value is manipulated; if RPM is placed on the Elbow POM, the cursor defining the minimum value is manipulated; and finally, if RPM is placed in the Middle POM of the forearm, the two cursors are moved simultaneously while maintaining the range length initially defined.

Characterization

User(s): the LS agent uses the system daily as his principal work tool. In this situation, the user is a Frequent user.

Service(s) used: To accomplish his task, the LS agent visualizes different types of data. The service provided by the system in this situation is a Visualization service.

Exploited data: the LS agent uses data relative to his activity to find a suitable conference room for the lecturer.

Deployment context: the system is not available to the public and is dedicated to the work of the LS agent. It is a Dedicated-access one.

Description diagram: The characterization translates to the following description diagram (Figure 6.5):

Abstract

Today's ever-growing data is becoming increasingly complex due to its large volume and high dimensionality: it thus becomes crucial to explore interactive visualization environments that go beyond the traditional desktop in order to provide a larger display area and offer more efficient interaction techniques to manipulate the data. The main environments fitting the aforementioned description are: large displays, i.e. an assembly of displays amounting to a single space; Multi-display Environments (MDEs), i.e. a combination of heterogeneous displays (monitors, smartphones/tablets/wearables, interactive tabletops…) spatially distributed in the environment; and immersive environments, i.e. systems where everything can be used as a display surface, without imposing any bound between displays and immersing the user within the environment. The objective of our work is to design and experiment original and efficient interaction techniques well suited for each of the previously described environments.

First, we focused on the interaction with large datasets on large displays. We specifically studied simultaneous interaction with multiple regions of interest of the displayed visualization. We implemented and evaluated an extension of the traditional overview+detail interface to tackle this problem: it consists of an overview+detail interface where the overview is displayed on a large screen and multiple detailed views are displayed on a tactile tablet. The interface allows the user to have up to four detailed views of the visualization at the same time. We studied its usefulness as well as the optimal number of detailed views that can be used efficiently.

Second, we designed a novel touch-enabled device, TDome, to facilitate interactions in Multidisplay environments. The device is composed of a dome-like base and provides up to 6 degrees of freedom, a touchscreen and a camera that can sense the environment. Having a unique device for interaction in these environments limits the homing effect when switching from one device to another and leads to a coherent set of interactions with the MDE, contributing to a more fluid task flow, a key element in such environments.

Finally, we introduced a new approach to interact in immersive environments with complex data.

It is based on the use of the forearm as a physical support to assist tangible interactions with a multidegrees of freedom device. We proposed a design space for this approach and we validated its feasibility through an experiment aimed at establishing the range, stability and comfort of gestures performed in this new paradigm.

All along this research work, resulting interaction techniques and environments have been concretely illustrated for exploring energy consumption data in the context of neOCampus, a project of the University of Toulouse 3 that aims at exploring the Campus of the Future, i.e. a smart, innovative and sustainable campus.