This manuscript would not exist in this state without other researchers that played a big part in the work that is presented here. To Géry Casiez, thank you for working with us on several occasions, for your "CHI touch", and for letting me continue your haptic legacy with the DigiHaptic! To Hiroyuki Kajimoto, thank you so much for welcoming me to your laboratory in Tokyo. Those were two fantastic months that I enjoyed greatly, both scientifically and culturally. I would also like to warmly thank the labmates in Kaji-lab for their kindness, the numerous parties, and the karaoke. More locally, I would also like to thank Ferran Argelaguet for his many advices and his mitten skills as well as Adrien Girard for his help in running extra-long user studies and his 3D modeling skills. I would also like to thank the many people that I met at Inria, researchers, engineers, staff, and other PhD students, especially in the Hybrid and Mimetic teams. Special thanks to my office mates, by order of appearance. Thank you Fabien for teaching me the hard truth about the kickflip button. Thank you Anthony for graduating a year before me and writing a thesis that served as a great inspiration! Thank you Adrien for the bike tips and many many parts. Thank you Benoît for your constant good mood and good luck with the physics simulation ;) To conclude, I thank my friends and family for their continuous support I would especially like to

1

Introduction

This thesis, entitled "Contributions to the Design of Novel Hand-based Interaction Techniques for Virtual Environments", presents research on improving 3D interaction with the hands in the context of Virtual Reality.

Using our hands in virtual environments

The hand, a primary tool of interaction

The human hand is an astonishing tool that serves as one of our proxy with the physical world. Indeed, it enables a variety of essential tasks, from manipulating objects to exploring our surroundings through touch, or interacting socially and communicating with gestures. In fact, the hands are rarely still and even actions that may appear passive, like reading a book, involve subtle finger motions, such as slightly bending pages and adjusting our grip. The finesse with which such actions are undertaken is due to two essential and interdependent features of the hand, its dexterity and its sensitivity to touch.

First, the hand possesses a great flexibility that allows us to act on our surroundings through a wide range of gestures and grasps. It is capable of accomplishing tasks that require either precision, such as writing with a pen, or strength, like using a screwdriver [START_REF] Bibliography Napier | Hands. Natural science[END_REF]. The combined use of both hands to conduct bimanual tasks such as opening a jar or playing a musical instrument demonstrates their capabilities furthermore.

Secondly, the hand lets us feel our environment thanks to its acute sensitivity to touch. At the smaller scale, its skin embeds a dense distribution of receptors for perceiving minute features such as texture and roughness [START_REF] Kolb | An Introduction to Brain and Behavior, Third Edition[END_REF]. With the fingers, we are also capable of assessing the shape and material of objects without even looking at them. The wrist and the arm contribute too, as they actively participate in the perception of their weight and inertia during manipulation.

Introduction are produced by the simulation in response to the user's actions [START_REF] Sherman | Understanding Virtual Reality: Interface, Application, and Design[END_REF]. A convincing simulation within which one can interact naturally and receive coherent sensations may induce a sense of presence, reinforcing the "extent to which one feels present in the virtual environments, rather than in the immediate physical environment" [START_REF] Steuer | Defining virtual reality: Dimensions determining telepresence[END_REF].

Virtual Reality supports a variety of interaction modalities. For example, one can act in a virtual environment (VE) via typical computer peripherals, like a joystick, or through voice command. However, simulations that aim for realism may benefit from interaction modalities that relate more closely to our daily, physical lives. In that regard, it seems essential to enable the natural use of our hands and fingers in virtual environments.

Hand-based interaction refers to the field of research that deals with the transposition of the capabilities of the hand into virtual environments so that tasks can be carried out with the same ease and efficiency as in a real setting.

Hand-based interaction in Virtual Reality

Efforts for virtualizing the human hand date back to the early beginnings of computer graphics when, in 1972, Ed Catmull digitalized his own hand in the making of the first computer-animated movie (Figure 1.1a). The labor involved in this project is a testimony to the complexity of the hand, since capturing its shape required to manually tesselate a plaster cast and input each of its vertices to the computer [START_REF] Price | The Pixar Touch: The Making of a Company[END_REF]. Even though the produced animations were quite rough, this first model led the way for the more elaborate -and interactive -hand representations that followed.

As advances in computer graphics enabled real-time 3D interaction, representations of the users' hands were quickly incorporated into virtual environments. At first, such models kept a rigid posture and mostly served a positional role (Figure 1.1b). Even so, giving users the ability to directly reach into the virtual world to manipulate its content was a significant step forward in terms of engagement. Subsequently, articulated models with mobile digits were introduced to leverage the dexterity of the human hand and let users reproduce the behaviors of their daily lives (Figure 1.1c and 1.1d). (a) Catmull's hand was digitalized for the first computer-animated movie [START_REF] Price | The Pixar Touch: The Making of a Company[END_REF]. (b) Early virtual environments involved static hands for reaching virtual objects [START_REF] Poupyrev | The go-go interaction technique: Non-linear mapping for direct manipulation in VR[END_REF]. (c,d) Articulated hand models leverage the individual mobility of the digits to enable actions such as pointing [START_REF] Fisher | Virtual environment display system[END_REF]] and pinching [START_REF] Bibliography Iwata | Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator[END_REF]. (e) Deformable models provide realistic visuals and a more truthful interaction by reproducing the softness of the skin [START_REF] Gourret | Simulation of object and human skin formations in a grasping task[END_REF]].

Hence, dedicated haptic interfaces seem necessary to establish a physical bridge between users and virtual environments. They serve the dual role of transmitting their motor actions to the simulation and then sending back stimuli that relate to our sense of touch [START_REF] Srinivasan | Haptic interactions in the real and virtual worlds[END_REF], as illustrated in Figure 1. [START_REF] Gribnau | Comparing single-and two-handed 3D input for a 3D object assembly task[END_REF]. Such interfaces may take various forms, the most common being desktop haptic arms that apply forces to the user's hand via a handle (Figure 1.2c). Alternatively, multi-finger exoskeletons that are attached to the hand and constrain the fingers individually are able to simulate more intricate contacts with virtual objects (Figure 1.2d). Yet, as of today, hand-based haptic interaction seems in practice restricted to a handful of select applications because of the difficulty for interfaces to accommodate the complexity of the hand.

Challenges of hand-based haptic interaction

For improving 3D interaction with virtual environments, tailored interaction techniques that match the capabilities of the hand, both in terms of control and in terms of perception are required. In this thesis, we focus on two main challenges of hand-based interaction: (1) accommodating the many degrees of freedom of the hand, and (2) providing convincing haptic sensations with common interfaces. (c) Desktop haptic arm that outputs forces to the user's hand [START_REF] Massie | Design of a three degree of freedom force-reflecting haptic interface[END_REF]. (d) Multifinger exoskeleton for simulating grasping forces on several fingers (CyberGlove Systems, USA).

of the user's hands. Even so, their complexity and high cost restrict their use to professional applications. In consequence, there is a need for novel user interfaces that efficiently handle the many degrees of freedom of the hand.

A complementary question is the relevancy of the different degrees of freedom of the hand for 3D interaction. In fact, we consistently employ a limited number of grasps, which makes exhaustive controls over all DoF unnecessary. This is further motivated by anatomic factors, like the strong dependency between successive finger joints, which makes controls over their individual states redundant for most applications. There are also functional factors to consider, such as the specific role of each finger and the respective duties of dominant and non-dominant hands. Hence, there is a need for interaction techniques that take into account the specificities of the hand to ease virtual manipulation.

Providing convincing haptic sensations

Hand-based interaction, and especially object manipulation, involves complex haptic sensations, the simulation of which requires dedicated interfaces. Cutaneous sensations are typically provided by small actuators, like vibration motors, which limits their bulkiness. Conversely, proprioceptive sensations, such as weight and collisions, call for more cumbersome interfaces for delivering forces to the whole hand.

Affordable desktop devices that provide forces through a handle can constrain the position of the hand, but this is insufficient for simulating the complex efforts involved in grasping. Moreover, their grounded nature and small range generally confines users in a small workspace. Contrarily, multi-finger exoskeletons provide more mobility and render complex sensations to each finger but the bulkiness of such apparatus does not make for a seamless Virtual Reality experience and their higher complexity entails a high cost. Thus, in practice, such interfaces are restricted to a handful of select applications.

Haptic feedback can alternatively be supplied by simpler passive props that gener-

.3 -Hand-based interaction between a user and a virtual environment. The haptic interface maps the user's motor actions to a virtual representation of his hand.

The physical interaction between virtual hand and virtual objects is sent back to the user as forces.

ate haptic cues only through their shape and material. Nevertheless, passive haptics is limited by a lack of control over the simulated haptic properties as, unlike with active interfaces, the provided feedback cannot be controlled in real-time by the simulation. For that purpose, multimodal effects that play on the tight coupling between visual and haptic channels, like pseudo-haptic feedback, can influence the users' perceptions and produce a richer interaction. Such techniques are however mostly limited to simple interaction cases and pseudo-haptics have been scarcely used for hand-based manipulation.

Thesis objectives

The goal of this thesis is to improve hand-based interaction in virtual environments. We consider two main axes of research: (I) the design of novel methods for improving the control of articulated hand models and (II) the design of new approaches that combine passive haptics and pseudo-haptics for hand-based interaction.

Those axes of research and the resulting contributions are illustrated in Figure 1. [START_REF] Mcneely | Voxel-based 6-DOF haptic rendering improvements[END_REF].

Axis I: Improving the control of articulated hand models

Harnessing the many degrees of freedom of realistic hand models requires matching human-computer interfaces. Nevertheless, general-purpose tracking systems suffer from practical limitations when dealing with the hand and costly specialized hardware is currently required. As a result, domains such as computer animation, video games, or desktop Virtual Reality cannot readily benefit from multi-finger interaction. Thus, a general theme of this first axis is to leverage more accessible hardware for controlling virtual hands. To do so, we propose two strategies: (1) reducing the degrees of freedom of virtual hand models in order to ease their control, and

(2) separating their degrees of freedom between several interfaces to better

.4 -Our contributions to hand-based interaction. Our first approach (Axis I) is to ease the control of virtual hands by reducing or separating their degrees of freedom so that alternative interfaces can be leveraged. Our second approach (Axis II) is to combine passive haptics and pseudo-haptic feedback to provide sensations when interacting with the hands in virtual environments.

distribute controls and feedback.

First, controlling virtual hands could be achieved with consumer-ready interfaces or easy-to-assemble setups. For instance, multi-touch input, nowadays commonplace in mass-market electronics, seems a promising candidate since it enables to interact with several fingers in parallel. Still, it has not been exploited for that purpose yet because of the lack of an obvious correspondence between 2D input and the actions of an articulated virtual hand. Hence, our first objective is to reduce the degrees of freedom of hand models to adapt them to multi-touch input.

Secondly, haptic feedback plays an essential part in object manipulation but costly force feedback exoskeletons are required for simulating the sensations involved in multifinger interaction. Conversely, desktop haptic interfaces are reasonably accessible but they only apply forces to the whole hand, without considering the separate fingers. However, combining several such desktop devices might cover more degrees of freedom and produce a more truthful interaction. Hence, our second objective is to propose methods that separate the controls of a virtual hand, as well as the resulting force feedback, between several interfaces handled in parallel.

Introduction

Axis II: Combining passive haptics and pseudo-haptics for hand-based interaction

The use of force feedback devices is currently limited by their complexity, cost, and bulkiness. Passive feedback, on the other hand, only relies on minimalist props to provide haptic cues but it has a low flexibility compared to active haptics. Pseudo-haptic feedback can alleviate those issues and complement passive props with visual effects but it has been scarcely used for hand-based manipulation. Hence, a general theme of this second axis is to leverage a combination of passive haptics and pseudohaptics as an alternative to the use of complex active devices. We consider three types of interaction, from a low to a high level of detail: (1) coarse arm-based interaction, (2) object grasping with the hand, and (3) fine multi-finger manipulation.

First, haptic tasks that rely on haptic devices to provide forces to the whole hand and arm are often limited by their narrow workspace and grounded nature, as they are mostly tailored to desktop interaction. Some "human-scale" haptic interfaces can accompany users in large workspaces but they are usually unwieldy, such as bulky exoskeletons or room-sized systems. On the other hand, passive haptics could provide a similar feedback as those active devices, with added comfort thanks to the lightweight components it relies on. Thus, our third objective is to propose a body-mounted passive interface that reproduces the feedback of grounded haptic devices in larger workspaces.

Secondly, most haptic devices are limited in their ability to simulate object grasping with the fingers as well as the resulting manipulation. Indeed, basic desktop arms do not support such interaction at all and multi-finger exoskeletons are burdened with complicated mechanical designs. However, generic prehension does not always require such precision as some postures, especially those bearing strength, keep the fingers grouped together. In such cases, both the user interface and the virtual hand representation can be substantially simplified. Thus, the next objective is to simulate grasping with a passive grip force interface, associated to pseudo-haptics in order to simulate variable haptic properties and enrich the interaction.

Finally, some manipulation scenarios do require the inputs of all digits. This is the case for haptic exploration tasks in which the properties of an object must be evaluated by touch, like medical palpation. Thus our last objective is to explore how the combination of passive and pseudo haptics can apply to interaction with all the digits in parallel. This could be achieved by designing a passive exoskeleton that constrains each finger individually, associated to a multi-finger pseudo-haptic effect that varies the perceived stiffness over the surfaces of virtual objects.

Introduction

Approach and contributions

The remainder of this manuscript is organized as follows. Chapter 2 presents related work on hand-based interaction. First, the underlying mechanics of the human hand are described to outline its mobility and its sensitivity to touch. Secondly, we give an overview of existing representations of the users' hands in virtual environments as well as methods for simulating their interaction with virtual objects. Then, user interfaces that couple the users' hands and their virtual counterparts are presented, with an emphasis on devices that provide haptic feedback. Finally, additional 3D interaction techniques that expand the range of actions and the perceptions of users are described.

Then, in the remainder of the manuscript, each of our two axes of research is addressed in a separate part. In Part I, we focus on the first objective, improving the control of articulated hand models.

In Chapter 3, we focus on reducing the many degrees of freedom of realistic hand models to exploit common interfaces. We present THING, a tablet-based approach that leverages multi-touch interaction for a quick and precise control of a 3D hand's pose. The flexion/extension and abduction/adduction of the virtual fingers can be controlled for each finger individually or for several fingers in parallel through sliding motions on the surface of the tablet. Two variants of this method are proposed: an integrated version that maps the spatial location and orientation of the tablet to the virtual hand, and a hybrid version with joint mouse controls for a desktop use. The usability of THING is compared to traditional mouse-only controls and to a data glove in a user evaluation.

In Chapter 4, we focus on separating the degrees of freedom of articulated hand models between several interfaces. We thus propose DesktopGlove, a bimanual setup that distributes controls and feedback so that one virtual hand could be controlled by both user's hands. Hence, one hand is in charge of global displacements while the other handles finger movements for grasping. Force feedback is similarly distributed between them to denote various haptic effects, which allows for rich haptic sensations that could otherwise only be achieved with costly grounded exoskeletons. A user study evaluates the performance and user appreciation of this approach compared to a traditional data glove. We also investigate which hand should control which device as well as the most appropriate distribution of force feedback between both hands.

In Part II of this manuscript, we address the second axis, improving haptic sensations resulting from hand-based interaction. To do so, we leverage passive haptics and pseudo-haptic feedback to enhance tasks of increasing complexity, from coarse arm-based interaction to precise manipulation with the fingers.

Chapter 5 considers coarse interaction with the hand and tackles the small workspace issue often associated with desktop haptic devices. It introduces the Elastic-Arm, a modile armature that constrains the user's hand relatively to his body and generate haptic feedback without restraining his freedom of movement. Several illustrative use Introduction cases based on well-known interaction techniques that initially relied on grounded devices are reimplemented with our approach to expand their original workspace. First, methods for reaching distant objects and navigating in the virtual environment are presented. Then, perceptual effects are proposed for simulating collisions with virtual obstacles and variable efforts when interacting with virtual objects thanks to pseudohaptic feedback. Finally, a user study evaluates a pseudo-haptic effect as well as the user appreciation for the Elastic-Arm.

Chapter 6 then considers grasping and object manipulation with the whole hand. It introduces the Virtual Mitten, an interaction metaphor that uses an elastic handheld device for grabbing objects. User compress the device through grasping gestures to hold virtual objects, which generates haptic cues. We leverage pseudo-haptic feedback to simulate different levels of grip force through this elastic device. Several use cases are presented to illustrate how the Virtual Mitten can apply to a variety of haptic tasks and to more complex bimanual scenarios. Perceptual and subjective evaluations are conducted to assess the capabilities and user appreciation of this approach, as well as the output range of the elastic device.

Chapter 7 considers multi-finger manipulation. It introduces ElasticGlove, an elastic exoskeleton that enables multi-finger interaction and delivers haptic feedback to the digits separately. An accompanying pseudo-haptic approach enriches its passive feedback and emulates various levels of stiffness for each finger, so that virtual objects with heterogeneous stiffness could be simulated. To illustrate this approach, several practical use cases, like a medical palpation simulator and a musical learning scenario are proposed.

Finally, Chapter 8 concludes this manuscript and discusses short-term future work for each of our contributions as well as long-term perspectives that relate to the field of hand-based interaction.

This thesis was partly supported by the MANDARIN project1 from the French National Research Agency (ANR-12-CORD-0011), whose aim is to develop new software and hardware solutions to support dextrous interaction for training in car maintenance procedures through Virtual Reality.

2

Related work

Related work

Figure 2.1 -Hand-based interaction between a user and a virtual environment. 1 The user interacts directly with his hand through a dedicated interface. 2 A virtual representation of the user's hand conducts the interaction in the virtual environment. [START_REF] Gribnau | Comparing single-and two-handed 3D input for a 3D object assembly task[END_REF] The haptic interface couples the actions of the user's hand with those of the virtual hand and provides force feedback. [START_REF] Mcneely | Voxel-based 6-DOF haptic rendering improvements[END_REF] Additional techniques expand the possibilities of interaction and the perception of the virtual environment.

This chapter presents an overview of previous work that relates to hand-based interaction. It follows the steps outlined in Figure 2.1. Firstly, the mechanical and sensory aspects of the human hand are described in order to identify which capabilities haptic interfaces and Virtual Reality simulations must accommodate. Then, we present possible virtual representations for the users' hands as well as simulation techniques that enable interaction with virtual objects. Next, the various types of interfaces that couple the users' hands with their virtual counterparts, for driving their actions and simulating touch, are presented. Finally, we present additional 3D techniques for improving the possibilities of interaction as well as user's perception.

The human hand

This section describes the complex underlying structure and function of the human hand. First, the anatomic features of the hand that grant it its many degrees of freedom (DoF) are presented and taxonomies that classify common human grasps and postures are discussed. Then, the hand as a perceptive organ is considered and we describe the two main sensory channels involved in object manipulation: cutaneous sensibility and proprioception.

The human hand

Mechanics of the hand

Structure and mobility

The human hand is a tightly packed structure housing many bones and joints, which grants it a high degree of articulation [START_REF] Bibliography Napier | Hands. Natural science[END_REF]. As shown in Figure 2.2a, the wrist comprises eight small bones that connect the hand to the forearm. Then, metacarpal bones under the palm begin the chains of joints leading to the fingertips. At this point however, interlocking prevents them from moving independently from each other, except for the thumb which can oppose the other digits [START_REF] Taylor | The anatomy and mechanics of the human hand[END_REF]. Finally, the proximal phalanxes branch out to form the fingers, each one followed by a medial and then a distal phalanx.

The hand is a powerful appendage, capable of tight grips, and yet it benefits from a small volume thanks to remote muscles located in the forearm [START_REF] Taylor | The anatomy and mechanics of the human hand[END_REF]]. Among those extrinsic muscles are the flexors and extensors, which are responsible for finger curling and emanate from the humerus, near the elbow [START_REF] Moran | Anatomy of the hand[END_REF]. They are followed by tendons that run through the wrist up to the fingertips (Figure 2.2b). Likewise, the muscles that govern wrist rotations, for approaching objects from a variety of angle, lie in the same area. However, lateral finger movements, called adduction and abduction (respectively toward and away from the middle finger), rely on smaller intrinsic muscles that are located between the metacarpal bones. As for the thumb, it has both extrinsic muscles, for handling its extension and abduction, and intrinsic muscles, for realizing flexion and opposition.

Those degrees of freedom are bound by anatomic constraints. For example, proximal phalanxes can rotate no more than 90°in average whereas medial phalanxes can reach 110° [START_REF] Lin | Modeling the constraints of human hand motion[END_REF]. Muscular interdependencies between adjacent parts of the hand also alter certain movements of the fingers. For instance, bending a distal joint makes the preceding joints bend too. Similarly, curling the little finger makes the neighboring fingers slightly curl.

The many degrees of freedom of the hand are enumerated in Figure 2.2c. Every finger has two degrees of freedom at its root for flexion/extension and abduction/adduction. The two joints that follow and lead to the fingertip each have one DoF in flexion, for a total of four degrees of freedom per finger. The thumb is more complex and possesses five DoF thanks to a special saddle joint that enables rotations toward the palm and plays an essential role in prehension [START_REF] Bibliography Napier | Hands. Natural science[END_REF]. Finally, the wrist has six DoF defining its position and orientation in space, which gives a total of 27 degrees of freedom in the human hand [START_REF] Elkoura | Handrix: animating the human hand[END_REF]].

Common grasps

The hand is a high-dimensional structure which mechanical properties enable many configurations. However, only a reduced number of manipulation postures are consistently used in our day-to-day lives. Consequently, taxonomies have been proposed to formalize the manner in which we manually interact with our environment. In his early classification, Schlesinger identified six recurring patterns: the cylindrical, tip, hook, palmar, spherical and lateral grasps [START_REF] Taylor | The anatomy and mechanics of the human hand[END_REF]] (Figure 2.3). This taxonomy suggested that the posture of the hand depends on the shape and size of the manipulated object. [START_REF] Napier | The prehensile movements of the human hand[END_REF] later proposed that grasps were also a function of the nature of the intended task. The example of a wooden rod was given, as it can be used both for hammering and for writing, with very different postures. Therefore, Napier distinguished power grasps that provide stability and intensity with the palm, from precision grasps that provide dexterity through the thumb and the fingertips.

Mixed classifications covering a wider range of grasping postures have since been constructed using those foundational elements. [START_REF] Cutkosky | Modeling manufacturing grips and correlations with the design of robotic hands[END_REF], for example, first differentiate power grasps from precision grasps and then consider shape and size. Non-prehensile postures such as holding a plate with an open hand were also introduced in this classification (Figure 2.4). [START_REF] Bullock | Classifying human manipulation behavior[END_REF] built another extensive taxonomy, which additionally encompasses grasps involving subtle "withinhand" motion like when writing with a pen or rolling a ball on a surface. At the higher level, tasks are separated between those with contact (manipulation) and those without (gestures) and then, are taken into account: prehension, the global hand motion, and the relative motion of the manipulated object (Figure 2.5).

Contrary to those exhaustive classifications, some authors rather focused on the geometry of grasps and especially on the contact patterns between hand and object. For example, [START_REF] Lyons | A simple set of grasps for a dextrous hand[END_REF] proposed three primitive grasps: the encompass grasp when the object is wholy enveloped, the lateral grasp when the object is held between the fingerpads, and the precision grasp when the fingertips are involved. He additionally introduced the concept of virtual finger, a functional block of digits acting on the The human hand [START_REF] Taylor | The anatomy and mechanics of the human hand[END_REF]. This classification differentiates grasps depending on the shape and size of the manipulated objects. same part of an object that can be abstracted as a single finger in order to simplify grasp analysis furthermore [START_REF] Arbib | Coordinated control programs for movements of the hand. Hand function and the neocortex[END_REF]. Another taxonomy taking into account the geometry of the contacts between hand and objects is the contact web [START_REF] Kang | Grasp recognition using the contact web[END_REF].

Other taxonomies focused on the efforts involved in object manipulation. For instance, [START_REF] Iberall | The nature of human prehension: Three dextrous hands in one[END_REF] considered the opposing forces at play in grasps and proposed three categories, depending on the parts of the hand that apply forces: pad opposition, palm opposition, and side opposition. [START_REF] Bloomfield | A taxonomy and comparison of haptic actions for disassembly tasks[END_REF] constructed another classification that considers interaction forces for generic haptic tasks. They classified actions according to the mechanical forces that they involve: aligned forces (e.g. pushing an object), non-aligned forces (e.g. sanding a surface), aligned torques (e.g. using a screwdriver), and non-aligned torques (e.g. pulling a lever). This particular taxonomy is further developed in Chapter 6.

A comprehensive comparison of various classifications counted a total of 33 different grasps [START_REF] Feix | A comprehensive grasp taxonomy[END_REF]. However, grasp usage primarily depends on the context, and they are not all used equally. For instance, the daily work of a machinist often involves precision grasps whereas housekeeping tasks rather require power grasps [START_REF] Bullock | Grasp frequency and usage in daily household and machine shop tasks[END_REF]. Moreover, the digits are not used equally either: the index finger and the thumb are the most used digits, and the frequency of use of the others decreases from the middle finger to the little finger [START_REF] Gonzalez | Analysis of hand contact areas and interaction capabilities during manipulation and exploration[END_REF].

Haptic senses

Haptics comes from the Greek word haptós which means "able to come in contact with". The haptic senses refer to the perception of the variety of sensations that relate to touch: tactile sensibility, the perception of our limbs in space and the stress that they undergo, but also equilibrium, pain, and temperature [START_REF] Kolb | An Introduction to Brain and Behavior, Third Edition[END_REF]. Those various signals are all integrated by the sensomatory system, giving us a global haptic image of our physical state and that of our environment.

This section presents an overview of the sensory aspect of the human hand and focuses on the predominant sensations involved in object manipulation: cutaneous sensibility and proprioception. The human hand

Cutaneous sensibility

Cutaneous sensibility enables the perception of small-scale features such as texture and roughness. These sensations stem from mechanoreceptors, nerve endings encapsulated inside of corpuscles that deform under stress, located below the superficial layer of the skin [START_REF] Dubin | How the Brain Works[END_REF] (Figure 2.6, Left). Structural variations in the capsule tissue make the four types of receptors react to different stimuli [START_REF] Kolb | An Introduction to Brain and Behavior, Third Edition[END_REF]. For instance, Meissner's corpuscles are sensible to changes in velocity. Pacinian corpuscles are sensible to vibration and light touch. Ruffini's corpuscles are sensible to skin stretch and detect the direction of forces. Merkel's disks are sensible to pressure and smallscale shapes, which helps in feeling the edges of objects. These mechanoreceptors can be found all over the body in varying quantities but the hands are among the most densely populated areas, which grants them a particularly acute sensibility [START_REF] Maciel | Multifinger haptic rendering of deformable objects[END_REF].

The types of mechanoreceptors differ on other properties than the stimuli they respond to (Table 2.1). First, they can be distinguished according to their adaptation rate, which is the speed of the transition from excited state to neutral state. Rapidly adapting receptors quickly return to their neutral state. Thus, they are not fit to detect the material properties of a surface from a static observation; which explains why we slide our fingers over a surface in order to perceive its texture. On the contrary, slowly adapting mechanoreceptors detect if an event is continuously occurring. They play an essential role in gauging the weight, balance, and slippage of the objects that we manipulate in order to ensure secure grasps.

Mechanoreceptors are also characterized by their input frequency, which corresponds to the speed at which separate stimuli can be detected. Rapidly adapting mechanoreceptors can respond to events between 20 and 300 Hz whereas slowly adapting receptors are typically limited to 10 Hz [START_REF] Burdea | Virtual Reality Technology[END_REF]. Additionally, the size of the receptive field, which corresponds to the skin area in which a contact is detectable by a single touch receptor, is larger for Pacinian and Ruffini's corpuscles. However, the larger is the receptive field, the lower the spatial resolution is [START_REF] Vallbo | Properties of cutaneous mechanoreceptors in the human hand related to touch sensation[END_REF].

Proprioception

Proprioception, or kinesthesia, is the perception of the motion of our limbs in space as well as the perception of the stress that they undergo; sensations which are essential for interacting with our hands [START_REF] Biggs | Haptic interfaces. Handbook of virtual Environments[END_REF].

At the limb level, proprioceptive information comes from mechanoreceptors in the joint, the muscles, and the tendons. First, the same Pacinian and Ruffini's corpuscles that can be found below the superficial layers of the skin are also located in the joints between bones (Figure 2.6, Right). The amplitude of the signal that they send to the brain informs about the joint angles between limbs whereas its frequency informs about their angular velocities [START_REF] Burdea | Force and touch feedback for virtual reality[END_REF]. Then, internal and external forces that our muscles are subjected to are evaluated by the Golgi organs located at the junction between muscles and tendons. They measure muscle tension, stabilize heavy grasps, and additionally play an inhibiting role by relaxing muscles when tension is too high. Other receptors, the muscle spindles, are located inside of the muscles of the arms and hands, and measure their length [START_REF] Kolb | An Introduction to Brain and Behavior, Third Edition[END_REF], which determines the shape and rigidity of the objects that we manipulate. The various mechanoreceptors of the skin responsible for cutaneous sensibility also play an indirect role in proprioception since extension/flexion of the limbs may result in a stretching/bulging of the skin adding extra information about our haptic state. At a higher level, the global position and orientation of each of our limbs is evaluated from the vestibular system in the inner ear. Canals filled with fluid subject to gravity inform about the orientation of the head. The perceived state of the rest of the body then depends on the proprioceptive links between head, neck, trunk and limbs.

The maximum frequency at which a human finger can apply output forces is between 5 and 10 Hz [START_REF] Brooks | Telerobotic response requirements[END_REF]. Regarding the input bandwidth, variable numbers were proposed: [START_REF] Sharpe | Technical and human operational requirements for skill transfer in teleoperations[END_REF] suggested that it could go as high as 10 kHz and [START_REF] Brooks | Telerobotic response requirements[END_REF] supposed that a finger cannot discriminate two consecutive force signals above 320 Hz. It is however admitted that a input frequency between 20 and 30 Hz is a minimum for meaningful perception [Shimoga, 1993a].

Virtual hands

Conclusion

The hand is a powerful tool for both acting on our surroundings and perceiving their physical state. It benefits from a great flexibility thanks to a highly articulated skeleton coupled to remote muscles located in the arm. Hence, the hand can adopt a wealth of configurations but, in practice, only a few postures are consistently used and research showed that we choose which grasp to employ depending on the shape of the considered object but also on the nature of the task that is carried out.

The hand also benefits from a high sensitivity to touch that helps in coordinating movements. It can sense small-scale cutaneous stimuli, such as pressure, stretching, and vibrations via mechanoreceptors embedded in the skin. Thanks to other receptors in the muscles, tendons and joints, we perceive the shape, stiffness, and weight of manipulated objects.

Transposing the finesse of the human hand in virtual environments is quite a technical challenge. The next section addresses two aspects of this endeavor: the virtual hand models that serve as proxy to interact in the virtual world and the software simulation methods that reproduce the potentially complex interplay between those hand models and virtual objects.

Virtual hands

This section addresses the challenge of transposing in virtual environments the complex structure, mobility, and function of the human hand. First, possible virtual representations of the user's hands, with different degrees of fidelity, are presented. Then, we give an overview of software techniques for simulating the interplay between those hand avatars and virtual objects.

Representing the hand in virtual environments

One or both user's hands must be embodied in some manner in a virtual environment in order to make the manipulation of its contents possible. It might seem ideal for these models to reflect the exact topology and dynamics of a real human hand but other factors must be considered, such as computational cost, the user interface to be used, or the task to be carried out. Therefore, there exists a variety of hand representations, spanning from minimalist rigid proxies for simple haptic exploration to realistic virtual hands capable of interacting precisely with their fingers.

Rigid proxies

The simplest interaction scenarios may consist of basic manipulation tasks in which some virtual objects are selected and coarsely moved around. In such cases, minimalist rigid proxies that represent the user's hand may be sufficient. They can take the form of abstract 3D cursors [START_REF] Boeck | Using the nondominant hand for selection in 3D[END_REF][START_REF] Bibliography Vanacken | Evaluating the influence of multimodal feedback on egocentric selection metaphors in virtual environments[END_REF] for showing the position of the hand, or volumes like arrows or cones [START_REF] Boeck | Multisensory interaction metaphors with haptics and proprioception in virtual environments[END_REF][START_REF] Swapp | Interaction with co-located haptic feedback in virtual reality[END_REF] to additionally communicate where it points (Figure 2.7a).

For a more engaging and explicit interaction, those rigid proxies can be given the appearance of a human hand. For example, in a 3D editor, [START_REF] Houde | Iterative design of an interface for easy 3-d direct manipulation[END_REF] displayed hand-shaped 2D cursors taking various postures to illustrate possible actions such as picking or rotating objects. [START_REF] Jáuregui | Design and evaluation of 3D cursors and motion parallax for the exploration of desktop virtual environments[END_REF] displayed a 3D hand over a 2D image and changed its size and orientation to express depth and relief (Figure 2.7b). In an immersive 3D environment, [START_REF] Poupyrev | Manipulating objects in virtual worlds: Categorization and empirical evaluation of interaction techniques[END_REF] displayed a static hand model at the position of the users' hand so that they could directly reach and grab virtual objects. [START_REF] Lindeman | Towards usable VR: an empirical study of user interfaces for immersive virtual environments[END_REF] integrated a pointing hand with an extended finger as a proxy for selecting items in a 3D menu. Similarly, the MaxwellWorld scientific visualization application [START_REF] Craig | Developing Virtual Reality Applications: Foundations of Effective Design[END_REF][START_REF] Dede | Maxwellworld: Learning complex scientific concepts via immersion in virtual reality[END_REF]] featured a pointing hand, as well as a flat non-dominant holding a 3D menu (Figure 2.7c). With all those rigid models however, the interaction is limited to rough pick-and-place or pointing tasks since no real grasping strategies can be implemented as they lack separate fingers.

Fingertips only

Interacting through a single rigid proxy representing the whole hand does not reflect the richness of hand-based interaction since the individual actions of the fingers cannot be reproduced. Thus, some simulations may display a number of fingertips, each one represented by a separate proxy capable of interacting with the virtual environment, which enables new interaction possibilities, including grasping. For example, [START_REF] Alhalabi | Medical training simulation for palpation of subsurface tumor using HIRO[END_REF] simulated a breast palpation procedure by representing each fingertip as a rigid sphere visible in the virtual environment (Figure 2.8a). [START_REF] Popescu | Virtual reality simulation modeling for a haptic glove[END_REF] placed cylindrical interaction meshes at the positions of the user's fingertips to enable basic grasping. The visual and physical components of the virtual model were decoupled: the meshes responsible for the interaction are invisible and a detailed visual model is displayed instead (Figure 2.8b). Similarly, [START_REF] Zhou | A realistic force rendering algorithm for CyberGrasp[END_REF] modeled a fingertip as a cluster of line segments radiating toward the fingerpad in order to mimic its shape. For reproducing the surfacic contacts that are characteristic of finger-based interaction, Talvas et al. [2013] proposed the God-Finger method. This algorithm emulates the softness of real fingerpads by spawning additional contact points distributed around a proxy representing a fingertip (Figure 2.8c).

Articulated hands

Displaying realistic models to embody the user's hand rather than an abstract representation has been shown to contribute to task performance as well as to the feeling of presence [START_REF] Durlach | Effect of variations in sensory feedback on performance in a virtual reaching task[END_REF][START_REF] Pusch | Effects of hand feedback fidelity on near space pointing performance and user acceptance[END_REF]. Moreover, a larger range of grasps is available if the action of the digits and palm is taken into account. Consequently, articulated hand models that more closely match the appearance of real hands have been used in various applications, with various degrees of fidelity.

Point-based models that represent the main parts of the hands as interaction points have been proposed. For example, [START_REF] Bibliography Iwata | Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator[END_REF] used a virtual hand based on 16 strategically placed anchor points (4 per finger and 1 for the palm) and [START_REF] Maciel | Multifinger haptic rendering of deformable objects[END_REF] designed a molecular virtual hand made of spheres at its joints and fingertips (Figure 2.9b). Denser representations have been proposed to reduce gaps; for instance, [START_REF] Holz | Multi-contact grasp interaction for virtual environments[END_REF] built a virtual hand filled with spherical sensors that follow the skeleton posture and generate collisions (Figure 2.9c). Likewise, [START_REF] Hirota | Dexterous object manipulation based on collision response[END_REF] proposed to represent each digit as a dense set of interaction points (Figure 2.9e).

Other models are made up from the assembly of primitive volumes. For simplicity,

Related work

earlier hand models had clamp-like representations with only a thumb and index finger to support basic pinching [START_REF] Buchmann | Fingartips: gesture based direct manipulation in augmented reality[END_REF][START_REF] Maekawa | Haptic display for object grasping and manipulating in virtual environment[END_REF]] (Figure 2.9a). Alternatively, more versatile multi-fingered hands can be the combination of basic shapes, such as cylinders [START_REF] Bibliography Funahashi | A model for manipulation of objects with virtual hand in 3D virtual space[END_REF], spheres [START_REF] Hui | Hands on a virtually elastic object[END_REF], or boxes [START_REF] Fisher | Virtual environment display system[END_REF]] (Figure 2.9d). For a better visual rendering, those interaction models may be hidden while visual pleasing representations are overlaid on top of them [START_REF] Ullmann | Intuitive virtual grasping for non haptic environments[END_REF]]. More truthful models may comprise articulated parts that approach the shape of real phalanxes (Figure 2.9f and 2.9g), in which case the same model is used for the visuals and the physics simulation [START_REF] Borst | A spring model for whole-hand virtual grasping[END_REF][START_REF] Walairacht | 4 + 4 fingers manipulating virtual objects in mixed-reality environment[END_REF]. In other cases, for example in immersive VR rooms, users already see their own hands and displaying the virtual model is not necessary [START_REF] Lok | Effects of handling real objects and avatar fidelity on cognitive task performance in virtual environments[END_REF]. Alternatively, when the virtual environment is displayed through a head-mounted display, video capture of the user's hand may be overlaid in the virtual scene [START_REF] Pusch | Effects of hand feedback fidelity on near space pointing performance and user acceptance[END_REF]. To emulate the soft nature of the hand, some hand models support skin deformation, which provides more convincing visuals and reflects more accurately the physics of

Simulating the interaction between virtual hands and virtual objects

Software models are required to simulate the potentially intricate interplay between virtual hands and virtual objects. Those can be distinguished into two categories: (1) techniques based on heuristics that explicitly define the different steps of the interaction with custom rules, and (2) physically-based techniques that implement natural physical phenomena so that interaction naturally emerges.

Heuristic-based manipulation

Simulations based on heuristics, or pseudo-physical simulations, express rules that govern under which conditions the interaction begins, how it occurs, and when it ends.

In the simplest case, the motion of the user's hand is simply mapped to the grasped object [START_REF] Mine | Isaac : A virtual environment tool for the interactive construction of virtual worlds[END_REF] but more elaborate methods take into account the posture of the hand to reflect real manipulation more truthfully. For example, [START_REF] Wan | Realistic virtual hand modeling with applications for virtual grasping[END_REF] attached an object to the user's hand if its posture corresponds to the global shape of the object, such as a palmar pinch with a cylindrical object or a spherical grasp with a spherical object (Figure 2.11a).

Related work

Many heuristics-based methods consider the contacts between the hand and virtual objects. For instance, with his virtual hand made of 16 anchor points, [START_REF] Bibliography Iwata | Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator[END_REF] started a grasp when a point from the thumb and any other point were touching the same object (Figure 2.11b). [START_REF] Moehring | Enabling functional validation of virtual cars through natural interaction metaphors[END_REF] defined more generally a grasping pair as two digits in contact with the same object that satisfy the friction cone test, which checks if a pair of digits clamping an object can ensure its equilibrium or if the grasp is too fragile, depending on the contact normals and material properties of the object. The motion of the manipulated object during the grasp is then function of the motion of the digits and it ends when the distance separating them increases more than a threshold. A similar heuristic-based method in which thumb, fingers, as well as palm are precisely identified was proposed by [START_REF] Ullmann | Intuitive virtual grasping for non haptic environments[END_REF]. A grip is determined by a friction cone test too but it is only considered valid if it involves the thumb and one or more fingers or the palm and one or more fingers, which is reminiscent of Iberall's classification based on grasp primitives [START_REF] Iberall | The nature of human prehension: Three dextrous hands in one[END_REF]. The authors also introduce rules to enable bimanual manipulation when opposed palms clamp an object. Oppositely to explicit rules that identify hand parts, Holz et al. [2008] proposed a general heuristic based on spherical sensors located within the virtual hand. Friction cone tests also validate sensor pairs creating a clamp but the varied placement of sensors allow for a wider range of manipulation postures, like the cigarette grasp since neighbor sensors of adjacent fingers can create a grasping pair (Figure 2.11c). Grasping pairs are constantly recalculated to enable finger readjustments and several objects can be carried simultaneously. [START_REF] Kijima | Representative spherical plane method and composition of object manipulation methods[END_REF] developed a set of rules for handling complex manipulation. Without contact, the behaviors of virtual objects are governed by a physical simulation but when a two-or three-fingers grasp is detected, the objects follow the motion of a sphere defined by the fingertips. [START_REF] Talvas | Novel interactive techniques for bimanual manipulation of 3D objects with two 3DoF haptic interfaces[END_REF] proposed to simulate the manipulation of virtual entities through a hybrid technique based on a heuristic condition enforced by physical rules: if two antagonist forces pressing against an object are detected, then a magnetic pinch begins. The two interaction points that represent fingertips (or hands) are then pulled toward each other by an invisible spring so that they clench the virtual object.

Physically-based manipulation

Physically-based simulations use motion laws from Newtonian mechanics to confer a natural behavior to the virtual environment. Phenomena such as gravity, friction, and collision response are implemented, which enables the manipulation of objects. Contrary to heuristic-based simulations that require new rules to support more interaction cases, with physics simulation, manipulation possibilities naturally emerge. For example, such simulations generally allow users to grab several objects with the same hand or to interact with the rest of the environment through the grasped object. However, such simulations may be computationally costly and special care must be taken to obtain a stable simulation since grasps may involve many antagonist forces.

In general, hand models based on rigid proxies or assemblage of articulated bodies can thus be handled with generic physics software packages [START_REF] Borst | Realistic virtual grasping[END_REF]; an object is grasped if the hand adopts the posture that corresponds to its approaximate shape [START_REF] Wan | Realistic virtual hand modeling with applications for virtual grasping[END_REF]. (b) Anchor points: the object is grasped if it touches a point from the thumb and any other point [START_REF] Bibliography Iwata | Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator[END_REF]. (c) Sensors: the object is grasped if sensors form a valid grasping pair [START_REF] Holz | Multi-contact grasp interaction for virtual environments[END_REF]. [START_REF] Zaeh | Dexterous object manipulation in a physics based virtual environment[END_REF]. However, due to the nature of multi-finger interaction, simulation models specifically adapted to the nature of the virtual hand that conducts the interaction have been proposed. For instance, [START_REF] Bergamasco | A realistic approach for grasping and moving virtual objects[END_REF] handled the interaction between a virtual hand and other entities with grids of points on the palmar side of each phalanx and on the palm. The distance between all objects and a gross control point at the wrist is first computed to check if a more precise collision detection is warranted. In this case, intersection volumes are approximated from the penetration of colliding points and repulsive forces grouped into surfaces are generated (Figure 2.12b). [START_REF] Hasegawa | Real-time rigid body simulation based on volumetric penalty method[END_REF] explicitely calculated intersection volumes between two colliding entities. The separation forces calculated are then distributed over the contact areas to avoid the "jiggling" effect occurring when a separation force is applied to an arbitrary position during surface/surface collisions. One particularly essential aspect of dexterous manipulation is the softness of the hand, which allows for stable grasps and could thus be reproduced in virtual environments. To simulate the interaction with their virtual hand made of 1200 interaction points, Hirota and Hirose [2003] computed repulsive forces with respect to the penetration of each point into an object of the scene. [START_REF] Talvas | The god-finger method for improving 3d interaction with virtual objects through simulation of contact area[END_REF] proposed the god-finger method to simulate surface/surface contacts from a single point of interaction. Upon collision with an object, its geometry is traveled to dynamically place new sub-interaction points in a circular fashion around the original contact position. This area is deformed depending on the amplitude and the tangential component of the original force to better mimick the action of an elastic fingerpad against a rigid object, without costly deformation models.

Additional simulation techniques handle the interaction with deformable objects. [START_REF] Popescu | Virtual reality simulation modeling for a haptic glove[END_REF] employed a simple deformation model in which the geometry of an object is locally interpolated between a reference non-deformed version and a maximum deformation state depending on finger penetration. [START_REF] Burdea | Virtual reality graphics simulation with force feedback[END_REF] enabled deformation by a virtual hand by collapsing the object geometry according to finger penetration and a predefined object-dependent model. [START_REF] Maciel | Multifinger haptic rendering of deformable objects[END_REF] proposed a molecular model that represents non-rigid entities, and the hand, as large mass-spring systems. The global force applied on each mass is the sum of cohesion forces, gravity and external forces due to collisions with the rest of the virtual world.

Conclusion

For interacting in virtual environments, the user's hands must be represented in some manner in the virtual environment. For simpler interaction cases, rigid proxies taking the form of 3D cursors or static hand models might be enough to coarsely manipulate virtual objects. When more accurate interaction must be carried out, articulated models with fingers can be leveraged. Simpler models only comprise fingertips to grasp objects and the more elaborate ones feature articulations, realistic appearances, and deformable skin.

Simulation models are required to enable physical interaction between the virtual hands and virtual objects. Techniques based on heuristic rules aim at reproducing the dynamics of grasping and object manipulation from custom rules, which enables a plausible interaction with simple algorithms at the expense of physical correctness. Conversely, physically-based simulations implement the laws of physics to more realistic and versatile interaction such as manipulating several objects simultaneously but it requires specialized methods that take into account the specificities of hand-based manipulation, for instance the surfacic contacts of the fingertips.

The next section deals with the interfaces that couple the motion of the user's hands with their virtual representations, with an emphasis on devices that provide haptic feedback to simulate the touch of virtual objects.

User interfaces for hand-based haptic interaction

User interfaces for hand-based haptic interaction

This section focuses on the interfaces that enable haptic interaction with the hands. First, we consider motion capture interfaces that record the user's motor actions to transpose them in the virtual environment. Then, we present an overview of existing interfaces that provide haptic feedback through either tactile stimulation or force feedback.

Motion capture interfaces

Motion capture refers to "the process of capturing the motion of a human body at some resolution" [START_REF] Moeslund | Interacting with a virtual world through motion capture[END_REF]. Motion capture is well-known for its use for movie special effects, where actors wear full-body suits covered with markers to reconstruct the 3D trajectories of their limbs. It can similarly be used to interact in real-time within virtual worlds (Figure 2.13). However, it shows limitations in dealing with the fingers as their high-level of articulation requires a dense distribution of tracking markers, which creates occlusions. As a result, traditional motion capture of the hand may be limited to only one or two fingers [START_REF] Sheng | An interface for virtual 3D sculpting via physical proxy[END_REF] or to contexts in which wide hand motions are avoided [START_REF] Kry | Interaction capture and synthesis[END_REF].

For whole hand input, which requires only the position and orientation of the user's hand without considering the fingers, simpler 6-DoF spatial interfaces can be leveraged [START_REF] Laviola | Flex and pinch: A case study of whole hand input design for virtual environment interaction[END_REF]. Early examples of such devices include Ware's Bat [START_REF] Ware | Using hand position for virtual object placement[END_REF][START_REF] Ware | Using the bat: A six-dimensional mouse for object placement[END_REF]] and the Polhemus 6-DoF tracker [START_REF] Agrawala | 3D painting on scanned surfaces[END_REF][START_REF] Schmandt | Spatial input/display correspondence in a stereoscopic computer graphic work station[END_REF]. Nowadays, this type of spatial input is even incorporated in mass-market products, for instance in video game peripherals such as the Wiimote [START_REF] Kuntz | Low-cost and home-made immersive systems[END_REF][START_REF] Wingrave | The Wiimote and beyond: Spatially convenient devices for 3D user interfaces[END_REF].

When the motion of the fingers is needed, data gloves can be leveraged [START_REF] Dipietro | A survey of glove-based systems and their applications[END_REF]. Basic models detect pinching motions between different fingers [LaViola and Zeleznik, 1999] and more sophisticated gloves measure the continuous flexion and adduction of each finger to faithfully reflect the user's motion in the virtual environment [START_REF] Sturman | A survey of glove-based input[END_REF][START_REF] Zimmerman | A hand gesture interface device[END_REF] (Figure 2.13b). Although they seem ideal for hand-based interaction, the high cost of data gloves diminishes their accessibility. Moreover, their cumbersome nature also interferes with other tasks, such as typing on a keyboard or using a mouse in parallel.

Instead, markerless optical tracking enables multi-finger input without instrumenting the hand thanks to recent progress in computer vision. For instance, [START_REF] Wang | Real-time hand-tracking with a color glove[END_REF] used a regular camera to retrieve the user's posture from a database of pre-recorded hand postures, and [START_REF] Oikonomidis | Efficient model-based 3D tracking of hand articulations using kinect[END_REF] used a low-cost depth camera to directly drive a virtual hand. Several low-cost tracking systems specifically dedicated to hand tracking are also commercially available, like the LeapMotion (Leap Motion Inc, USA) and the CamBoard pico S (Pmdtechnologies GmbH, Germany). Such devices are made for casual applications like games but their lower level of accuracy makes them unfit for professional contexts [START_REF] Guna | An analysis of the precision and reliability of the Leap Motion sensor and its suitability for static and dynamic tracking[END_REF]. Another example of Related work multi-finger tracker with a simple design is Digits, a wrist-worn sensor making use of infrared tracking to drive a virtual hand [START_REF] Kim | Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor[END_REF] (Figure 2.13c).

It can be argued that motion capture provides a form of haptic feedback since it engages users physically and stimulates their sense of proprioception [START_REF] Mine | Moving objects in space: exploiting proprioception in virtual-environment interaction[END_REF]]. However, the sensations associated with object manipulation are missing and the virtual world lacks tangibility. The following sections present interfaces that deliver haptic feedback, either through small-scale tactile stimulation or through force feedback. Actuator arrays are grid of pins in contact with the fingertips, much like Braille displays. For instance, [START_REF] Wagner | A tactile shape display using rc servomotors[END_REF] developed such a device which 6x6 grid of pins supported by as many motors and the FEELEX is a larger tactile device, made of rods covered by a rubber screen on which an image of the touched texture is projected [START_REF] Iwata | Project FEELEX: adding haptic surface to graphics[END_REF]] (Figure 2.14a).

Tactile feedback interfaces

Pneumatic devices blow air toward the user's hand to generate haptic sensations. For instance, [START_REF] Sodhi | AIRREAL: tactile interactive experiences in free air[END_REF] developed AIREAL, a system that directs air vortices toward the user's hands to simulate contacts in 3D space, like the sensation of catching a ball (Figure 2.14b). Likewise, [START_REF] Carter | Ultra-Haptics: multi-point mid-air haptic feedback for touch surfaces[END_REF] developed UltraHaptics, a system that provides multi-point haptic feedback above an interactive surface by using focused ultrasounds. Other pneumatic devices rely on inflatable pockets for creating a sensation of touch on the fingers [START_REF] Calder | Design of a force-feedback touch-inducing actuator for teleoperator robot control[END_REF]. For example, Sato et al. [1991] designed a tactile glove equipped with an inflatable balloon at each fingerpad and the Teletact is a glove made of 30 inflatable bags that simulate contacts on various parts of the hand [START_REF] Stone | Haptic feedback: A brief history from telepresence to virtual reality[END_REF].

User interfaces for hand-based haptic interaction

Electrotactile stimulation consists in transmitting electric currents to the mechanoreceptors lying under the skin surface. It has been shown that different electrotactile patterns can simulate various sensations, like pressure or vibration [START_REF] Kajimoto | Tactile feeling display using functional electrical stimulation[END_REF][START_REF] Kajimoto | Electrocutaneous display as an interface to a virtual tactile world[END_REF]. For example, [START_REF] Kajimoto | Electrocutaneous display as an interface to a virtual tactile world[END_REF] delivered current to a fingerpad via a grid of electrodes that could display roughness, various patterns, and relative motion. [START_REF] Kajimoto | Design of cylindrical whole-hand haptic interface using electrocutaneous display[END_REF] also designed an electrotactile cylindrical interface that evaluates the posture of the user's hand via a measure of skin impedance and that delivers haptic cues to simulate the sensations resulting from object manipulation (Figure 2.14c).

Vibrotactile feedback produces vibrations at the fingertips to simulate haptic events. The price and small size of vibrotactile actuators make it a suitable option for applications that do not require high fidelity feedback, and they are commonly included in mass-market electronics, such as video game peripherals and mobile devices. For Virtual Reality, they have been embedded in portable devices, like TACTool, a spatial input device with vibrotactile feedback [START_REF] Regenbrecht | Virtual reality aided assembly with directional vibro-tactile feedback[END_REF], and the v-Glove, which has vibration motors at each fingertip to signal occuring contacts [START_REF] Gallotti | v-Glove: A 3D virtual touch interface[END_REF]. The CyberTouch, an extension of the CyberGlove sensing system (CyberGlove Systems, USA), is another example (Figure 2.15a). Vibrations can alternatively be used to simulate friction. For instance, [START_REF] Watanabe | A method for controlling tactile sensation of surface roughness using ultrasonic vibration[END_REF] made a roughly textured surface undergo ultrasonic vibrations to alter its perceived roughness, Ikei et al. Alternatively, some tactile feedback interfaces directly apply mechanical strain on the fingertips to simulate contacts with virtual objects. [START_REF] Provancher | Fingerpad skin stretch increases the perception of virtual friction[END_REF] designed the Gravity Grabber, which is made of two small motors at the ends of a belt surrounding a fingertip that clenches or laterally stretches the fingerpad, generating manipulation sensations [START_REF] Minamizawa | Gravity grabber: wearable haptic display to present virtual mass sensation[END_REF]. The GhostGlove is the assembly of five Gravity Grabbers and a larger palm-worn system into a single glove [START_REF] Minamizawa | GhostGlove: haptic existence of the virtual world[END_REF]

Force feedback interfaces

Force feedback devices deliver net forces to the user's arm, hand,or fingers in order to simulate the haptic properties of the virtual environment [START_REF] Srinivasan | Haptics in virtual environments: taxonomy, research status, and challenges[END_REF]. Early haptic devices handled few degrees of freedom; for instance a 1-DoF haptic door knob that resists the user's action [MaClean and Roderick, 1999], a 2-DoF haptic joystick for simulating machining [START_REF] Balakrishnan | Virtual hand tool with force feedback[END_REF], and the linear grasper which resists the motion of two pinching fingers [START_REF] Pang | Manual discrimination of force using active finger motion[END_REF]. However, object manipulation involves a variety of forces: global efforts acting on the arm, wrist, and the whole hand, like weight, as well as finer local forces acting on individual fingers when touching virtual objects. As outlined in the following sections, most force feedback interfaces support only one or the other type of forces due to technical limitations. Thus, we first distinguish between whole hand interfaces and multi-finger interfaces.

At the end of this section, we also present more complex and costly hybrid interfaces that do provide both types of force but are restricted to professional applications. Another level of distinction that will appear is that of the attachment of the force feedback devices. Ground-based, or desktop, interfaces have a fixed position whereas body-based interfaces are directly mounted on the users in order to acccompany them in larger environment [START_REF] Srinivasan | Haptic interactions in the real and virtual worlds[END_REF].

Whole hand interfaces

Most VR applications incorporating force feedback rely on compact desktop devices taking the form of robotic arms that end with a stylus or a handle. They let users control the position of a proxy typically shaped as a 3D cursor or a virtual tool, and User interfaces for hand-based haptic interaction they output global forces to the user's hand to reflect the physical interaction occurring in the simulation. For example, the Phantom 3-DoF device, a staple of desktop haptics, outputs linear forces [START_REF] Massie | The PHANTOM haptic interface: A device for probing virtual objects[END_REF]] (Figure 2.16a), and bulkier 6-DoF systems may additionally output torques (Figure 2.16b). The Spidar is a device that takes an alternative form, with a metallic frame surrounded by motors pulling strings attached to a prop or to the user's fingertips [START_REF] Kim | Tension based 7-DOF force feedback device: Spidar-g[END_REF][START_REF] Sato | Development of string-based force display: Spidar[END_REF] (Figure 2.16c). Desktop devices generally have small workspaces because of their grounded nature and the reduced range of their effector. Hence, "human-scale" haptic interfaces have been designed to accompany users within a larger physical space [START_REF] Bouguila | Scaleable SPIDAR: A haptic interface for human-scale virtual environments[END_REF][START_REF] Dominjon | Novel devices and interaction techniques for human-scale haptics[END_REF] (Figure 2.17a). For example, the INCA (Haption SA, France) is a Spidar-like frame that constrains a prop held by users with strings in a 1.5 m 3 volume (Figure 2.17b). Force feedback arms can also be mounted on mobile stands to follow users in their displacements [START_REF] Buss | Development of a multi-modal multi-user telepresence and teleaction system[END_REF][START_REF] Nitzsche | A mobile haptic interface mastering a mobile teleoperator[END_REF]. Alternatively, force feedback interfaces can be directly mounted on the user. For instance, HapticGEAR is a backpack-like device capable of providing force-feedback by pulling on the user's hand with strings [START_REF] Hirose | Hapticgear: The development of a wearable force display system for immersive projection displays[END_REF]] (Figure 2.17c). Similarly, [START_REF] Tsetserukou | Exointerfaces: Novel exosceleton haptic interfaces for virtual reality, augmented sport and rehabilitation[END_REF] proposed a minimalist haptic interface taking the form of a motorized cable that links the wrist to the arm to constrain the extension of the arm, mimicking a muscle. While this system only has one DoF, it can simulate various haptic effects such as weight and collisions with obstacles (Figure 2.17d).

Multi-finger interfaces

Generic desktop haptic devices are generally unable to provide multi-finger force feedback. They may be equipped with a thimble for inserting a fingertip [START_REF] Massie | The PHANTOM haptic interface: A device for probing virtual objects[END_REF][START_REF] Salisbury | Phantom-based haptic interaction with virtual objects[END_REF] Other desktop multi-finger interfaces have been specifically designed to support the actions of several fingers. For example, [START_REF] Ueda | Development of a mouse-shaped haptic device with multiple finger inputs[END_REF] developed a mouseshaped device that resists each digit individually when they curl in order to simulate the contact of virtual objects. [START_REF] Monroy | Masterfinger: Multi-finger haptic interface for collaborative environments[END_REF] designed the MasterFinger-2, a haptic interface made of two articulated arms, each one ending with a thimble (Figure 2.18c). The DigiHaptic is a 3-DoF desktop device that actuates three fingers through separate levers for manipulating object in 3D [START_REF] Casiez | The DigiHaptic, a new three degrees of freedom multi-finger haptic device[END_REF]] (Figure 2.18d). The HIRO III is a robotic hand that supports 21 degrees of freedom of the human hand [START_REF] Endo | Five-fingered haptic interface robot: HIRO III[END_REF] (Figure 2.18e). Variants of the Spidar interface also support several fingers, such as the Spidar-8, which actuates four fingers per hand [START_REF] Walairacht | 4 + 4 fingers manipulating virtual objects in mixed-reality environment[END_REF] (Figure 2.18f).

Multi-finger force feedback interfaces can be directly mounted on the user's hand. For instance, [START_REF] Bouzit | The Rutgers Master II-ND force feedback glove[END_REF] designed the Rutgers-Master, a pneumatic endoskeleton that provides force feedback with pistons nested inside of the palm (Figure 2.19a). Contrarily, haptic exoskeletons shift the bulk of their components on the back of the hand so as not to hamper the user's gestures. For example, [START_REF] Nishino | 20 DoF haptic device for the interaction with virtual environments[END_REF] designed a wire-driven system that constrains 20 DoF, [START_REF] Bullion | Haptic glove with MR brakes for distributed finger force feedback[END_REF] designed an exoskeleton with brakes filled with viscosity-changing MR fluid, and Fang et al. [2009] built MasterHand, an exoskeleton covering the hand with chains of mechanical phalanxes (Figure 2.19c). [START_REF] Nakagawara | An encounter-type multi-fingered master hand using circuitous joints[END_REF] designed an encounter-type glove is composed of robotic joints that block the fingers when a collision must be rendered (Figure 2.19f). The CyberGrasp (CyberGlove Systems, USA) is an exoskeleton that pulls the fingertips via cables (Figure 2.19a). It is the only device of this type that is commercially available at the moment but it comes with a high cost, which restricts its use to specific applications. Some haptic exoskeletons also incorporate tactile actuators to combine small-scale effects with larger net forces. For instance, the WHIPFI exoskeleton constrains the thumb and the index fingers to simulate pinching and provides tactile feedback by way

Hybrid interfaces

In order to provide efforts that are both external and internal to the hands and recreate convincing haptic sensations, some grip force interfaces have the ability to measure the grasping efforts applied by users and to return a resistive force. For example, the Freedom-7 is a 6-DoF haptic arm ending with a grip force end-effector that also has an additional axis of action to allow the actuation of a tool (scissors, forceps or, in our context, a closing hand) [START_REF] Hayward | Freedom-7: A high fidelity seven axis haptic device with application to surgical training[END_REF]]. The sigma.7 is a bimanual interface that provides a similar seventh DoF to simulate grasping [START_REF] Tobergte | The sigma.7 haptic interface for mirosurge: A new bi-manual surgical console[END_REF] (Figure 2.20a). For more lightweight systems, custom handles have been designed to be plugged on generic desktop devices to add an additional degree of freedom on their end-effector [Barbagli et al., 2003;[START_REF] Najdovski | Extending haptic device capability for 3d virtual grasping[END_REF]] (Figure 2.20b and 2.20c). While this enables the display of certain haptic properties like stiffness, fingers are not distinguished and the interaction is limited to basic claw-like grasping.

For enabling truly complete force-feedback comprising global forces on the arm or wrist and local forces on several fingers, hybrid systems combining grounded robotic arms and multi-finger exoskeletons are required. For instance, [START_REF] Loscos | The museum of pure form: touching real statues in an immersive virtual museum[END_REF] designed Pure Form, which is composed of a grounded robot that constrains the user's arm and ends with two thimbles for the thumb and index finger. The Haptic WorkStation (CyberGlove Systems, USA) is the commercial combination of a pair of Cybergrasp exoskeletons with two robotic arms for displaying global forces on both hands but, in practice, it is inaccessible because of its extremely high price (Figure 2.20d).

Passive haptic feedback

Passive haptics engages users haptically without relying on costly active devices. Instead, it leverages inert physical objects to simulate the touch of virtual ones. For instance, tangible surfaces have been used to simulate the touch of a virtual control panel [START_REF] Borst | Evaluation of a haptic mixed reality system for interactions with a virtual control panel[END_REF] [1998a] gave users tablets to respectively recreate the surface of a hand-held menu and to reproduce the support of a notepad in a virtual handwriting task (Figure 2.21a).

In some cases, the user's body can itself provide haptic cues. For example, Hummel Passive haptic feedback can also be leveraged to simulate manipulation tasks. For example [START_REF] Lok | Effects of handling real objects and avatar fidelity on cognitive task performance in virtual environments[END_REF] used real objects with the same shape as virtual objects to physicalize the interaction. Similarly, [START_REF] Chapoulie | Finger-based manipulation in immersive spaces and the real world[END_REF] designed different passive systems that support manipulation tasks with various degrees of freedom and constraints, like passive dials and sliders. Hinckley et al. [1994a] used a doll head prop to control the position of a human brain in a neurosurgical surgery application (Figure 2.21b). Deformable props can also be exploited, like a proxy sponge for sculpting virtual clay models [START_REF] Sheng | An interface for virtual 3D sculpting via physical proxy[END_REF] or a block of foam on the palm to provide haptic cues during 3D modeling tasks [START_REF] Hoang | Passive deformable haptic glove to support 3D interactions in mobile augmented reality environments[END_REF].

Passive haptic can also be incorporated into input interfaces. For example, Galyean and Hughes [1991] designed a poor man's force feedback unit made of rubber bands that Related work passively recenter a 3D cursor in order to conduct a sculpting task (Figure 2.21c) and Paljic and Coquillart [2004] developed a passive Spidar with a string-mounted brake that generates friction when gripped.

Other passive devices provide haptic feedback to separate digits. For instance, Pihuit et al. [2008] used a foam ball with a pressure sensor under each fingertip to control a virtual hand (Figure 2.22a). Kry and Pai [2006a] designed Tango, a ball covered with pressure sensors that recognizes the pose to apply on a 3D hand depending on its pressure distribution. HandNavigator [START_REF] Kry | Handnavigator: handson interaction for desktop virtual reality[END_REF]] is a 3D mouse customized with pressure sensors under each finger that allows users to control a virtual hand by slightly displacing the fingers inside of deformable thimbles (Figure 2.22b). [START_REF] Koyama | Multi-fingered exoskeleton haptic device using passive force feedback for dexterous teleoperation[END_REF] developed a partially passive exoskeleton that links fingertips to an elastic torsional shaft on the back of the hand (Figure 2.22c). However, they use active clutches to hold up the fingers when necessary.

Conclusion

A variety of user interfaces let us interact with our hands in virtual environments.

Traditional motion capture systems transpose the user's movements onto virtual avatars but their use with hands is limited because of occlusion issues. For a more accurate input, data gloves measure the flexion of the fingers of the hand, and recent markerless solutions even provide multi-finger tracking in a non-invasive manner. While such methods engage the users physically and stimulate their sense of proprioception, they do not provide true haptic feedback in the form of tactile stimuli or forces. Tactile feedback devices simulate the touch of virtual objects, through different forms: grid of effectors, pneumatic devices, electrocutaneous, and vibrotactile displays. For simulating the net forces resulting from object manipulation, force feedback interfaces are also available. Typical desktop interfaces deliver simple efforts to the whole hand and their grounded nature may restrain the user's freedom of movement. More elaborate multi-finger devices do provide forces to fingers separately, either in a desktop form or attached on the users, such as haptic exoskeletons. For providing both whole hand forces and individual forces to the fingers, hybrid workstations are required but their high cost and bulkiness restrict their use to professional settings.

Finally, passive haptic feedback is a lightweight alternative to active devices. It can simulate the sensations of object manipulation through inert props or interfaces but they are not flexible as they cannot be controlled in real-time by the simulation. For this reason, software techniques are necessary to enrich their feedback, as described in the next section, which focuses on 3D interaction techniques.

3D interaction techniques

In computer sciences, an interaction technique is the fusion of input and output that provides a way for the user to accomplish a task [START_REF] Tucker | Computer Science Handbook[END_REF]. In the specific context of virtual environments, [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF] defined 3D interaction techniques as: "[...] a method allowing a user to accomplish a task via the user interface. An interaction technique includes both hardware and software components. The interaction technique's software component is responsible for mapping the information from the input device into some action within the system, and for mapping the output of the system to a form that can be displayed by the output device."

In this section, we give of overview of existing 3D interaction techniques (3DIT) that relate to hand-based interaction. We first describe existing metaphors for interaction with the hand through natural gestures or pointing. Then we present multimodal feedback that leverages other sensory modalities than pure haptics to provide haptic information to users. Finally, we describe interaction techniques that were specifically designed to address the limitations of current haptic hardware.

Interaction metaphors for manipulating virtual objects

3D interaction techniques are categorized into four canonical tasks: navigation, selection, manipulation, and system control [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF]. In this manuscript, we focus on dexterous manipulation and address interaction techniques that assist in manipulation in the broad sense of [START_REF] Poupyrev | Manipulating objects in virtual worlds: Categorization and empirical evaluation of interaction techniques[END_REF], i.e. selection and positioning.

Furthermore, interaction techniques can be categorized depending on the general metaphor that they follow, which corresponds to their "fundamental mental model [...], a perceptual manifestation of what users can do (affordances), and what they cannot do (constraints)" [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF]. For example, the image plane metaphor consists in selecting objects at a distance by framing our fingers around them [START_REF] Pierce | Image plane interaction techniques in 3D immersive environments[END_REF]] -an analogy used by the authors is that of many tourists in Italy that mime supporting the Pisa tower through a simple perspective illusion (Figure 2.23a). The virtual hand metaphor for touching and grabbing objects with the hand [START_REF] Poupyrev | The go-go interaction technique: Non-linear mapping for direct manipulation in VR[END_REF].

(c) The virtual pointer metaphor for targeting objects with the hand [START_REF] Poupyrev | The go-go interaction technique: Non-linear mapping for direct manipulation in VR[END_REF].

In this section, we present 3D interaction techniques based on the virtual hand and virtual pointer metaphors. Then, we describe composite interaction techniques that combine them.

The virtual hand metaphor

The virtual hand metaphor allows users to naturally grab and position objects with their own hand. In recent applications requiring fine controls, the hand is fully articulated and interacts precisely with other virtual objects. However, for scenarios that do not demand such precision, manipulation is reduced to rough pick-and-place tasks where objects are simply attached to the frame of reference of the hand. This is the classical virtual hand metaphor (Figure 2.23b). In this section, we present additional interaction techniques based on this general metaphor that aim at easing the manipulation of distant or small objects.

A useful task in VR is to select out-of-reach objects, as enabled for example by the Go-Go technique, which amplifies the user motion to extend his reach [START_REF] Poupyrev | The go-go interaction technique: Non-linear mapping for direct manipulation in VR[END_REF]. In this way, the operator can manipulate objects in his close vicinity with precision and then extend his arm to grasp distant ones; however the farther the object is, the less precise is the selection. Stretch go-go is a variant that instead divides the user's reach into three areas [START_REF] Bowman | An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments[END_REF]]. In the middle area, the virtual hand matches the motion of the user but closer or farther from the body, the virtual hand moves toward and away from the user in rate control so that an infinite environment can be traveled with the same accuracy. [START_REF] Poupyrev | Non-isomorphic 3d rotational techniques[END_REF] also proposed similar non-homogeneous mappings to ease rotations in 3D environments.

Another necessity is to precisely control the virtual hand that perform such actions. For example, the intent driven selection method adjusts the selection of small objects depending on behavioral cues such as action persistence [START_REF] Periverzov | IDS: the intent driven selection method for natural user interfaces[END_REF]. Similarly, PRISM is a manipulation technique that decreases the Control/Display ratio of the hand depending on the detected user's intentions, which are inferred from the velocity of the hand [START_REF] Frees | Precise and rapid interaction through scaled manipulation in immersive virtual environments[END_REF].

3D interaction techniques

Other interaction techniques are designed to assist users in performing complex tasks that require precision. In the virtual assembly system ISAAC, a snap-to-grid mode constrains manipulated objects to positions on a regular grids instead of continuously following the user's motion [START_REF] Mine | Isaac : A virtual environment tool for the interactive construction of virtual worlds[END_REF]. For easing object assemblage, [START_REF] Kitamura | A sophisticated manipulation aid in a virtual environment using dynamic constraints among object faces[END_REF] attracted together parts that were likely to be attached depending on their orientation and geometry. For guiding users in moving objects along intricate paths, haptic fixtures can be manually placed in the environment to attract or repel the cursor with force feedback [START_REF] Sayers | An operator interface for teleprogramming employing synthetic fixtures[END_REF]. Similarly, [START_REF] Ladeveze | Haptic assembly and disassembly task assistance using interactive path planning[END_REF] developed a haptic path planning system that dynamically computes a collision-free path to guide the object that the user manipulates toward a target.

The virtual pointer metaphor

The virtual pointer metaphor mimics the act of using a finger or a laser pointer to direct toward an object or an area of interest. In its simplest form, the virtual pointer metaphor is realized by ray-casting: an imaginary line is fired in the direction of the hand and the first collided object is selected [START_REF] Poupyrev | Egocentric object manipulation in virtual environments: Evaluation of interaction techniques[END_REF]] (Figure 2.23c). With its bimanual variant, two-handed ray-casting, the ray emanates from the nondominant hand and it goes through the dominant one [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF].

The selection of small objects with such pointing techniques can be tiresome due to tremors of the hand and tracking inaccuracies. Thus, [START_REF] Wingrave | 3DUI flavors beyond vanilla[END_REF] proposed a snap-to-ray method that reorients the fired ray toward the closest object. Also, the flashlight technique uses a conical volume instead of a ray in order to expand the selection space [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF]. With its variant, the aperture technique, the cone starts at the user's eyes and pass through his hand; users then alter the radius of the cone in a manner analogous to the aperture of a photo camera [START_REF] Forsberg | Aperture based selection for immersive virtual environments[END_REF]. [START_REF] Steinicke | Object selection in virtual environments using an improved virtual pointer metaphor[END_REF] proposed an improved pointing technique that bends the ray toward the closest virtual object, which eases targeting.

Selecting occluded objects is another challenge. [START_REF] Grossman | The design and evaluation of selection techniques for 3d volumetric displays[END_REF] presented several methods to pick hidden objects, such as the depth ray, which allows users to slide a depth marker along the ray to go through occluded objects. The flexible pointer technique is similar but the fired ray can be curved to reach objects hidden between obstacles [START_REF] Olwal | The flexible pointer: An interaction technique for augmented and virtual reality[END_REF]] (Figure 2.24a). The distance between hands determines the length of the ray and the separate orientations of each hand place a conceptual control point bending it much like a Bézier spline. Some 3D interaction techniques deal with ambiguous cases that arise when many objects are in close proximity. The smart ray, for instance, selects the virtual object that is the closest to the intersection of rays continuously fired over a period of time [START_REF] Grossman | The design and evaluation of selection techniques for 3d volumetric displays[END_REF]. [START_REF] Kopper | Rapid and accurate 3D selection by progressive refinement[END_REF] proposed SQUAD, a progressive selection technique that distributes copies of the candidate objects on the screen and requires users to refine the selection until there is only one item left. This method was later extended with the expand SQUAD variant to reduce the number of steps required to refine a selection [START_REF] Cashion | Dense and dynamic 3d selection for game-based virtual environments[END_REF]. de Haan et al. [2005] proposed intenselect, which selects a moving object depending on the sustained focus of the user during several frames. Although these interaction techniques based on the virtual pointer metaphor were mainly intended for selection tasks, they are also capable of positioning objects once they are selected by simply moving the ray or cone to another position, like with the action-at-a-distance technique [START_REF] Mine | Isaac : A virtual environment tool for the interactive construction of virtual worlds[END_REF]. However, only interaction techniques where the length of the ray or cone can be varied enable to move it in the direction between the user and the object. The bent pick ray technique allows the ray linking a user to an object to be bent like a fishing reel when he points upwards [START_REF] Riege | The bent pick ray: An extended pointing technique for multi-user interaction[END_REF]] (Figure 2.24b) and with the iSith technique, the object is positioned at the intersection of two rays fired from each hand [START_REF] Wyss | iSith -Intersection-based spatial interaction for two hands[END_REF]] (Figure 2.24c).

Composite interaction techniques

Interaction techniques based on the virtual hand and virtual pointer metaphors are primarily used to manipulate virtual objects. However, they can be combined with other techniques more often used for pure selection tasks or navigation tasks in order to provide additional interaction schemes.

For example, the world in miniature metaphor places a scaled down copy of the virtual environment in the user's hand so that he can benefit from an additional point of view (Figure 2.28c) [START_REF] Stoakley | Virtual reality on a wim: Interactive worlds in miniature[END_REF]. This metaphor can assist in executing various tasks; for example, the user can select occluded objects (from his first-person point of view) by simply touching their miniature versions. Likewise, he can move objects more quickly across the virtual scene. He can even navigate through the environment by manipulating his own avatar in the miniature scene as well as leverage other interaction techniques from this elevated perspective. The scaled world grab technique shares similarities with the world in miniature. Once a virtual object (that may be large or distant) has been selected by means of any selection methods, it is scaled down and placed within the user's hand [START_REF] Mine | Moving objects in space: exploiting proprioception in virtual-environment interaction[END_REF] , 1997]. The initial selection is done with ray-casting but instead of attaching the picked object to the ray in order to move it around, a virtual hand is instantly placed at its position so that it can be more precisely manipulated.

Multi-sensory feedback for manipulation tasks

Most of the perception resulting from manipulation passes through the haptic channel but current haptic devices are not sophisticated enough to provide users with an accurate haptic image of the efforts involved in complex cases of interaction. To compensate for this limitation, haptic properties can be expressed by means of other sensory modalities. This section presents an overview of alternative feedback that can reinforce haptic perception using visuals cues, auditory cues, and pseudo-haptics.

Visual feedback

Providing a clear visual feedback to the user so that he can more efficiently act in the virtual world starts by reproducing visual phenomena from the real world. For instance, lighting effects such as interreflection and shadows help in evaluating the distances between objects and anticipating contacts [START_REF] Hu | Visual cues for imminent object contact in realistic virtual environment[END_REF][START_REF] Naemura | Virtual shadowsenhanced interaction in mixed reality environment[END_REF].

3D interaction techniques

Abstract representations conveying similar information can be used. For example, de Sá and Zachmann [1999] changed the color of intersecting objects to inform users about collisions, [START_REF] Mcneely | Voxel-based 6-DOF haptic rendering improvements[END_REF] colored the surface of manipulated objects with a proximity map to better anticipate imminent contacts, and Sreng et al. [2006] placed light sources near collision points to illuminate the surrounding areas. For expressing interaction efforts, glyphs denoting the direction and intensity of interaction forces can be displayed (Figure 2.26a), or colors drawn and particles projected to illustrate friction forces (Figure 2.26b). For communicating the efforts between virtual objects, they also drew colors on colliding objects to illustrate friction forces and emitted particles to inform about the direction of a frictional contact and the pressure between two objects [START_REF] Sreng | Using an event-based approach to improve the multimodal rendering of 6dof virtual contact[END_REF] (Figure 2.26b). [START_REF] Koutek | Spring-based manipulation tools for virtual environments[END_REF] linked manipulated objects to the user's hand with a visual spring so that efforts could be inferred from its apparent bending, stretching, and torsion (Figure 2.26c). Similarly, Otsuki et al.

[2014] connected 3D objects with elastic links so that users could gauge the importance of their relationships by pulling them.

Visual feedback can also be directly applied on the hand model that conducts the interaction. For example, [START_REF] Lindeman | The effect of 3D widget representation and simulated surface constraints on interaction in virtual environments[END_REF] colored the tip of a virtual finger when it intersected another entity, [START_REF] Moehring | Effective manipulation of virtual objects within arm's reach[END_REF] colored the segments of a virtual hand that contribute to a valid grasp, and Ullmann and Sauer [2000] colored the colliding segments as well as the whole hand when an object was grasped. [START_REF] Prachyabrued | Visual feedback for virtual grasping[END_REF] compared different types of visual feedback overlayed on a hand model, including finger coloring depending on the grasp intensity, object coloring, arrows representing forces, and a vibration effect (Figure 2.26e). Rusák et al.

[2009] made the virtual hand transparent and explicitly displayed the contact areas between fingers and objects.

Visual feedback often relies on the assumption that there is a direct line of sight between the user and the area of interaction but in practice they may be occluded by other objects or the user's own hands. 3DIT have thus been designed for this specific purpose. [START_REF] Argelaguet | Improving co-located collaboration with show-through techniques[END_REF] clipped or faded part of the occluding objects blocking the user's view. [START_REF] Otaduy | User-centric viewpoint computations for haptic exploration and manipulation[END_REF] dynamically changed the viewpoint of the virtual camera so that the area of interaction was always visible. Hachet et al.

[2009] developed the Navidget system, which displays several points of view of the same scene and Raghupathy and Borst [2011] used virtual mirrors to keep hidden parts of the environment in view.

Auditory feedback

The auditory channel is also capable of conveying information about the interaction occurring within the virtual world. The impact on the user's performance is not as clear as with visual feedback [START_REF] Edwards | The use of force feedback and auditory cues for performance of an assembly task in an immersive virtual environment[END_REF] but users generally prefer the addition of sound effects as it strengthens the impression of realism [START_REF] Yano | Audiohaptics: Audio and haptic rendering based on a physical model[END_REF].

The more straightforward option when the aim is to provide realistic auditory effects in order to enhance a VR simulation is to play prerecorded sounds [START_REF] Difranco | The effect of auditory cues on the haptic perception of stiffness in virtual environments[END_REF]. To add more variety, the nature of these sounds can be adjusted to translate the occurring interaction with more fidelity. For example, [START_REF] Altinsoy | The effect of auditory cues on the audiotactile roughness perception: Modulation frequency and sound pressure level[END_REF] modulated the frequency of a scraping sound depending on the properties of a grooved surface being touched and [START_REF] Barrass | Interactive granular synthesis of haptic contact sounds[END_REF] modulated simple sound grains in accordance with the interaction between a haptic interface and a virtual object.

Audio cues can also be generated by physical models. O'Brien et al. [2001] proposed a finite element model simulating the pressure waves induced by the contact between objects. [START_REF] Raghuvanshi | Interactive sound synthesis for large scale environments[END_REF] approximated virtual objects as spring-mass systems to capture the small-scale external and internal vibrations that create sounds. Likewise, van den Doel et al. [2001] simulated the small-scale interplay between objects to compute impact, scraping, and rolling "audio-forces" to produce realistic auditory effects in real-time.

More abstract sound representations can alternatively be used to inform the user about the objects that he manipulates. To express the proximity between a tool held by the user and points of interest in a medical simulation, Müller-Tomfelde [2004] used a reverbered sound which becomes clearer with the proximity. Murayama et al.

[2004] modulated the intensity of a monotone sound to convey information about the pressure exerted by the user on virtual objects. [START_REF] Richard | Human perceptual issues in virtual environments: sensory substitution and information redundancy[END_REF] used a similar technique to inform about the deformation undergone by a malleable virtual ball. [START_REF] Edwards | The use of force feedback and auditory cues for performance of an assembly task in an immersive virtual environment[END_REF] played specific sounds to denote important events in the simulation, for example to confirm the grasp on a virtual object or the interlocking of two parts.

Cross-modal sensory illusions

Cross-modal feedback consists in leveraging interactions between different, and sometimes conflicting, sensory modalities to communicate properties of a virtual environment to users [START_REF] Biocca | Visual touch in virtual environments: An exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions[END_REF]. In this manuscript, we focus especially on pseudohaptics, a type of cross-modal effects that exploits the tight bounds between the visual and haptic modalities in order to provide haptic sensations -or modulate existing ones -through visuals. Pseudo-haptic feedback has been defined as "a technique meant to simulate haptic sensations in virtual environments using visual feedback and properties of human visuo-haptic perception, [...] and verges on haptic illusions" [START_REF] Lécuyer | Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback[END_REF]. In two dimensions for instance, [START_REF] Bibliography Lécuyer | Feeling bumps and holes without a haptic interface: the perception of pseudo-haptic textures[END_REF] demonstrated that perturbing the motion of a cursor controlled with a computer mouse conveys information about the texture of the hovered surface. Similarly, [START_REF] Watanabe | Visualhaptics: generating haptic sensation using only visual cues[END_REF] altered the trajectory and size of a cursor to create feelings of texture, resistance, and depth.

In three dimensions, pseudo-haptics has initially been introduced as a dynamic change of the gain between the force applied on an isometric device and the corresponding movement of the virtual object being manipulated (the Control/Display ratio, or C/D ratio) [START_REF] Lécuyer | Pseudohaptic feedback: Can isometric input devices simulate force feedback?[END_REF]. This effect was illustrated with a task involving the insertion of an object into a duct: when the manipulated object reaches the duct, the on-screen speed is slowed down and users tend to increase the force applied on the passive device which makes them perceive stronger haptic cues.

Pseudo-haptic feedback has also been leveraged to simulate other haptic properties, like weight. For example, The stiffness of virtual objects is another haptic property that has been successfully simulated with pseudo-haptic feedback. For example, [START_REF] Argelaguet | Elastic images: Perceiving local elasticity of images through a novel pseudo-haptic deformation effect[END_REF] deformed 2D textures with different dynamics to express the stiffness of materials. Similarly, Sof-tAR is an augmented reality application that deforms the texture projected on a real object depending on the simulated stiffness when users press it with their fingers [Punpongsanon et al., 2015] (Figure 2.27b). [START_REF] Ridzuan | Direct touch haptic display using immersive illusion with interactive virtual finger[END_REF] designed an effect for interacting with pseudo-haptic objects displayed on tactile tablets: they elongated the user's fingers beyond the screen and applied visual transformations to this virtual extension to simulate various levels of stiffness. [START_REF] Argelaguet | Collaborative pseudo-haptics: Two-user stiffness discrimination based on visual feedback[END_REF] also explored how pseudo-haptics applies to cooperative interaction with two users pushing on the same object with a single finger. [START_REF] Kimura | Pseudo-haptic feedback on softness induced by grasping motion[END_REF] equipped a smartphone with pressure sensors on its side to modulate the deformation of a displayed shape; the on-screen object was squished with various speed to express different levels of stiffness.

Recent research efforts also focused on the application of pseudo-haptic feedback to interaction with the hand. For example, [START_REF] Pusch | Hemp-hand-displacement-based pseudo-haptics: A study of a force field application[END_REF] proposed HEMP, a pseudohaptic effect simulating the influence of force fields on a user's hand by offsetting their virtual position in space (Figure 2.27c). [START_REF] Bibliography Ban | Modifying an identified curved surface shape using pseudo-haptic effect[END_REF] simulated different shapes by slightly warping the virtual representation of a real prop as well as the position of the user's finger while he follows its outline (Figure 2.27d). [START_REF] Kohli | Redirected touching: The effect of warping space on task performance[END_REF] introduced redirected touching, a spatial warping that leverages discrepancies between the haptic feedback of the real world and the visual feedback of the virtual world in order to distort shape perception in a similar manner (Figure 2.27e).

Handling the limitations of haptic devices

Current haptic devices do not provide realistic feedback of manipulation tasks due to technical limitations. Grounded devices have a small workspace whereas multi-finger exoskeletons cannot exert external forces on the user's hand and are significantly underactuated. Additional 3D interaction techniques must thus be developed to circumvent such hardware limitations.

Small workspace

Most grounded haptic devices limit the amplitude of the user's movements due to their grounded nature and small range. This hampers the manipulation of virtual objects within a large environment as users cannot move away from the device nor perform wide motions with the hand. To circumvent this issue, the Control/Display ratio can be adjusted in order to amplify the virtual hand motion and thus expand the user's reach; which however results in a lessened precision. Clutching provides another solution to this issue by briefly deactivating the coupling between haptic interface and manipulated object while the users recenters the input device (much like a computer mouse can be moved by lifting it so that it does not change the cursor position). However, this method forces the user to bring his device back and forth for movements over long distances.

The Bubble technique proposed by Dominjon et al. [2005b] also enables environmentwide movements and uses different control schemes depending on the relative positions of an interaction point and a control sphere (Figure 2.28a). When inside of the sphere, the point is position-controlled but when outside of the sphere, it is rate-controlled. A spring-like constraint links the point and its bubble such that the user can haptically feel the switch between the two modes. In [START_REF] Talvas | Novel interactive techniques for bimanual manipulation of 3D objects with two 3DoF haptic interfaces[END_REF], two hands have separate bubbles and the viewport is recomputed when they move in order to keep them visible on the screen at all time (Figure 2.28b). A joint control mechanism was implemented to coordinate the motion of both hands even if they are controlled through different devices.

Underactuation

Most haptic interfaces suffer from underactuation, i.e they are not able to actuate as many degrees of freedom as they sense. [START_REF] Barbagli | The effect of sensor/actuator asymmetries in haptic interfaces[END_REF] formalized the measure of these asymmetries by using the concepts of controllability and observability. With n, the number of degrees of freedom of an arbitrary virtual avatar, s the number of sensors and r the number of actuators of the device, the controllability k equals s n and the observality o equals r n . For example, the Desktop PHANToM senses 6 DoF but can only exert translational forces so k = 1 and o = 3 6 when interacting through a 6DoF

Related work

cursor. This asymmetry between input and output capabilities reduces the usability of haptic interfaces. With human hands and haptic devices, the discrepancy between controllability and observability is even worse since most degrees of freedom of the hand can be tracked but output forces are generally only exerted to the fingertips along a fixed axis.

To allow 6DoF manipulation of an object with a 3DoF device, Lécuyer et al.

[2001] separated translation and rotation into distinct phases. The translation is handled in a classical manner and the manipulated object follows the motion of the device. In rotation mode, the position of the haptic interface is mapped onto the surface of a sphere encircling the object. Another technique, called A 4 , is focused on point-based interaction with underactuated device [START_REF] Lécuyer | a4": A technique to improve perception of contacts with under-actuated haptic devices in virtual reality[END_REF]. The principle is to rotate the whole scene such that the forces to be rendered are aligned to the axis of actuation of the employed haptic device and thus can be rendered realistically.

Lack of external constraint

A major drawback of stand-alone haptic exoskeleton gloves is the lack of external constraint that they apply on the user's hands. This proves challenging when the goal is to realistically render absolute interaction forces such as gravity or collisions. However, perceptual effects can be leveraged to make the user aware of constraints that cannot be physically imposed on him.

To let the user perceive absolute influences like gravity and inertia, a basic possibility is to employ multi-sensory representations of these forces, as described in Section 2.4.2. Visual, auditory, and pseudo-haptic cues can also give an idea of the efforts occurring in the simulation.

Another issue is the inability to forcefully stop the user's hand to faithfully reproduce the infinite stiffness of a wall or a rigid object. Visually, this results in a sinking effect in which the fingers penetrate a virtual object if the user does not make a conscious effort to keep them at its surface. [START_REF] Burns | The hand is more easily fooled than the eye: Users are more sensitive to visual interpenetration than to visual-proprioceptive discrepancy[END_REF] suggested that users are more sensitive to visual interpenetration than to visual-proprioceptive discrepancy and preventing interpenetration is generally preferred [START_REF] Bibliography Prachyabrued | Visual interpenetration tradeoffs in wholehand virtual grasping[END_REF]]. Hence, many authors preferred keeping the virtual hand at the object surface even if it does not precisely correspond to the state of the real hand. For example, [START_REF] Boulic | Multi-finger manipulation of virtual objects[END_REF] avoided the sinking effect by unfolding the finger joint by joint until the constraint violation was solved (Figure 2.29a).

Borst and Indigula proprosed a coupling scheme that avoids that the virtual hand sink into virtual object [START_REF] Borst | Realistic virtual grasping[END_REF]. Two hands are actually simulated in the virtual environment: the tracked hand, matching the configuration of the user's hand, is invisible while the spring hand represents the configuration that the virtual hand should plausibly have (Figure 2.29b). The spring hand is linked to the tracked one by linear and torsional spring-dampers at each articulation so that their postures, positions and orientations are as close as possible. However, the spring hand is constrained to stay at the surface of objects. This technique removes the sinking effect that could disturb the user but the displayed virtual hand no longer perfectly matches 3D interaction techniques the configuration of its real counterpart. As a consequence, dropping objects is difficult since slightly opening the fingers only reduces the interpenetration but may not completely stop the contact so objects tend to "stick" to the hand.

To solve this new issue, incremental motion can be employed [START_REF] Zachmann | Natural and robust interaction in virtual assembly simulation[END_REF]. With this method, the virtual hand moves by the same amount as the real hand and it is easily unstuck from objects but the technique creates a greater discrepancy between virtual and real hands. [START_REF] Burns | MACBETH: Management of avatar conflict by employment of a technique hybrid[END_REF] proposed the MACBETH method that is based on incremental motion but progressively readjusts the hands configurations so that they are colocated again after being separated from the object (Figure 2.29c). [START_REF] Prachyabrued | Dropping the ball: Releasing a virtual grasp[END_REF] tackled the sticking issue by extending their models to a system of three virtually coupled hands, the new one being subject to incremental motion and attracting the spring hand.

Conclusion

3D interaction techniques add supplementary layers of control and feedback to enrich the actions and perceptions possible in virtual environments. In their simplest forms, interaction techniques based on the virtual hand and virtual pointer metaphors allow users to select and manipulate remote, small, or cramped objects. Then other 3DIT enhance these tasks by resolving ambiguous situations with occluded or cluttered environments, or providing haptic or visual assistance to guide users in a task. Combining 3DIT together creates composite methods that expand the range of interaction furthermore. Alternative feedback based on visual and auditory stimuli can complement haptic feedback, through either explicit or abstract cues. Crossmodal feedback, and more specifically pseudo-haptics, helps in generating haptic sensations from other sensory modals. For instance, it has been leveraged to simulate a variety of physical properties such as weight or stiffness. However, as of now, it has only been scarcely used for hand-based interaction.

Finally, certain are designed to circumvent the limitations inherent to haptic devices. For instance, methods like the Bubble technique enlarge the small workspace of desktop devices and other techniques deal with underactuation. Another problem is the lack of external constraint on the user's hands, which can be partially solved via the use of visual effects.

Conclusion

The hand is a major tool of interaction that acts both as an output and input link between us and our environment. In one direction, its highly articulated skeleton, paired with remote muscles, allows us to grasp and precisely manipulate objects. In the other direction, the structural and physical properties of the manipulated objects can be perceived through mechanoreceptors located in the skin, muscles, tendons, and joints.

The virtual hand that is the embodiment of the user's in the virtual world can wear different forms, from simple interaction points at the fingertips to realistic deformable models. The higher the complexity of a virtual hand is, the larger its potential range of interaction, but a middle-ground has to be found between realism and computational cost. As for the interaction between these hands and the objects populating the virtual world, it is either governed by physical or by heuristic rules. With the former, mechanical laws from the real-world are implemented to provide objects with a unified behavior while with the latter, they are reduced to simpler mechanisms at the expense of physical correctness.

In Virtual Reality simulations, haptic interfaces provide the ability to touch virtual objects and feel their physical properties. However, most desktop devices cannot accommodate the complexity of the human hand and the use of more sophisticated multi-finger interfaces is restricted by their cost and bulkiness. Another distinction between haptic devices is their attachment, either to the ground, which restricts the users' freedom of movement, or to the body, which limits the display of absolute forces. Alternatively, passive props and interfaces can provide haptic cues through their shape and material, but their lesser flexibility call for additional software techniques to enrich their feedback.

3D interaction techniques provide an additional layer of methods to carry out complex tasks such as selecting distant or occluded objects, or performing complex tasks with haptic guidance. Additionally, multi-sensory feedback can give the user information about the physics of the interaction occurring within the simulation through visual and auditory stimuli. Most notably, pseudo-haptic feedback provides and modulates haptic sensations through visual cues, and presents a promising alternative to traditional hardware haptic feedback.

Even if we are not yet able to fulfill Sutherland's vision of Virtual Reality in which virtual and real objects cannot be distinguished [START_REF] Sutherland | The ultimate display[END_REF], computer haptics Conclusion and its applicability to hand-based interaction made consequent advances in the last decades. A variety o hardware devices have been designed but more research is still necessary in order to conceive interfaces that could more satisfyingly fit the structure and dynamics of the human hand without burdening users. In the meantime, software solutions relying on novel interaction techniques and multi-sensory feedback could improve what we perceive of virtual worlds and thus bring tangible computer simulations closer to our fingertips. Hands are, along with the face, among the most expressive body parts, and subsequently their virtual representations are among the most difficult 3D models to animate for computer-generated movies or video games. In practice, a realistic animation of an action as simple as grasping an object or pointing a finger requires to define numerous successive hand poses, each of which is obtained through the careful manipulation of multiple articulations, for each individual finger. As of today, time-consuming manual methods or costly motion capture systems are used, and it seems that there is no satisfying solution for 3D animators to achieve complex hand poses in a simple, yet effective manner.

Part I

Improving the Control of Articulated Hand Models

Therefore, in this chapter, we address our first objective, reducing the complexity of hand models to leverage common user interfaces. To do so, we propose THING, a novel solution that is more lightweight than motion capture, yet benefits from the flexibility and accuracy of hand-based interaction. This approach is based on a tablet with which the animator controls finger configurations through multi-touch input. Each THING virtual finger corresponds on the tablet to a slider that the animator manipulates to control flexion and adductionn. THING therefore leverages the direct mapping between the animator's own fingers and the virtual hand through a morphologicallyconsistent user interface that can be rapidly assimilated by users. For example, a grasping motion is naturally obtained by sliding the fingers closer together, as illustrated in Figure 3.1.

Built upon THING, we additionally propose two variants with different controls for the global position of the virtual hand. MobileTHING capitalizes on the mobility of the tablet and exploits it as a prop to manipulate the whole hand model. With DesktopTHING, the location of the hand is controlled with a traditional mouse. This decoupling of controls allows for a less physically demanding manipulation and leaves one hand free for performing other tasks, such as invoking keyboard commands.

The contributions of this chapter can be summarized as follows:

a novel approach for controlling a virtual hand model in an intuitive manner using accessible multi-touch interfaces, the design and implementation of two variants of THING, a mobile version that integrates all DoF on a single device and a desktop version paired with a computer mouse, which were integrated into an animation tool, the evaluation of our techniques through two controlled user experiments that compare THING to traditional animation techniques and compare both variants in terms of performance and user appreciation.

In the remainder of this chapter, an overview of the traditional framework of computer animation as well as alternative animation techniques are presented. Then, we describe our approach and its current implementation. Finally, the formal evaluation of our techniques through two user studies is detailed and the possible evolutions of THING are discussed. Our approach is based on a morphologically-consistent multi-touch interface that applies the real motion of the user on the virtual model. For example, performing a grasping gesture or a pointing gesture replicates it on the hand model.

Hand animation: framework and techniques

This section reviews the traditional framework of computer animation, as well as beyond-the-desktop approaches for animating hand models, including multi-touch and tangibles.

From the 3D mesh to the animated character

Before a 3D model can be animated, its mesh must be bound to a structure resembling a digital skeleton, made up of joints and bones that act as "handles" that the animators manipulate to bend the model into a desired pose. This articulated system, with its degrees of freedom and its constraints, is called a rig. The general framework of computer animation is based on the manipulation of such rigs that govern the pose of 3D meshes [START_REF] Komatsu | Human skin model capable of natural shape variation[END_REF].

The most common approach for manually controlling rigs relies on forward kinematic animation, where joints are manipulated sequentially with a computer mouse [Parent, 2012]. For a highly-articulated hand rig supporting all 27 DoF, this makes posing an impractical and labor-intensive task. Rather than controlling each joint individually, inverse kinematics allows animators to manipulate a target point at each limb's extremity, which is used to automatically guide the rest of the joints chain [START_REF] Zhao | Inverse kinematics positioning using nonlinear programming for highly articulated figures[END_REF]. While it requires fewer steps than forward kinematics, individual fingers must still be manipulated one after the other.

A less direct approach consists in using drivers, i.e. custom rig components bound to UI widgets. For example, the flexion/extension of each finger might be bound to separate sliders. However, the lack of parallelism remains and the decoupling of the controls and model may cause the animator to split their attention. Our approach aims at providing more appropriate and parallelizable inputs by letting the animator perform the desired finger gestures himself.

Multi-touch

The use of multi-touch devices, which offer a simple and expressive way to control parameters in real time, has been explored for computer animation purposes. For example, [START_REF] Kin | Eden: a professional multitouch tool for constructing virtual organic environments[END_REF] built virtual stages for computer-generated movies with a multi-touch system. For animating virtual characters, Finger Walking is a technique that consists in recording a two-finger "walk" on a touch surface to pantomime leg movements [START_REF] Lockwood | Finger walking: Motion editing with contact-based hand performance[END_REF]. [START_REF] Gutiérrez | The mobile animator: Interactive character animation in collaborative virtual environment[END_REF] used a mobile animation system to map stylus motions on a PDA to rotations of a virtual avatar joints. [START_REF] Kipp | Multitouch puppetry: creating coordinated 3D motion for an articulated arm[END_REF] used two-finger input to swivel an articulated arm and position its hand. In their approach, tactile input is used to determine the pose of the hand by blending between several pre-defined poses and joints are not manipulated directly. However, finger configurations remain limited to variations of the reference poses. In contrast, THING we propose to leverage the directness of tactile input and benefits of multi-touch by mapping the controls of each virtual finger to the animator's own fingers so that more degrees of freedom can be manipulated simultaneously.

Tangible Interfaces

Other techniques rely on the animators manipulating tangible interfaces to control 3D models like puppeteers. [START_REF] Held | 3D puppetry: a Kinectbased interface for 3D animation[END_REF] used a depth camera to track real objects during natural toy-like interaction, but articulated objects are not supported. For articulated figures, physical armatures with flex sensors [START_REF] Johnson | Sympathetic interfaces: Using a plush toy to direct synthetic characters[END_REF] or joint encoders [START_REF] Knep | Dinosaur input device[END_REF] have been designed. These are generally employed for animating high-level dynamics such as gait and are not suited for finer finger movements. [START_REF] Jacobson | Tangible and modular input device for character articulation[END_REF] developed a modular tangible interface with pluggable joints and splitters to build a variety of skeletons, including hands. However, with this assemblage, users must use several of their own fingers to manipulate a single virtual finger so the number of DoF controllable simultaneously is reduced.

THING

In this section, we present THING, a multi-touch technique for manipulating a hand rig with a tablet as main input device. First, the simplified hand rig that is tailored for our system is presented. Then, we describe the associated multi-touch interface to control virtual fingers. Lastly, we present the two variants of THING that we designed to control the global position of the hand in space.

Rigged hand model with reduced degrees of freedom

The human hand has 27 degrees of freedom: each finger has three DoF for extension and flexion (one per phalanx) and one for abduction and adduction, the thumb has five DoF, and the wrist has six DoF for position and orientation. Due to physiological constraints, all these DoF cannot be controlled individually. For example finger flexion involves a close combination of each DoF of a finger. Considering these constraints, we designed a purpose-built rig with 16 DoF in order to simplify the formulation of hand poses while preserving their essential kinematics: flexion/extension, abduction/adduction and global position and orientation. Thus, we describe the pose H of our hand model with:

H = [p p p, o, F 1 , F 2 , F 3 , F 4 , F 5 ] F n = [f n , a n ], n ∈ [1, 5]
Position p p p and orientation quaternion o form the global frame of the hand model and encompass six DoF. Each parameter F n describes the individual state of a finger, which corresponds to the local frame of the hand: f n defines the flexion, where f n = 0 corresponds to an extended finger and f n = 1 corresponds to a fully flexed finger; a n describes the abduction, where a n = 0 means that the finger is in its neutral position,

b 1 ,c 1 c 1 c 1 ,b 2 b 2 b 2 ,c 2 c 2 c 2 (x) with x ∈ [0, 1] (b 1 b 1 b 1 and b 2 b 2 b 2 are extremities, c 1 c 1 c 1 and c 2 c 2 c
2 are control points). Such curves are common components in 3D editors which makes them easy to use and tweak to accommodate different morphologies, from realistic hands to disproportionate cartoon hands.

Thus flexing or extending a virtual finger is a matter of moving the target guiding its motion along its trajectory (Figure 3.2b). If we only consider flexion then the position of a target is defined as p p p target (F n ) = B(f n ). Abduction/adduction is realized by adding a lateral offset to the target's position with r a a scalar value bounding the extent of the abduction/adduction:

p p p target (F n ) = B(f n ) + a n ((c 1 c 1 c 1 -b 1 b 1 b 1 ) × (c 2 c 2 c 2 -b 2 b 2 b 2 )) r a

Controlling the model via multi-touch input

Leveraging multi-touch input for animating the hand model has several benefits: it provides efficient and natural input since users perform finger gestures that correspond to the transformations applied on the hand model, THING it provides parallel input so that all fingers can be controlled simultaneously whereas traditional techniques require to manipulate each finger one after another, it is based on common devices that can be nowadays found for cheap in a variety of sizes, it leverages the pre-established familiarity that users may have with tablets or other common tactile interfaces, which is particularly relevant for seasoned 3D artists who already use graphics tablets with similar tactile capabilities in their workflow.

Tactile surfaces provide two DoF per contact point (10 DoF if all fingers touch the tablet) and they contain sensors that can be used to track them in space, providing six additional DoF. It is thus possible to map these 16 DoF to the 16 parameters of our hand model. Similarly, [START_REF] Kim | Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor[END_REF] mapped 2D input (depth data from a postprocessed image) to finger motions with their Digits system. With THING however, multi-touch input avoid inaccuracies due to optical tracking and adapt such principle to consumer-ready hardware.

Regarding rotational tracking, most tablets embed sensors accurate enough to support real-time 3D interaction. Regarding positional tracking, current commercial tablets are not yet equipped with appropriate sensing capabilities but it is reasonable to expect that future generations of devices will provide such capability. In the meantime, we used a GameTrak positional tracking device (In2Games, United Kingdom) in our prototype.

Layout

The user interface displayed on the screen of the tablet consists of five sliders, each one associated to a virtual finger. Users control the flexion and abduction of a virtual finger by sliding their own finger along and away from the slider. The morphologicallyconsistent layout shown in Figure 3.3a enables a direct symmetry between real and virtual motions, and all virtual fingers can be controlled in parallel rather than sequentially. Additionally, the associated proprioceptive cues allow users to focus on the screen where the editor is displayed rather than looking at the multi-touch interface.

Each slider

S n = [c n c n c n , s n s n
s n ] associated with the finger F n goes through the 2D position c n c n c n of the user's finger when it touches the tablet for the first time. Its direction s n s n s n points toward a weighted barycenter of all the contacts (approximately at the center of the palm).

The first task of the user is to place his whole-hand on the tablet so that it detects handedness and associates each slider to its corresponding finger before displaying them. It is both possible to perform whole-hand input to control all virtual fingers simultaneously or to apply precise adjustments through single-finger input. For whole-hand input, users touch the tactile surface with all their fingers at the same time. This moves the sliders under their fingers. For single-finger input, they simply touch the slider associated with the desired finger. Each time five contacts are detected at the same time, a finger identification step is triggered. A distance-based heuristic identifies the thumb as the finger having the greatest distance with its closest neighbor. The other fingers are then identified by ordering them according to their horizontal position. Handedness is detected by comparing the horizontal position of the thumb with respect to the other fingers.

Controls

The displacement d d d of a user's finger on the tablet is sampled at 50 Hz to infer finger flexion/extension and abduction/adduction.

To control the flexion of a virtual finger, we measure the displacement of the user's finger along a slider: d d d is projected along the slider axis s s s to obtain a vector

d f d f d f effectively representing a change in flexion (Figure 3.3b). The direction of d f d f d f determines the direction of the virtual finger motion (if d f d f d f • s s s > 0 then the virtual finger is flexed, otherwise it is extended).
To control the abduction of a virtual finger, we measure the distance of the user's finger to the associated slider. Similarly, the direction of d a d a d a with respect to s s s ⊥ determines if an adduction or an abduction is to occur. The magnitude of these motions is determined by the distance to the slider.

d f d f d f = d d d • s s s ||s s s|| 2 s s s d a d a d a = d d d • s s s ⊥ ||s s s ⊥ || 2 s s s ⊥ THING

Mapping

To obtain the final f n parameter that defines the flexion of a virtual finger, the projected vector d f d f d f from the tactile input is modulated by a transfer function Θ with respect to the sliding speed

v f v f v f and a scaling coefficient k f : ∆f = Θ(d f d f d f , v f v f v f ) = { 0, if ||d f d f d f || < d min k f (||d f d f d f || -d min )G(v f v f v f ), otherwise
Θ thresholds the user's input so that small unintended slidings are discarded if they are less than the d min threshold. It also scales the magnitude of the sliding measured in pixel on the tablet by k f to the normalized [0, 1] range of the trajectory. Finally, it applies a gain G that modulates the resulting flexion depending on v f v f v f (a logistic function in our prototype).

To obtain the a n parameter that defines the abduction of the virtual finger, the projected vector d a d a d a from the tactile input is first scaled by a coefficient k a and then fed into the abduction parameter of the virtual finger:

a = sign(d a d a d a • s s s ⊥ ) k a ||d a d a d a ||

Mobile and desktop variants

For controlling the global frame of the virtual hand (its global position and orientation in space), we distinguish two flavors of our technique. Those two variants are compared in terms of performance and user appreciation in the user study in Section 3.3.

MobileTHING

With the MobileTHING variant, all degrees of freedom are handled with the tablet, even global position and orientation (Figure 3.4a). The position p p p of the virtual hand is obtained from the position of the tablet such that p p p hand = k p p p p tablet with k p a scaling coefficient increasing the virtual workspace volume. The orientation quaternion o of the virtual hand is controlled by rotating the tablet such that o hand = o ko tablet with k o a scaling coefficient. We subsequently apply a 1€ filter [START_REF] Casiez | 1€ filter: a simple speed-based low-pass filter for noisy input in interactive systems[END_REF] over the obtained quaternions to reduce noise. Additionally, two buttons were placed on the tablet's screen to lock translations and/or rotations. In this way, a complex configuration can be reached and then locked so that the tablet could be rested on a desk to avoid awkward and tiring postures.

DesktopTHING

With the DesktopTHING variant, only the fingers are controlled through the tablet.

The global frame is controlled via traditional mouse input in the 3D editor (Figure 3.4b).

While MobileTHING is fully integrated on the multi-touch device and allows to focus on a single device for direct spatial interaction, the need to hold the tablet in the air may be detrimental to performance over long work sessions. For this reason, DesktopTHING leverages a computer mouse which is expected to reduce physical fatigue. An additional motivation for this variant is its lack of requirement for external tracking devices; especially positional tracking devices since current-generation tablets are not yet able to provide this information. Moreover, DesktopTHING allows users to keep using the keyboard along the tablet to input shortcuts in their animation software.

Proof-of-concept implementation

We integrated THING in the Blender 3D editor1 in order to demonstrate its capabilities in a real animation context. Figure 3.5 features several examples of poses created with THING and frames from an animation sequence involving object manipulation can be seen in Figure 3.6. Figure 3.7 shows a use case in which a 3D character performs gestures and object manipulation. The user simply puts either his left or right hand on the tablet to control the corresponding virtual hand.

The additional screen space offered by the tablet is a valuable feature since it displays user interface elements to help the production of animations. In our prototype, we added several buttons to create keyframes and move in the 3D editor's timeline so that a complete animation could be crafted without leaving the tablet. More generally, the screen offers a variety of promising ways to ease hand animation; for example by adding buttons to save and load user-defined poses.

User evaluation

We conducted two user studies to evaluate the performance of both variants of THING compared to two other input devices classically used for hand animation techniques: a mouse/keyboard combination and a data glove. For both studies, participants had to reproduce predefined hand poses in a 3D editor. We measured the time to complete this posing task with each technique and participants had to answer a subjective questionnaire.

We designed a first experiment comparing only MobileTHING to a data glove and mouse/keyboard. MobileTHING was chosen since it is closest to the glove technique. The glove technique is expected to define an upper bound for performance to assess how our approach compares to a specialized and direct control method. In a second experiment we compare the two variants of THING with a mouse/keyboard, used to cross-validate the results with the ones of the first experiment.

Experiment #1: MobileTHING Apparatus

The evaluation was presented within the Blender 3D editor using Python scripting to retrieve real-time data from the various input devices used. The editor was displayed on a 24 inch screen placed approximately 80 centimeters in front of participants. The realistic hand model used is part of the LibHand library2 .

THING

In the experimental condition involving a data glove, participants wore a righthanded 5DT glove (Fifth Dimension Technologies, South Africa) whose finger movements were mirrored on the hand model (Figure 3.8b). Additional sensors tracked the position and orientation of their hand: a GameTrak device (In2Games, United Kingdom) was attached to their wrist for the position and a Trivisio Colibri inertial tracker (Trivisio Prototyping GmbH, Germany) was glued to the back of the glove for the orientation. All the devices were calibrated at the beginning of each session.

In this experiment, we used MobileTHING for which both the local and the global frames of the virtual hand are controlled via the multi-touch interface. Participants were given a 7 inch Galaxy Tab 3 tablet (Samsung, North Korea) that was also connected to a GameTrak for positional tracking. For a fair comparison, the same rotational tracker as the data glove was used instead of the tablet's sensors.

For the control condition, participants used a computer mouse and a keyboard to manipulate the different degrees of freedom of the hand model using standard widgets [Hess, 2010, p. 41] available in Blender (Figure 3.8a). For positioning the wrist, they could select and drag a translation widget along one of its three axes. Alternatively, they could type a keyboard shortcut triggering an unconstrained translation and drag the cursor. For orienting the wrist, they could select and drag a rotation widget around one of its three axes. Another keyboard shortcut triggered an unconstrained rotation. For controlling the finger, they could manipulate the translation widgets of the appropriate inverse kinematics target or press the translation shortcut once it was selected.

Participants

16 participants took part in the experiment. The results of four of these participants were discarded because they could not finish the evaluation due to issues with the data glove: the glove did not fit their hands and its sensor readings were inaccurate which made hand control very difficult. The 12 remaining participants were all right-handed males aged from 22 to 31 (M = 26, SD = 2.8). Regarding their level of familiarity with Blender, six had no prior experience, four stated that they were moderately familiar with it or with similar modeling tools, and two stated that they were familiar with it.

Procedure

Participants had control over a virtual hand and they were asked to match its pose with that of a target hand displaying four predefined configurations (Figure 3.9). These configurations were presented in increasing order of complexity: Pose #1 only required a simple flexion of the thumb, Pose #2 required a rotation around one axis and different degrees of flexion, Pose #3 required a rotation around two axes and different degrees of flexion, and finally Pose #4 required a rotation around three axes and different degrees of flexion. All poses required a translation. Each pose was presented 5 times. For each trial, the hand model was reset to its initial position and orientation, with all fingers extended. A pose was considered valid once the controlled virtual hand was superimposed with the ghost hand under a tolerance threshold. For reference, the threshold is equivalent to 10% of the length of the virtual hand. Visual feedback was provided by colored spheres placed at each fingertip and a larger sphere at the base of the hand. When the associated hand part was below the proximity threshold, the sphere color was modified. The editor interface was divided into four different views of the 3D scene (top, left, front, and a 3/4 perspective) so that participants could complete the task without controlling a camera (Figure 3.8c).

Participants were instructed to complete the task as fast as possible, they had to successfully complete the current trial before moving to the next one, and they could take break at anytime.

Experimental Design

The independent variables were the Technique (Mouse, Glove, MobileTHING), Pose (#1, #2, #3, #4) and Repetition (5 different trials). The four poses to reproduce were always presented in the same ascending level of difficulty since pretests showed that less experienced users would experience difficulties if they started directly with the more complex poses. The order of the techniques was counterbalanced among the participants using a Latin square. The total duration of the experiment was approximately 75 minutes including breaks. In total we had 12 remaining participants × 3 techniques × 4 poses × 5 repetitions = 720 total trials. At the end of the evaluation, participants also indicated which interaction technique they preferred and filled a questionnaire (Appendix C1) to rate ease, speed, comfort, precision, and fatigue on a 7-point Likert scale. A score of 7 for these criteria means a very positive rating except for fatigue where it means that the technique was very tiring (a rating of 1 means that it was not tiring at all).

Results

Completion time is the main dependent measure and is defined as the time taken between a three seconds countdown displayed at the beginning of each trial and the time when participants successfully matched the virtual hand with the ghost hand within the tolerance thresholds.

A repeated measures ANOVA showed a significant effect of Repetition repetitions and the others (we used Bonferroni correction for all post-hoc analysis).

We found no significant interaction for Repetition. We thus removed the two first repetitions from subsequent analysis. A repeated measures ANOVA showed a significant effect of Technique (F 2,22 = 16.8, p < 0.0001), Pose (F 3,33 = 16.3, p < 0.0001) and a significant Technique × Pose interaction (F 6,66 = 13.7, p < 0.0001) on completion time (Figure 3.10). Pairwise comparisons revealed significant differences (p < 0.002) between Mouse and the two other techniques (Mouse: 53.7s, Glove: 23.5s, MobileTHING: 26.8s). Pairwise comparisons showed significant differences (p < 0.04) between all poses except between #3 and #4, which confirmed that they were presented in ascending order of difficulty (#1: 15.6s, #2: 31.9s, #3: 44.1s, #4: 47.0s). Pairwise comparisons revealed no significant differences across techniques for pose #1 but Mouse revealed to be significantly (p < 0.02) slower compared to Glove and MobileTHING for poses #2 and #3. Pose #4 only revealed significant differences (p = 0.001) between Glove and Mouse.

A Friedman analysis on the subjective questionnaire showed no significant effect for global appreciation, ease, speed, and comfort criteria. A significant effect was found for the precision criterion (χ 2 = 2.19, p = 0.03). Post-hoc analysis revealed that the Mouse (M edian = 6) was more precise than the Glove (M edian = 4, p = 0.03). A significant effect was also found for the fatigue criterion (χ 2 = 2.41, p = 0.04). Participants felt more fatigue when using the Glove (M edian = 4) compared to the Mouse (M edian = 2, p = 0.04). No significant differences were found for MobileTHING. Finally, when they were asked to state which technique they prefered, three participants chose MobileTHING, four chose Glove, and four chose the Mouse technique.

Summary

As we hypothesized, the Glove technique performs overall well compared to the Mouse as it is more than twice faster on average. However, this comes at the cost THING of more fatigue and less precision as highlighted by the questionnaire. The Mo-bileTHING technique also outperformed the Mouse technique with an overall performance similar to the glove and the questionnaire did not reveal subjective differences with the other techniques.

Experiment #2: DesktopTHING

This second experiment aims at comparing the performance of the two variants of THING: DesktopTHING and MobileTHING. The experimental procedure and design are the same as for the first experiment except for the following differences.

For DesktopTHING, the local frame was controlled by the multi-touch interface while the global frame of the hand was controlled by a computer mouse. For controlling the global frame, the control scheme was identical to the mouse technique but participants could only manipulate the wrist and not the fingers.

Participants

Twelve participants who did not participate in the first experiment, took part in this second experiment. They were all right-handed males aged between 22 and 30 (M = 25, SD = 2.4). Regarding their level of familiarity with Blender, four had no prior experience, four were moderately familiar with it or with similar modeling tools and four were familiar with it.

Procedure and experimental design

We used the same procedure as the first experiment except that we removed Pose #1 from the configuration pool since the first evaluation showed that it was easy to reproduce, which allowed us to reduce the duration of the experiment. The independent variables of the experiment were the Technique (Mouse, MobileTHING, Desk-topTHING), Pose (#2, #3, #4, as represented in Figure 3.9) and Repetition (5 different trials). The presentation order for Technique was counterbalanced across participants and Pose was presented in ascending order of difficulty. In total, we had 12 participants × 3 techniques × 3 poses × 5 repetitions = 540 trials.

Results

A repeated measures ANOVA showed a significant effect of Repetition (F 4,44 = 67.8, p < 0.0001) on completion time, showing the presence of a learning effect. Pairwise comparisons revealed a significant difference (p < 0.0001) between the two first repetitions and the others. We found no significant interaction for Repetition. We thus removed the two first repetitions from subsequent analysis.

A repeated measures ANOVA showed a significant effect of Technique (F were found between all techniques for pose #3. For pose #4, significant differences were found between Mouse (p < 0.03) and the two other techniques.

A Friedman analysis on the subjective questionnaire showed a significant effect of Technique on global appreciation (χ 2 = 3.16, p = 0.004), ease (χ 2 = 2.48, p = 0.036), speed (χ 2 = 2.62, p = 0.02), precision (χ 2 = 2.95, p = 0.009) and fatigue (χ 2 = 2.52, p = 0.03) criteria. Figure 3.12 shows the answers to the subjective questionnaire for the significant criteria. Post-hoc analysis revealed that DesktopTHING (M edian = 7) was preferred to the Mouse (M edian = 5, p = 0.004). DesktopTHING (M edian = 6.5) was also found to be easier than the Mouse (M edian = 5, p = 0.035). Post-hoc analysis revealed that DesktopTHING (M edian = 6.5) was felt as being faster than the Mouse (M edian = 5, p = 0.02). In addition MobileTHING (M edian = 4.5) was found as less precise than DesktopTHING (M edian = 7, p = 0.009). Participants felt more fatigue when using MobileTHING (M edian = 3) compared to DesktopTHING (M edian = 1, p = 0.03). Finally, when they were asked which technique they preferred, 10 participants chose DesktopTHING, two chose MobileTHING, and none chose the Mouse technique.

Summary

In this experiment, the Mouse still showed much lower performance compared to the two versions of THING, which are similar in terms of completion times. Compared to the first experiment, the completion times for each pose are similar, except for MobileTHING and pose #2. We hypothesize that this may be due to a learning effect as participants started with pose #1 in the first experiment while they started with pose #2 in this experiment.

The subjective questionnaire shows an overall preference for DesktopTHING over THING the two other techniques. DesktopTHING was also rated higher than the Mouse in terms of global appreciation, ease, and speed. DesktopTHING was also better rated than MobileTHING for precision and fatigue.

Discussion

Through these two experiments, the mouse technique consistently showed the lowest performance compared to the glove and the THING variants: participants were more than 50% faster using the glove or MobileTHING in the first experiment and they were 40% faster using the THING techniques in the second one. The ability to control several degrees of freedom simultaneously with the glove and THING techniques is likely the main reason for this improved performance. Indeed, the mouse shows performance similar to those of the glove and MobileTHING with the first pose that only required the flexion of a single finger. The first experiment revealed similar performance between the glove and the Mo-bileTHING technique. However, THING worked consistently for all participants while the data glove was incompatible for four users (out of 16) who had smaller hands and had to stop the experiment since they could not control the hand model accurately. Additionally, several participants verbally expressed that pose #4 was difficult to replicate with the glove because they lacked the appropriate flexibility. However, they were not limited by THING since it is not dependent on the animator's own range of motion. Regarding the subjective questionnaire, participants felt significantly more tired and less precise with the glove compared to the mouse/keyboard. There was no significant effect with regards to these two criteria for THING, although some participants verbally mentioned that the tablet was heavy.

The second experiment focused on the evaluation of DesktopTHING, the variant that does not require to hold the tablet. The two versions of THING provided similar Conclusion performance but participants preferred DesktopTHING. It also scored higher for the ease, speed, precision, and fatigue criteria.

Overall, these results show that DesktopTHING is a good alternative to the mouse and keyboard in desktop environments. In scenarios requiring the user to stand up, MobileTHING appears as a potential alternative to data gloves: it shows similar performance and fits all users while the glove was incompatible with several of them. However, the tablet must be held with two hands and it is heavier than a data glove which could induce discomfort in certain configurations (e.g. with complex wrist rotations). Ultimately, we can imagine that users would not have to choose between MobileTHING and DesktopTHING. Instead DesktopTHING's control scheme would be used when the tablet lays on the desktop and it would switch to MobileTHING's as soon as the tablet is lifted.

Conclusion

This chapter proposed THING, an approach that reduces the degrees of freedom of articulated hand models to enable their control with common multi-touch interfaces. It introduced THING, a tablet-based interaction technique dedicated to the animation of 3D hand models. This approach leverages a direct mapping between the animator's own fingers and the virtual hand's through a morphologically-consistent user interface. We designed two variants of THING: (1) a mobile version taking advantage of spatial input to provide an animation system integrated in a single device and (2) a desktop version that delegates the control of the global frame of the hand to a computer mouse to reduce fatigue.

A user evaluation was conducted for assessing the performance benefits granted by our technique. In a first experiment, MobileTHING was found to be 50% faster than the traditional computer mouse technique with performance close to a data glove. In a second experiment, we found that DesktopTHING provided similar performance as MobileTHING and that it was predominantly preferred by participants.

These results suggest that THING is a valuable interaction technique that could prove useful to 3D animators. It could relieve expert users from the tediousness of traditional hand posing but it could also be relevant to non-experts since it provides a less intimidating alternative to classic forward/inverse kinematic rigs. Moreover, its barrier of entry is low compared to methods depending on less common and more costly hardware and its underpinning on common computer graphics primitives makes it easily integrable into 3D editors.

4

Separating the degrees of freedom of virtual hands for haptic manipulation: DesktopGlove Haptic interfaces that stimulate the sense of touch are needed to appreciate the physical properties of the virtual objects that we manipulate through the fingers. However, accommodating the many degrees of freedom of the human hand is a significant technical challenge and generic desktop interfaces are ill-suited to multi-finger interaction. Conversely, complex grounded multi-finger exoskeletons may provide a consistent feedback to the digits, wrist, and arm, but their use is restricted to a handful of professional applications due to a very high cost. Hence, it seems that there is currently no way to enjoy dextrous interaction with force feedback in an affordable manner.

In this chapter, our objective is to separate the degrees of freedom between several interfaces to better distribute controls and feedback. We thus designed Desktop-Glove, a novel approach that consists in combining two accessible haptic interfaces. Through different hands, and in parallel, each interface drives a subset of the degrees of freedom involved in object manipulation (Figure 4.1). Hence, one DekstopGlove hand is responsible for spatial displacements while the other handles grasping with the fingers. The force feedback resulting from the interaction is accordingly split between the hands depending on their respective duties. In this way, users can leverage affordable hardware to experience a compelling multi-finger force feedback through both hands, which was otherwise restricted to costly haptic interfaces.

The contributions of this chapter can be summarized as follows:

The separation of the degrees of freedom involved in multi-finger manipulation between two interfaces used in parallel, as an alternative to complex input devices handling all DoF in an integrated manner. We conducted a user study to assess how users integrate the separated DoF and how they perform comparatively to a traditional data glove. We additionally investigated which allocation of controls between dominant/non-dominant hands yields the best performance.

The separation of the force feedback resulting from multi-finger interaction between the two interfaces. In this way, each hand is subjected to forces that relate to its own frame of reference, and users perceive an exhaustive force feedback involving efforts on the wrist and the fingers. We compared different distributions of forces across the two hands in an object manipulation task and highlighted the user preferences in a second user study.

The following section presents an overview of two-handed interaction and then the current implementation of DesktopGlove. Then two user studies that evaluate the controls and feedback granted by our approach are presented.

Two-handed interaction

Two-handed interaction

Our approach separates the degrees of freedom involved in multi-finger manipulation between two hands. Thus, a careful examination of the foundations of bimanual interaction and its applicability to human-computer interaction is necessary.

Benefits of using both hands

First, two hands imply twice as many degrees of freedom and the possibility to split a single task into parallel sub-tasks, which can be quite beneficial for human-computer interaction. For example, [START_REF] Buxton | A study in two-handed input[END_REF] evaluated two-handed manipulation of 2D graphical elements as well as two-handed navigation/selection in a text document; each outperformed their one-handed counterpart. [START_REF] Gribnau | Comparing single-and two-handed 3D input for a 3D object assembly task[END_REF] found similar results in favor of two-handed interaction in a 3D manipulation task in which one hand moves selected objects in space while the other hand moves the rest of the environment.

Secondly, two-handed interaction enhances spatial comprehension. Indeed, proprioception gives a sense of the position of the hands relatively to each other and relatively to the body [START_REF] Hinckley | A survey of design issues in spatial input[END_REF][START_REF] Mine | Moving objects in space: exploiting proprioception in virtual-environment interaction[END_REF], which additionally reduces the need for continuous visual attention. [START_REF] Guiard | Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model[END_REF] developed a theoretical framework of two-handed tasks that outlines the complementarity between the dominant hand (DH) and the non-dominant hand (NDH). It states that a first component of bimanual interaction is the right-to-left reference, which states that the NDH acts as a spatial frame of reference for the DH. This relationship is often reflected in two-handed user interfaces. For instance, Balakrishnan and Kurtenbach [1999] controlled the view of a 3D scene with a computer mouse held in the NDH while the DH performed other tasks in parallel with a second mouse. Similarly, Hinckley et al. [1998] used both hand for a neurosurgical visualization application, with the NDH rotating a doll-head prop to change the user's view and the DH holding a cutting-plane prop.

Specificities of each hand

Another essential principle of Guiard's framework concerns the asymmetry of the hands: the DH typically acts at a finer temporal and spatial scale than the NDH. In general, the DH has been found to perform better in tasks that require precise, small-scale, displacements, while the NDH handles motions with a large amplitude better [START_REF] Kabbash | Human performance using computer input devices in the preferred and non-preferred hands[END_REF].

Current limitations of multi-finger haptics

The inadequacy of current haptic hardware to support multi-finger interaction is attributable to the overwhelming number of degrees of freedom of the human hand. Because of technical considerations including bulkiness, mechanical complexity, and cost, handling them all through a single device is unpractical.

Simplifications may occur in favor of leaner designs, generally by considering only the essential functions of the hand. For example, the flexion of individual finger joints is rarely, if ever, constrained by haptic interfaces. Instead, multi-finger devices may simply pull the fingertips, which amounts to reducing the chain of joints that make a finger to a single conceptual DoF corresponding to its global curling. Still, even with such simplifications, most haptic devices only support a limited number of DoF that apply either to the whole hand or to the fingers only.

Those practical limitations can be summarized as a size problem: designers of haptic interfaces cannot fit so many degrees of freedom on the small volume of the hand without burdening users. For this reason, we propose to distribute the degrees of freedom involved in multi-finger manipulation between two hands that operate in clearly separated workspaces. Each hand controls a specific aspect of object manipulation and receives appropriate force feedback in return. In the end, our system exposes users to the same controls and feedback as a grounded exoskeleton would, while being much more accessible.

DesktopGlove

Separating the degrees of freedom

For practical reasons related to the sophisticated nature of the hand, individual haptic interfaces cannot support the many degrees of freedom involved in multi-finger manipulation without complex and costly mechanisms. Consequently, we propose Desktop-Glove, a bimanual setup that distributes multi-finger controls and feedback between two distinct interfaces.

This general idea raises the question of the most appropriate distribution of degrees of freedom between the hands. In this work, we chose to distinguish (1) the DoF that are part of the global frame of reference of the hand from (2) the DoF that are part of its local frame of reference.

Global frame of reference

We designate one of the user's hand to control the global frame of reference of the virtual hand he is interacting through, which consists of its position and orientation in space. Thus in practice, this user's hand is in charge of performing large-scale actions such as reaching for virtual objects or pushing them. The forces that are displayed DesktopGlove through this hand are those that relate to the interaction of the hand as a whole with the virtual environment, such as collision or gravity forces.

In our implementation, a Geomagic Touch haptic arm (3D Systems, USA) provides the controls and feedback described above (Figure 4.1, Right). Users move the virtual hand by moving the stylus of the device in a similar manner to maintain an isomorphism between the real motion and its restitution in the virtual environment. A rotational offset is also applied to the hand model so that the alignment of the user's hand holding the stylus and that of the virtual hand match.

Local frame of reference

We designate one of the user's hand to control the local frame of reference of the virtual hand, which consists of the precise motion of its fingers. This hand is responsible for a variety of actions that require precision such as grasping, brushing fingers against an object, or squeezing another to feel its material.

In our implementation, we used a custom variant of the DigiHaptic device [START_REF] Casiez | The DigiHaptic, a new three degrees of freedom multi-finger haptic device[END_REF], with motors arranged in a parallel layout to handle two pinching fingers (Figure 4.2). Additional 3D-printed rings were installed on the levers to insert the fingers in and the device provides forces that bring the finger together or apart to simulate haptic manipulation. Comparatively to gripper handles that output a single separation force to keep the fingers apart, this interface applies individual forces to each finger. Moreover, different layouts could accommodate more fingers to support more complex grasps. Note that any multi-finger force feedback device would be appropriate to serve this purpose; for instance, other systems from the literature that handle different numbers of fingers would also be compatible.

Motivations

Separating the local and global frames of reference is motivated by hardware reasons and previous results from the study of two-handed interaction:

1. Force feedback devices tend to specialize in dealing with a specific frame of reference. As outlined in Section 2.3, desktop arms provide whole-hand forces whereas multi-finger interfaces deliver within-hand efforts. In order to capitalize on the current supply of haptic hardware, we thus split the degrees of freedom in a manner that is consistent with the available technology.

2.

There is a part of common sense in this division, as moving the fingers and moving the hand can reasonably be thought of as distinct actions. Such intuition is rooted the work of Jacob et al. [1994] on the separability of tasks and interfaces, which states that the perceptual structure of a task must match the control structure of the input device used. Hence, well-designed interfaces should not separate integral components that are inherently coupled, like the displacements along different axis. However, the finger movements can be perceived as separate from the hand position and orientation.

3.

Studies in the field of psychophysics have shown the asymmetry of two-handed interaction and highlighted the strengths of each hand. Thus, it seems appropriate to associate each hand with a frame of reference that fits its capabilities. For instance, the dominant-hand is more suited to precise small-scale movements [START_REF] Kabbash | Human performance using computer input devices in the preferred and non-preferred hands[END_REF]]. The user study described below aims at determining which hand is best suited to the control of each frame of reference.

Benefits

Separating the degrees of freedom in this manner presents the following benefits over the existing solutions:

Exhaustive force feedback DesktopGlove exposes users to a variety of haptic effects acting both on the wrist and on the fingers, to express the various forces involved in hand-based interaction. Traditionally, such effects are individually obtained via separate haptic devices that do not work in concert but our approach enable them all through a single two-handed setup.

Common hardware The implementation that is described below is accessible and easily reproducible. It leverages a common haptic arm that is affordable and often found in research laboratories. Regarding the multi-finger interface, it is easily to replicate, as its minimalist design suggests. Moreover, various other multi-finger interfaces would be appropriate to fill this role.

Isomorphism Even though users wield two separate interfaces, they still perform the gestures that are reflected in the virtual environment. In Virtual Reality applications involving only haptic arms, non-natural commands are necessary to grab an object, like pushing a button. With our approach however, users do perform the grasping motion with their own fingers and move the wrist in space to travel in the virtual environment.

Distributing the force feedback between the interfaces

The virtual hand that conducts the interaction is driven by a physical model that distributes the efforts resulting from object manipulation between the two interfaces. The hand model itself is supported by an underlying skeleton made from an assemblage of rigid bodies forming the palm and the phalanxes (Figure 4.3, Left). Physical constraints connect those segments together and apply realistic limits to the motion of each joint. Hence, in the end, the skeleton adapts to the shape of the objects it touches while keeping a plausible configuration.

Coupling this articulated model to the pair of input devices is done via an intermediary model that is not visible, in a manner analogous to [START_REF] Borst | A spring model for whole-hand virtual grasping[END_REF]. This intermediary model, which consists only of a palm and fingertips (Figure 4.3, center), is responsible for driving the motion of the virtual hand. It only serves a positional role and does not react with the rest of the environment. The position of the palm is governed by the haptic arm handling the global frame whereas the position of the fingertips depends on the multi-finger device that handles the local frame. However, each of its part is linked through a spring-like constraint to the corresponding one in physical hand model. In consequence, the physical model is attracted toward the intermediary model and tends to adopt a similar configuration while respecting the constraints of the virtual environment.

The intermediary model is also responsible for the computation of the forces to be rendered through each interface The intensity of the force feedback sent to the devices then depends on the distance between corresponding parts. In free space for instance, both physical and intermediary model should overlap and no feedback is sent to the interfaces. If the user pushes against a virtual object however, the intermediary model penetrates it while the physical one stays at its surface. The distance between the palms determines the feedback to be provided by the haptic arm while the distance between corresponding fingertips determine the intensity of the feedback on the appropriate lever of the multi-finger interface. Figure 4. [START_REF] Mcneely | Voxel-based 6-DOF haptic rendering improvements[END_REF] shows examples of interaction cases and the resulting force feedback sent to the user.

User evaluation

We conducted two user studies to evaluate how users handled multi-finger manipulation with separate degrees of freedom. For both studies, we measured the time to complete given tasks and participants had to answer a subjective questionnaire.

The goal of the first experiment was to evaluate if users could control separated DoF in an effective manner, when no force feedback is involved. To do so, their per- formance for controlling a virtual hand was evaluated in a posing task. Our approach, DesktopGlove, was compared to a traditional data glove that integrates all degrees of freedom on a single hand. A side-goal of this evaluation was two assess which of the global or local frames should be controlled by the dominant and non-dominant hands.

The goal of the second experiment was to assess how force feedback and its distribution between the two interfaces affects performance and user appreciation. Participants had to complete a docking task to evaluate in a quantitative and qualitative manner both their control and their appreciation of the force feedback. The results from this experiment are expected to provide insights into which configuration is preferable: no forces at all, forces applied on both hands or partial feedback on a specific hand.

Experiment #1: separated controls vs. integrated controls Apparatus

The evaluation was presented on a 24 inch screen displaying a fixed view of the virtual environment (Figure 4.5). The environment featured shadows providing additional depth cues helping in the task. The virtual hand controlled by participants was a realistic-looking model from the LibHand library1 .

In one experimental condition, participants wore a 5DT glove (Fifth Dimension Technologies, South Africa). Only the movements of the thumb and index fingers were considered and input from the other digits was ignored. Since the hardware is sensible to user morphology (see the user study of the Virtual Mitten, Chapter 6), only participants whose hand size fit the glove were considered and the system was individually calibrated.

For controlling the global position and orientation of the virtual hand, a Razer Hydra (Sixense, USA) 6-DoF tracker with a 1 mm/1°spatial resolution was used. It was attached to the wrist when the data glove was used and held in the appropriate hand for the other conditions.

Our custom multi-finger device controlled the finger curling in the experimental conditions making use of our approach. The levers were arranged in a triangle layout with the central motor slightly offset to the front of the device so that users could insert their thumb in it comfortably, and insert the index finger in the left or right lever depending on the hand they had to use at the moment.

Participants

Twelve participants took part to the experiment. All the participants were males aged from 23 to 30 (M = 26, SD = 3) who identified their right hand as the dominant one.

Procedure

Participants were asked to complete a posing task that consisted in quickly reproducing certain postures with their virtual hand. Those postures were illustrated by a semi- transparent ghost hand that had to be overlapped. Seven postures involving various positions, orientations, and finger configurations were presented to participants in an ascending order of difficulty (Figure 4.6). All postures except for the first one had an horizontally mirrored variant placed on the opposite side of the environment in order to ensure fairness between the experimental conditions. Thus, pose #1 only required a translation of the virtual hand. Poses #2 and #2 ′ additionally required a rotation around one axis and the flexion of a single digit. Poses #3 and #3 ′ required a rotation around two axes and the flexion of the other digit. Finally, poses #4 and #4 ′ required a complex rotation around three axes and different amounts of flexion for each digit.

A trial was considered valid once the virtual hand overlapped the ghost hand within a tolerance threshold of 1.5 cm in position, 15°in rotation and 0.15 in normalized flexion. Under those conditions, the ghost hand disappeared and a blinking marker prompted the participant to go back to its starting position at the bottom of the screen, which triggered a countdown leading to the next trial.

Conditions

The goal of this first evaluation was to assess if users can efficiently control a virtual hand with separated degrees of freedom. Thus, our approach is compared in terms of performance to a traditional method that integrates all DoF on a single interface. An additional objective of this user study is to determine which hand -dominant or non-dominant -is best suited to the control of the local and global frames of reference. Therefore, participants are subjected to the three following experimental conditions that vary the roles attributed to each hand (Table 4.1):

Integrated. All of the controls are handled by the dominant hand, equipped with a data glove. Curling the fingers curls those of the virtual hand correspondingly. Translating and rotating the hand in space moves the virtual hand accordingly. This direct and natural mode of interaction is expected to provide an upper bound in terms of performance.

SeparatedA. In this variant of our approach, controls are split between two hands. The non-dominant hand controls the flexion of the virtual fingers through the multifinger device that has been described previously. The position of the virtual hand is updated from the real position of the participant's dominant hand, which holds a spatial sensor.

SeparatedB. In this second variant, controls are also split between two hands but the interfaces are reversed: the non-dominant hand controls the global position and orientation while the dominant hand controls curling. Even though the devices are swapped, the virtual hand is still right-handed and the user's thumb (respectively index finger) still controls the virtual thumb (respectively index finger).

Experimental design

The independent variables of the experiment were the Technique (Integrated, Sep-aratedA, SeparatedB), Pose (#1, #2, #2 ′ , #3, #3 ′ , #4, #4 ′ ) and Repetition (4 different trials). The seven poses to reproduce were always presented in the same ascending level of difficulty since pre-tests showed that less experienced users would experience difficulties if they started directly with more complex poses. The order of the techniques was counterbalanced among the participants using a Latin square. The duration of the experiment was approximately 45 minutes including breaks. In total, we had 12 participants × 3 techniques × 7 poses × 4 repetitions = 1008 trials.

Collected data

Completion time is the main measure and is defined as the time taken between the beginning of each trial and the time when participants successfully matched the virtual hand with the ghost hand within the tolerance thresholds. At the end of the evaluation, participants filled a subjective questionnaire (Appendix C2) to rate each technique in terms of global appreciation, ease, speed, accuracy, comfort and fatigue on a 5-point Likert scale. A score of 5 for these criteria means a very positive rating except for fatigue where it means that the technique was very tiring (a rating of 1 means that it was not tiring at all). The participants were also asked to order the techniques by personal preference.

Results

A mixed repeated measures ANOVA showed a significant effect of Technique (F (2, 976) = 8.36, p < 0.001) and Pose (F (6, 976) = 26.81, p < 0.001) on completion time (Figure 4.7) as well as a significant Technique × Pose interaction (F (12, 976) = 4.21, p < 0.001).

A post-hoc analysis using a Tukey test revealed significant differences between the Integrated technique and the two other techniques (p < 0.001 for SeparatedA, p < 0.004 for SeparatedB) (Integrated=9.63s, SeparatedA=7.69s, SeparatedB=7.48s).

Post-hoc analysis also showed significant differences between Pose#2 and the other poses (p < 0.001) as well as between Pose#3 and the other poses (p < 0.001).

There is no significant different between Pose#2 and Pose#3. There is also a significant difference between Pose#1 and Pose#2 ′ (p < 0.001) and Pose#1 and Pose#3 ′ (p = 0.01) (Pose#1=5.32s, Pose#2=11.58s, Pose#2 ′ =7.73s, Pose#3=11.67s, Pose#3 ′ =7.54s, Pose#4=6.26s, Pose#4 ′ =7.79s).

Concerning the interaction effect, post-hoc analysis revealed that the Integrated technique was significantly slower than the SeparatedA technique for Pose#2 (p = 0.009) and slower than the SeparatedB technique for Pose#3 (p < 0.001). There was no other significant effect.

A Friedman analysis on the subjective questionnaire showed no significant effect for the global appreciation, ease and speed criteria. A significant effect was found for the accuracy criterion (maxT=2.90, p = 0.01). Post-hoc analysis revealed that the Integrated technique (Median=3) was perceived as less accurate than SeparatedA (Median=4, p = 0.01). A significant effect was also found for the comfort criterion (maxT=2.83, p = 0.01). Participants felt that the two Separated techniques (Me-dian=4 and 3) were more comfortable than the Integrated technique (Median=2, Finally, when they were asked to state which technique they preferred, eight participants chose SeparatedA as their favorite technique, two participants chose Sep-aratedB and two participants chose the Integrated condition.

Summary

Overall, the two Separated techniques performed well compared to the Integrated technique. Subjective answers also showed that the Separated techniques were both felt as more comfortable than the Integrated technique, and that the SeparatedA technique was less tiring than the Integrated technique.

Pose#2 and Pose#3 took significantly more time to complete than the other poses, especially because the Integrated technique was significantly slower for these specific tasks compared to the Separated techniques.

Finally, the SeparatedA technique, i.e. controlling the fingers with the nondominant hand and controlling the position with the dominant hand, was mostly preferred by participants.

Experiment #2: object manipulation with force feedback

Apparatus

This experiment was conducted in the same desktop environment as the previous evaluation. In all conditions, users interacted through a single pair of devices, with the layout that was preferred by participants in the first user study. Thus, the virtual fingers were controlled with our custom multi-finger device, as in the previous experiment, and the position and orientation of the virtual hand was controlled through a Geomagic Touch force-feedback arm (3D Systems, USA) with the dominant hand (Figure 4.8b).

Population

Twelve participants took part to the experiment. All the participants were right-handed males aged from 22 to 31 (M = 26.5, SD = 2.9) who identified their right hand as the dominant one.

Procedure

Participants were asked to complete a docking task which consisted in placing a virtual toy cube in certain configurations indicated by a semi-transparent target. They first had to grasp the cube which was on the floor and then dock it on the target (Figure 4.8). They could drop the cube and grab it from another angle to complete complex rotations in several steps, if necessary.

Three pairs of poses involving different orientations had to be reproduced. Each pair consisted of a same pose, either placed on the left of the environment (#n) or on the right (#n ′ ), at the same distance from the starting position. Poses #1 and #1 ′ only required to grab and translate the cube. Poses #2 and #2 ′ additionally required a leftward rotation of 90°so that the left side faced the participant. Poses #3 and #3 ′ required a downward rotation of 90°so that the top side faced the participant. The cubes featured colored numbers on their faces as an additional visual cue.

A trial was considered valid when the toy cube overlapped the target within a tolerance threshold identical to the first experiment. Then, the cube and the target disappeared and users had to return to their initial position to trigger a countdown leading to the next trial.

Conditions

The goal of this second experiment was to evaluate how users could handle the separation of force feedback between two interfaces both in terms of control and appreciation. Regarding control, it is indeed necessary to assess if force feedback has an impact on performance. Regarding appreciation, it is necessary to ensure that users integrate those separated degrees of freedom well and that the resulting sensations are clearly understandable. We exposed participants to the following distributions of force feedback between the two hands: Local forces correspond to opposing forces delivered on the fingers of the nondominant hand, to simulate the sensation of grasping the cube. When an object was grasped, it delivered opposing forces to the thumb and index finger via the DigiHaptic interface to push them to its surface.

Global forces correspond to the efforts delivered to the wrist of the dominant hand via the desktop haptic arm, and include the weight of the cube and collisions with the floor. The intensity of the downward force was adjusted to be consistent with the visual appearance of the object (a wooden toy cube). A large upward force was applied by the device if participants attempted to pass through the floor.

Experimental design

The independent variables were the Feedback (None, Fingers, Wrist, Both), Pose (#1, #1', #2, #2', #3#3', ) and Repetition. The order of the types of feedback and poses were counterbalanced among the participants using a Latin square. The total duration of the experiment was approximately 20 minutes. In total, we had (M edian N one = 1, M edian F ingers = 1, M edian W rist = 4, M edian Both = 4) for the perception of contact. Concerning the perception of shape, the post-hoc analysis revealed that the Both feedback (Median=4) was found as better to perceive the shape compared to the None (Median=1, p < 0.001) and Wrist (Median=1.5, p < 0.001) feedbacks.

Finally, when they were asked to state which technique they preferred, 7 participants chose Both as their favorite feedback, 3 participants chose Fingers, 1 participant chose Wrist and 1 participant chose None.

Summary

The quantitative evaluation did not reveal any significant effect on task performance between the different types of feedback. However, the subjective evaluation gave significant information concerning the participants appreciation with respect to different criteria. Thus, force feedback in both hands was found to be more realistic and more accurate than no haptic feedback in both hands. Haptic feedback in the wrist of the dominant hand was found to be more tiring than with the Fingers feedback, whatever the feedback in the non-dominant hand. Thus, the use of a haptic device increased the perception of fatigue of the participants. On the contrary, the use of haptic feedback in the dominant hand seemed to increase the perception of weight, contact and shape compared to no haptic feedback in the dominant hand, thus improving the perception of the objects when manipulating them in the virtual scene. Participants thus commented that "it was easier to grab the cube and place it" and "it felt natural" for the use of the Both feedback. The use of the Fingers feedback was felt to "ease the grasping, especially when the cube is occluded by the hand".

Discussion

Controlling separated DoF. Our design of DesktopGlove introduced the main question of the users' ability to coordinate both their hands to control the separated degrees of freedom of one virtual hand. The results of the first experiment showed that users are able to simultaneously control the separated degrees of freedom without loss of performance compared to a data glove. They also showed a higher preference for Desk-topGlove in terms of fatigue and comfort. In addition, we found that the preferred configuration relies on the use of the DigiHaptic on the non-dominant hand and a spatial tracker on the dominant hand.

Perceiving separated forces. The second experiment assessed the quality of perception of separated forces. We found that users were not disoriented by the force feedback and there was no degradation of performance. Participants rated the force feedback on both hands higher than the other conditions, in terms of precision and realism. Moreover, the use of a haptic feedback on both hands was felt as better to perceive the weight, the contact, and the shape of the manipulated objects.

Conclusion

In this chapter, we addressed the challenge of providing multi-finger control and force feedback for hand-based interaction. To do so, our objective was to separate the many degrees of freedom of virtual hands between two interfaces. We thus introduced Desk-topGlove, a new bimanual multi-finger force feedback setup that provides the control of fingers grasping, hand position, and orientation together with force feedback both on the user's hand and fingers. This approach is built on the core idea of controlling in parallel the global frame of reference (position and orientation) of a virtual hand with one device and the local frame of reference (finger movements) with another.

The results of a first experiment showed that users were able to control these separated degrees of freedom with an overall higher performance and preference compared to a data glove. A second experiment showed that the distribution of force feedback between both hands did not degrade performance and that users preferred a consistent force display on both their hands.

The results of this work opens the way to the design of mechanically simplified haptic interfaces that distribute the degrees of freedom without sacrificing performance or expressiveness. A limitation often encountered with existing haptic devices is their limited workspace, as they are often designed for a desktop use. Several "human-scale" haptic interfaces provide larger workspaces [START_REF] Dominjon | Novel devices and interaction techniques for human-scale haptics[END_REF] but the resulting hardware is often expensive and cumbersome. Thus, in most cases, haptic devices limit the mobility of the users, which prevents their use with VR setups such as immersive rooms or head-mounted displays.

In this chapter, our objective is to leverage lightweight passive components to provide a similar force feedback without restraining the user's mobility. To do so, we propose the Elastic-Arm, a simple and cost-effective approach for incorporating haptic feedback in immersive virtual environments. This system is based on an elastic armature that is mounted on the user's body and linked to his hand. The Elastic-Arm is built from simple and inexpensive components (arm exercisers, 3Dprinted parts and a gaming tracking system) and it provides a progressive passive Elastic-Arm egocentric haptic feedback when the user extends his arm with the goal of performing 3D interaction tasks. Such passive force-feedback can be exploited in virtual environments for improving either 3D interaction or the perception of haptic properties, as illustrated through several use cases presented in this chapter.

The contributions of this chapter can be summarized as follows:

the design of a novel body-mounted interface, made of inexpensive components, that provides haptic feedback during 3D interaction with the hand, two illustrative use cases enhancing existing interaction techniques that originally used grounded haptic interfaces for selecting out of reach objects and navigating in virtual environments, two illustrative use cases enhancing the perception of the virtual environment by first simulating its physical boundaries through the Elastic-Arm, and then simulating the various levels of stiffness of virtual objects with pseudo-haptics, a pilot user study that assesses the user's appreciation of the Elastic-Arm as well as the effectiveness of the pseudo-haptic effect that we propose to associate with the elastic armature.

In the remainder of this chapter, we first introduce the general concept of the Elastic-Arm and detail its current implementation. Then, we present different use cases that illustrate how this approach can be combined with existing and proven interactive techniques, first for enhancing the interaction possibilities, and secondly for enhancing the perception of the virtual environment. Then, the a preliminary user study that evaluates the Elastic-Arm and our pseudo-haptic effect is presented and discussed. Finally, we propose possible extensions of the general concept of the Elastic-Arm to simulate other haptic properties.

Figure 5.1 -The Elastic-Arm is a body-mounted armature that provides egocentric passive haptic feedback. It presents an alternative to more complex active haptic devices that are generally less adapted to large immersive environments. In this example, the user performs a selection task by stretching his virtual arm using a combination of the Bubble [START_REF] Dominjon | A comparison of three techniques to interact in large virtual environments using haptic devices with limited workspace[END_REF]] and Go-Go [START_REF] Poupyrev | The go-go interaction technique: Non-linear mapping for direct manipulation in VR[END_REF] techniques reimplemented with our system.

The Elastic-Arm

The Elastic-Arm

Concept

The system that we propose is a novel body-mounted elastic armature that enhances interaction in virtual environments by providing passive haptic feedback to the user's arm. It relies on an elastic cable that links the user's hand to his body (Figure 5.1). When stretching out the arm in order to perform interaction tasks, an effort proportional to the stiffness of the cable is felt. This egocentric resistance force can then be leveraged in order to incorporate haptic feedback either into interaction techniques without any haptic components or into interaction techniques that originally relied on active haptic devices or passive props. Various illustrative applications are presented in Section 5.2.

Implementation

For providing the haptic feedback of the Elastic-Arm, elastic cables sold as arm exercisers were used. Each end of the elastic cable is attached through a hook to 3D-printed straps; one on the shoulder and the other on the back of the hand. Velcro bands wrapped around the arm and around the torso ensure that the shoulder strap remains securely fixed when the user stretches his arm and pulls the elastic cable (Figure 5.2a).

As illustrated in Figure 5.2b, this setup inherently creates a dichotomy between two states: when the elastic is relaxed and when it is taut. The transition between these states can be controlled by the user by extending his arm as well as gauged thanks to the provided haptic feedback. Designing interaction techniques around this feature requires both the position of the hand h h h and the position of the shoulder s s s in order to obtain the reach vector r r r = h h h -s s s. In our prototype, a Razer Hydra (Sixense, USA) was used for tracking these positions. However, any other tracking system for desktop interaction or for large physical spaces can also be used, camera-based setups for instance.

The rest length of the elastic cable is defined as d e . Thus, when the hand is close to the body (||r r r|| < d e ), the elastic cable is not taut and the user's arm moves in an unconstrained manner. However, when ||r r r|| > d e , an effort proportional to the cable's stiffness and the arm extension pulls the arm back. Another potentially useful value is the maximum reach of the user's arm, d m . These thresholds can be obtained through a short calibration step by, first, asking the user to stretch his arm until the elastic is taut (d e ) and then asking him to stretch his arm as much as possible (d m ). The thresholds can then be exploited for the design of relevant control schemes.

Discussion

The design of the Elastic-Arm is motivated by established literature showing that passive haptic feedback is involved in enhancing both performance [START_REF] Borst | Evaluation of a haptic mixed reality system for interactions with a virtual control panel[END_REF]; Viciana-Abad et al., 2010] and perception [START_REF] Kohli | Redirected touching: Warping space to remap passive haptics[END_REF][START_REF] Lécuyer | Pseudohaptic feedback: Can isometric input devices simulate force feedback?[END_REF]Paljic et al., 2004]. It also provides a stronger sense of presence [START_REF] Insko | Passive Haptics Significantly Enhances Virtual Environments[END_REF]. Through this system, our aim is to provide a mobile, low-cost, and easily reproducible mean to leverage such an essential feedback. Thus, existing interaction techniques could be augmented with the use of our system; which additionally opens the gate for all-new interaction methods specifically designed around its egocentric haptic capabilities.

In order to illustrate the possibilities offered by the Elastic-Arm, the following section presents several application examples, focused on control and perception. For these illustrative use cases, we chose to leverage the virtual hand metaphor since it provides an engaging and meaningful representation for descriptive purposes. This metaphor also fits well with the egocentric nature of the Elastic-Arm. Therefore, some of the demo applications display a bright orange hand (inspired by Mr. Tickle, a cartoon character capable of stretching his arms to perform various feats) and others display a realistic avatar.

Illustrative use cases

In this section, we present several illustrative use cases based on existing interaction techniques. These techniques were originally designed and evaluated with haptic feedback in mind and rely either on grounded haptic devices or one static passive props. Here, we adapt them to the Elastic-Arm in order to demonstrate how our system can make such methods egocentric and more mobile whilst providing a similar haptic feedback.

Illustrative use cases

The use cases are divided into two categories. Firstly, we explore how interaction tasks such as object selection and navigation can benefit from the Elastic-Arm. Then, we explore perceptual effects made possible by our system and propose examples of methods for enhancing the users' perception of a virtual environment.

Using the Elastic-Arm to expand the virtual workspace

Here, we propose two illustrative use cases that enable (1) selection of distant objects by stretching a virtual arm and (2) navigation toward out-of-view areas in order to reach occluded objects.

Selection of distant objects with the Bubble technique

The first example that we propose enables users to select virtual objects that are out of reach by stretching their virtual arm (Figure 5.3). This example is based on the hybrid position/rate control of the Bubble technique [START_REF] Dominjon | A comparison of three techniques to interact in large virtual environments using haptic devices with limited workspace[END_REF]. It is also reminiscent of the Go-Go technique [START_REF] Poupyrev | The go-go interaction technique: Non-linear mapping for direct manipulation in VR[END_REF] with regards to its arm-extension mechanics.

Figure 5.3 -The virtual arm stretches to grab out-of-reach objects. In rate-control mode, the virtual arm is extended beyond the limit of the user's real reach (gray dots).

In this scenario, users control a virtual hand and select objects by touching them. Similarly to the Bubble technique, two control modes can be differentiated. To begin with, the virtual hand is position-controlled with a 1:1 mapping if ||r r r|| < d e , which is relevant when the user performs interaction tasks in his close vicinity. However, when the elastic cable is taut (||r r r|| ≥ d e ), the virtual hand switches to rate-control and stretches to reach faraway targets (Figure 5.1 and 5.3). Flexing the arm below the elastic threshold at any moment triggers a rewinding animation that quickly rolls the arm backward to its initial position.

In rate-control mode, when the virtual arm stretches, users have control over the speed of the virtual hand by extending their arm farther away from their shoulder. They also steer the hand by pointing their arm in the direction that they wish to go. These relations are described in the following equation and Figure 5.

4, with h v h v h v the position of the virtual hand and k p a scaling coefficient:

h h h v (t + 1) = h h h v (t) + k p r r r (||r r r|| -d e )
This implementation of the Bubble technique with the Elastic-Arm shares similar principles with its original version that made use of active haptic feedback. Distant targets can similarly be reached by leveraging an hybrid control scheme. However, even without an active device, the interaction is still assisted by the same feedback. Notably, the transition between position and rate control (the boundary of the bubble) can be perceived by users. Similarly, the haptic feedback perceived when stretching the virtual hand is proportional to its speed. As shown by Zhai [START_REF] Zhai | User performance in relation to 3D input device design[END_REF], rate-control provides better performance with self-centering elastic devices such as ours. Thus, the Elastic-Arm seems well adapted to control schemes like that of the Bubble.

Navigation toward occluded objects with the BubbleCam technique

We propose a variation of the previous method that additionally enables to select occluded objects by navigating across the virtual environment and around obstacles. The associated controls are similar to the previous method but the displayed view is similar to the BubbleCam technique [START_REF] Dominjon | A comparison of three techniques to interact in large virtual environments using haptic devices with limited workspace[END_REF].

Here, the virtual camera displaying the scene is fixed to the virtual hand. In this way, the precision of the control does not decrease as the hand moves away since the user keeps the same relative viewpoint. In order to accommodate the egocentric nature of the Elastic-Arm, we also complemented the Bubble's original control scheme with rotations: users can rotate their virtual hand by pointing away from the forward direction F F F . These relations are summed up by the following equations and Figure 5.4, with h v h v h v and o v the position and orientation of the virtual hand, k p and k o scaling coefficients and q the quaternion representing the rotation from F F F to r r r:

h h h v (t + 1) = h h h v (t) + k p F F F (||r r r|| -d e ) o o o v (t + 1) = o o o v (t) q ko
The combination of a fixed camera and rotational controls enables to navigate along more intricate paths in order to reach occluded objects. For instance, Figure 5.5 illustrates how an object hidden behind a wall can be selected. In order to provide true navigation capabilities, users are also able to set the current position of the hand as a new starting position by pressing a button (the trigger of the Razer Hydra in our prototype). In this way, they can navigate to different points of interest and then stretch their arm to grab objects before rolling back to the chosen position.

Using the Elastic-Arm to improve perception

The second set of illustrative use cases that we propose explores how the Elastic-Arm can improve user's perception of the virtual environment. Two different examples are proposed: (1) a redirection effect to touch the virtual environment through our elastic armature and (2) a pseudo-haptic effect allowing users to perceive varying levels of effort when interacting with virtual objects.

Perception of virtual boundaries with Redirected Touching

For this illustrative use case, our aim is to provide users with haptic cues related to the physical bounds of the virtual environment so that they can perceive its limits by This example is inspired by Redirected Touching [START_REF] Kohli | Redirected touching: The effect of warping space on task performance[END_REF], an haptic effect that leverages a passive prop and alters its virtual appearance as well as the position of the users' hand on its surface to make them perceive a different shape when they touch it. Similarly, the method proposed here relies on a discrepancy between visual and real positions: we alter the position of the user's virtual hand so that its encounter with an obstacle is correlated with the elastic cable being taut. In this way, users are able to perceive a clear resistance when "touching" virtual objects. Since this effect relies on a visual discrepancy, users wear a head-mounted display so that only their virtual hand is visible.

The alteration of the virtual hand's position consists in varying its Control/Display ratio with respect to the distance to facing obstables. In order to obtain this distance d o , rays are continuously cast in the r r r direction. The Control/Display ratio between real and virtual hands is then adjusted so that the cable tension matches the potential collision with the obstacle, as illustrated in Figures 5.6a and 5.6b. In other words, the distance to the obstacle is mapped on the rest length of the elastic cable. The following The object is stiff so the hand motion is slowed down and the user must stretch his arm to a greater degree; the haptic feedback is stronger. equation sums up this principle (s s s is the shoulder position):

h h h v (t) = s s s + r r r d o min(1, ||r r r|| d e )
Figure 5.6c presents a scene featuring this redirection effect. Users are seated in a control room and they can touch its different parts. The Elastic-Arm then lets them perceive the collisions of their virtual hand with the control panel. This application of the idea of Redirected Touching with the Elastic-Arm enables to feel virtual obstacles that are within the user's reach. However, contrary to the original implementation, users do not have to stay in front of a grounded passive prop since the haptic feedback is here provided by the body-mounted armature. In principle, this technique could thus make a large virtual environment tangible.

Perception of variable levels of effort with pseudo-haptics

The second perception-oriented use case that we present is based on pseudo-haptic feedback [START_REF] Lécuyer | Pseudohaptic feedback: Can isometric input devices simulate force feedback?[END_REF]] and leverages the Elastic-Arm to simulate different levels of effort when interacting with virtual objects.

Pseudo-haptics is an alternative means of delivering haptic sensations that simulates haptic properties by relying on visual feedback coupled with the actions of the user. Here, we apply this principle and alter the speed of the user's virtual hand depending on the haptic properties of the object it is interacting with. In this way, users have to stretch their arm to different degrees depending on the object. In consequence, users perceive different levels of effort thanks to the elastic nature of our armature.

This effect builds on the previous technique based on Redirected Touching, since the virtual hand must first collide with an object and the elastic cable must be just taut in order to start the effect. Then, once the interaction begins, the motion of the virtual hand is altered depending of the object properties. In practice, each object is associated with a different interaction coefficient k i that governs how the motion of the virtual hand is scaled during the interaction (k i ∈ [0, 1[ slows down the motion,

k i ∈ ]1, ∞[ amplifies it).
The following equation describes this principle:

h v h v h v (t) = s s s + r r r d o min(1, ||r r r|| d e ) + r r r k i ||r r r|| -d e d m -d e
Through this effect, users can interact with different objects and distinguish different levels of effort. In Figure 5.7, two deformable objects are pushed on and the one with the smaller k i bends to a lesser degree, even whilst the arm is stretched more, and the resulting haptic feedback is greater. Figure 5.8 shows a demonstration application in which users can push two different buttons and perceive different effort requirements to activate them. More generally, this effect could simulate various haptic properties related to pushing actions, such as closing a drawer or pushing on a wheeled cart.

User evaluation: distinguishing virtual buttons with different stiffness

We conducted a preliminary user study in order to verify that users could indeed perceive different levels of effort through the pseudo-haptic effect presented in the previous section. We thus presented participants with a stiffness discrimination task in which

Illustrative use cases they had to interact with a collection of virtual buttons and to sort them according to their level of stiffness.

Apparatus and participants

The environment of the experiment was displayed through an Oculus Rift head-mounted display (Oculus VR, USA). Participants were embodied by a virtual avatar and the virtual camera was placed at the level of their eyes. The avatar was seated with the same position as participants and they could freely look around the environment. Participants wore the elastic arm armature described in Section 5.1.1. The positions of their shoulder and their dominant-hand were tracked in space with a pair of Razer Hydra sensors. They could move the right arm of the avatar by moving their own arm with a 1:1 mapping (except when the pseudo-haptic effect was enabled). The input devices and head-mounted display were calibrated for each participant at the beginning of the experiment.

Eight participants took part in the experiment; 10 were male and two were female, aged from 22 to 31 (M = 24.6, SD = 6.2). Seven of them were right-handed and one was left-handed.

Procedure

Participants were asked to sort virtual buttons according to the amount of effort that they require to push on. Buttons were placed in front of the participants one at a time (Figure 5.9a) and they were identified by prominent bold letters (A, B, or C). To ensure a consistent amount of interaction for each button, participants had to push the current button to at least 80% of its course, which switched on a red light as a visual cue (Figure 5.9b) Then, participants were asked to release the button and bend the arm to display the next button.

Once the three buttons had been interacted with, participants had to select which one they considered as requiring the most effort to push. Three floating letters appeared in front of the users and they chose an answer by placing their hand on the corresponding letter (Figure 5.9c). Then the selected answer disappeared and participants had to select the button that required the least effort to push among the remaining options. Finally, participants were asked to return to their initial position, with the arm close to the body, to trigger the next trial.

Conditions

The preliminary user study has different goals. First, it aims at gathering subjective input from users about their experience with the Elastic-Arm. Secondly, its aims at assessing if users do perceive different levels of effort when interacting through the elastic armature, with the proposed pseudo-haptic effect.

For this reason, participants are subjected to the two following conditions. In the With condition, users wear the Elastic-Arm to constrain their arm during the interaction. In the Without condition, users interact without wearing the elastic armature.

In both conditions, participants control their avatar to interact with the buttons in the same manner, by moving their own arm in space. Likewise, the pseudo-haptic effect that simulates different levels of effort is always enabled and uses the same parameters. The only difference is the addition of passive egocentric haptic feedback. Hence, this condition will help verify that the different levels of effort perceived by participants are not only due to the visual feedback (altered speed of the hand when pushing the buttons) but also to the contribution of our elastic armature.

Experimental design and collected data

The independent variables of the experiment were the Condition (With, Without), the Ordering of the three buttons, and the Repetition.

The buttons were associated to three different interaction coefficients that governed the pseudo-haptic effect: k low = 0.5, k mid = 1, k high = 2. The attribution of these coefficients was counterbalanced across all trials.

The answers of participants when sorting the buttons is the main measure. We consider an answer as valid if the interaction coefficients of the corresponding buttons are ordered from lower to higher.

We additionally asked participants to fill a subjective questionnaire (Appendix C4) to rate each condition in terms of appreciation, ease of use, accuracy, comfort, fatigue, as well as their perception of the task on a 5-Likert scale. The questions are listed in Figure 5.10. A score of 5 for these criteria means a very positive rating except for fatigue where it means that the technique was very tiring (a rating of 1 means that it was not tiring at all). The participants were also asked to indicate which condition they preferred: visual feedback only (Without condition) or both haptic and visual feedback (With).

The total duration of the experiment was approximately 15 minutes. In total, we had 8 participants × 2 conditions × 6 combinations × 2 repetitions = 192 trials.

Results

Ordering of the virtual buttons -Regarding the answers of participants, we performed a generalized likelihood test and we found a significant difference between the types of Condition (χ 2 = 10.24, p = 0.001). The orderings were more often correct when using the With condition, with a 94% probability, whereas they were correct with the Without condition with a probability of 80%.

Subjective questionnaire -Regarding the subjective answers of the participants, a t-test showed no significant effect for the appreciation, ease of use, and accuracy criteria (Figure 5.10).

We found a significant effect for the comfort criterion in favor of the Without condition (W ith = 3.625, W ithout = 4.5, p = 0.041) as well as for the fatigue criterion (W ith = 3.125, W ithout = 2.375, p = 0.019). There was also a significant effect for the perception of different levels of effort criterion in favor of the With condition (W ith = 4.875, W ithout = 3.875, p = 0.049). Regarding user's preferences, two participants declared that they preferred the Without condition and six participants preferred the With condition.

Discussion

The preliminary evaluation gave insights about the appreciation of the Elastic-Arm by users as well as the effectiveness of the pseudo-haptic effect that we proposed.

First, the subjective questionnaire revealed that participants rated our armature as less comfortable and more tiring that the unconstrained condition. This opinion is reflected in some comments from participants: "[It was] a bit tiring at the end of the experiment", "I prefer not to get tired by moving the arm that much". Those results are to be expected from such modes of interaction that engage users physically. Thus, in order to reduce fatigue, applications making use of the Elastic-Arm should carefully assess the range and frequency of movements that users would have to perform.

However, the Elastic-Arm showed to provide a richer understanding of the virtual environment than simple "in-the-air" unconstrained interaction. Indeed, participants provided correct orderings significantly more often when they were equipped with our armature, and they also rated the Elastic-Arm as being significantly better to distinguish the stiffness of the virtual buttons. Those elements suggest that the association of elastic armature and pseudo-haptic effect do provide an advantage in terms of sensations, that cannot be provided by visual feedback alone. Additionally, some users mentioned in the questionnaire that they appreciated that the Elastic-Arm provides a sense of tangibility to the virtual environment: "It really feels like pushing a button", "This is more realistic". Finally, participants predominantly chose the Elastic-Arm as their favorite technique.

Alternative designs for simulating new haptic properties

In this section, we present alternative setups using the Elastic-Arm concept to simulate other haptic sensations. The first extension explores how to physically provide several levels of effort with a multi-string armature. The second extension illustrates how additional haptic effects can be enabled by considering the waist as another anchor point for the rubber band to provide downwards forces and simulate weight.

Multi-layer rendering with several rubber bands

With the current prototype of the Elastic-Arm, the magnitude of the haptic feedback is limited to the stiffness of the rubber band that is used. However, more elaborate rigs with several elastic cables of different lengths could be attached on the same arm. In this way, more "layers" of stiffness could divide the user's reach.

This technique generalizes the redirected touching technique previously presented by synchronizing the tension of each rubber band with one change of stiffness. For example, the "button pushing" task that we evaluated could be enriched if the shortest elastic Alternative designs for simulating new haptic properties was just tightened when the user hand touches the surface of the button (Figure 5.11b), and the longer rubber band (or a rigid string) is tightened when the button meets the end of its course (Figure 5.11c).

This multi-string armature progressively accumulates the stiffness of several rubber bands, and could thus be used to simulate virtual objects with complex material layers (e.g. a soft layer followed by a stiffer one), or interaction in heterogeneous environments. An example of game-like scenario using this type of feedback would be a task in a basin of water with different levels of density. For instance, users would start above the basin (no stiffness), immerse their hand in the water (low stiffness), reach the sand at the floor to find buried objects (medium stiffness), and finally hit the glass at the bottom (high stiffness).

Weight simulation from the waist

Currently, the haptic feedback provided by the Elastic-Arm is unidirectional, in the sense that the armature always pulls the user's hand toward his shoulder. Specific haptic properties could be provided by considering alternative anchor points for the elastic cables. Here, we illustrate how to simulate weight by attaching the rubber cable to the the waist and altering the vertical motion of a manipulated virtual object. This extension is inspired by an early work on pseudo-haptic feedback, which simulated different weights by only altering the vertical motion of a sphere controlled through a haptic interface [Dominjon et al., 2005a]. Recent work also focused on the simulation of mass through deformations of the visual feedback; for instance [START_REF] Ban | Augmented endurance: controlling fatigue while handling objects by affecting weight perception using augmented reality[END_REF] changed the color of virtual dumbbells to change the perceived endurance during a lifting task and Jáuregui et al. [2014] altered the lifting animation of virtual avatars to express various weights.

In the implementation that we propose, the rubber band pulls the user's hand downward when he lifts the arm, which provides a first haptic feedback. Then, a pseudo-haptic effect modulates this sensation by scaling the vertical motion of the hand, depending on the weight of the manipulated object. For example, in Figure 5.12, a user interacts with two different virtual dumbbells. The motion of the lighter dumbbell (Figure 5.12a) is mapped on the user's real motion. However, the motion of the heavier dumbbell (Figure 5.12b) is scaled down vertically, and users have to raise their real hand higher to reach the same position in the virtual environment, which increases the intensity of the downward force.

In a second example, we consider both hands, linked to the waist with separate rubber bands. Each hand is associated to a different Control/Display ratio in order to simulate an asymmetric weight distribution in large objects held with both hands. For instance, in Figure 5.12c, weights on the right side of the user are heavier, thus the motion of the right virtual hand is slowed down. The user keeps the barbell straight by raising his right hand to compensate this difference, which provides stronger haptic cues on the heavier side.

Conclusion

In this chapter, we tackled the small workspace associated with many haptic interfaces and we proposed the Elastic-Arm, a body-mounted armature that provides passive egocentric haptic feedback to the user's hand. The simple design of our system enables a mobile and accessible haptic feedback compared to cumbersome and costly active devices and compared to static passive props.

Several use cases were presented to illustrate the capabilities of our approach. First, two examples focusing on interaction respectively allowed to select distant virtual objects and to navigate in virtual environments by stretching a virtual arm. Then, two use cases focused on user's perception and delivered haptic cues about the boundaries of the virtual environment through a redirection effect, and about the haptic properties of virtual objects through pseudo-haptics. A preliminary user study evaluated this pseudo-haptic effect by exposing users to a stiffness discrimination task in which they had to order virtual buttons. Results showed that participants answered correctly significantly more often when wearing the Elastic-Arm, and that they predominantly preferred to use the elastic armature compared to free, unconstrained interaction.

This work leads us to believe that the Elastic-Arm could be leveraged for a wide range of tasks and haptic properties, as illustrated with a multi-string setup that simulates layers of stiffness and by a waist-based design for simulating weight. it could also be incorporated in a wide range of contexts. For instance, perception-oriented methods such as those presented in this paper could be used for ergonomics studies. Another field of application that would fit the Elastic-Arm is medical rehabilitation since tasks of increasing difficulty could be proposed to patients in order to progressively enhance their physical performance, either by equipping them with armature with increasing levels of stiffness or by leveraging our pseudo-haptic approach.

6

Haptic Manipulation of Objects using Grip Force:

The Virtual Mitten Grasping virtual objects is a fundamental task that can be greatly improved with the addition of haptic feedback. In practice however, common force feedback arms do not reflect the true dynamics of grasping. Conversely, multi-finger exoskeletons enable users to directly grasp and feel virtual objects with their own hands but the associated mechanical complexity entails a high cost. Alternative means of haptic stimulation have been proposed, such as passive haptic feedback, but they seem currently limited for providing a convincing haptic perception in the context of manipulation tasks.

Thus, in this chapter, we introduce the Virtual Mitten, a novel interaction paradigm to naturally grasp and manipulate virtual objects with haptic sensations. This paradigm is based on a handheld elastic device that enables object manipulation through a simplified hand model -the mitten. Upon clenching actions from the user, the mitten is operated in 3D space to grasp and release virtual objects. A pseudo-haptic effect that takes as input the user's grip force is added to simulate various levels of effort when performing manipulation tasks.

The contributions of this chapter can be summarized as follows: a low-cost handheld input device that generates elastic grip force and preserves freedom of movement due to its low weight and small size, an interaction metaphor that takes the form of a mitten, bound to a control scheme that allows users to manipulate virtual objects through natural grasping motions, a novel pseudo-haptic approach based on grip force that varies the effort perceived when performing object manipulation tasks.

In the next section, we first present the concept of the Virtual Mitten and detail its components. Then, we present the user evaluation that investigated the subjective user appreciation of the Virtual Mitten as well as the effectiveness of the pseudo-haptic feedback that we propose. Each hand holds an elastic device to control a corresponding mitten. The grip force applied by the user is measured to grasp virtual objects and to generate pseudo-haptic feedback.

The Virtual Mitten

The Virtual Mitten

In this section, we first introduce the general concept of the Virtual Mitten. Then, we detail its individual components: the input device it relies on, its control scheme, as well as the visual and pseudo-haptic feedback that it provides. Then, several illustrative use cases are presented.

Concept

The Virtual Mitten is based on a handheld elastic input device that maps the motion of the user's hand to a mitten model capable of interacting with virtual objects (Figure 6.1). The grasping of an object and the ensuing interaction depends on the grip force applied on the device, which provides passive force feedback thanks to its internal elasticity. Due to its internal elasticity, the device provides passive force feedback, which enables the perception of efforts that relate to the manipulation occurring within the virtual environment.

The elastic device used is simple in nature as well as low-cost. Moreover, its small dimensions and its low weight do not impede the freedom of movement of users. Nevertheless, it provides controls and sensations similar to those experienced when grasping objects. Additionally, the visual metaphor -a mitten with a generic folding animation (Figure 6.2) -provides a natural mapping between real and virtual environments.

The Virtual Mitten is composed of four main components:

1. an elastic input device that maps the position and grip force of the user's hand to a mitten model, 2. a control scheme to naturally select and manipulate virtual objects with the mitten,

two types of visual feedback graphically expressing the efforts occurring during the interaction,

4. pseudo-haptic feedback to modulate the perceived haptic sensations.

Elastic input device for simulating grasping

For the prototyping of the Virtual Mitten, a spring-loaded hand exerciser (ProHands, USA)1 that is both inexpensive and commonly available was used as an input device. We take advantage of its low weight (62 g), its small size (8 × 7 × 1.5 cm) and its shape that perfectly fits the hand due to its original purpose (Figure 6.3). Its elasticity is an essential feature since it provides passive haptic feedback, with a stiffness of approximately 4400 N.m -1 over a range of two centimeters. We attached optical tracking markers on the device to retrieve its position and orientation in 3D space (6 DoF) as well as its compression (1 DoF). Since only coarse grasping motions are measured through compression, there is no need to track individual fingers and the device can be more easily and reliably tracked than a fully articulated hand.

Interaction in the virtual environment depends on the grip force applied on the elastic device. This quantity is formalized as a normalized compression ratio r such that r = 0 when the device is relaxed and r = 1 when the device is fully compressed. The ratio r is computed from the area A defined by four reflective markers and varies linearly with respect to the minimum area A min (when the device is fully compressed) and the maximum area A max (when relaxed) such that:

r = 1 - A -A min A max -A min
A filtering is established to flatten tracking inaccuracies and smooth the interaction with virtual objects, with r real the compression ratio optically measured, k a stiffness coefficient (empirically set to 0.15 in our prototype in order to provide both stability and responsiveness) and r virtual (or simply r) the output virtual compression ratio effectively used in the simulation:

r t = r virtual,t = r virtual,t-1 + k × (r real,t -r virtual,t-1 )

Control scheme

The Virtual Mitten is associated to a control scheme that allows users to control mitten models representing their own hands (both unimanual and bimanual scenarios are possible). This control scheme is consistent with the real dynamics of grasping: first, users move their in space to reach an object, then they clenched their hand to select it, and finally the grip force must be maintained to ensure a secure grasp.

The Virtual Mitten

Figure 6.3 -Elastic input device for controlling a virtual mitten. It consists of a hand exerciser equipped with markers for optical tracking. Here, the elastic device is fully relaxed so A = A max (green overlay) and the compression ratio r is 0.

Moving the mitten in space

The mitten naturally follows the user's hand: by moving his hand, a user directly controls its position and orientation in 3D space. Since the virtual environment is physically simulated, it is thus possible to interact coarsely with virtual objects by touching or pushing them.

A virtual coupling is established between the virtual mitten and the user's real hand via a 6-DoF virtual spring [START_REF] Borst | Realistic virtual grasping[END_REF]. In this way, the mitten applies the user's motion as closely as possible while respecting the constraints of the virtual environment and sliding against hard surfaces.

Selecting objects by clenching the mitten

In its most basic form, the classical virtual hand metaphor requires to touch objects to glue them to the hand. With the Virtual Mitten, a more expressive approach that reproduces the true dynamics of grasping is proposed: to select an object for further interaction, the mitten has to be placed in a valid grasping configuration around the targeted object, that is, there must be at least one contact between each side of the mitten and the targeted object.

To validate this condition and bring the digits of the mitten closer to an object, users have control over the clenching of the mitten. A finger folding animation is triggered when a slight compression of the device is detected (r ≥ r f olding ). Conversely, when the device is relaxed (r < r f olding ), the mitten automatically unfolds. The speed of this animation is constant and does not vary with the compression.

Virtual Mitten

Closing the mitten should only use a small part of the compression range of the elastic device since most of it is reserved for the manipulation part of the interaction. As such, a small r f olding threshold must be chosen, low enough so that it does not take up too much spring length and large enough so that it does not unintentionally trigger the animation due to tracking inaccuracies or false positive detection of the user's intent (it is set to 0.05 in our prototype). We used incremental joint-unfolding to adapt the folding animation to the shape of the objects [START_REF] Boulic | Multi-finger manipulation of virtual objects[END_REF].

Using grip force to interact through the mitten

Once an object is selected, users have to apply a sufficient amount of force on the elastic device to hold the targeted object securely and not let it slip. In concrete terms, the compression ratio r must be greater than another fixed threshold r grasping so that the object becomes attached to the mitten. Once a virtual object has been grasped with the mitten, the exerted compression must be maintained above r grasping in order not to drop it.

Objects can thus be manipulated and moved around the virtual environment. Additionally, tools capable of executing actions benefit from an additional degree of freedom in that their inherent function can be modulated with respect to the grip force applied on the elastic device. For example, the closing of a pair of scissors can be mapped to the remaining compression range such that the user has fine control over a paper cutting task. A virtual spray can that varies the amount of projected paint depending on the compression of the device is another possible example.

Visual feedback for expressing the level of effort

We propose two different types of visual feedback to graphically communicate the amount of force exerted on the elastic device when grasping an object: a Boolean feedback and a Progressive feedback.

Boolean feedback

While the compression ratio is less than the r grasping threshold, the appearance of the mitten remains unchanged. However, when the grasping threshold is reached, the mitten turns a different color (blue in our prototype) in order to indicate that the exerted force is sufficient to hold the object (Figure 6.4, top).

Progressive feedback While the compression ratio is less than r grasping , the mitten is continuously filled from its tip to its base with a primary color (blue) representing the effort required to reach this threshold and grasp the object. When r ≥ r grasping , the mitten is continuously filled in the other direction with a secondary color (yellow) representing the excess of compression with respect to the grasping threshold. In other words, the mitten acts as a gauge displaying first the compression required to start the interaction and then the excess compression (Figure 6.4, bottom). Boolean feedback. Bottom: Progressive feedback. The r f olding threshold is fixed but the r grasping threshold varies in order to enable our pseudo-haptic effect.

Pseudo-haptic feedback for modulating the perceived grip forces

The Virtual Mitten, as described until this section, allows users to manipulate virtual objects and provides passive haptic feedback through the elastic device. However, since the compression threshold r grasping that must be reached to hold an object is constant, the perceived effort is similar for each virtual object. Here, we introduce a novel pseudo-haptic approach to simulate different levels of effort.

The general principle is that the higher the magnitude of the simulated haptic property is, the higher the r grasping threshold that has to be reached is. This pseudohaptic feedback thus replaces the unique r grasping with object-specific r grasping (object) thresholds that associate a haptic property with a required grip force. The appearance of the visual feedback (Boolean or Progressive) is correlated with these object-specific thresholds so that they demonstrate different dynamics depending on the targeted object. For example, when grasping an object with a higher threshold, the mitten would change color more slowly and the users would tend to apply more force to speed up the grasp. Due to the elasticity of our input device, this would induce stronger haptic cues.

The value of the r grasping threshold bound to each manipulable object is calculated via a context-based mapping function that takes as input the magnitude of the physical property being simulated. For example, when simulating heaviness, the masses of the interactive objects are considered and mapped to the [r f olding , 1] range of the elastic device. Other haptic properties such as stiffness, friction or the effort to operate a tool can be simulated. To deliver an optimal pseudo-haptic feedback, the mapping function must consider the order of magnitude of the simulated property for all the interactive objects in the virtual environment. Indeed, using a linear mapping in a scene populated with objects bearing a high difference of magnitude (e.g. feathers and cars) would incur a loss of resolution for the elastic device when comparing light objects between them whereas a non-linear mapping could enhance haptic perception.

Illustrative use cases

To conclude this presentation of the Virtual Mitten, here follow several illustrative use cases that demonstrate the range of tasks that it supports. First, we implemented with the Virtual Mitten a series of primitive haptic tasks taken from an existing taxonomy. Then, we developed a playful scenario in which users can experience various haptic sensations.

Taxonomy of haptic tasks. [START_REF] Bloomfield | A taxonomy and comparison of haptic actions for disassembly tasks[END_REF] classified haptic actions according to the forces and torques that they involve. In order to illustrate how the Virtual Mitten applies to a wide range of tasks, we implemented the four main actions that this classification describes (Figure 6.5).

Force I

The applied force is aligned with the motion of the hand (e.g. pushing a box).

As an example, we implemented a drawer that is opened by grasping its handle and performing a linear displacement.

Force II The applied force is not aligned with the motion. As an example, we simulated the action of pulling a pin stuck on a wall. At first glance, the Force I and Force II scenarios seem similar but when pulling the pin, grip force (toward the head of the pin) are perpendicular to the direction of the motion.

Torque I The axis of the applied torque passes through grip-space. As an example, we implemented a screwing task that is completed by grasping a cylinder and rotating the wrist.

Torque II The axis of the applied torque passes outside of grip-space. As an example, we implemented the act of pulling a lever (the axis of rotation passes through the base of the lever, not the hand).

The user study described in the next section is based on the four categories of this force-centric taxonomy. It will allow us to validate the use of the Virtual Mitten for these high-level categories of tasks that cover most interaction cases.

Fruit-o-Matic.

Another illustrative use case, called the Fruit-o-Matic, involves a variety of haptic subtasks (Figure 6.6). The goal of the scenario is to prepare fruit juice with a blender. Users are provided with two elastic devices so that both their hands can interact with the virtual environment and perform actions in parallel. Several virtual objects can be grabbed and moved around with the mittens: fruits, the lid of the blender, and a glass. Other elements of the environment can be activated: a dial to switch on the blender and a lever to pour the prepared juice into the glass. The fruits can be squeezed above the blender to extract their juice. Each fruit has a different internal stiffness and requires a different amount of effort to be pressed. This effort is perceived through the elastic grip force device. Similarly, the glass requires more or less efforts to hold depending on the volume of liquid that it contains.

User evaluation

The evaluation of the Virtual Mitten was decomposed in two different experiments: (1) a subjective evaluation to compare the two types of visual feedback (Boolean versus Progressive) and to assess the appeal of the Virtual Mitten for general hand-based interaction, and (2) a preliminary psychophysical evaluation to assess the resolution of our elastic device when specifically simulating a screwing effort. Both experiments considered the tasks defined in the taxonomy of haptic actions from [START_REF] Bloomfield | A taxonomy and comparison of haptic actions for disassembly tasks[END_REF], as depicted in Figure 6.5, and they were conducted in the same experimental environment.

Apparatus and participants

Apparatus

The virtual environment was displayed on a 55 inches screen placed at 2 meters from participants. In order to allow participants to rest their elbows, they were seated in front of a table (Figure 6.7). They answered the tests by pressing marked keys on a keyboard.

The elastic device was tracked with a Vicon Bonita system (Vicon, USA) with ten infrared cameras arranged in a circle surrounding the participant. The tracking data was further processed with the Vicon Nexus reconstruction software and streamed into our application which used OpenSceneGraph 2 as a rendering engine and Bullet 3 for Figure 6.6 -Operating the "Fruit-o-Matic" with virtual mittens. The preparation of the fruit juice involves sub-tasks such as grabbing objects, squeezing fruits, rotating a dial to mix and pulling a lever to pour the juice. Different levels of stiffness are associated with each type of fruit and the glass requires different amount of grip force to hold depending on the volume of juice that it contains. the physics simulation. It is most important to mention that this setup does not describe the minimal requirement to use the Virtual Mitten since it would contradict the promise of a low-cost interaction paradigm. Thanks to the basic tracking model of our elastic device, setups with simpler tracking solutions and pressure sensors to measure compression would also be appropriate.

In order to avoid occlusions between the real user's hand and the visual content of the scene, the mitten was offset by approximately 50 centimeters in front of the real hand. In addition, as the placement of the cameras was done to maximize the tracking accuracy of the user's hands and not the tracking of the head, we used monoscopic rendering from a static point of view.

Participants

Twelve participants, all male and right-handed, ranging from 21 to 28 years old (M = 25.2; SD = 2.6), took part in the experiments. Regarding their experience with Virtual Reality, three were experienced users, six had a moderate experience and three had no prior experience. No participants had any prior knowledge about the Virtual Mitten. They started each experiment by a training session and could take breaks at any time. The virtual scene contained a single object to be interacted with.

Experiment #1: subjective evaluation of the Virtual Mitten

The first experiment was based on the four primitive haptic tasks defined by Bloomfield et al. [2003]. Its goal is to compare the two types of visual feedback in terms of precision and user preference.

Procedure

Participants had to carry out classical Two Alternative Forced Choices (2AFC). For each trial, they had to perform two repetitions of a same task but each repetition was associated to a different r grasping threshold. Once the user had completed both steps, she had to answer the question "Which task required more effort to perform?". Their answers were considered correct if they chose the interaction with the highest grasping threshold.

Design and Hypotheses

The independent variables of the experiment were: the Task (Drawer, Pin, Screw, Lever), the visual Feedback (Boolean, Progressive) and the grasping Threshold (Soft, Hard). The soft grasping threshold r grasping (Soft) was 0.5 and the hard grasping threshold r grasping (Hard) was 0.625. The thresholds were determined through informal evaluations, ensuring that users will be able to perceive the difference but with a certain level of error. While the four tasks were always presented in the same order, the ordering of the types of visual feedback and the ordering of the grasping thresholds were counterbalanced. In total, we had 4 tasks × 2 types of feedback × 3 repetitions × 12 participants = 288 trials. The dependent variables were the answers entered by the participants and the mean compression exerted during each grasp. At the end of the evaluation, participants also indicated which visual feedback they preferred and answered the questions listed in Table 6.1 and 6.2 on a 7-point Likert scale (Appendix C5).

Virtual Mitten seven users preferred the Progressive feedback. The comments for each group of users are consistent. Users who preferred the Boolean feedback state that it let them better focus on the task ("I could focus on the task", "It made me act faster"). On the contrary, users who preferred the Progressive feedback state that it allowed them to be more precise and to adjust the level of grip force ("the drop of the object is more predictable").

Experiment #2: psychophysical assessment for a screwing task

The second experiment of the user study focused on assessing the perceptual resolution of the elastic device for simulating a specific pseudo-haptic property with the Virtual Mitten. We focused on one primitive haptic task from Bloomfield's taxonomy: the screwing effort (Torque I). We also restricted this second study to the Progressive feedback which was preferred by participants and showed to be more accurate in the first experiment.

Procedure

A Just Noticeable Difference (JND) psychophysical study [START_REF] Gescheider | Psychophysics : Method, Theory, and Application[END_REF] was conducted to measure the minimum difference between two r grasping thresholds that can be discriminated by users and thus assess the resolution of the elastic device. Participants had to follow a 2AFC procedure: each trial was a comparison between a fixed reference grasping threshold and a comparison grasping threshold (from a set of precomputed thresholds). Participants had to answer the question "Which object requires more effort to screw?".

Design

The independent variable was the grasping Threshold, with a reference value of r grasping (Reference) = 0.45 and six comparison thresholds r grasping (Comparison) = {0.288, 0.342, 0.396, 0.504, 0.558, 0.612}. The comparison thresholds were computed as r grasping (Reference) × (1 + ∆) with ∆ ∈ {-0.36, -0.24, -0.12, 0.12, 0.24, 0.36}. The values of the reference and comparison thresholds serve as a baseline for this evaluation and were chosen empirically so that the covered range contains JND values already studied in the literature. For each trial, participants had to determine which condition (comparison versus reference) required more effort to perform. In total, we had 6 pairs × 5 repetitions × 12 participants = 360 comparisons.

Results

The goal of the perceptual evaluation was to compute the JND between two grasping thresholds when performing a screwing task.

First, we computed the percentage of answers in which the repetition using the reference compression threshold was considered as the one requiring additional effort. As expected, as the value of r comparison decreases, the reference is chosen more often and vice versa. Then, using Weber's law, we computed the Weber Fraction (k) as k = ∆I/I, where ∆I refers to the JND threshold and I the grasping threshold for the reference. The JND threshold can be determined as the value of the stimuli in which the recognition ratio is 84% [START_REF] Gescheider | Psychophysics : Method, Theory, and Application[END_REF]. In order to compute the compression ratio for the recognition rate of 84%, we fit the psychometric curve f (x) = 1/(1 + e (x+α)/β ) to the data (Weibull function with α = 0.0042 and β = 0.0692) (Figure 6.10). Furthermore, we observe that the Point of Subjective Equality (the user provides a random answer) is consistent since the fit shows that it is approximately 0. The Weber fraction of the compression ratio for the screwing task was k = ∆I/I = 0.267 which means that users are capable of discerning approximately four different levels of effort with the elastic device in this specific context.

Discussion

The user study provided insights about the potential of the Virtual Mitten both in terms of appeal and perception. Concerning the subjective evaluation, two types of visual feedback (Progressive and Boolean) were proposed and tested for various manipulation tasks. The results are similarly good for the different tasks, which suggests that the Virtual Mitten can apply successfully to the contexts covered by Bloomfield's taxonomy.

For the comparison tasks, users gave a greater number of correct answers when using the Progressive visual feedback thus supporting hypothesis H1. The measure of the compression exerted by participants shows that their grasping is more precise with the Progressive feedback: the mean compression applied is closer to the grasping threshold associated with virtual objects, which validates hypothesis H2. This result is consistent with those of [START_REF] Fabiani | Human interface using the Rutgers Master II force feedback interface[END_REF] who evaluated that a combination of visual and haptic feedback helps in achieving precise grasps. However, in our case, the visual feedback is coupled with the pseudo-haptic approach and the simpler visual information relates to the whole mitten rather than individual fingers. The results on discrimination and precision are consistent thus the higher amount of correct answers could stem from the enhanced precision while grasping virtual objects. Indeed, a more precise compression implies a more accurate haptic feedback due to the elastic nature of the input device. However, even though performances were globally better with Progressive feedback, several participants preferred the Boolean feedback (42%). The Boolean feedback was found to be less prone to distract the users and allowed them to focus on the task. Thus, for future use of the Virtual Mitten it seems thus preferable to let users choose their visual feedback.

Concerning the second evaluation, a psychophysical protocol was used to assess the perceptions of users in the specific context of a screwing task, as a pilot experiment. The results of a series of discrimination trials yielded a JND close to 26%, which seems consistent with the values given in the haptic literature for force (12%), torque (16%), and stiffness (22%) [START_REF] Burdea | Force and touch feedback for virtual reality[END_REF]. The lower resolution found here (i.e., higher JND) could be due to the fact that the perceived effort corresponds in our case to a more complex context involving a sequence of actions (selecting an object and then rotating it). This pilot experiment could be followed by other evaluations applying this psychophysical protocol to the three other categories from Bloomfield's taxonomy. Other evaluations could also follow in order to compare the Virtual Mitten with existing interaction techniques: users could be asked to perform various manipulation tasks sequentially with the Virtual Mitten and with other types of interfaces so that differences in performance (speed and precision) and in perception could be assessed.

Conclusion

In this chapter, we addressed the difficulty to provide convincing grasping sensations with most haptic interfaces and presented the Virtual Mitten, a novel interaction paradigm for manipulating virtual objects in a manner that is faithful to the dynamics of grasping and does not require active haptic feedback or complex input devices. Our approach is based on the passive haptic feedback provided by a handheld elastic input device (a modified hand-exerciser). The grip force exerted on the device enables to grasp objects and to achieve various manipulation tasks by means of a virtual mitten. A pseudo-haptic effect was also introduced to generate the haptic perception of different levels of effort.

A user study was conducted to assess the acceptance of our novel interaction paradigm by naive participants and the perception of the pseudo-haptic feedback. The results suggest that the Virtual Mitten allows us to reliably manipulate virtual objects in various primitive manipulation tasks. A psychophysical test confirmed that different levels of effort could be successfully perceived in a basic screwing task. An entertaining application (the Fruit-o-Matic) involving bimanual interaction and a sequence of manipulation tasks was also provided to illustrate the versatility of the Virtual Mitten.

Conclusion

Taken together, our results suggest that the Virtual Mitten could be applied to various manipulation cases and used in multiple Virtual Reality applications in which a simple haptic information is required such as for virtual prototyping, sport training, rehabilitation procedures, or video games. Enabling users to conduct complex manipulation tasks with their fingers and to feel the properties of virtual objects would enhance interaction within virtual environments. However, generic haptic devices are unfit for dextrous manipulation since they do not consider the fingers separately, and the only systems capable of doing so are cumbersome and costly multi-finger exoskeletons. Simpler passive interfaces based on minimalist props can be used to stimulate the user's sense of touch during 3D interaction and offer a low-cost and lightweight alternative to active feedback. For instance, Chapter 6 presented the Virtual Mitten, an interaction metaphor based on a passive hand-exerciser for manipulating virtual objects. While this technique incorporated haptic feedback in an accessible manner, it was limited to coarse grasping and fingers were not considered individually.

In this chapter, we explore how passive haptics and pseudo-haptics can contribute to multi-finger interaction by proposing ElasticGlove, a novel elastic interface that handles the digits individually (Figure 7.1). Its structure is reminiscent of typical multifinger exoskeletons but it only leverages passive and affordable components to provide haptic cues when interacting with the fingers. We additionally propose an accompanying pseudo-haptic effect that modulates the perceived efforts. In this way, users can perform multi-finger tasks and perceive varying efforts on each finger.

The contributions of this chapter can be summarized as follows:

the design of a novel elastic exoskeleton that enables multi-finger interaction and delivers individual forces to the digits in an affordable and portable manner, an accompanying pseudo-haptic approach that enriches the passive feedback of the elastic device and simulates various levels of stiffness when manipulating deformable objects, the design of illustrative use cases to provide examples of contexts in which our approach could prove valuable: medical training and musical learning.

In the next section, we first present our elastic exoskeleton and the considerations that motivated its design. Then we describe the pseudo-haptic effect that applies individually to the fingers and simulate various levels of effort. Finally, we present several use cases that illustrate how multi-finger pseudo-haptics can contribute to handbased interaction.

The ElasticGlove

The ElasticGlove

Design

We propose a passive hand-worn device that delivers haptic feedback to individual digits (Figure 7.1). In the previous chapter, the Virtual Mitten was based on a hand exerciser that provided haptic cues when users performed grasping motions. Although, this allowed for coarse object manipulation, finer tasks requiring precise finger movements could not be achieved. Thus, in this chapter, we extend this original principle to multi-finger interaction and introduce the Elastic Glove, an exoskeleton that has a similar form as the typical multi-finger force feedback devices but leverages simple passive components.

The main components are independent finger modules made of a bendable metal strip ending with a thimble (Figure 7.2a). A number of those modules is plugged on a base plate attached to the back of the hand with a velcro band. A smaller separate plate handles the thumb. Each module is anchored to the plate on a rail slider so that it can accompany curling motions without blocking the finger (Figure 7.2b).

The finger modules are essential to haptic feedback since they constrain the digits when they curl, which generates forces consistent with real-world grasping. The bendable strips that they are made of are reminiscent of the bow spring used by [START_REF] Lawrence | Bow spring/tendon actuation for low cost haptic interfaces[END_REF]. In our case however, we use one bow per finger with an entirely passive setup instead of a single one for the whole hand driven by a motor. In consequence, fingers undergo an opposing elastic force when they curl. We can then exploit this feedback for simulating haptic effects related to object manipulation such as grip force or stiffness.

Due to the elastic nature of this device, the intensity of the feedback depends on the degree of curling of each finger. The pseudo-haptic effect described in a later section exploits this fact to induce different sensations depending on the properties of the virtual object that the user is interacting with.

Motivations

The elastic exoskeleton aims at circumventing the drawbacks of existing multi-finger haptic interfaces. Hence, the following considerations motivated its design.

Size and weight. Wearable haptic interfaces should be compact and lightweight enough so as not to interfere with the tasks being carried out. Here, the lack of motorization of passive haptics significantly reduces the weight of the device: a threefinger configuration only weights 100 grams. Concerning size, the bulk of the system occupies a vertical space of 4 centimeters on the back of the hand. Moreover, the thin bendable strips run close to the finger so as not to hamper users with a bulky support armature.

Cost and complexity. The device should be made of accessible components so that it remains affordable compared to commercial haptic systems as well as easy to produce. Our current prototype is made of simple parts which total price is under 30 $.

Its key components, the deformable strips that provide haptic feedback, are made of phosphor bronze, a common and affordable alloy. The base plates and thimbles are 3Dprinted which makes them cheap to produce as well as easy to tweak to accommodate different morphologies.

Modularity. The device should be able to accommodate different numbers of fingers depending on the context. Indeed, pushing a virtual button may only require a single finger while sophisticated precision grasps may require three fingers or more (Figure 7.1). In our implementation, up to four finger modules can be plugged on the base plate depending on the requirements of the interaction scenario. The intensity of the haptic feedback can also be tuned either by choosing bendable strips with an appropriate stiffness or by stacking several strips together. Another approach that we explored to vary the perceived haptic feedback is to leverage pseudo-haptics, as detailed in the following section.

Pseudo-haptic effect for varying the perceived efforts

The elastic device that we propose leverages passive haptics for providing forces opposing curling on the fingertips, yet its stiffness is constant. Therefore, we introduce an additional pseudo-haptic feedback that renders different levels of stiffness so that varying haptic properties can be simulated. We first present the coupling between users and their virtual hand, then we describe the visual mismatch that makes our effect possible. Finally, we expand on how to bind haptic properties to virtual objects.

Pseudo-haptic effect for varying the perceived efforts

Coupling the user's motion to a virtual hand

In the virtual environment, the 3D hand model that conducts the interaction reproduces the gestures of the user's hand. This model supports the main degrees of freedom of a real hand: it moves like the user's and the flexion of the virtual digits depends on the flexion of the user.

We describe the state of each virtual finger n ∈ {1, 2, 3, 4, 5} with a virtual flexion parameter v n ∈ [0, 1] such that v n = 0 corresponds to an extended finger and v n = 1 corresponds to a curled finger. Those parameters are reflected on the 3D model according to a curling rule similar to Rijpkema and Girard [1991]: the proximal joint of finger n rotates with respect to v n within predefined bounds and the following joints are rotated by 2 3 of their parent joint's rotation. Coupling the virtual hand is then a matter of appropriately setting those flexion parameters with respect to multi-finger input from the user.

For our prototype, hand tracking was handled by a LeapMotion sensor (Leap Motion Inc., USA) that measures the position and orientation of the user's wrist as well as the orientation of the finger joints. The wrist data is directly mapped with a 1:1 scale in the virtual environment. For finger curling, we compute real flexion parameters r n ∈ [0, 1] corresponding to the actual flexion of the user. As it stands, a truthful 1:1 coupling between real and virtual hands can thus be simply formulated as ∀ n, v n = r n .

Rendering the stiffness of virtual objects

Pseudo-haptics generally consists in altering the visual feedback resulting from the user's actions in order to express haptic properties [START_REF] Lécuyer | Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback[END_REF]. Here, we focus on simulating the stiffness of virtual objects so that users can discriminate between soft and firm materials.

We induce this sensation of varying stiffness by altering the Control/Display ratio [Dominjon et al., 2005a] of the virtual fingers controlled by the user while they press an object. In practice, increasing the ratio slows down finger curling, which suggests a stiffer object. Users need to adapt to the mismatch by compensating their own posture to reach the same amount of visual deformation for different objects. Thus, a harder object requires to amplify one's curling and, in consequence, the elastic exoskeleton constraining the fingers generates stronger haptic cues. We call s n the stiffness coefficient that amplifies or reduces the virtual flexion of finger n, such that:

∀ n, v n = r n • s n
A potential issue with this mapping between real and virtual hands arises from the fact that the elastic exoskeleton provides force feedback as soon as users curl their fingers. In consequence, users already perceive an effort while closing the hand around a virtual object even though they are ostensibly not touching its surface yet. If needed, this unrealistic step can be skipped by automatically curling the virtual fingers around objects when they are at reach. In this way, the neutral flat posture of the user's real ElasticGlove hand is mapped to the grasping posture of the virtual hand. The flexion parameter c n corresponding to finger n just touching the surface of the object is obtained by casting rays along the trajectory of each finger. The finger then curls within the remaining flexion span 1 -c n such that:

∀ n, v n = c n + (1 -c n ) • r n • s n

Authoring pseudo-haptic objects

Rendering the stiffness of an object with this pseudo-haptic approach requires to bind it to appropriate haptic parameters. For instance, the designer of the virtual environment may want to simulate the stiffness of a given material or to make it vary over the surface of a non-homogeneous object with softer parts. Thus, there is a need for a practical workflow for authoring such interactive objects.

We embed the haptic data governing this effect into the representation of the virtual object, at each vertex of its mesh. In this way, a stiffness value can be obtained for any point on its surface by interpolation, as customarily done in computer graphics for colors or normal vectors. In practice, the s n coefficient associated to finger n thus varies depending on the position of the finger over the object, which yields varying sensations. If the resolution of the haptic data must be greater than the level of detail of the polyhedral mesh, then alternative methods such as procedural haptic texturing [Siira and Pai, 1996] and image-based mapping [START_REF] Ruspini | The haptic display of complex graphical environments[END_REF]] could be used in a similar fashion.

This setup makes the authoring of pseudo-haptic objects accessible since haptic data can be painted on a virtual object in any 3D editor. For example, Figure 7.3 shows a virtual fruit with rotten spots and the associated stiffness map that makes those parts softer to the touch.

Illustrative use cases

In this section, we present two use cases that were developed to illustrate how our approach can enhance 3D interaction and simulate virtual objects with different levels of stiffness. The first example consists of a musical proof of concept in which passive haptics help in learning piano chords. The second use case is an educational scenario in which users are introduced to medical palpation.

Learning piano chords

The learning of musical instruments has been shown to improve when students are exposed to haptic cues guiding them. For instance, rhythm skills necessary to play drums have been taught with haptic feedback delivered to the wrists and ankles [START_REF] Holland | Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit[END_REF] and piano players have been guided via vibrations on the fingers [START_REF] Huang | Mobile music touch: mobile tactile stimulation for passive learning[END_REF]]. As a first example, we explore how our elastic exoskeleton and the associated pseudo-haptic feedback can help in learning piano chords. In our scenario, a virtual piano which keys can be pressed to produce notes is presented to users. Those keys are given either a weak or a strong stiffness value depending on the chord to practice. In this way, keys that are not part of the studied chord require more effort to push on, and thus it is more difficult to play wrong notes (Figure 7.4). The influence of this assistance can then be progressively diminished as the training advances, until the player can perform the chord without errors nor assistance. This scenario could be developed furthermore by adding visual feedback to guide the student. The stiffness coefficients could also be changed in real-time to teach sequences of chords and song sections.

Training in medical palpation

Haptic feedback has a wide range of application in the medical field, especially when dealing with bodily examinations such as palpation which goal is to detect anomalies in the shape or consistency of some body part. For instance, [START_REF] Ullrich | Haptic palpation for medical simulation in virtual environments[END_REF] developed a simulator that let trainees touch a virtual body through a desktop haptic interface. [START_REF] Bibin | SAILOR: a 3-D medical simulator of loco-regional anaesthesia based on desktop virtual reality and pseudo-haptic feedback[END_REF] developed a similar application that instead leveraged pseudo-haptic by altering the motion of a cursor to suggest relief on the patient's anatomy. Here, we explore how such procedures can be conducted directly with the fingers thanks to our multi-finger pseudo-haptic approach.

In our scenario, a virtual body part is presented to users. They can study the organ through palpation by exploring its surface and squeezing it with their fingers. Their goal is to detect an invisible anomaly that can take the form of suspiciously soft or hard tissue. Those targets can be painted on the stiffness map of the body part as described previously or taken from a database of medical samples to simulate different conditions. For instance, Figure 7.5 shows an arm with a harder spot inside of the elbow that must be detected by the trainee.

Conclusion

In this chapter, we focused on multi-finger manipulation and we explored how passive haptics and pseudo-haptic feedback could be combined for interacting with the fingers. We first presented a novel elastic exoskeleton, the Elastic Glove, that constrains the fingers separately so that each of them can move in an independent manner and be subjected to varying forces when manipulating virtual objects. Thanks to its passive components, the resulting system is a simple and low-cost alternative to complex haptic devices usually exploited for dextrous interaction. We then proposed an accompanying pseudo-haptic effect that leverages visual feedback to vary the sensations provided by the elastic device. In this way, soft or firm material can be distinguished and virtual objects with heterogeneous stiffness can be simulated.

Two practical use cases, related to musical learning and medical training, illustrated how this approach can be incorporated into various Virtual Reality scenarios. Those examples give insights into the concrete applications that could make use of our approach. For instance, domains necessitating simple and low-cost multi-finger input, such as training simulations, video games, or any applications that would require a portable multi-finger interface could benefit from its simplicity.

Conclusion

In this manuscript, entitled "Contributions to the Design of Novel Handbased Interaction Techniques for Virtual Environments", we focused on the topic of interacting with the hands in the context of Virtual Reality. This type of interaction is typically enabled by haptic interfaces that allow users to act in virtual environment and to get relevant haptic sensations in return, but they impose restrictions on users because of their reduced workspace, mechanical complexity, and cost. We tackled those limitations by following two main axes of research. The first axis focused on improving the control of articulated hand models. The second axis focused on improving the sensations resulting from hand-based interaction.

In Part I of this manuscript, we proposed novel methods for handling the many degrees of freedom of realistic hand models. Two approaches were proposed: (1) reducing the number of degrees of freedom of virtual hands to ease their control with multi-touch interfaces, and (2) separating the degrees of freedom between two desktop interfaces.

Chapter 3 focused on reducing the degrees of freedom of virtual hands and presented THING, a multi-touch interaction technique that leverages the user's gestures to animate a 3D hand model. This approach is more light-weight than motion capture and yet it benefits from the flexibility and accuracy of hand-based interaction. Moreover, it is based on common and affordable hardware so that it can be readily integrated into exisiting applications. Two variants were proposed: MobileTHING, which integrates all the degrees of freedom of hand models on a tablet, and DesktopTHING, a hybrid setup that conjointly uses a computer mouse. We conducted a user evaluation in order to compare those two variants to traditional techniques relying on the computer mouse or motion capture. In the end, both variants provided performance close to those of a data glove and the DesktopTHING variant was predominantly preferred by users.

Chapter 4 focused on separating the degrees of freedom involved in object manipulation and introduced DesktopGlove, a bimanual setup that enables the control of one virtual hand with two interfaces operating in clearly separated workspaces. In practice, one device drives the global motion of the virtual hand and the other handles the motion of its fingers. The force feedback that results from the interaction is accordingly split between the two interfaces so that users are exposed to forces on the whole hand and on individual fingers, which was otherwise restricted to costly grounded exoskeletons. We conducted a user evaluation to compare DesktopGlove to a traditional data glove and to assess how controls and force feedback should be split between dominant and non-dominant hands. Our approach showed similar performance to the data glove. Moreover, users preferred to use their non-dominant hand to control virtual fingers, their non-dominant hand to control the global position, and being exposed to a complete feedback on both hands.

In Part II of this manuscript, we proposed new interaction techniques to improve the sensations resulting from hand-based interaction in virtual environments. Our general approach was to combine passive interfaces with pseudo-haptic feedback as an alternative to complex and bulky active devices. We considered a variety of tasks with different scales: (1) coarse arm-based interaction, (2) object grasping with the hand, and (3) precise multi-finger manipulation.

Chapter 5 first focused on arm-based interaction and addressed the small workspace issue of desktop haptic interfaces. It introduced the Elastic-Arm, a simple and costeffective elastic armature that links the user's hand to his body. As a result, a progressive resistance force is perceived when extending the arm. This haptic feedback has been incorporated with various 3D interaction techniques that originally relied on active haptic devices to conduct selection and navigation tasks. We also demonstrated how the Elastic-Arm could contribute to a better comprehension of the virtual environments with a redirection effect for perceiving its boundaries and a pseudo-haptic effect for perceiving different levels of effort. This pseudo-haptic effect was evaluated in a preliminary user study that confirmed that the Elastic-Arm augments perceptions and showed that it was well received by participants.

Chapter 6 focused on object grasping and manipulation with the whole hand and introduced the Virtual Mitten. This approach relies on an elastic handheld device that provides passive grip force and a mitten interaction metaphor that enables to grasp and manipulate objects. The grasping performed by the mitten is directly correlated with the grip force applied on the elastic device. A supplementary pseudohaptic feedback modulates the visual feedback of the interaction to simulate varying haptic sensations. Our approach has been evaluated within two experiments focusing on subjective appreciation and perception. Results showed that participants were able to well perceive different levels of effort during basic manipulation tasks thanks to our pseudo-haptic approach and that they could also rapidly appreciate how to achieve different actions with the Virtual Mitten.

Finally, Chapter 7 focused on multi-finger interaction and presented the Elastic-Glove, an elastic multi-finger device that constrains the digits individually and produces passive force feedback during object manipulation. It is combined with a pseudo-haptic approach that simulates different levels of stiffness for each finger separately when interacting with virtual objects. We illustrated how this combination of Conclusion passive haptics and pseudo-haptics could benefit multi-finger interaction through several use cases related to music learning and medical training. Those examples suggest that this kind of passive exoskeleton could find numerous applications in domains that require an accessible and portable way of providing multi-finger haptic feedback.

Future work

This section addresses the current limitations of the interaction techniques and interfaces that were presented in this manuscript and proposes possible solutions that could be investigated as future work.

A Multi-touch Interaction Technique for Controlling 3D Hand Models: THING Additional input. THING controls only 16 degrees of freedom out of the 27 of the hand because the virtual fingers are bound to fixed trajectories. In consequence, their joints cannot be controlled individually and certain poses cannot be achieved (e.g. irregular joint flexion when fingers are pushed against objects or other fingers, or special rotations of the thumb). Giving control over the missing degrees of freedom would allow users to leverage the full range of motion of the hand through THING. One possible solution to achieve this goal would be to consider additional input to adjust the trajectories of the fingers. For example, pressure sensing, which is supported by most modern tablets, could add one supplementary DoF per finger. Another possibility would be to leverage mid-air interaction techniques to allow users to interact above surfaces [START_REF] Hilliges | Interactions in the air: adding further depth to interactive tabletops[END_REF][START_REF] Marquardt | The continuous interaction space: Interaction techniques unifying touch and gesture on and above a digital surface[END_REF], which would add a third dimension to the current finger tracking.

Homing time.

A limitation that is specific to the hybrid variant, DesktopTHING, is the need to switch from tablet to mouse to control different degrees of freedom. The homing times inherent to these transitions have a negative impact on performance: in the evaluation, users performed on average 1.8 transitions per pose. In practice, the nature of computer animation forces users to switch back and forth between hand models and other objects in order to build sequences incrementally so these homing times are bound to occur but it would be beneficial to reduce them. One solution could be to display additional manipulation widgets on the screen of the tablet to set the position and orientation of the virtual hand; but this would require to frequently look at the tablet. An alternative solution would be to discard the computer mouse entirely and use the tactile surface to control the cursor on the main screen, similarly to a touchpad.

Separating the degrees of freedom of virtual hands for haptic manipulation: Desk-topGlove

Greater number of fingers. The implementation of DesktopGlove that we presented combines a generic haptic arm with a custom multi-finger interface de-rived from the DigiHaptic [START_REF] Casiez | The DigiHaptic, a new three degrees of freedom multi-finger haptic device[END_REF]. It currently supports two pinching fingers that are controlled and receive force feedback independently. However, as illustrated in Section 2.1.1.2, some grasps employ from three to five fingers, like the cylinder and tripod grasps. Thus, our approach could be extended to support a greater number of fingers, either by leveraging existing multi-finger interfaces [START_REF] Endo | Five-fingered haptic interface robot: HIRO III[END_REF][START_REF] Leuschke | High fidelity multi finger haptic display[END_REF] or by designing new devices to handle more digits. In that respect, it would be necessary to conduct user evaluations to quantify the number of fingers effectively required for a meaningful interaction, and if they all require the same level of control and haptic fidelity.

Bimanual interaction. DesktopGlove distributes the degrees of freedom of one virtual hand between two hands. In consequence, users are restricted to unimanual tasks in the virtual environment. Nevertheless, a variety of actions requires the coordinated use of both hands, for instance using a hammer with one hand and holding a nail with the other. This limitation is critical for virtual training applications in which trainees must learn procedures through practice, as interacting through a single hand would teach only half the procedure. A possible solution could be to display a second, autonomous virtual hand that would assist the dominant one during bimanual tasks and illustrate the other half of the procedure. This first idea could be extended furthermore with new bimanual control schemes for either switching between hands depending on the focus of the task, or leveraging our bimanual setup to control and perceive the action of each virtual hand during two-handed tasks.

Human-Scale Haptic Feedback for Augmenting 3D Interaction: the Elastic-Arm

Alternative designs. We leveraged the Elastic-Arm to conduct coarse manipulation tasks with the hand. However, we could consider equipping other body parts with similar elastic rigs in order to provide haptic cues for other types of interaction. For example, fingers could be connected by short elastic cables in order to design new manipulation techniques enhanced by a portable passive feedback, for constraining curling in the manner of the ElasticGlove (Chapter 7), or contrarily to constrain the opening of the hand. Additionally, new setups could be designed to handle both hands and constrain their relative positions when conducting bimanual tasks like lifting a virtual object. While the hand is a major tools for 3D interaction, other body parts could be equipped with similar elastic armatures, like the legs. In this specific instance, the design of novel navigation techniques based on the provided elastic feedback could be considered.

Haptic Manipulation of Objects using Grip Force: The Virtual Mitten

Low-cost tracking. The setup used for the evaluation of the Virtual Mitten has ten optical cameras surrounding the user, which creates quite an imbalance between our elastic device and tracking system in terms of complexity. However, the simple tracking model of the elastic device (six DoF in space and one Conclusion additional DoF for compression) could allow for simpler hardware setups. For instance, recent markerless tracking solutions leverage regular cameras to capture the motion of the hand [START_REF] Oikonomidis | Efficient model-based 3D tracking of hand articulations using kinect[END_REF][START_REF] Wang | Real-time hand-tracking with a color glove[END_REF]. As for the measure of the compression of the elastic interface, it could be handled with pressure sensors incorporated in the device, which would simplify tracking furthermore and could enhance the accuracy of the input, and thus the quality of the pseudo-haptic rendering. Going a step further, adding pressure sensors under each digit could provide new input DoF and enable multi-finger interaction with the mitten.

Simulating the use of tools. In its current implementation, the Virtual Mitten enables the control of a mitten model representing the user's hand. Other fields of application could benefit from this type of interaction, especially when the action of tools is required. Thus, future approaches could be twofold. First, new tool-based props enhanced by springs and other elastic components could be incorporated into Virtual Reality simulations; for instance pliers or surgical tools. Then, new multimodal effects could be developed alongside those "augmented" tool, for instance to simulate the stiffness of body tissue in a surgery simulation or the efforts required to use a power tool in virtual manufacturing training.

In-depth evaluation for each type of task. We evaluated the resolution of our elastic device for a specific screwing task to which users responded positively in pre-tests. Other categories of haptic actions were presented in Section 6.1.5 (i.e. Force I, Force I, and Torque I [START_REF] Bloomfield | A taxonomy and comparison of haptic actions for disassembly tasks[END_REF]) and they could be evaluated in the same manner to validate the applicability of the Virtual Mitten to a wider range of tasks. Then, more complex composite tasks consisting of sequences of actions could be evaluated (e.g. first grasping an object and then screwing it) to assess the extent to which the compression range of the elastic device can be split between several sub-tasks and still provide a consistent feedback.

Multi-Finger Interaction Combining Passive and Pseudo-Haptics: the ElasticGlove

Addition of other low-cost components.

The ElasticGlove provides haptic cues when users curl their fingers to manipulate virtual objects thanks to bendable strips. This structure is particularly appropriate for simulating soft sensations such as stiffness but it seems unfit for the simulation of abrupt haptic events, like sudden collisions. Consequently, the elastic exoskeleton could be extended with other low-cost components to expand the range of haptic effects that it supports. One such example would be to incorporate vibration motors on the finger modules to provide active haptic cues to the separate digits. Another possibility could be to design simple clutching mechanisms that would block the sliders supporting the finger modules in order to simulate contacts with virtual objects. The multifinger tracking that drives the interaction could also be integrated on the device by putting bend sensors on the deformable strips rather than relying on an external system.

Long-term perspectives Adaptative hand postures with low-DoF input devices

In this manuscript, we proposed methods for reducing the many degrees of freedom of the hand and distributing them between several input devices. However, those approaches limit the use of certain grasps and postures since users only have control over a subset of the effective DoF of the hand. Consequently, techniques for unlocking the control of the missing degrees of freedom could be explored. Rather than supporting all DoF with hardware, which showed to be challenging because of practical limitations, an alternative strategy could be to handle those missing DoF via software techniques.

A first approach would be to consider the semantics of the interaction. For example, explicit coordinated gestures could be mapped to specific actions, such as only moving the index finger and thumb to produce a pinch for which the virtual thumb would automatically rotate appropriately, even if the input device cannot measure such subtleties. Another possibility could be to leverage implicit mechanisms that extrapolate missing DoF from the user's actions. For example, when grasping a virtual object, the finger trajectories could adapt to its shape and function, in the manner of smart objects [Kallmann and Thalmann, 1999] with a higher level of detail. In that regard, previous work from other fields could be valuable starting points. For example grasp planning is an active topic of research in robotics [START_REF] Miller | Automatic grasp planning using shape primitives[END_REF], and it could be transposed from simplified robotic hands to human hands. Similarly, computer animation techniques that reconstruct realistic grasps with behavioral data and shape matching from posture databases [START_REF] Li | Data-driven grasp synthesis using shape matching and task-based pruning[END_REF] could be adapted to a real-time interactive setting. In the end, the fusion of the semantic, geometric, and behavioral data could empower users by driving complex virtual hands via simpler input devices.

Evaluating pseudo-haptic feedback for the simulation of complex sensations

In this manuscript, we proposed methods that provide haptic sensations with a combination of passive haptics and pseudo-haptics. There are still open questions about some aspects of this type of feedback that must be investigated: its application to complex parallel stimuli, its reliance on vision, and its merits compared to active haptics.

Firstly, in most of the literature, pseudo-haptic feedback has been evaluated by exposing users to a single visual stimulus, such as a moving object or cursor. With some of our approaches however, several pseudo-haptic stimuli may be presented in parallel: on both hands for the Virtual Mitten or on several fingers for the ElasticGlove. Thus, it is essential to study how these parallel stimuli affect the overall perception of the interaction.

Secondly, the reliance of pseudo-haptics on visuals could be investigated. The user evaluations conducted in this manuscript often featured simple environments with sin-Conclusion gle tasks, without much room for distraction; thus users focused on the pseudo-haptic effect. However, in practice, we often do not look at our hands at all when performing manipulation tasks. Moreover, real-world applications might involve a variety of objects, tasks, and events, that could detract the user's attention from the pseudohaptic effects. Consequently, pseudo-haptics could be evaluated in practical settings to measure its efficiency as well as its long-term efficiency. Then, solutions to reinforce pseudo-haptic effects could be designed, for instance by leveraging other sensory modalities, such as sound or touch, which could be dynamically enabled depending on the user's focus.

The relative merits of active haptics and passive/pseudo-haptics could also be investigated. The passive solutions that we proposed granted advantages in terms of workspace size, complexity, and general accessibility, but it would be quite insightful to compare each of our interface with its active equivalent. For example, Desktop-Glove could be compared to a complex haptic workstation and ElasticGlove could be compared to active multi-finger exoskeletons. In this way, the trade-off between the fidelity of the interaction and the accessibility of the input devices could be more accurately quantified, and guidelines could be drawn for choosing appropriate interfaces depending on the application context.

Combining active haptics, passive haptics, and multimodal feedback

In this manuscript, we presented approaches for conducting 3D interaction with the hands for a variety of techniques and haptic interfaces. The ad hoc use cases that were proposed applied to specific tasks, such as stretching the arm via an egocentric armature and inspecting objects through palpation with a multi-finger passive exoskeleton. However, hand-based interaction in general could benefit from combining those various contributions into novel hybrid haptic interfaces.

A first step in that direction could be to combine several passive haptic interfaces together. For example, a complete elastic armature involving the whole motor chain from the shoulder to the fingers could be designed to support generic hand-based interaction. As of now, this could be achieved by combining the Elastic-Arm and ElasticGlove, for example. More generally, this could lead to the design of novel elastic exoskeletons that constrain several body parts to conduct specific interaction tasks. We can for instance envision useful uses of such systems for video games, sport training, and medical rehabilitation.

Another perspective is to combine passive and active devices together. Indeed, while passive interfaces are lightweight and affordable, active devices provide more flexibility. Bringing those two paradigms together with hybrid interfaces could provide accessible, and yet powerful forms of haptic feedback. A first possibility in that direction could be to separate the feedback of different DoF depending on their importance, with the primary DoF handled by active devices and complementary DoF with passive haptics. More generally, hand-based interaction could greatly benefit from combinations of hardware and software solutions, with multimodal haptics comprising active, passive, and cross-modal components.

B

Résumé long en français Introduction

Dans ce manuscrit de thèse, intitulé « Contributions à la conception de nouvelles techniques d'interaction avec les mains en environnement virtuel », nous présentons des travaux de recherche se plaçant dans le contexte de la réalité virtuelle.

La main humaine est un outil formidable qui nous sert d'interface avec le monde qui nous entoure. En effet, la main rend possible de nombreuses tâches essentielles : manipuler les objets de notre quotidien, explorer nos environs par le toucher et même interagir et communiquer par des gestes. La finesse avec laquelle de telles actions sont menées est due notamment à deux propriétés essentielles de la main : sa dextérité et sa sensibilité au toucher. Tout d'abord, la dexterité de la main permet d'agir à travers de nombreuses prises qui fournissent aussi bien de la précision que de la force [START_REF] Bibliography Napier | Hands. Natural science[END_REF]. Ensuite, la main nous permet de percevoir notre environnment proche grace à sa sensibilité au toucher. En effet, à la plus petite échelle, elle contient une dense distribution de récepteurs pour ressentir de minuscules détails comme la texture ou la rugosité d'un matériau. Quant aux doigts, au poignet et au bras, ils permettent de déterminer des propriétés physiques à plus grande échelle, comme la forme et le poids des objets que nous manipulons [START_REF] Kolb | An Introduction to Brain and Behavior, Third Edition[END_REF].

Le terme « réalité virtuelle » (VR) fait référence à des simulations immersives par ordinateurs qui affichent un environnement artificiel à un ou plusieurs utilisateurs comme s'ils s'y trouvaient réellement. Pour maintenir cette illusion, des retours visuels, sonores et haptiques sont produits par la simulation en fonction des actions de l'utilisateur [START_REF] Sherman | Understanding Virtual Reality: Interface, Application, and Design[END_REF]. Dans de tels environnements virtuels (EV), nous pouvons interagir à travers de nombreuses modalités (joystick, contrôle vocal) mais les simulations se voulant réalistes peuvent nécessiter des modes d'interaction plus fidèles à la réalité. Ainsi, il semble essentiel de permettre aux utilisateurs d'utiliser leurs mains de façon naturelle au sein d'environnements virtuels.

Défis de l'interaction avec les mains en environnement virtuel

Dans ce manuscrit de thèse, nous abordons deux défis liés à l'interaction avec les mains :

(1) gérer les nombreux degrés de liberté de la main humaine et (2) fournir des sensations haptiques lors de la manipulation virtuelle. Tout d'abord, pour maîtriser la complexité de la main, les utilisateurs doivent être pourvus de moyens de contrôler leurs nombreux degrés de liberté (DdL). Des interfaces de capture de mouvements permettent de transposer les gestes des utilisateurs dans les EV mais elles sont limitées par la tendance des mains à générer des occlusions. D'autres interfaces, en lien mécanique direct avec les doigts, peuvent aussi mesurer la posture des mains mais leur prix elévé et leur complexité les réservent à un usage professionnel. En conséquence, de nouvelles interfaces sont nécessaires pour gérer efficacement et de façon accessible les nombreux degrés de liberté de la main.

La manipulation 3D d'objets implique aussi des sensations haptiques complexes, dont la simulation nécessite des interfaces haptiques [START_REF] Biggs | Haptic interfaces. Handbook of virtual Environments[END_REF] ; Srinivasan and Basdogan, 1997]. Les interfaces dites « de bureau » fournissent un retour d'effort par un effecteur unique, souvent sous forme de poignée, mais cela est insuffisant pout simuler des forces complexes liées à la préhension. De plus, elles reposent souvent sur des supports fixes et leur espace de travail s'en trouve réduit. Au contraire, les interfaces haptiques de type exosquelette bénéficient d'une plus grande liberté de mouvement ainsi que d'un retour de force sur les différents doigts, mais elles sont généralement encombrantes et coûteuses.

Le retour haptique passif, qui consiste à exploiter des objets inertes pour fournir un retour haptique par leur forme ou leur matériau, est une alternative possible à ces interfaces complexes [START_REF] Borst | Evaluation of a haptic mixed reality system for interactions with a virtual control panel[END_REF][START_REF] Lok | Effects of handling real objects and avatar fidelity on cognitive task performance in virtual environments[END_REF]. Néanmoins, ce type de retour est peu flexible comparé aux interfaces actives puisque qu'il ne peut pas être contrôlé en temps réel par la simulation. Cependant, des effets multimodaux qui jouent sur les liens étroits entre les canaux visuel et haptique peuvent influencer les perceptions des utilisateurs ; le retour pseudo-haptique en est un exemple [START_REF] Lécuyer | Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback[END_REF]. Le retour pseudo-haptique est pourtant limité à des cas d'interaction simples et n'est que peu utilisé pour la manipulation en environnement virtuel avec les mains. Ensuite, nous abordons le problème du retour haptique pour la manipulation virtuelle avec des mains articulées, qui nécessite actuellement de lourdes interfaces de type exosquelette. Les interfaces haptiques de bureau sont quant à elles plus accessibles mais elles n'appliquent sur la main que des forces globales sans séparer les doigts. Ainsi, notre second objectif est de combiner plusieurs interfaces de ce type et de distribuer les contrôles liés à la manipulation d'objets ainsi que les retours de force entre ces différentes interfaces. Notre contribution est une interface bimanuelle nommée DesktopGlove qui permet de gérer les nombreux DdL d'une main virtuelle grâce à deux interfaces : l'une contrôle les déplacements globaux et l'autre est chargée des mouvement des doigts. Le retour de force est aussi distribué entre les interfaces, Résumé long en français ce qui permet de fournir des sensations réalistes aux utilisateurs avec des interfaces de bureau. Nous évaluons cette approche à travers une étude utilisateur qui la compare à un gant de données et détermine quelle main, dominante ou non dominante, devrait être attribuée à chaque interface.

Objectifs de la thèse et contributions

Le thème principal de notre second axe de recherche est de combiner retour haptique passif et retour pseudo-haptique en tant qu'alternative à l'usage d'interfaces à retour de force complexes. Dans ce contexte, nous abordons plusieurs catégories d'interaction 3D, avec différentes échelles : (1) l'interaction avec le bras, (2) la préhension d'objets avec la main, et (3) la manipulation fine avec les doigts.

Premièrement, nous abordons le problème de l'espace de travail réduit lié aux interfaces haptiques de bureau. Bien que quelques interfaces à « taille humaine » existent, elles sont souvent imposantes et coûteuses. Cependant, des composants passifs pourraient nous permettre de fournir des sensations similaires aux utilisateurs, dans un espace de travail tout aussi grand. Ainsi, notre contribution est une armature élastique que nous appelons Elastic-Arm. Cette interface relie la main de l'utilisateur à son propre corps et fournit un retour de force égocentrique lorsqu'il tend le bras, ce qui permet d'implémenter de nombreuses techniques d'interaction 3D de façon mobile. Plusieurs cas d'usage sont présentés, notamment des méthodes permettant de saisir des objets distants et de naviguer dans un vaste environnement virtuel. Nous proposons aussi des techniques visant à enricher la perception des utilisateurs par des effets pseudo-haptiques. Une étude utilisateur évalue l'utilisation de l'Elastic-Arm ainsi que l'efficacité d'un effet pseudo-haptique simulant différents niveaux d'effort.

Le second niveau de précision auquel nous nous intéressons est la saisie d'objet avec la main ; l'objectif étant de reproduire la sensation naturelle de préhension. De par les limitations des interfaces haptiques de bureau, il est impossible de les exploiter pour fournir de tels efforts sur les doigts. Cependant la saisie d'objets ne requiert pas toujours une telle précision et certaines postures, notamment celles comportant de la force, gardent les doigts groupés. Dans de tels cas, la représentation virtuelle de la main et l'interface utilisateur peuvent être simplifiées. Pour ce faire, nous proposons, la Virtual Mitten, une nouvelle approche pour saisir les objets virtuels à l'aide d'une interface élastique que l'on compresse pour contrôler un modèle de moufle. Nous exploitons ensuite un effet pseudo-haptique jouant sur une modulation du retour visuel pour simuler différents niveaux d'effort lors de la manipulation d'objets. Une étude utilisateur est réalisée pour évaluer les capacités de cette approche ainsi que la résolution de l'interface élastique que nous proposons.

Finalement, certains scénarios nécessitent l'usage précis de tous nos doigts, notamment lors de phases de manipulation haptique précises. Ainsi, notre dernier objectif est d'explorer de quelle façon une combinaison de retour passif et de retour pseudohaptique pourrait s'appliquer à tous les doigts en parallèle. Pour ce faire, nous proposons l'ElasticGlove, un exosquelette passif qui contraint chaque doigt séparément grâce à des bandes déformables attachées sur le dos de la main. Un retour pseudo-haptique est ajouté pour varier la sensation perçue sur chaque doigt et simuler des matériau hétérogènes. Nous proposons plusieurs cas d'usage qui exploitent cet effet THING, une technique d'interaction multi-touch pour l'animation de mains virtuelles dans des scénarios concrets, comme une application d'apprentissage du piano et un simulateur de palpation médicale. Elastic-Arm, retour haptique à échelle humaine pour améliorer l'interaction 3D

B.1 THING, une technique d'interaction multi-touch pour l'animation de mains virtuelles

La seconde expérience consistait à saisir un objet virtuel et à le placer dans des configurations prédéfinies en étant exposé à différentes distributions de retour haptique : aucun retour, un retour partiel sur une seule main, ou un retour complet sur les deux mains. Selon un questionnaire, les participants ont majoritairement préféré ressentir du retour de force sur les deux mains à la fois, ce qui a par ailleurs contribué à une meilleure perception du poids des objets, des contacts entre doigts et objets et de la forme des objets. bien pour rendre plus mobiles des techniques initialement basées sur des interfaces de bureau.

B.3.1 Techniques améliorant la sélection et la navigation

Nous proposons deux extensions de techniques existantes pour sélectionner les objets éloignés et naviguer dans un large environnement virtuel. La première technique est basée sur la Bubble [Dominjon et al., 2005b]. Son principe est de permettre aux utilisateurs d'allonger leurs bras virtuel pour saisir les objets distants. Deux modes de contrôle existent selon l'extension du bras : près du corps, lorsqu'il n'est pas tendu, un contrôle en position est activé, et au delà de la longueur de repos de l'élastique, un contrôle en vitesse est activé. Dans ce dernier cas, le bras virtuel s'allonge jusqu'à atteindre les objets éloignés. Lorsque l'utilisateur replie le bras, le bras virtuel est « rembobiné » jusqu'à lui. Grâce à l'armature passive, les utilisateurs peuvent percevoir haptiquement ces transitions. La seconde technique fixe la caméra virtuelle au bout du bras virtuel et ajoute des contrôles en rotation pour permettre de se déplacer le long de chemins complexes et donc de contourner les obstacles pour atteindre des objets initialement hors de vue.

B.3.2 Techniques améliorant les perceptions

Nous proposons deux techniques d'interaction pour améliorer les perceptions de l'utilisateur dans l'environnement virtuel. La première technique est basée sur le redirected touching [START_REF] Kohli | Redirected touching: Warping space to remap passive haptics[END_REF], une technique qui simule différentes formes en altérant le rendu virtuel d'un objet réel ainsi que la position de la main virtuelle de l'utilisateur lorsqu'il en inspecte le contour. Notre implémentation reprend ce principe initial pour simuler le contact avec les surfaces qui forme l'environnement virtuel. Nous modifions la position de la main virtuelle lorsqu'elle se dirige vers des obstacles de façon à ce que la collision coincide avec la tension du cable élastique (Figure B.9). Ainsi les utilisateurs peuvent « toucher » les surfaces qui forment l'environnement virtuel et ressentir une résistance physique. La seconde technique est une application du retour pseudo-haptique pour simuler différent niveaux d'effort lors d'interaction 3D impliquant des mouvements du bras. Nous altérons la vitesse de la main virtuelle selon les propriétés physiques de l'objet avec lequel on interagit, par exemple un bouton virtuel. En conséquence, les utilisateurs allongent le bras à jusqu'à différents degrés, et perçoivent différents niveaux d'effort grâce à l'élasticité de l'armature.

B.3.3 Évaluation

Nous avons réalisé une étude utilisateur dans le but d'évaluer l'efficacité de notre effet pseudo-haptique ainsi que l'appréciation des utilisateurs pour l'Elastic-Arm. Les participants avaient pour consigne d'interagir avec trois boutons virtuels associés à différents

B.4 Virtual Mitten, manipulation haptique d'objets virtuels et simulation de la force de saisie

La manipulation d'objets virtuels est une tâche fondamentale qui peut être grandement améliorée par l'addition de retour haptique. Cependant, en pratique, les interfaces à retour de force classique ne reflètent pas la véritable dynamique de la saisie d'objets, poussant à utiliser à la place de coûteux exosquelettes. Nous proposons donc la Virtual Mitten, ou « moufle virtuelle », un nouveau paradigme d'interaction qui permet de saisir et de manipuler des objets virtuels avec des sensations haptiques.

B.4.1 Moufles virtuelles et interface élastique

B.4.2 Retours visuels et pseudo-haptiques

Nous proposons deux types de retours visuels pour exprimer les efforts d'interaction : un retour booléen avec lequel la mitten change de couleur quand la compression appliquée est suffisante pour saisir un object et un retour progressif avec lequel elle se remplit progressivement, telle une jauge.

Pour simuler différent niveaux d'effort lors de l'interaction 3D, nous proposons un effet pseudo-haptique qui fait varier la compression nécessaire par objet. Ce retour altère la vitesse des retours visuels, ce qui pousse les utilisateurs à adapter la pression qu'ils appliquent sur l'interface élastique, et donc l'intensité du retour de force. Par exemple, nous avons simulé plusieurs tâches issues d'une taxonomie d'actions haptiques [START_REF] Bloomfield | A taxonomy and comparison of haptic actions for disassembly tasks[END_REF]

B.5 ElasticGlove, interaction multi-doigt combinant retour haptique passif et retour pseudo-haptique

Permettre aux utilisateurs de ressentir des objets virtuels avec leurs propres doigts est nécessaire pour simuler des tâches de manipulation complexes. Cependant, les interfaces haptiques couramment utilisées en réalité virtuelle ne sont pas adaptées à la manipulation dextre et sont donc incapables de fournir ces sensations. Les seuls systèmes haptiques pouvant fournir de tels effets sont des exosquelettes multi-doigt qui sont encombrants et dont l'usage est reservé à des application professionelles. Nous proposons donc d'explorer une nouvelle façon de combiner retours passif et pseudo-haptique à travers un exosquelette élastique : l'ElasticGlove.

B.5.2 Retour pseudo-haptique

Nous proposons un effet pseudo-haptique pour faire varier l'effort ressenti sur chaque doigt séparément. Cet effet repose sur une modulation du gain entre mouvements du doigt réel et mouvements du doigt virtuel. Selon les propriétés physiques des objets virtuels, encodées dans leur maillage, la vitesse de chaque doigt est altérée. En conséquence, les utilisateurs adaptent leur mouvement pour atteindre les mêmes niveaux de flexion, ce qui génère un retour de force proportionnel à la magnitude de la propriété haptique simulée. Par exemple, une zone plus dure nécessite d'amplifier le mouvement de son doigt car sa représentation virtuelle est ralentie.

B.5.3 Cas d'usage illustratifs

Nous proposons plusieurs exemples illustratifs de scénarios que nous avons implémentés avec l'ElasticGlove. Le premier scénario est une application de formation à la palpation médicale dans laquelle les utilisateurs peuvent examiner un organe virtuel. Le retour pseudo-haptique multi-doigt décrit précedemment permet ici de simuler différentes pathologies sous formes de zones suspectes au toucher. Le second scénario illustratif est une application d'apprentissage musical dans laquelle les utilisateurs peuvent jouer d'un piano virtuel. Selon l'accord qui doit être appris par l'utilisateur, les touches du piano sont associées à des coefficients d'interaction différents de façon à favoriser les accords corrects n'incluant pas de touches incorrectes. Cette assistance peut ensuite être progressivement diminuée au cours de la leçon.

B.6 Conclusion

Dans ce manuscrit de thèse, nous avons étudié l'interaction haptique avec les mains en environnement virtuel. Dans ce contexte, des interfaces à retour d'effort sont traditionnellement utilisées pour permettre aux utilisateurs d'interagir et de ressentir les objets virtuels manipulés. Elles peuvent prendre la forme d'interfaces de bureau se limitant à fournir des forces à la main sans distinguer les doigts, à travers une poignée par exemple, ou bien de complexes exosquelettes multi-doigt qui permettent de simuler une interaction plus riche, la saisie d'objets par exemple. Dans ce cadre, nos objectifs étaient (1) d'améliorer le contrôle des mains virtuelles articulées et (2) d'améliorer les sensations haptiques liées à l'interaction avec les mains.

Pour améliorer le contrôle des mains virtuelles articulées, nous avons tout d'abord voulu réduire leurs nombreux degrés de liberté afin de les coupler à des interfaces utilisateurs plus accessibles. Pour ce faire, nous avons proposé THING, une interface multi-touch permettant d'animer un modèles 3D de main en éxecutant des gestes sur la surface d'une tablette tactile. Une étude utilisateur a demontré que cette approche permet des performances similaires à un gant de données, tout en utilisant du matériel accessible et courant. Ensuite, lors d'une seconde contribution, nous avons proposé une approche alternative qui consiste à répartir les degrés de liberté entre plusieurs interfaces. Le sytème que nous avons conçu, DesktopGlove, permet un contrôle bimanuel d'une main virtuelle ainsi que des retours de force distribués sur les deux mains de l'utilisateur. DesktopGlove a été évalué à travers deux expériences démontrant des performances similaires à de la capture de mouvements et soulignant un accueil favorable de la part des utilisateurs.

Pour répondre au second objectif, qui consistait à améliorer les sensations liées à l'interaction avec les mains, nous avons proposé plusieurs combinaisons de retours haptiques passifs et pseudo-haptiques. Dans un premier temps, nous avons abordé le problème des espaces de travail réduits lié aux interfaces de bureau. Dans ce but, nous avons conçu l'Elastic-Arm, une interface passive mobile qui contraint la main de l'utilisateur par rapport à son corps et permet de générer des retours haptiques égocentriques. Plusieurs cas d'usage ont illustré les possibilités de cette approche et une étude utilisateur a démontré qu'un effet pseudo-haptique que nous avons proposé permet d'améliorer les perceptions des utilisateurs. Ensuite, nous avons considéré la manipulation d'objets virtuels et les sensations de saisie qui nécessitent habituellement l'usage d'interfaces multi-doigt complexes. En tant qu'alternative, nous avons proposé la Virtual Mitten, une technique d'interaction qui associe une interface élastique, un modèle de moufle et un retour pseudo-haptique pour simuler des sensations variables de saisie. Nous avons ensuite réalisé une étude utilisateur qui a validé l'applicabilité de la Virtual Mitten pour l'exécution de tâches variées et qui a mesuré la résolution de l'interface élastique utilisée. Finalement, nous avons considéré les tâches de manipulation fine nécessitant plusieurs doigts. Pour ce faire, nous avons proposé l'ElasticGlove, un exosquelette passif qui contraint chaque doigt séparement associé à un effet pseudo-haptique s'appliquant à chaque doigt en fonction de sa position sur la surface de l'objet manipulé. Plusieurs cas d'usage ont été proposé : un outil d'apprentissage musical exploitant l'effet pseudo-haptique pour favoriser certaines configurations des doigts ainsi qu'un simulateur de palpation médicale.

Les travaux présentés dans ce manuscrit de thèse pourraient faire l'objet de recherches additionnelles. Premièrement, dans l'optique de contrôler des mains virtuelles articulées, l'usage de données comportementales, géométriques et sémantiques pourrait automatiser la manipulation de certains degrés de liberté et donc favoriser l'utilisation d'interfaces plus simples. Ensuite, concernant les retours haptiques, les approches pseudo-haptiques proposées pourraient être évaluées dans des contextes applicatifs réels afin de mesurer leur effet sur le long-terme ainsi que l'effet sur les performances de présenter plusieurs retours de ce type en parallèle. Finalement, des interfaces hybrides mettant en oeuvre des composants à la fois matériels et logiciels pourraient combiner retours actifs, passifs et pseudo-haptiques pour enrichir l'interaction avec les mains en environnement virtuel

Résumé

Faire directement usage de nos mains pour explorer des environnements virtuels et interagir avec leur contenu permet une interaction à la fois naturelle et convaincante. Dans ce manuscrit de thèse, nous visons à améliorer l'interaction avec les mains dans le contexte de la Réalité Virtuelle en abordant deux défis majeurs : (1) faciliter le contrôle de modèles de mains articulées et (2) fournir des sensations haptiques au travers d'interfaces accessibles.

Nous abordons tout d'abord l'interaction au travers de mains virtuelles articulées et proposons deux méthodes pour faciliter leur contrôle. Premièrement, nous réduisons leurs nombreux degrés de liberté de façon à pouvoir exploiter des interfaces tactiles courantes. Le système qui en résulte permet aux utilisateurs de contrôler une main virtuelle en réalisant des gestes sur la surface de la tablette. Ensuite, nous adoptons une autre approche et séparons les degrés de liberté des mains virtuelles entre deux interfaces haptiques contrôlées en parallèle. Par cette distribution des contrôles et des retours de force, les utilisateurs sont exposés à des effets haptiques variés, autrement réservés à des interfaces haptiques coûteuses.

Nous abordons ensuite le sujet du retour haptique pour différents types d'interaction avec les mains. Pour cela, nous combinons des retours passifs à des retours pseudo-haptiques en tant qu'alternative à l'usage d'interfaces actives complexes et encombrantes. Dans un premier temps, nous considérons l'interaction avec les bras et proposons une armature élastique attachés à l'utilisateur qui fournit un retour de force égocentrique tout en conservant sa mobilité. Nous abordons ensuite le sujet de la saisie d'objets virtuels et proposons un nouveau paradigme d'interaction basé sur une interface élastique qui reproduit les mouvements de préhension et fournit un retour adapté et modulable par un effet pseudo-haptique. Finalement, nous considérons la manipulation fine avec les doigts et proposons un exosquelette passif qui les contraint séparément, associé à un retour pseudo-haptique multi-doigt simulant l'interaction avec des matériaux hétérogènes.

IntroductionFigure 1 . 1 -

 11 Figure 1.1 -Virtual hands, from pre-rendered models to interactive, articulated, hands. (a) Catmull's hand was digitalized for the first computer-animated movie [Price, 2008]. (b) Early virtual environments involved static hands for reaching virtual objects [Poupyrev et al., 1996].(c,d) Articulated hand models leverage the individual mobility of the digits to enable actions such as pointing[START_REF] Fisher | Virtual environment display system[END_REF]] and pinching[START_REF] Bibliography Iwata | Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator[END_REF]. (e) Deformable models provide realistic visuals and a more truthful interaction by reproducing the softness of the skin[START_REF] Gourret | Simulation of object and human skin formations in a grasping task[END_REF]].

IntroductionFigure 1 . 2 -

 12 Figure 1.2 -User interfaces for interacting with the hand. (a) Data glove for transposing the motion of the user's hand in the virtual world [Zimmerman et al., 1987]. (b) Markerless multi-finger tracking with the Leap Motion (Leap Motion Inc., USA). (c) Desktop haptic arm that outputs forces to the user's hand [Massie, 1993]. (d) Multifinger exoskeleton for simulating grasping forces on several fingers (CyberGlove Systems, USA).

Figure 2 . 2 -

 22 Figure 2.2 -Structure and mobility of the human hand. (a) Skeletal view revealing a highly articulated system [Taylor and Schwarz, 1955]. (b) Muscular layout composed of intrinsic muscles and tendons connected to extrinsic muscles located in the forearm [Napier and Tuttle, 1993]. (c) Kinematic model detailing the 27 degrees of freedom of the hand.

Figure 2 . 3 -

 23 Figure 2.3 -Patterns from Schlesinger's taxonomy of human grasps [Taylor and Schwarz, 1955]. This classification differentiates grasps depending on the shape and size of the manipulated objects.

Figure 2 . 4 -

 24 Figure 2.4 -Mixed grasp classification from Cutkosky and Wright [1986] (illustration from Zheng et al. [2011]), which takes into account shape and size, as well as the precision/power component of the task.

Figure 2 . 5 -

 25 Figure 2.5 -Mixed grasp classification from Bullock and Dollar [2011], which incorporates gestures as well as grasps involving within-hand motion.

Figure 2 . 6 -

 26 Figure 2.6 -Mechanoreceptors involved in haptic perception [Kolb and Whishaw, 2005]. Left: Cross-section of the skin; the hand has a particularly dense distribution of mechanoreceptors, which grants it a high sensitivity to touch. Right: Receptors in the muscles, joints, and tendons that participate in proprioception.

Figure 2 . 7 -

 27 Figure 2.7 -Rigid proxies for whole hand interaction. (a) 3D cursor for selecting items in a 3D menu [Boeck et al., 2004]. (b) Static hand model for exploring 2D images of 3D content [Jáuregui et al., 2012]. (c) Rigid hands with postures that correspond to their function, either holding a menu or selecting entries [Craig et al., 2009].

Figure 2 . 8 -

 28 Figure 2.8 -Rigid proxies for interacting with the fingertips. (a) Rigid spheres representing fingertips in a breast palpation procedure [Alhalabi et al., 2005]. (b) Interaction meshes located at the fingertips of a visually convincing hand model [Popescu et al., 1999]. (c) God-finger method for simulating the surfacic contacts of fingerpads from single interaction points [Talvas et al., 2013].

Figure 2 . 9 -

 29 Figure 2.9 -Articulated hand models. (a) Clamp-like model with only a thumb and an index finger [Maekawa and Hollerbach, 1998]. (b) Interaction spheres at the joints and fingertips [Maciel et al., 2004]. (c) Spherical sensors in the whole hand [Holz et al., 2008]. (d) Rough model made of primitive shapes [Hui and Wong, 2002]. (e) Hand made of a dense set of interaction points [Hirota and Hirose, 2003]. (f,g) Hand Models made of rigid segments that approximate the real shape of a hand [Borst and Indugula, 2005; Walairacht et al., 2002].

  Virtual hands real-world grasping. For example, Jacobs and Froehlich [2011] conceived a virtual hand made of deformable phalanxes attached to a rigid palm (Figure 2.10a) and other models compriseentirely deformable fingers [Pérez et al., 2013; Talvas et al., 2015]. Lee et al. [2006] modeled a entire deformable hand with the palm as a freeform surface that realistically stretches and bulges according to its posture (Figure 2.10b). Some deformable models simulate the underlying anatomy of the hand; for instance Albrecht et al. [2003] built a hand model housing virtual skeleton and muscles (Figure 2.10c).

Figure 2 . 10 -

 210 Figure 2.10 -Deformable hand models. (a) Soft fingertips on a rigid armature [Jacobs and Froehlich, 2011]. (b) Deformable skin that bulges when fingers curl [Garre et al., 2011]. (c) Deformable model simulating the underlying skeleton and muscles [Albrecht et al., 2003].

Figure 2 . 11 -

 211 Figure 2.11 -Virtual manipulation based on heuristic rules. (a) Shape-based grasping:an object is grasped if the hand adopts the posture that corresponds to its approaximate shape[START_REF] Wan | Realistic virtual hand modeling with applications for virtual grasping[END_REF]. (b) Anchor points: the object is grasped if it touches a point from the thumb and any other point[START_REF] Bibliography Iwata | Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator[END_REF]. (c) Sensors: the object is grasped if sensors form a valid grasping pair[START_REF] Holz | Multi-contact grasp interaction for virtual environments[END_REF].

Figure 2 . 12 -

 212 Figure 2.12 -Virtual manipulation based on physics simulation. (a) Rigid-body simulation [Zaeh et al., 2004]. (b) Penetration volumes that generates forces depending on the object intersection with the hand [Bergamasco et al., 1994]. (c) Molecular model to interact with deformable objects [Maciel et al., 2004].

Figure 2 . 13 -

 213 Figure 2.13 -Motion capture for hand-based interaction. (a) Optical tracking with infrared markers [Kry and Pai, 2006b]. (b) Data glove measuring the flexion of the digits (CyberGlove System, USA). (c) Markerless multi-finger tracker [Kim et al., 2012].

  [1997] used vibrations to simulate various textures and patterns, and TeslaTouch varies the perceived friction on a surface with electrovibrations [Bau et al., 2010] (Figure 2.14d).

Figure 2 . 14 -

 214 Figure 2.14 -Tactile feedback devices. (a) FEELEX: a surface on which is projected the image of a material is deformed to simulate various shapes [Iwata et al., 2001]. (b) AIRREAL: air vortices are blown toward the user's hand to simulate the touch of virtual objects [Sodhi et al., 2013]. (c) Electrocutaneous cylinder that measures the posture of the hand and stimulates the mechanoreceptors. [Kajimoto, 2012]. (d) Tesla-Touch: electrovibration can simulate various textures [Bau et al., 2010].

Figure 2 . 15 -

 215 Figure 2.15 -Tactile interfaces stretching the fingerpads. (a) The CyberTouch has a vibration motor on each finger (CyberGlove Systems, USA). (b) The GhostGlove stretches the fingerpads and palm to simulate contacts with virtual objects [Minamizawa et al., 2008]. (c) Wire-driven system contracting around the fingerpads [Scheibe et al., 2007].

Figure 2 . 16 -

 216 Figure 2.16 -Whole hand force feedback interfaces. (a) Geomagic Touch, a typical haptic desktop arm with 3-DoF output (3D Systems, USA). (b) Virtuose arm, a larger haptic device with 6-DoF output (Haption SA, France), here used in a cooking simulator [Cirio et al., 2011]. (c) String-based Spidar G&G with three DoF per hand [Luo et al., 2003].

Figure 2 . 17 -

 217 Figure 2.17 -Human-scale feedback interfaces. (a) Virtuose arm mounted on a ceiling rail to accompany users in a large workspace (Haption, France). (b) INCA 6D, a stringbased interface with an expanded workspace [Dominjon et al., 2007] (c) HapticGEAR, a backpack-like string-based device that actuates a tool held by the user [Hirose et al., 2001]. (d) Body-mounted device that constrains the user's arm on one degree of freedom [Tsetserukou et al., 2010].

Figure 2 . 18 -

 218 Figure 2.18 -Multi-finger grounded interfaces. (a) Combination of two desktop devices that handle one finger each [Pacchierotti et al., 2012]. (b) Combination of three desktop devices to simulate three buttons [Michelitsch et al., 2002]. (c) MasterFinger-2: two fingers are constrained with three DoF each [Monroy et al., 2008]. (d) DigiHaptic: three fingers are constrained with one DoF each [Casiez et al., 2003]. (e) HIRO III: a mirrored robotic hand [Endo et al., 2011]. (f) Spidar-8: a string-based system actuating four fingers per hand [Walairacht et al., 2002].

Figure 2 .

 2 Figure 2.19 -Multi-finger body-mounted exoskeletons. (a) CyberGrasp, a commercial haptic exoskeleton (CyberGlove Systems, USA). (b) Rutgers Master II, a pneumatic endoskeleton [Bouzit et al., 2002]. (c) MasterHand, a full-hand multi-finger exoskeleton [Fang et al., 2009]. (d) WHIPFI, a two-finger exoskeleton with tactile stimulation [Gosselin et al., 2005]. (e) HAPTEX, a two finger exoskeleton for simulating the touch of fabric [Magnenat-Thalmann et al., 2007]. (f) Encounter-type glove that accompanies the fingers until a collision must be rendered [Sato et al., 2007].

Figure 2 . 20 -

 220 Figure 2.20 -Hybrid force feedback devices. (a) Sigma.7 grip force device [Tobergte et al., 2011]. (b,c) Gripper handles plugged on dekstop arms; they actuate two [Najdovski and Nahavandi, 2008] or three different fingers [Barbagli et al., 2003]. (d) Haptic WorkStation, a complex interface that delivers external forces to the wrist and internal forces to the fingers (CyberGlove Systems, USA).

Figure 2 . 21 -

 221 Figure 2.21 -Whole hand passive haptic interfaces. (a) Tangible surfaces to simulate the support of a virtual control panel [Borst and Volz, 2005]. (b) Passive props to conduct a neurosurgery visualization task [Hinckley et al., 1998]. (c) The poor man's force feedback device for conducting a sculpting task [Galyean and Hughes, 1991].

Figure 2 . 22 -

 222 Figure 2.22 -Multi-finger passive interfaces. (a) Foam ball with pressure sensors to control the fingers of a virtual hand [Pihuit et al., 2008]. (b) HandNavigator for controlling a virtual hand [Kry et al., 2008]. (c) Partially passive exoskeleton [Koyama et al., 2002].

Figure 2 . 23 -

 223 Figure 2.23 -Interaction metaphors for manipulating virtual objects. (a) The image plane metaphor for selecting objects with the fingers [Pierce et al., 1997]. (b) The virtual hand metaphor for touching and grabbing objects with the hand [Poupyrev et al., 1996]. (c) The virtual pointer metaphor for targeting objects with the hand [Poupyrev et al., 1996].

Figure 2 . 24 -

 224 Figure 2.24 -Selection techniques based on the Virtual Pointer metaphor. (a) Flexible pointer: users bend the ray with both hands to reach occluded objects [Olwal and Feiner, 2003]. (b) Bent pick ray: the ray can be lifted like a fishing rod to move virtual objects [Riege et al., 2006]. (c) iSith: virtual objects are moved to the intersection of two rays fired from each hand [Wyss et al., 2006].

Figure 2 . 25 -

 225 Figure 2.25 -Composite interaction techniques. (a) The scaled world grab: objects are scaled and put in the user's hand [Mine et al., 1997]. (b) The voodoo doll: manipulated objects undergo the same transformations as the dominant and non-dominant hands [Pierce et al., 1999]. (b) The object in hands: the dominant hand interacts through a haptic interface with virtual objects brought forward with the non-dominant hand [Boeck et al., 2004].

Figure 2 . 26 -

 226 Figure 2.26 -Different visual feedback for object manipulation. (a) Explicit glyphs displaying the direction and magnitude of interaction forces [Sreng et al., 2006]. (b) Particles displaying the intensity and direction of friction forces [Sreng et al., 2007]. (c) Spring-loaded virtual device that bends and twists to visually express the weight of manipulated objects [Koutek and Post, 2001]. (d) Contact area and forces displayed through transparent objects [Rusák et al., 2009]. (e) Various types of visual feedback that apply to a virtual hand or to manipulated objects [Prachyabrued and Borst, 2014].

Figure 2 . 27 -Figure 2 . 28 -

 227228 Figure 2.27 -Pseudo-haptic feedback for hand-based interaction. (a) Pseudo-haptic avatar to express the weight of lifted objetcs [Jáuregui et al., 2014]. (b) SoftAR, a projective display for simulating stiffness [Punpongsanon et al., 2015]. (c) HEMP simulates force fields by offsetting the position of the user's hand [Pusch et al., 2008]. (d) Simulation of curvatures through an augmented reality display[START_REF] Bibliography Ban | Modifying an identified curved surface shape using pseudo-haptic effect[END_REF]. (e) Redirected touching, a method that warps the physical space to simulate different shapes[START_REF] Kohli | Redirected touching: Warping space to remap passive haptics[END_REF].

Figure 2 . 29 -

 229 Figure 2.29 -Interaction techniques to handle interpenetration. (a) Joint unfolding: the virtual hand is progressively opened until it exits the objects [Boulic et al., 1996]. (b) Virtually coupled hands: a physically-based virtual hand is attracted to the exact position of the user but it respects the constraints of the virtual objects [Prachyabrued and Borst, 2011]. (c) Sticking issue occurring with virtually coupled hands [Prachyabrued and Borst, 2012a].

Figure 3 . 1 -

 31 Figure 3.1 -Controlling the posture of a hand model with THING. Our approach is based on a morphologically-consistent multi-touch interface that applies the real motion of the user on the virtual model. For example, performing a grasping gesture or a pointing gesture replicates it on the hand model.

THINGFigure 3 . 2 -

 32 Figure 3.2 -Rigged hand model with reduced degrees of freedom. (a) It is composed of a hand mesh, a skeletal armature (grey bones), IK targets (colored spheres) and predefined trajectories (colored curves). (b) The flexion/extension of a finger is realized by sliding the IK target along the curve and the adduction/abduction is realized by moving it away from the curve. and a n = 1 (respectively -1) signifies that the finger is fully abducted (respectively adducted). The local frame has thus 10 DoF. In 3D animation, hand models are rigged with a skeletal armature consistent with human anatomy. Regarding the global frame, the skeleton root is placed at position p p p with orientation o. Regarding the local frame, each finger is guided by an inverse kinematics target. Editable curves define the trajectory of each target (and thus of each finger) (Figure 3.2a). Those trajectories are parametrically defined as cubic Bézier curves B b 1 b 1 b 1 ,c 1 c 1 c 1 ,b 2 b 2 b 2 ,c 2 c 2

THINGFigure 3 . 3 -

 33 Figure 3.3 -Graphical user interface of displayed on the tablet. (a) Graphical user interface displayed on the tablet. Sliders are placed under each user's finger to control a corresponding finger of the hand model. (b) Slider controlling the motion of a finger. A displacement along the slider controls flexion and a displacement away from the slider controls adduction.

THINGFigure 3 . 4 -

 34 Figure 3.4 -The two variants of our approach. (a) MobileTHING integrates all the degrees of freedom on the tablet and leverages its internal sensors to rotate the virtual hand. (b) DesktopTHING conjointly uses a computer mouse to handle the global motion of the hand. The user can either switch between tablet and mouse with his hand or control one device with each of his hand.

THINGFigure 3 . 5 -

 35 Figure 3.5 -Postures realized with THING. (a,b) Grasping and juggling with a ball. (c) A walking hand reminiscent of the Thing character from the Addams Family, which actually inspired THING.

Figure 3 .

 3 Figure 3.6 -A ball throwing sequence animated with our technique.

Figure 3 . 7 -

 37 Figure 3.7 -Controlling both hands of a virtual character. (a) The user switches between his own left and right hands to control the respective hand of the virtual character. (b) Example of character posture realized with THING.

Figure 3 . 8 -

 38 Figure 3.8 -Traditional techniques compared with THING in the first experiment. (a) The participant uses a mouse and a standard keyboard. (b) The participant wears a data glove and additional tracking devices (a string-based GameTrak behind the screen for position tracking and a Colibri sensor on the hand for orientation tracking). (c) View of the editor in which the experiment was conducted.

Figure 3 . 9 -

 39 Figure 3.9 -Hand poses to reproduce in the user study (top view and side view).

Figure 3 . 10 -

 310 Figure 3.10 -Completion time for Technique and Pose, error bars representing 95% confidence interval.

Figure 3 . 11 -

 311 Figure 3.11 -Completion time for Technique and Pose, error bars representing 95% confidence interval.

Figure 3 . 12 -

 312 Figure 3.12 -Answers to the subjective questionnaire of the second experiment, on a 7-point Likert scale. Each boxplot is delimited by the quartile (25 % quantile and 75% quantile) of the distribution of the effect over the individuals. The median is also represented for each Technique.

Contents 4 . 1

 41 Two-handed interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 4 . 1 -

 41 Figure 4.1 -Manipulating a virtual object with DesktopGlove. The left hand handles a multi-finger device that controls the flexion of the virtual fingers, in order to grasp objects and feel their shape. The right hand controls a classic haptic desktop device, to control its position and orientation and to feel global forces like weight or collisions.

Figure 4 . 2 -

 42 Figure 4.2 -Multi-finger device for controlling the fingers of the virtual hand, derived from the DigiHaptic [Casiez et al., 2003]. Users insert their thumb in the middle ring and their index finger into either the left or right ring depending on hand that is used (dominant or non dominant). The device applies rotational efforts through each ring, and it can separate or brind the fingers together to provide haptic feedback that relate to grasping.

Figure 4 . 3 -

 43 Figure 4.3 -Coupling between the virtual hand and the input devices. Left: the hand model has an underlying physical skeleton (green) that interacts with the virtual environment.Center: a non-physical intermediary model is directly driven by the input devices; spring-like constraints attracts the virtual hand which tends to adopt the same configuration without penetrating virtual objects. Right: the input devices controls the intermediary model and output forces that depend on the discrepancy between virtual hand and target hand.

Figure 4 . 4 -

 44 Figure 4.4 -Forces delivered by each haptic interface. Orange arrows illustrate global forces applying to the whole hand that are provided by the haptic arm. Pink arrows illustrate local forces applying to the digits that are provided by the multi-finger device.Some manipulation tasks involve both types of feedback, like feeling the shape of an object through the fingers as well as its weight through the wrist.

Figure 4 . 5 -

 45 Figure 4.5 -Interfaces featured in the different experimental conditions. (a) With the Integrated condition, users wore a data glove that replicated their gestures in the virtual environment. (b) With the SeparatedA (pictured) and SeparatedB conditions, one hand controlled the global motion while the other controlled the fingers.

Figure 4 . 6 -

 46 Figure 4.6 -Hand poses to reproduce in the first experiment (presented one at a time during the experiment). Each posture is mirrored along the horizontal axis except for #1 which is centered. Users start each trial with their hand at the bottom of the screen. Condition Non-dominant hand Dominant hand Integrated Global + Local SeparatedA Global Local SeparatedB Local Global

Figure 4 . 7 -

 47 Figure 4.7 -Completion time for the first experiment.

Figure 4 . 8 -

 48 Figure 4.8 -Docking procedure of the second experiment. (a) Participants had to pick and place a toy cube over a target position. (b) Depending on the experimental condition, participants were subjected to various distributions of force feedback between their two hands. The provided forces were the collisions with the floor, the weight of the cube, and the contact between cube and fingers. Condition Local forces Global forces None ✗ ✗ Fingers ✓ ✗ Wrist ✗ ✓ Both ✓ ✓

Figure 4 . 9 -

 49 Figure 4.9 -Answers to the subjective questionnaire of the second experiment on a 5-point Likert scale. Each box plot is delimited by the quartile (25% quantile and 75% quantile) of the distribution of the effect over the individuals. The median is also represented for each Feedback.

  Elastic-Arm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.1.

Figure 5 . 2 -

 52 Figure 5.2 -Elastic armature that constrains the user's hand relatively to his body. (a) It is composed of an elastic cable (an arm exerciser in our prototype), 3D-printed straps, and tracking devices (a Razer Hydra). (b) Different states depending on the extension of the arm: relaxed cable (yellow), nearly taut (orange) and extended arm (red). The dotted radius corresponds to the spherical boundary of the elastic cable beyond which haptic feedback is felt (||r r r|| > d e ).

Figure 5 . 4 -

 54 Figure 5.4 -Control scheme for steering the virtual arm. The control mode depends on the extension of the arm: the virtual hand is position-controlled below the elastic threshold (yellow) and the hand is rate-controlled beyond the elastic threshold (red).

Figure 5 . 5 -

 55 Figure 5.5 -Navigating around obstacles to reach occluded objects. Top: Successive frames of the user's view when reaching for an occluded object. The camera follows the virtual hand to maintain a constant level of precision. Bottom: Top view of the path traveled by the virtual hand.

Figure 5 . 6 -

 56 Figure 5.6 -Simulating virtual boundaries with the Elastic-Arm. (a)The virtual hand (blue) does not collide with any obstacle yet and the user's arm is below the elastic threshold; there is no feedback. (b) The virtual hand now collides with an obstacle. The Control/Display ratio was adjusted to match this event with the tension of the elastic cable and a resistance is felt. (c) Users can explore the environment and "touch" virtual objects at various distances like the screens or the keyboard.

Figure 5 . 7 -

 57 Figure 5.7 -Simulating different levels of effort with the Elastic-Arm. Two different deformable objects are interacted with. (a) The object is soft so the hand motion is amplified and the user moderately stretches his arm to bend the surface; the haptic feedback is moderate. (b)The object is stiff so the hand motion is slowed down and the user must stretch his arm to a greater degree; the haptic feedback is stronger.

Figure 5 . 8 -

 58 Figure 5.8 -Pseudo-haptic effect simulating different levels of effort (Top: virtual avatar, Bottom: real hand movements). (a) The button on the left is associated to an interaction coefficient k i = 1 so the motion of the user's hand is unaltered. (b)The button on the right is associated to a lower interaction coefficient k j < k i so the motion of the user's hand is reduced. The user compensates this difference by extending the arm furthermore, which generates stronger haptic cues.

Figure 5 . 9 -

 59 Figure 5.9 -Different steps of the experimental procedure (Top: external perspective, Bottom: view of the participant). (a) The participant pushes on the virtual button until the light is switched on. (b) The participant releases the button to display the next button. (c) The participant selects the button with the highest stiffness.

Figure 5 . 10 -

 510 Figure 5.10 -Answers to the subjective questionnaire on a 5-Likert scale. Q1: You liked this technique. Q2: This technique was easy to use. Q3: This technique was accurate. Q4: This technique was comfortable. Q5: This technique was tiring. Q6: The buttons required some effort to push. Q7: The buttons required different levels of effort to push. Q8: You were confident about your answers.

Figure 5 . 11 -

 511 Figure 5.11 -Multi-layer rendering for providing different levels of stiffness. (a) The elastic is not taut when there is no collisions with the virtual hand. (b) The shortest elastic is tightened just when the hand touches the virtual button. (c) The second and longest elastic is tightened when the button reaches the end of its course.

ConclusionFigure 5 . 12 -

 512 Figure 5.12 -Simulating weight with rubber bands anchored on the waist. (a) Lighter dumbbell associated to a neutral Control/display ratio. (b) Heavier dumbbell associated to a lower ratio. To lift it, the user has to raise his arm higher, which extends the rubber band furthermore and produces stronger downward forces. (c) Simulating imbalance by scaling the vertical motion of each hand asymmetrically. The user compensates this difference to keep the barbell straight, which generates stronger downward forces on the heavier side.

Figure 6 . 1 -

 61 Figure 6.1 -Grasping virtual objects in a bimanual scenario with the Virtual Mitten. Each hand holds an elastic device to control a corresponding mitten. The grip force applied by the user is measured to grasp virtual objects and to generate pseudo-haptic feedback.

Figure 6 . 2 -

 62 Figure6.2 -Mitten model that represents the user's hands. The fingers are merged together since only coarse grasping is considered with our approach, which makes hand tracking simpler and more reliable while keeping the interaction expressive.

Figure 6 . 4 -

 64 Figure 6.4 -Boolean and Progressive visual feedback expressing grasping efforts. Top: Boolean feedback. Bottom: Progressive feedback. The r f olding threshold is fixed but the r grasping threshold varies in order to enable our pseudo-haptic effect.

Figure 6 . 5 -

 65 Figure 6.5 -Primitive haptic tasks from Bloomfield et al. [2003]. (a) Force I: pulling a drawer. (b) Force II: removing a pin. (c) Torque I: screwing an object. (d) Torque II: pulling a lever.

Figure 6 . 7 -

 67 Figure 6.7 -Setup of the user study. (a) Users were seated in front of a screen displaying the virtual environment; they entered their answers by pressing marked keys on a keyboard. (b) The virtual scene contained a single object to be interacted with.

Figure 6 . 10 -

 610 Figure6.10 -Psychometric curve for the screwing task. It plots the average percentage of answers in which participants considered the task using the reference threshold as the one requiring more effort to perform.

  ElasticGlove . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 7 . 1 -

 71 Figure 7.1 -Performing various tasks with the ElasticGlove. (a) Single-finger configuration for simulating basic tasks such as pushing a button. (b) Three-finger configuration for simulating more complex manipulation tasks such as clay sculpting with several digits.

Figure 7 . 2 -

 72 Figure 7.2 -Schematics of the elastic exoskeleton. (a) Bendable finger modules are attached to the back of the hand to provide force feedback when the fingers curl. (b) Each module is mounted on a rail slider to accompany the digit when it curls.

Figure 7 . 3 -

 73 Figure 7.3 -Haptic data embedded in the mesh of a virtual object. (a) Visual model of a virtual apple, darker areas are rotten spots. (b) Stiffness map that associates each surface point to a stiffness coefficient input to our pseudo-haptic algorithm.

Figure 7 . 4 -

 74 Figure 7.4 -Playing the piano with the ElasticGlove. The passive interface reproduces the resistance of the piano keys. Depending on the chord to learn, the levels of stiffness of the key are updated to favor the motion of the appropriate fingers.

Figure 7 . 5 -

 75 Figure 7.5 -Training in medical palpation with the ElasticGlove. Users must detect suspicious tissues by exploring a body part. Anomalies can be "painted" on the body part to represent various conditions, using the workflow presented previously.

  Le but de cette thèse est d'améliorer l'interaction avec les mains en environnement virtuel. Nous abordons deux axes de recherche : (I) la conception de nouvelles méthodes pour améliorer le contrôle des mains virtuelles articulées et (II) la conception de nouvelles approches pour combiner retour haptique passif et retour pseudo-haptique dans le cadre de la manipulation 3D. Ces axes de recherche et nos contributions sont illustrés sur la Figure B.1. Puisqu'interagir à travers des modèles de mains réalistes nécessite des interfaces complexes, l'objectif du premier axe de recherche est d'exploiter des interfaces plus accessibles, à différents niveaux de détails. Dans ce but, nous proposons deux stratégies : (1) réduire le nombre de DdL des modèles de main pour faciliter leur contrôle et (2) séparer les DdL entre plusieurs interfaces afin de mieux répartir les contrôles et les retours de force.Premièrement, contrôler des mains virtuelles pourrait être réalisé par des interfaces tactiles, aujourd'hui courantes et accessibles, puisqu'elles permettent d'interagir avec

Figure B. 1 -

 1 Figure B.1 -Organisation de nos contributions. Notre premier axe de recherche est de faciliter le contrôle de mains virtuelles en réduisant et en distribuant les degrés de liberté liés à la manipulation d'objets. Notre second axe de recherche consiste à combiner retour passif et retour pseudo-haptique pour fournir des sensations haptiques convaincantes.

  Les mains sont parmi les parties du corps les plus expressives et donc leurs représentations virtuelles font partie des modèles les plus difficiles à animer. Les méthodes traditionnelles d'animation nécessitent de spécifier une succession de postures, chacune étant obtenue par l'ajustement précis de tous les segments de la main. Ainsi, ce chapitre présente THING, une nouvelle technique d'animation qui est plus accessible que la capture de mouvements mais bénéficie de la flexibilité et de la précision de l'interaction avec les mains. Cette approche est basée sur une tablette tactile sur laquelle l'animateur réalise des mouvements avec ses doigts pour contrôler un modèle de main virtuelle dont les degrés de liberté ont été réduits (Figure B.2).

Figure B. 2 -

 2 Figure B.2 -Animation d'une main virtuelle avec THING. Notre approche est basée sur une interface tactile qui applique les mouvements de l'utilisateur sur le modèle virtuel. Par exemple, réaliser un mouvement de saisie ou pointer le doigt reflète ce même mouvement sur le modèle de main.

Figure B. 3 -B.1. 3 Évaluation

 33 Figure B.3 -Modèle de main aux degrés de liberté réduits. Chaque doigt virtuel se déplace par rapport à une trajectoire éditable. L'animateur contrôle les doigts en réalisant des mouvement similaires sur la surface d'une tablette tactile.

Figure B. 4 -

 4 Figure B.4 -Résultats de l'évaluation de THING et de ses variantes. (a) Performances obtenues pour les différentes techniques lors de la première expérience. (b) Résultats du questionnaire subjectif de la seconde expérience.

Figure B. 5 -B. 2 . 1

 521 Figure B.5 -Interface bimanuelle séparant les degrés de liberté d'une main virtuelle entre les deux mains de l'utilisateur. À gauche, une interface de bureau multi-doigt contrôle les doigts virtuels et perçoit les efforts de saisie. À droite, un bras haptique contrôle la position de la main et perçoit le poids et les collisions.

Figure B. 6 -B. 2 . 2 Évaluation

 622 Figure B.6 -Distribution des forces lors de tâches de manipulation. Les flèches oranges illustrent les forces externes qui s'appliquent à toute la main et qui sont fournies par le bras haptique dans notre implémentation. Les flèches roses illustrent les forces internes qui s'appliquent aux doigts et qui sont fournies par notre interface dérivée du DigiHaptic [Casiez et al., 2003].

Figure B. 7 -B. 3

 73 Figure B.7 -Première expérience de l'évaluation de DesktopGlove. (a) Postures à reproduire (présentées une par une pendant l'évaluation). (b) Performances de chaque technique ; Integrated correspond au gant de données, avec SeparatedA les doigts virtuels étaient contrôlés par la main dominante et avec SeparatedB les doigts virtuels étaient contrôlés par la main non dominante.

Figure B. 8 -

 8 Figure B.8 -Atteindre des objets distants avec l'Elastic-Arm. Ici, un mode de contrôle hybride inspiré de techniques d'interaction utilisant initialement des interfaces haptiques [Dominjon et al., 2005b] permet d'étirer son bras de façon précise.

Figure B. 9 -Figure B. 10 -

 910 Figure B.9 -Simuler les surfaces de l'environnement virtuel avec l'Elastic-Arm. (a) La main virtuelle (bleue) ne touche pas encore l'obstacle et le cable élastique n'est pas pas tendu. (b) La main virtuelle touche l'obstacle. Sa vitesse de déplacement a été ajustée pour que ce contact coincide avec la tension du cable élastique, ce qui provoque une résistance physique. (c) Simulation mettant en oeuvre cet effet : l'utilisateur peut, à travers son avatar, toucher les éléments d'un panneau de contrôle placés à différentes distances.

  niveaux de raideur et de les trier en fonction de l'effort perçu (Figure B.11) et ce dans deux conditions : avec et sans armature élastique. Au final, lorsqu'ils étaient équipés avec notre système, les participants ont fournis significativement plus de réponses correctes que lorsqu'ils interagissaient sans retour haptique, ce qui démontre l'apport en sensations de l'Elastic-Arm comparé à un retour visuel seul. De plus, les participants ont majoritairement préférés l'Elastic-Arm à la condition sans retour de force, en mentionnant qu'il « rend le monde virtuel plus réaliste » et qu'il « donne l'impression de toucher de véritables boutons ».

Figure B. 11 -

 11 Figure B.11 -Étapes de l'évaluation utilisateur de l'Elastic-Arm. (a) Le participant appuie sur le bouton virtuel pour percevoir sa raideur, jusqu'à ce que la lumière rouge s'allume. (b) Le participant ramène la main vers son corps pour passer au bouton suivant. (c) Une fois les trois boutons poussés, le participant sélectionne celui qui a nécessité le plus d'effort, puis celui qui en a demandé le moins.

  Figure B.12 -Scénario bimanuel faisant usage de la Virtual Mitten. Chaque main tient une interface élastique pour contrôler un modèle de moufle. La force de saisie appliquée sur l'interface est mesurée pour attraper les objets virtuels et générer un retour pseudohaptique.

  (Figure B.13) ainsi qu'un scénario ludique dans lequel des fruits préssés possèdent différents niveaux de raideur (Figure B.12).

Figure B. 13 -B. 4 . 3 Évaluation

 1343 Figure B.13 -Exemples d'actions haptiques réalisables avec la Virtual Mitten, issus de la taxonomie de Bloomfield et al. [2003].

Figure B. 14 -

 14 Figure B.14 -Évaluation de la Virtuel Mitten. (a) Les participants devaient interagir avec des objets virtuels et les classer selon l'intensité de l'effort d'interaction. (b) Courbe psychométrique issue de la seconde expérience et caractérisant la résolution de l'interface élastique utilisée.

  ElasticGlove, interaction multi-doigt combinant retour haptique passif et retour pseudo-haptiqueB.5.1 Interface élastique multi-doigtL'ElasticGlove est un exosqelette fait de composants uniquement passifs. Son retour haptique est fourni par des modules de doigt prenant la forme de tiges déformables montées sur le dos de la main. Ces modules contraignent le mouvement de flexion des doigts lorsque l'utilisateur ferme sa main, ce qui permet de fournir des sensations qui correspondent à la saisie d'objets ou bien à leur palpation. Afin de ne pas bloquer les doigts, des glissières supportent chaque module et accompagnent les mouvements de flexion. L'interface est modulaire et peut accueillir différent configurations selon l'usage qui va en être fait(Figure B.16).

Figure B. 15 -

 15 Figure B.15 -Différentes configurations de l'exosquelette élastique. (a) Une configuration à un doigt pour simuler une tâche basique d'appui sur un bouton virtuel. (b) une configuration à trois doigts pour simuler des tâches plus complexes telles que la déformation d'un objet virtuel.

Figure B. 16 -

 16 Figure B.16 -Scénarios de palpation médicale et d'apprentissage musical. (a) L'utilisateur doit détecter des anomalies and inspectant un organe virtuel avec les doigts. (b) Pour favoriser l'apprentissage d'accords de piano, les touches incorrectes sont associées à un coefficient d'interaction plus faible.

It was confusing * 1 =Condition 4 :2 3 4 5

 144 strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 It was simple to use * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 = strongly agree 1 You perceived the weight of the cube * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 You perceived the contact of the floor * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 You perceived the shape of the cube * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 What did you feel with your right hand? What did you feel with your left hand? Force feedback on the wrist and fingers Did you like this condition? * 1 = don't like it at all / 2 = don't like it / 3 = neutral / 4 = like it / 5 = like it a lot 1 What did you like about it? What did you dislike about it? It was natural * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 = strongly agree 1 disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 = strongly agree 1 It was simple to use * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 = strongly agree 1 2 3 4 5 strongly disagree strongly agree It was clearly understandable * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5

Table 2 .

 2

	Related work
	Meissner's corpuscle
	Merkel's receptor
	Ruffini corpuscle
	Pacinian corpuscle

1 -Properties of the different types of mechanoreceptors. The adaptation rate is the speed for a receptor to go back to its neutral state, the input frequency corresponds to the frequency at which separate stimuli are distinguished, and the receptive field is the area in which a single receptor is sensible to stimuli.

(1) actuators arrays, (2) pneumatic feedback, (3) electrotactile feed- back, and (4) vibrotactile feedback.

	Dedicated tactile interfaces have been designed for simulating the touch of virtual ob-jects and for communicating their material properties to users. Four different types of actuation are generally exploited for tactile stimulation [Benali-khoudja et al., 2004; Shimoga, 1993b]:

  (Figure 2.15b). Similarly, Scheibe et al. [2007] designed a thimble surrounded

  but this only enables basic single-finger tasks, such as pushing or probing virtual objects. At most, objects can be pinched by pairing two separate devices together, each one handling a finger[START_REF] Kawai | Stable haptic display of 1-DoF grasping with coupling impedance for internal and external forces[END_REF][START_REF] Pacchierotti | Two finger grasping simulation with cutaneous and kinesthetic force feedback[END_REF] (Figure2.18a and 2.18b).

  and a virtual board-game [Viciana-Abad et al., 2010]. Hand-held props can also be leveraged: Lindeman et al. [1999] and Poupyrev et al.

  Dominjon et al. [2005a] altered the C/D ratio of a virtual object on the vertical axis only to simulate gravity.[START_REF] Issartel | Perceiving mass in mixed reality through pseudo-haptic rendering of newton's third law[END_REF] incorporated a pseudo-haptic simulation of weight in augmented reality by shifting the position of a virtual proxy used to push weighted virtual objects. For simulating the mass of different dumbbells, Jáuregui et al.[2014] designed a pseudo-haptic avatar which animation parameters (speed profile, posture) are altered to express different levels of effort (Figure2.27a). Similarly,[START_REF] Ban | Augmented endurance: controlling fatigue while handling objects by affecting weight perception using augmented reality[END_REF] altered the visual color of real objects in augmented reality to affect the user's perception of endurance.

3 A Multi-touch Interaction Technique for Controlling 3D Hand Models: THING Contents 3.1 Hand animation: framework and techniques

  . . . . . . . . . . . . . . . . . 57 3.2 THING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.2.1 Rigged hand model with reduced degrees of freedom . . . . . . . . . . 58 3.2.2 Controlling the model via multi-touch input . . . . . . . . . . . . . . . 59 3.2.3 Mobile and desktop variants . . . . . . . . . . . . . . . . . . . . . . . . 62 3.2.4 Proof-of-concept implementation . . . . . . . . . . . . . . . . . . . . . 63 3.3 User evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.3.1 Experiment #1: MobileTHING . . . . . . . . . . . . . . . . . . . . . . 65 3.3.2 Experiment #2: DesktopTHING . . . . . . . . . . . . . . . . . . . . . . 70 3.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

72 3.4 Conclusion

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

  Mouse: 61.9s, DesktopTHING: 39.1s, MobileTHING: 37.3s). Pairwise comparisons showed significant differences (p < 0.03) between poses #2 and #3 (#2: 38.0s, #3: 51.4s, #4: 48.9s). Pairwise comparisons revealed significant differences (p < 0.01) between DesktopTHING and Mouse for pose #2. Significant differences (p < 0.04)

2,22 = 8.4, p < 0.01), Pose (F 2,22 = 4.3, p < 0.03), and a significant Technique × Pose interaction (F 4,44 = 9.5, p < 0.0001) on completion time (Figure 3.11). Pairwise comparisons revealed significant differences (p < 0.01) between Mouse and the two other techniques

(

.3 User evaluation

  DesktopGlove . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.2.1 Separating the degrees of freedom . . . . . . . . . . . . . . . . . . . . . 78 4.2.2 Distributing the force feedback between the interfaces . . . . . . . . . 81 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 4.3.1 Experiment #1: separated controls vs. integrated controls . . . . . . 83 4.3.2 Experiment #2: object manipulation with force feedback . . . . . . . 88 4.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

. . 77 4.1.1 Benefits of using both hands . . . . . . . . . . . . . . . . . . . . . . . . 77 4.1.2 Specificities of each hand . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.1.3 Current limitations of multi-finger haptics . . . . . . . . . . . . . . . . 78 4.2

Table 4 .1 -

 4

Experimental conditions for the first user study. Dominant and nondominant hands were attributed different frame of references. The Integrated condition leverages a data glove whereas the Separated condition leverages a variant of our bimanual setup.

Table 4 . 2

 42

-Experimental conditions of the second user study. Dominant and nondominant hands were subjected to different distributions of force feedback. In the None condition, force feedback was disabled. In the Both condition, force feedback was delivered to the two hands. In the Fingers and Wrist conditions, a partial force feedback was respectively delivered to the non-dominant or dominant hand.

104 5.3 Alternative designs for simulating new haptic properties .

  Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

1 Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.1.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.1.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.2 Illustrative use cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.2.1 Using the Elastic-Arm to expand the virtual workspace . . . . . . . . 99 5.2.2 Using the Elastic-Arm to improve perception . . . . . . . . . . . . . . 101 5.2.3 User evaluation: distinguishing virtual buttons with different stiffness . . . . . . . . 108 5.3.1 Multi-layer rendering with several rubber bands . . . . . . . . . . . . . 108 5.3.2 Weight simulation from the waist . . . . . . . . . . . . . . . . . . . . . 109 5.4

  User evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 6.2.1 Apparatus and participants . . . . . . . . . . . . . . . . . . . . . . . . . 121 6.2.2 Experiment #1: subjective evaluation of the Virtual Mitten . . . . . . 123 6.2.3 Experiment #2: psychophysical assessment for a screwing task . . . . 126 6.2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Contents 6.1 The Virtual Mitten . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.1.1 Elastic input device for simulating grasping . . . . . . . . . . . . . . . 115 6.1.2 Control scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6.1.3 Visual feedback for expressing the level of effort . . . . . . . . . . . . . 118 6.1.4 Pseudo-haptic feedback for modulating the perceived grip forces . . . 119 6.1.5 Illustrative use cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 6.2

2 Pseudo-haptic effect for varying the perceived efforts

  . . 133 7.1.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7.1.2 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7.. . . . . . . . . . . 134 7.2.1 Coupling the user's motion to a virtual hand . . . . . . . . . . . . . . 135 7.2.2 Rendering the stiffness of virtual objects . . . . . . . . . . . . . . . . . 135 7.2.3 Authoring pseudo-haptic objects . . . . . . . . . . . . . . . . . . . . . . 136 7.3 Illustrative use cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 7.3.1 Learning piano chords . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 7.3.2 Training in medical palpation . . . . . . . . . . . . . . . . . . . . . . . 137 7.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

http://www.anr-mandarin.fr/

Blender 3D editor, http://www.blender.org/

LibHand, http://www.libhand.org

LibHand, http://www.libhand.org

http://www.prohands.net/products/gripmaster.php

OpenSceneGraph graphics toolkit, http://www.openscenegraph.org/

Bullet physics library, http://bulletphysics.org/wordpress/

Acknowledgments

First and foremost, I would like to sincerely thank my two advisors, Maud Marchal and Anatole Lécuyer, for supervising me and for sharing their brilliant ideas during those three years. Beyond Virtual Reality and haptics, Maud taught me how to say more with less words and Anatole showed me how to take a step back and look at the big picture. Those are two very precious skills that I will carry in other areas of my professional and personal life, so thank you to both of you. I would also like to thank the members of the PhD. committee for taking the time to read this manuscript, for traveling to Rennes in order to participate to the defense despite their busy schedules, and for discussing this work so thoroughly.

DekstopGlove

12 participants × 4 types of feedback × 6 poses × 3 repetitions = 864 trials.

Collected data

Completion time is the main measure and is defined as the time taken between the beginning of each trial and the time when participants successfully matched the virtual cube with the target cube within the tolerance thresholds. At the end of the evaluation, participants filled a subjective questionnaire (Appendix C3) to rate each technique in terms of global appreciation, realism, accuracy, comfort, fatigue, ease, the perception of weight, perception of contact and perception of shape on a 5-point Likert scale. A score of 5 for these criteria means a very positive rating except for fatigue where it means that the technique was very tiring (a rating of 1 means that it was not tiring at all). The participants were also asked to order the techniques by personal preference.

Results

A mixed repeated measures ANOVA showed a significant effect of Pose (F (5, 844) = 59.83, p < 0.001) only on completion time. A post-hoc analysis using a Tukey test revealed significant differences between all the poses, except between Pose#1 and Pose#2, and between Pose#2 and Pose#3. The mean values were : None=10.84s, Wrist=11.73s, Fingers=11.18s, Both=10.54s for the Feedback, (Pose#1=8.72s, Pose#1'=6.46s, Pose#2=11.61s, Pose#2'=9.73s, Pose#3=17.43s, Pose#3'=12.47s.

A Friedman analysis on the subjective questionnaire showed no significant effect for the ease criterion. A significant effect was found for the global appreciation criterion (maxT=2.58, p = 0.0048). Post-hoc analysis revealed that the Fingers feedback (Me-dian=4) had a higher rating compared to the Wrist feedback (Median=3, p = 0.0048). A significant effect was also found for the realism and the accuracy criteria (maxT=3.46, p = 0.003) and (maxT=2.83, p = 0.03) respectively. Post-hoc analysis revealed that the Both feedback (Median=4,) was found more realistic than the None feedback (Me-dian=2, p = 0.003). For the accuracy criterion, the same significant effect was found, the Both feedback (Median=4,) being more accurate than the None feedback (Median=3, p = 0.02). The Friedman analysis revealed also a significant effect for the comfort criterion (maxT=2.78, p = 0.03). The post-hoc analysis showed that the Fingers feedback (Median=4) was found more comfortable than the Wrist feedback (Median=3, p = 0.03). Finally, a significant effect was found for the fatigue criterion (maxT=2.71, p = 0.03). Post-hoc analysis revealed that both the Wrist feedback (Median=4) and the Both feedback (Median=4) were felt as more tiring than Fingers feedback (Median=2, p = 0.03 and p = 0.02). These two types of feedback have in common a haptic feedback in the dominant hand. Boxplots of the different criteria are shown in Figure 4.9.

Concerning the perception criteria, we found a significant effect for all of them: perception of weight (maxT=4.21, p < 0.001), perception of contact (maxT=3.54, p = 0.002), perception of shape (maxT=4.11, p < 0.001). For both the perception of weight and the perception of contact, the post-hoc analysis revealed that: the Wrist feedback and the Both feedback were found as significantly better to perceive the weight

Part II

Improving Hand-based Interaction with Passive Haptics and Pseudo-haptics

Virtual Mitten

The results from this evaluation will permit to validate the following hypotheses: H1 Participants will provide more correct answers when the Progressive visual feedback is used.

H2

The mean compression will be lower for the Progressive feedback than for the Boolean feedback.

Results

The mean compression applied by participants was analyzed using a repeated measures three-way ANOVA with the factors Task, visual Feedback and grasping Threshold.

The data followed a normal distribution (Anderson-Darling test with a p < 0.05).

Regarding the post-hoc comparisons, we used Bonferroni pairwise tests adjusted for α = 95%. Only significant post-hoc comparisons are mentioned (p < 0.05). The threeway ANOVA showed a main effect for Visual Feedback (F (1, 11) = 41.1, p < 0.001, η 2 P = 0.415) and Grasping Threshold (F (1, 11) = 148.8, p < 0.001, η 2 P = 0.857). Bonferroni posthoc tests confirm the main effects, showing differences among several levels. The mean compression is significantly lower for the Progressive feedback (M = 0.648, SD = 0.091) compared to the Boolean feedback (M = 0.691, SD = 0.077). Regarding the grasping threshold, post hoc-tests also showed significant differences among the two levels r grasping (Soft) (M = 0.628, SD = 0.09) and r grasping (Hard) (M = 0.710, SD = 0.06). Figure 6.8 summarizes the mean compression for each visual feedback and each task. The percentage of correct answers for each task considering the two types of visual feedback evaluated are showed in Figure 6.9. Pairwise t-tests showed a significant difference regarding the amount of correct answers between the Progressive feedback (M = 0.85, SD = 0.19) and the Boolean feedback (M = 0.73, SD = 0.29), User evaluation (t(2) = -2.8, p < 0.01). In contrast, pairwise t-tests did not show any significant differences among interaction tasks. Regarding potential learning effects, the analysis of the evolution of the mean compression and the participants' answers did not show any correlation. Concerning the questionnaire, we performed pairwise Wilcoxon's tests for the results of each question considering the four tasks and the two types of visual feedback. Regarding Q1, two pairwise tests were significant: (1) "Pulling the drawer" vs "Pulling the Pin" (p < 0.01) and "Pulling the drawer" vs "Screwing the cylinder" (p < 0.01). Users found "Pulling the drawer" an easier task than "Pulling the Pin" and "Screwing the cylinder" (Table 6.1). The analysis for Q2, Q3 and Q4 (Table 6 The buttons required some effort to push * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 = strongly agree 1 2 3 4 5 strongly disagree strongly agree

Task

You perceived that the different buttons required different amounts of force * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 = strongly agree 1 2 3 4 5 strongly disagree strongly agree

You were confident about your answers * 1 = strongly disagree / 2 = disagree / 3 = neutral / 4 = agree / 5 = strongly agree 1 2 [START_REF] Gribnau | Comparing single-and two-handed 3D input for a 3D object assembly task[END_REF] The Virtual Mitten experiment

Thank you for your participation. During this experiment, you will be asked to manipulate various objects in a virtual environment with a device held in your hand. You will go through two tests in a random order.

In the first test, you will interact with various objects to perform specific tasks, e.g. opening a drawer (see table below). Two objects will be presented sequentially for a fixed amount of time. After manipulating both of then, answer through the designated keys on the keyboard about which one required more effort to interact with (more precise instructions will be displayed on screen depending on the current task, e.g. Whi h drawer is harder to pull? ). This sequence will be repeated twice with a different type of visual feedback each time. Is this task easy to accomplish? 1 = very difficult, 7 = very easy 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 5 6 7

Did you perceive a difference between the two objects? 1 = absolutely not, 7 clearly 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 5 6 7 Do you consider the haptic feedback realistic? 1 = absolutely not, 7 = absolutely 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 5 6 7

Did you answer with confidence? 1 = never, 7 = always 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 5 6 7

Abstract

Directly using our hands to explore virtual environments and interact with their contents produces a natural and compelling interaction. In this thesis, we propose contributions to improve hand-based interaction in the context of Virtual Reality by considering two main challenges: (1) improving the control of articulated hand models, and (2) providing haptic sensations with accessible techniques.

We first address the challenge of interacting through realistic, articulated virtual hands and propose two methods for easing their control. As a first step, we reduce the degrees of freedom of complex hand models in order to make multi-finger interaction possible with common multi-touch interfaces. The resulting system allows users to control a virtual hand by performing gestures over a tactile tablet. Then, we take another approach and separate the degrees of freedom of one virtual hand between two haptic interfaces handled in parallel. Through this distribution of controls and feedback, users are exposed to a variety of haptic effects, otherwise restricted to complex haptic workstations.

We then address the challenge of providing haptic sensations during hand-based interaction. To do so, we introduce different techniques that combine passive haptic feedback and pseudo-haptics as an alternative to complex and cumbersome active interfaces. We consider various types of interaction at different scales, starting with coarse interaction with the arm through an elastic armature that provides an egocentric and mobile haptic feedback. We then focus on object grasping and manipulation and propose an interaction paradigm that relies on elastic input devices for reproducing grasping gestures and perceiving modulable haptic properties through crossmodal feedback. Finally, we consider fine multi-finger manipulation and we propose a passive exoskeleton that constrains the digits individually, associated to a multi-finger pseudo-haptic feedback for simulating complex interaction with heterogeneous materials.