Thierry Duval
email: thierry.duval@irisa.fr

Chadi El Zammar
email: chadi.zammar@gmail.com

Managing Network Troubles while Interacting within Collaborative Virtual Environments

We are interested in real time collaborative interactions within Collaborative Virtual Environments (CVE). This domain relies on the low latency offered by high speed networks. Participants of networked collaborative virtual environments can suffer from misunderstanding weird behavior of some objects of the virtual universe, especially when low level network troubles occur during a collaborative session. Our aim is to make such a virtual world easier to understand by using some graphic visualizations (dedicated 3D metaphors) in such a way that the users become aware of these problems and can go on working even during these troubles. In this paper we present how two independent mechanisms may be coupled together for a better management and awareness of network troubles while interacting within a networked collaborative virtual environment. The first mechanism is an awareness system that visualizes, through the use of special metaphors, the existence of a network trouble as strong delay or disconnection. The second mechanism is a virtual object migration system that allows the migration of an object from one site to another to ensure a non interrupted manipulation in case of network troubles. We will detail only this awareness system and we will show how it uses the migration system to allow users to go on interacting while network breakdowns occur.

Introduction

Within a virtual environment, interacting with virtual objects means that a user can manipulate and control these objects [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF]. Furthermore, if this virtual environment is a collaborative one, an interactive object may either be controlled by one user in an exclusive way or by several users in a collaborative way. In both cases the evolutions of the shared environment are more difficult to understand than in a single-user environment because these evolutions can be caused by the other users. And of course, it is more difficult when the evolution is caused by collaborative interactions. We suppose here that these collaborative interactions can occur in real time thanks to the low latency offered by high speed networks. This is the reason why many researchers have focused on making the users aware of the other users' interactions with the objects of the shared virtual worlds [START_REF] Fraser | Supporting Awareness and Interaction through Collaborative Virtual Interfaces[END_REF] [3] [START_REF] Gutwin | Design for Individuals, Design for Groups: Tradeoffs Between Power and Workspace Awareness[END_REF]. All these studies consider that data transmission over the network is always reliable, so they never matter about the problems that could arise if there were troubles with the network. However, when network troubles happen such as the increase of the network delay or the disconnection of some sites, some problems of consistency can arise between the different instances of the shared universe. We think that we need visual metaphors to make the users aware of these inconsistencies and that we need rescue solutions to allow them to go on working anyway. These users must be aware that there is no possibility to interact with a set of objects, or that the actions of some other users are not updated any longer, so that they can go on interacting with objects of the world that are not disconnected from their site.

We will focus on networked virtual environments that connect a few number of participants who collaborate closely to achieve a specific task, for example to assemble some parts of a car mockup [START_REF] Duval | Interactions 3D coop ératives sur des objets techniques avec OpenMASK[END_REF]. With such constraints, a network perturbation or disconnection can cause damages to the remote interactions that users are performing. The solutions usually proposed in this domain do not resolve this problem in a global way, despite they try to reduce the effect of network troubles by reducing the amount of circulating updates and messages on the network. These solutions are not sufficient as long as networks are the base structure of distributed virtual environments. This is the reason why we have decided to look at this problem from a different point of view: we admit that these network troubles can arise, so we have to make all the users of a collaborative virtual environment aware of these problems. This way, users may understand and deal with some temporarily network troubles, especially when they are interacting with objects of the virtual universe. Furthermore, we want to provide a prevention method based on virtual object migration that prevents a user from losing the control of a remotely calculated object in case of a network breakdown.

Next section gives a view of related work in this domain, it is a brief glance to different approaches and metaphors that have been used in existing networked virtual environment systems. In section 3 we detail how we manage network troubles at the distributed virtual environment system level, and which visual metaphors we propose to make the users aware of these troubles. In section 4 we explain how we use a mechanism allowing virtual objects to migrate from one site to another. This migration mechanism can be useful to a user, in order to move a distant object from a distant site towards his own site, when he becomes aware of potential network troubles with the site where the object in interaction is initially located. An example of such a scenario is given in section 5. To conclude, we present what more could be done to improve our work.

Related work

Network delays and side effects

The network affects directly the performance of distributed virtual environments systems. For example when interacting remotely, a user can take the control of a virtual object and manipulate it. Low latency offers a real time interaction by minimizing the delay between user's actions and the object's responses that may be on a very distant site. If a data transmission problem arises on the network, this problem will directly affect the remote interaction. The most frequently types of problems envisaged are the delay when transmitting data over a network and some little disconnections from time to time due to dynamic rerouting systems. If the disconnection time of a site increases during a distributed virtual simulation, the consequences can be catastrophic.

A network disconnection can drive different kinds of perturbations to a shared distributed virtual reality session. Whatever the kind of the data model the virtual reality system is using (centralized, distributed, ...) we can not avoid a periodic communication between all the sites that are sharing the same virtual world, especially when users interact with objects of the shared universes. So, from the interactivity point of view, when a network disconnection occurs, the user's interactions that exist on the disconnected site are not seen any longer by the other sites. This same user is not able any longer to see the interactions of the other users and it also produces collaboration breakdowns.

When an object is evolving linearly, we can tone down the communication problem between different sites by using a prediction system that achieves local computing to estimate for example a future position of a moving object. NPSNET implements this method using the "Dead Reckoning" algorithm [START_REF] Macedonia | NPSNET: a Network Software Architecture for Large Scale Virtual Environments[END_REF]. But when we are facing a complex evolution or totally unpredictable actions like users' interactions with virtual objects (interaction may depend on user skills or mood), such prediction systems are unable to manage the situation in a virtual world, so we find ourselves powerless facing a technical challenge. Some other systems like SPIN [START_REF] Dumas | SpIn: a 3D Interface for Cooperative Work[END_REF] duplicate all universes objects and perform parallel calculations locally on each site. In this case network delay is not meaningful to a virtual session from the animation point of view, but this architecture does not resolve the main problem when objects are interactive because the possible interactions of a user can not be "duplicated", so we can not avoid the anomalies provoked by delays and disconnections. OpenMASK [START_REF] Margery | OpenMASK: Multi-Threaded or Modular Animation and Simulation Kernel or Kit : a General Introduction[END_REF] implements a predictive architecture based on the concept of referentials and mirrors, which is similar to the distribution concept in NPSNET. A referential is an independent entity that can evolve through internal state calculus. A mirror is some kind of light copy of a referential that performs no internal state evolution. The evolution of a mirror relies completely on updates received from its associated referential. The referential/mirror paradigm of the open-source OpenMASK3 architecture will be the base of our studies, so we have decided to inform users of the presence of a network problem (if any) referring to referentials and mirrors points of view. First, we signal to users who interact on sites having referentials that other users may not see modifications and interactions performed on the referential side. Second, we signal to users who interact on sites having mirrors that some of these mirrors may not have updated values.

Providing awareness of network troubles

In [START_REF] Vaghi | Coping with Inconsistency due to Network Delays in Collaborative Virtual Environments[END_REF] Vaghi et Al. have presented an experiment of a collaborative two players ball game where one player was subjected to an increasing amount of delay. They observed that as the network delay increases, the users (being aware) modify their strategies in an attempt to cope with the situation. Fraser et Al. have made a step forward in [START_REF] Fraser | Revealing the Realities of Collaborative Virtual Reality[END_REF] by giving visibility to what they have called "delay induced phenomena". They have implemented a system that estimates the maximum difference between the "objective" position of a user avatar, and where another might perceive it to be, according to network delay times between the users and speed of motion. For example, in case of network delay, an avatar is shown surrounded by a sphere that represents all uncertain possible positions, and this sphere evolves according to the network delay.

In [START_REF] El Zammar | Management and Awareness of Delay Perception when Interacting within Networked Virtual Environments[END_REF] we have presented two different methods to make network troubles visible and also to make a user aware of possible inconsistencies in the virtual environment. The first method is a marker system that marks all mirror objects existing on a particular site to make a user aware that these objects are neither interactive nor updated anymore. The other method is the creation of an echo object that is associated to each mirror object in the virtual world. This echo is visualized at the same place than the original object's place. In case of delay between the process of the referential and the process of its mirror, we can see a spatial gap between the motion of the referential and the motion of the echo associated with this mirror that shows in this case the position variation between these two processes. This last method presents some limitations when a large number of sites are participating to a simulation due to the very important number of echoes, which will be harmful to visualization.

Detection and awareness of network troubles

Whatever the kind of the data model the virtual reality system is using (centralized, distributed, ...), we can not avoid a communication between all the sites that are sharing the same virtual world, especially when users interact with the shared universes. One of the common synchronization methods in distributed systems is the use of periodic synchronization messages. This method ensures a hard real-time synchronization between all sites. Hard real-time applications require a response to events within a predetermined amount of time in order to function properly. Network delays or troubles will deny events and updates from coming in time, which will cause the breaking of the real time concept. In a distributed virtual reality simulation context, the consequence of breaking real time may be one of the two following scenarios: freeze the whole virtual world on all sites until a synchronization message shows up, which is very harmful for the session especially if the delay is important; or let the distributed virtual world goes on even in case of delay or disconnection. This second scenario will split up the virtual world into several parallel worlds, due to different users interactions on the same virtual object, which is a very bad choice. In this paper we present our work, which is a mix of the two scenarios. We choose to let the simulation continue by freezing only the parts of the world whose state is uncertain for consistency considerations. We let the virtual simulation evolve even in case of presence of non updated values due to latency or disconnection. So, the virtual world is not out of use while waiting the reception of updates as it is in hard real time synchronization based distributed systems. Even objects interactivity will be preserved but only for objects that are calculated locally. Remote calculated objects lose their interactivity as long as the disconnection remains.

Detection of network delay or disconnection

Usually, all participant sites to a virtual simulation using OpenMASK are sending each other synchronization messages. Synchronization between different sites aims to co-ordinate different processes to evolve similarly in the virtual world. Synchronization messages are used to synchronize an object with its distant mirrors and to carry the updates from a referential to its mirrors. A synchronization message may even contain only a dating element in case of no new changes in the virtual world. Once a participant site is not receiving synchronization messages from one or more sites any longer, these sites will be declared as disconnected. Each site conserves a list of disconnected sites that is not necessarily the same on each site. The synchronization message time-out threshold has to be carefully determined according to the characteristics of the network, otherwise it could downgrade the system performance, either by too frequent creation of unnecessary echoes or by detecting the troubles too late.

The awareness provider system

Our aim is to make the participants aware of any possible network trouble, this is why we have implemented, in the kernel of OpenMASK, some services that provide useful informations to the higher level applications. In [START_REF] El Zammar | Management and Awareness of Delay Perception when Interacting within Networked Virtual Environments[END_REF] we did a first step for realizing an information system provider implemented in the kernel of OpenMASK. Next we will detail the limitations of this first implementation and we will explain the solutions that we have proposed and realized to enhance our system.

The echoing system. We want to make users aware that sometimes their interactions are not seen by other users as they should be, because of network delay or disconnection. The idea is to use an echo object that represents the state of its associated real distant object. Initially, the echoing system had been implemented in a static way.

The static echoes management. On each process, for each mirror object in the virtual world, we associate an echo object that follows the motion of the mirror. The echoes association is made in a special configuration file where we specify manually which objects in the world will have echoes. The echo is a referential, it will have mirrors on the other sites as all OpenMASK simulated objects and it is visualized at the same place than the original object (e.g. the referential of the mirrors it is echoing). Physically the echo has the shape of its original object but is a little bit smaller and half-transparent so that we can not see it when a simulation is going on normally. We can associate as many echoes as we have mirrors in the virtual world. When the delay between the process of the referential and the process of its mirror is important, we may encounter some inconsistencies between the state of the two processes. For example, in case of spatial motion we can see a gap between the motion of the referential and the motion of the echo associated with its mirror (Fig. 1 (a)). In case of a disconnection, the echo associated with the mirror existing on the disconnected site is frozen on the screen and it does not evolve any longer. This means that the mirror concerned with this echo is not receiving updates any longer because of the disconnection.

The disadvantage of this method is that each referential object of a scene will have as many echoes as there are mirrors (participant sites). This will overcrowd the scene especially when moving objects while interacting, and it will increase the calculus time that the system will need in case of a huge number of users. For example, consider that N sites are participating to the same virtual simulation. If we have X referential objects within the shared universe, so we will obtain X(N-1) mirror objects distributed on the different participating sites. By applying the static echoes method, we will have as many referential echoes as we have mirrors (X(N-1) referential echoes). These referential echoes will generate a supplementary X(N-1)(N-1) mirror echoes. We can easily imagine the impact in case of an important number of sites beside the hard work of associating manually X(N-1) echo objects.

For the above reason we have decided to change the whole way of creating and managing echoes. Detailed explanation of our new method is given below.

The dynamic echoes management. A better solution to the above cited limitations is to create echoes dynamically in an automatic way, but where and when exactly do these echoes have to be created ? To answer these questions, two approaches will be presented next.

The first approach is to activate dynamically the creation of echoes after the detection of a first network perturbation, and disable the echoes as long as the distributed simulation is going on normally. This method is a clear amelioration relatively to the static creation method, but what if the first perturbation encountered during a simulation was the loss of a participating site ? In this case it is too late to create echoes because we are not able to communicate with this site any longer.

We resolve this problem by providing a new concept that avoids the creation of the referentials echoes on the disconnected site: local objects. Once a site detects the lost of another distant site, it activates locally the creation of local echo objects that appear exactly with the current states (position and orientation for example) of existing referentials. Only one simulation step time separates the physical disconnection of a site from the creation of associated echoes on other sites, so the lost of the last exact value is not really significant (in the order of few milliseconds). Echoes may also appear with the current state of some mirrors in the scene. Actually the dynamic echoes creation system detects not only referentials existing on a site but also mirrors that have "brothers" located on a disconnected site. "Brother" mirrors are mirrors associated to the same referential. This way, a user becomes aware that his interactions with a mirror are not perceived any longer by some other users.

We have implemented this last approach in the kernel of OpenMASK and we are now in the evaluation phase of this system. We could also present more informations about disconnected or delayed sites by displaying the name of the disconnected site above the associated echo, or may be by using different echo colors in case of a few number of participating sites. Each color could then represent a particular site.

The marker system. In [START_REF] El Zammar | Management and Awareness of Delay Perception when Interacting within Networked Virtual Environments[END_REF] the marker system is a very limited service that marks a specific kind of objects existing on a particular site (for example the mirror objects): we use a 3D button that launches the marker system once pressed by the user. The marker system surrounds all mirror objects (or may be another kind of object) by a half-transparent sphere that makes it clear to the user that these marked objects are not holding the last updated values (Fig. 1 (b)). The limitation of this version of the marker system is that it was not able to be more specific by designing only mirrors damaged by a delay, so it was marking all the mirrors.

We have extended this system in order to be more specific about objects that we need to mark. For example, the new marker system is able now to mark only the mirrors associated to referentials that exist on a disconnected site without marking all mirror objects. This new realization offers two main advantages. First we can give to the user a very specific idea concerning the disconnected sites. Second we do not charge the scene by undesirable marks unless the user really wants to mark all the objects belonging to a particular type. We can take also a supplementary advantage from the fact of surrounding frozen mirrors, which is the protection from the user attempts for interaction. Actually, even if the user is no longer able to interact with a disconnected mirror, all his attempts are stocked in a special PVM buffer 4 . Once the site holding this mirror is reconnected, all orders stocked in this buffer are transmitted to the referential that may be confused at this time because of some contradictory orders. So, as we surround mirrors with non-interactive objects, it prevents order attempts issued from 3D direct interaction of being transmitted to the PVM buffer.

Migration of virtual objects

As we have mentioned earlier in the introduction, beside detecting and visualizing network troubles to the users, we want to provide them a rescue technique based on virtual objects migration. Some few virtual reality systems like AVIARY [START_REF] West | AVIARY -A Generic Virtual Reality Interface for Real Applications[END_REF][START_REF] Snowdon | The AVIARY VR System: A Prototype Implementation[END_REF] and WAVES [START_REF] Kazman | Load Balancing, Latency Management and Separation of Concerns in a Distributed Virtual World[END_REF] have implemented object migration, but only to ensure load balancing.

In our case we use object migration to ensure a non-interrupted control of a specific object chosen by the user [START_REF] Duval | A Migration Mechanism to Manage Network Troubles while Interacting within Collaborative Virtual Environments[END_REF]. Let us remind the reader that, on each site, when network troubles occur, a user can only control referential objects. Mirror objects lose their interactivity as they perform no calculation and receive their updates from distant referentials. If a user is interested in keeping the control of a specific object or a set of objects, he can claim the need of these objects to the migration system that ensures that the user will own these objects locally on his site.

We have implemented the object migration system at the kernel level of Open-MASK, as detailed in [START_REF] El Zammar | Interactions coopératives distantes en environnements virtuels : gestion des problèmes réseau[END_REF]. We choose to migrate an object by changing the state of his referential and mirrors rather than recreating it on the destination site and destroying the original object on the source site. For example, to migrate an object from site1 to site2 the migration system changes the state of the mirror of the object on site2 to make a referential of it and then it changes the state of the referential on site1 to make a mirror of it. If no mirror exists on site2, the migration system will first create one. This way there is no destruction of existing objects when they migrate. If there where some other mirrors of this object on other sites, they are informed that they have a "new" referential and the OpenMASK kernel ensures that they receive now their updates from the new site of the object.

A scenario example

To illustrate the interest of coupling the migration system with the awareness system we are going to present a use case of these mechanisms. Three librarians located on distant sites are invited to arrange different sections of a virtual library. Network troubles begin to appear progressively while the librarians are performing their job. Let's see how the awareness system and the migration system will help to allow librarians to continue their job even when such network troubles occur:

1. as soon as network troubles (as delay or disconnections) begin to show up, librarians will be aware of these troubles thanks to the awareness system (marker and echoes systems). These troubles will be seen by users thanks to the appearing and disappearing of echoes and marks. 2. moreover, awareness system will provide users informations about the objects upon which they will lose control (mirror objects) in case of network disconnection: user will lose the control of all marked objects. 3. each librarian may be interested by keeping the control of all objects (books) that he must arrange in his section. This way he will not be interrupted by a strong delay or disconnection on the network. Thus, each librarian can ask the migration toward its site of all the distant books he is interested in. 4. once migration procedure is over, users may disable, if they want, the awareness system and continue working without being disturbed by network troubles. 5. when a user has arranged his section, he may move onto another one hoping that network troubles are over. If it is not the case and if troubles are only strong delays on the network, the user may migrate once again objects he wants to manipulate for a better local manipulation.

This way, our contributions will allow the users to go on working, even in case of network troubles as delays or disconnections, by focusing on tasks that may be realized by one user and does not require a collaborative work.

Of course, if these librarians have to realize some tasks that need simultaneous interactions of several users on the same object at the same time (for example moving cooperatively a desk or some shelves), these tasks can not be realized efficiently while there are some network troubles. And in case of a temporarily network breakdown, such interactions are impossible to achieve, because it would need to make the user himself migrate from one site to another before the breakdown. Of course it is possible only if the user is a virtual one (for example an autonomous agent), but it is impossible for a real user, unless he physically goes by himself to a site that is not disconnected. So, for such kind of closely coupled interactions, we can only give an advice to the users: within a shared virtual environment that can suffer from some network troubles, they should realize the cooperative tasks that need closely coupled interactions as soon as possible, and delay them when network troubles appear by doing then the work that can be realized alone.

Conclusion

We have presented our mechanisms to detect and visualize network problems while interacting within a networked virtual environment. Two kinds of metaphors are used to inform users about the availability of the updates of an object: echoes for users having referentials, and marks for those having mirrors. Those metaphors are coupled with a virtual object migration mechanism implemented within the OpenMASK platform. This migration mechanism is used to ensure a non interrupted control and manipulation of an object by migrating this object onto the site of a user who wants to interact with it, since local interactions are not sensitive to a network problem.

Our contributions, especially virtual objects migration, allow several kinds of extensions such as the dynamic management of processes and users by adding or removing sites to an already running simulation. Another possible extension is the dynamic management of areas of interest by migrating automatically a set of objects to a particular site depending on the interest of users, for example depending on the 3D position of the user, to ensure that most of the objects he can interact with will be located on his site.

There are also interesting research perspectives about how to make the users aware of the differences between referentials and mirrors. Indeed, in this article we have talked mainly about visualizing differences between geometrical parameters of our virtual objects (such as position, orientation, scale factor or color). What about other kind of parameters ? Of course, our simulation kernel is able to notice the differences between any kind of values, and if we are able to visualize any of them, we can show to a user the last known value of any of them. For example a value of pressure or temperature could be visualized with a 3D device or simply thanks to a 2D or 3D text near the virtual object. If the virtual object has also moved, the user will see at two different positions these different values, thanks to our echoing system. But what if the virtual object did not move ? We will have to propose a way to visualize both values "at the same place": the current one and the last sent to the disconnected mirror.

What would be more difficult would be to show to a user, with our marker system, the possible range of values that could have been taken by the parameter during a network breakdown. For some 3D parameters such as the position of an object, we could visualize a 3D area within which the virtual object could be (according to its last known 3D position, speed and acceleration for example) as proposed in [START_REF] Fraser | Revealing the Realities of Collaborative Virtual Reality[END_REF]. But how could we show other kind of areas of possibilities ? We can imagine 3D intervals with bounds that would evolve dynamically for real values like pressure or temperature, thanks to the record of their last known variations, but what could be used to show areas of possibilities for colors ? Or to show the current target of a tracker that would change its target randomly from time to time ? Of course, for this last example we can not propose anything, but there could be interesting fields of investigation with more predictable kinds of parameters.

Fig. 1 .

 1 Fig. 1. (a) The echoing system and (b) the Marker System

www.openmask.org

OpenMASK's distributed version rely on PVM: www.csm.ornl.gov


Thierry Duval
email: thierry.duval@irisa.fr

Chadi El Zammar
email: chadi.zammar@gmail.com

A Migration Mechanism to Manage Network Troubles while Interacting within Collaborative Virtual Environments

Keywords: H.5.1 [INFORMATION INTERFACES AND PRESENTATION]: Multimedia Information Systems-Artificial, augmented, and virtual realities; H.5.3 [INFORMA-TION INTERFACES AND PRESENTATION]: Group and Organization Interfaces-Computer- ]: Three-Dimensional Graphics and Realism-Virtual reality I.3.2 [COMPUTER GRAPHICS]: Graphics Systems-Distributed/network graphics Collaborative Virtual Environments, Computer-Supported Cooperative Work, Virtual Reality, Augmented Virtuality, Awareness of Network Problems, Object Migration

Real time collaborative interactions within CVEs rely on the low latency offered by high speed networks. When low level network troubles occur during a collaborative session, participants of networked collaborative virtual environments can suffer from misunderstanding weird behavior of some objects of the virtual universe. We want to make such a virtual world easier to understand by using some graphic visualizations in such a way that the users become aware of these problems. It is a kind of augmentation of the Virtual Environment thanks to dedicated 3D metaphors associated to some objects of the virtual environment. We also want to allow users to work while these troubles occur, maybe with only restricted interaction possibilities, by making some interactive objects migrate onto the same site than the user who wants to interact with them. This is why we present then how two independent mechanisms may be coupled together for a better management and awareness of network troubles while interacting within a networked collaborative virtual environment: an awareness system that visualizes, through special metaphors, the existence of a network trouble as strong delay or disconnection; and a virtual object migration system that allows the migration of an object from one site to another to ensure a non interrupted manipulation in case of network troubles.

Introduction

Manipulation of objects is the most fundamental tasks of 3D interaction in Virtual Reality (VR) [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF], it means that a user can manipulate and control objects. Within a collaborative virtual environment, an interactive object may either be controlled by one user in an exclusive way or by several users in a collaborative way. In both cases the evolutions of the shared environment are more difficult to understand than in a single-user environment because these evolutions can be caused by the other users. It is even more difficult when the evolution is caused by collaborative interactions. These collaborative interactions can occur in real time thanks to the low latency offered by high speed networks. This is the reason why many researchers focus on making users aware of other users' interactions with the objects of the shared virtual worlds [START_REF] Gutwin | Design for Individuals, Design for Groups: Tradeoffs Between Power and Workspace Awareness[END_REF]] [START_REF] Fraser | Supporting Awareness and Interaction through Collaborative Virtual Interfaces[END_REF][START_REF] Fraser | Revealing the Realities of Collaborative Virtual Reality[END_REF]]. These studies consider that data transmission over the network is always reliable, so they never matter about the problems that could arise if there were network troubles. However, when network troubles happen such as the increase of the network delay or the disconnection of some sites, some problems of consistency can arise between the different instances of the shared universe. We need visual metaphors to make the users aware of these inconsistencies and rescue solutions to allow them to go on working anyway. They must be aware that there is no possibility to interact with a set of objects, or that actions of some other users are not updated any more, so that they can go on interacting with objects of the world that are not disconnected from their site.

We will focus on networked virtual environments that connect a few number of participants who collaborate closely to achieve a specific task, for example to assemble some parts of a car mockup [START_REF] Duval | Interactions 3D coopératives sur des objets techniques avec OpenMASK[END_REF]]. With such constraints, a network perturbation or disconnection can cause damages to the remote interactions that users are performing. The solutions usually proposed in this domain do not resolve this problem in a global way, despite they try to reduce the effect of network troubles by reducing the amount of circulating updates and messages on the network. These solutions are not sufficient as long as networks are the base structure of distributed virtual environments. This is why we look at this problem from a different point of view: we admit that these network troubles can arise so we have to make all the users of a collaborative virtual environment aware of them. This way, users may understand and deal with some temporarily network troubles, especially while they are interacting with virtual objects. Furthermore, thanks to virtual object migration, we want to prevent a user from losing the control of a remotely calculated object in case of network problem.

Related work

Network delays and side effects

The network affects directly the performance of distributed virtual environments systems. For example when interacting remotely, a user can take the control of a virtual object and manipulate it. Low latency offers real time interaction by minimizing the delay between user's actions and distant object's responses. A data transmission problem on the network will directly affect the remote interaction. Most frequently types of problems envisaged are: delay when transmitting data over a network [START_REF] Vaghi | Coping with Inconsistency due to Network Delays in Collaborative Virtual Environments[END_REF][START_REF] Fraser | Revealing the Realities of Collaborative Virtual Reality[END_REF]; short disconnections from time to time due to dynamic rerouting systems; or temporarily breakdown of the network [START_REF] Macedonia | NPSNET: a Network Software Architecture for Large Scale Virtual Environments[END_REF]]. Consequences of the increase of the disconnection time of a site can be catastrophic because the users can not understand any longer the global behavior of the shared virtual universe [START_REF] Gutwin | Revealing Delay in Collaborative Environments[END_REF], and several studies has been done to try to manage collaborative interactions even with some network delay as detailed in [START_REF] Gutwin | Effects of Network Delay on Group Work in Shared Workspaces[END_REF]] and [START_REF] Park | Effects of Network Characteristics on Human Performance in the Collaborative Virtual Environment[END_REF]. So, from the interactivity point of view, when a network disconnection occurs, the user's interactions that exist on the disconnected site are not seen any more by the other sites. This same user is not able any more to see the interactions of the other users and it also produces collaboration breakdowns.

When we can predict an object evolution, we can tone down the communication problem by using a prediction system that achieves local computing to estimate the future status of an object. NPSNET implements this method using the "Dead Reckoning" algorithm [START_REF] Macedonia | NPSNET: a Network Software Architecture for Large Scale Virtual Environments[END_REF]. But when we are facing totally unpredictable actions like users' interactions, such prediction systems are useless. Some other systems like SPIN [START_REF] Dumas | SpIn: a 3D Interface for Cooperative Work[END_REF]] duplicate all universes objects and perform parallel calculations locally on each node. In this case network delay is not meaningful from the animation point of view, but this architecture does not resolve the main problem when objects are interactive, because users' interactions can not be "duplicated" any longer during a network breakdown. OpenMASK 1 [START_REF] Margery | OpenMASK: Multi-Threaded or Modular Animation and Simulation Kernel or Kit : a General Introduction[END_REF] implements a predictive architecture based on the concept of referentials and mirrors similar to the distribution concept in NPSNET. A referential is an independent entity that can evolve through internal state calculus. A mirror is a light copy of a referential, its evolution relies completely on updates received from its associated referential. This referential/mirror paradigm is the base of our work. [START_REF] Vaghi | Coping with Inconsistency due to Network Delays in Collaborative Virtual Environments[END_REF]] experimented a collaborative two players ball game where one player was subjected to an increasing amount of delay. They observed that as the network delay increases, the users (being aware) modify their strategies in an attempt to cope with the situation. [START_REF] Fraser | Revealing the Realities of Collaborative Virtual Reality[END_REF]] made a step forward by giving visibility to what they have called "delay induced phenomena". They have implemented a system that estimates the maximum difference between the "objective" position of a user avatar, and where another might perceive it to be, accordingly to network delay times between the users and speed of motion. For example, in case of network delay, an avatar is shown surrounded by a sphere that represents all uncertain possible positions, and this sphere evolves accordingly to the network delays. [START_REF] Gutwin | Revealing Delay in Collaborative Environments[END_REF] proposed visual metaphors called "decorators", to obtain a kind of Augmented Virtuality, in order to show either the value of the delay, or the past or the future of a virtual object. Our solutions to provide awareness of network troubles will be inspired from these studies.

Providing awareness of network troubles

Managing network troubles

One of the common synchronization methods in distributed systems is the use of periodic synchronization messages. It ensures a hard real-time synchronization between all sites and requires a response to events within a predetermined amount of time to function properly. Network delays or troubles deny events and updates from coming in time, which causes the breaking of the real time concept. In a distributed virtual reality simulation context, the consequence of breaking real time may be one of the two following scenarios: freeze the whole virtual world on all sites until a synchronization message shows up, which is very harmful for the session especially if the delay is important; or let the distributed virtual world goes on 1 www.openmask.org even in case of delay or disconnection, which will split up the virtual world to several parallel worlds due to different users interactions on the same virtual object. That is what can be obtained using a Dead-Reckoning algorithm. It can be a good choice if the users do not collaborate closely, but not if several users' interactions are closely coupled. Our work [START_REF] Duval | Managing Network Troubles while Interacting within Collaborative Virtual Environments[END_REF]] is a mix of the two scenarios: we choose to let the simulation continue by freezing up only parts of the world which state is uncertain for consistency considerations. So the virtual world is not out of use while waiting the reception of updates. Even objects interactivity will be preserved for objects that are calculated locally.

Detection of network delay or disconnection

All the processes involved in an OpenMASK shared simulation send synchronization messages to each other. It allows to coordinate the different processes so that they evolve equally in the virtual world. These messages are used to synchronize an object with his distant mirrors and to carry the updates from a referential to his mirrors. When a site has not received synchronization messages from one site for too long, this site is declared as temporarily disconnected. Each site conserves its own list of disconnected sites. The synchronization message time-out threshold has to be carefully determined according to the characteristics of the network, otherwise it could downgrade the system performance, either by too frequent creation of unnecessary echoes or by detecting the troubles too late.

The awareness provider system

To visualize differences between a referential and some of its mirrors because of network delay or disconnection, we use an echo object that represents the state of its associated real distant object: the mirror of a referential. An echo has the shape of its associated object but is a little bit smaller and half-transparent so that we can not see it when a simulation is going on normally. When the delay between the referential process and its mirror process is important, we may see a gap between the motion of the referential and the motion of the echo of its mirror (figure 1), and in case of a disconnection, the echo is frozen on the screen and does not evolve any more. This means that the mirror concerned with this echo is not receiving updates any longer because of the disconnection. We use local objects (a kind of referential that can not have mirrors) to create echoes dynamically. Once a site has detected the loss of another distant site, it enables the creation of local echo objects that appear exactly with the last known states (position and orientation for example) of existing referentials. Only one simulation step time separates the physical disconnection of a site from the creation of associated echoes on other sites, so the loss of the last exact value is not really significant. Echoes may also appear with the current state of some mirrors in the scene, since the dynamic echoes creation system detects also mirrors that have "brothers" (mirrors associated to the same referential) located on a disconnected site. This way, users become aware that their interactions with some objects are not perceived any longer by some other users.

(a) (b) Figure 1: The echoing system (a) and the marker system (b)

The marker system surrounds some mirror objects by a halftransparent sphere, which visualizes that these marked objects are not holding the last updated values (figure 1). It marks only the mirrors associated to referentials that exist on a disconnected site. This gives to the user a very specific idea concerning the disconnected sites, and does not charge the scene by undesirable marks.

Migration of virtual objects

Beside detecting and visualizing network troubles, we want to provide to users a rescue technique based on virtual objects migration. Some few virtual reality systems like AVIARY [START_REF] West | AVIARY -A Generic Virtual Reality Interface for Real Applications[END_REF] and WAVES [START_REF] Kazman | Load Balancing, Latency Management and Separation of Concerns in a Distributed Virtual World[END_REF]] have implemented object migration to ensure load balancing. Our object migration will ensure a non-interrupted control of a specific object chosen by the user, as on each site, when network breakdowns occur, a user can only control referential objects. Mirror objects lose their interactivity as their evolution depends on the reception of their updates from distant referentials. If a user wants to keep the control of a set of objects, he claims the need of these objects to the migration system that ensures that the user will own these objects locally on his site.

How do objects migrate ?

We have implemented the object migration system at the kernel level of OpenMASK, it migrates an object by changing the state of its mirror and referential, because this ensures a good continuity of the shared virtual environment. For example, to migrate an object from site1 to site2 the migration system changes the state of the mirror of the object on site2 to make a referential of it, and then it changes the state of the referential on site1 to make a mirror of it. If no mirror exists on site2, the migration system will first create one. This way there is no destruction of existing objects when they migrate. The different steps to apply in order to make an object migrate (by transformation) to another process are:

1. the computation of the object is temporarily stopped, 2. if it exists, the target mirror is unplugged from its referential, 3. this mirror is transformed into a referential, 4. all the other mirrors are unplugged from the initial referential (their processes unsubscribe to the sending of updates from the process of the initial referential), 5. these mirrors are then plugged to the new referential (their processes subscribe to the sending of updates from the process of the new referential), 6. the initial referential is transformed into a mirror, 7. this last mirror is plugged to the new referential, 8. the computation of the object is enabled.

We have studied two different possibilities allowing to switch on and off between referentials and mirrors. The first solution can be achieved by implementing the migration thanks to an internal mutation of the object. The second solution enables migration by changing the nature of the object. Referentials and mirrors have many common features. Their implementation within OpenMASK is completely transparent to users. The simulated object associated to a referential is the same than the one associated to a mirror except that only referentials are initialized and evolve: their inputs, internal parameters and published parameters can have new values at each simulation step. On the contrary, only the published parameters of a mirror will evolve, to reflect the public values of their associated referential. So, when initializing an object, a user may define a special behavior for this object, and the mirrors may not be informed of all the details of this behavior. The main difference between referentials and mirrors is their associated object manager: a referential object is an association between a simulated object and a referential manager, and a mirror object is an association between a simulated object and a mirror manager.

The first method consists in reconstructing the internal structure of referentials and mirrors so that they implement exactly the same interface. This means that a referential will contain the same functionality than a mirror, in other terms it is considered as a referential and mirror at the same time. Referential or mirror interface will be enabled or disabled accordingly to the need of the migration system. The advantage of this method is that the mutation of an object from a referential into a mirror is very easy to realize at run-time and conversely. The disadvantage, mainly because of our OpenMASK context, is the important structure modifications that may change deeply the software architecture of a simulated object. Moreover this will generate a heavy object structure that contains referential interface and mirror interface at the same time.

The second method does not change the structure neither the interface of referentials and mirrors. Migrating an object by changing its nature consists in removing the manager of an object and replacing it without destroying the object itself. For example, to transform a mirror into a referential, we only destroy its mirror manager and we replace it by a referential manager. In case a user has given some specific behavior to an object, he can migrate this behavior by redefining two extra methods emigrate() and immigrate(), which are kinds of serialization and deserialization methods that allow to determine all the important information to transmit from the old referential to the new one, within a dedicated message.

Although the first method would probably have been the more efficient one, we adopted the second method to implement object migration within OpenMASK, because the first method would have imposed too much modifications within the OpenMASK kernel.

Migration protocol

A special event is sent to start the migration of an object. It contains the name of the object and the name of the destination process that may be on a distant site. On each process, the local controller has to process this event accordingly to one of the following cases:

• the controller owns the referential of the migrating object. In this case, this referential must be transformed into a mirror by destroying its referential manager and creating a mirror manager. The controller will transmit to this mirror manager the name of the process where the referential is located.

• the controller is located at the destination process and it controls a mirror of the migrating object. In this case, this mirror must be transformed into a referential by destroying its mirror manager and creating a referential manager instead. The controller will record that from now on, it will have to send updates to all the mirrors of its new referential.

• the controller is located at the destination process but it does not have a mirror of the migrating object. In this case, a mirror should be created. Then, it will be transformed into a referential as explained in the previous case.

• the controller has a mirror copy of the migrating object without being located on the destination process. In this case, this mirror should be directed towards the new referential so that it can go on receiving updates.

As previously detailed, the migration steps must be done in one particular order, so all the local controllers will not receive these different migration events at the same simulation step. As some of the simulation steps can be parallelized, our OpenMASK implementation enables a migration in two simulation steps, which is quite a good optimization of the migration process.

Evaluation of the migration mechanism

We have just explained that the migration of an object to another process is carried out in only two OpenMASK's simulation steps, which duration is around 13ms for a 75Hz simulation. To evaluate our migration mechanism, we realized an experiment to measure its efficiency in term of visual perception, i.e. to answer this question: can a user perceive the migration of an object ?

The application we have realized for the experiment contains 100 moving objects with a 75Hz frequency. These objects are following each other, and the first object is following an arbitrary trajectory.

The objects are associated to two processes A and B distributed on two different computers. All referential objects were on process A at the beginning of the simulation, and the experiment consisted in making some objects migrate from process A to process B and to study the visual impact of the migration while increasing progressively the number of migrating objects. The measured results of this experiment show that we need around 13ms + n*0.25 ms to migrate n objects. The computers we used were Pentium IV activated at a 3 GHz frequency, connected by a gigabyte local network.

From the visual perception point of view, we can observe that the application freezes for a very short moment while the migration is on, this is true whatever the number of migrating objects, as the lower cost of migration is about 13ms. This 13 ms threshold could be reduced thanks to one particularity of our OpenMASK Kernel that allows each simulated object to have its own activation frequency. It is possible to give a higher activation frequency to the OpenMASK controllers (for example 150 Hz, 300 Hz or even 600 Hz), but it would also have its cost (as the controllers would be activated at a higher frequency, there would be an increase of computation weight) and some experiments have to be made to find what could be the optimal frequency value for a given set of computers.

Conclusion

We provide a virtual object migration mechanism, which is used to ensure a non interrupted control and manipulation of an object by migrating this object to the local site of a user, since local interactions are not sensitive to a network problem. This mechanism is coupled with a synchronization message time-out detection mechanism and two kinds of metaphors to inform users about the availability of the updates of an object (echoes and marks).

Our contributions allow users to go on working even when network troubles appear because they can become aware of the problem as soon as the OpenMASK kernel detects the increase of the delay. So they can temporarily delay closely coupled cooperative interactions, to focus upon interaction they can manage locally, may be with objects that can migrate from one process to another. They can also go on with cooperative interactions with users whose sites are not affected by the troubles. As soon as the network problems disappear, the users can come back to their previous collaborative tasks. The migration is possible only if the network is not totally down, so if a user wants to make some objects migrate because of network troubles, we hope that he could have been aware of the problem thanks to the echoing and marking systems, which is possible only if the network does not breakdown too suddenly and with a good synchronization message time-out threshold.

Virtual objects migration allows several kinds of extensions such as the dynamic management of processes and users by adding or removing sites to an already running simulation. Another possible extension is the dynamic management of areas of interest by migrating automatically a set of objects to a particular site depending on the interest of users, for example depending on the 3D position of the user, to ensure that most of the objects he can interact with are located on his site.


Thierry Duval
email: thierry.duval@irisa.fr

Alain Chauffaut
email: alain.chauffaut@irisa.fr

La cabine virtuelle d'immersion (CVI) : un mode de transport des outils d'interaction dans les univers 3D

Keywords: Navigation en univers virtuels, Interaction 3D, Immersion 3D, Environnements Coopératifs .5.1 [Information Interfaces and Presentation (e.g. HCI)] : Multimedia Information Systems -Artificial, augmented, and virtual realities ; I.3.6 [Computer Graphics] : Methodology and Techniques -Interaction techniques 3D Navigation, 3D Interaction, 3D Immersion, Collaborative Environments

Tout comme le maître de philosophie essayait d'expliquer à monsieur Jourdain qu'il faisait de la prose sans le savoir, cet article tente de poser et d'explorer un concept que de nombreux réalisateurs d'applications de réalité virtuelle ont programmé et exploité sans jamais l'expliciter clairement. La cabine virtuelle d'immersion (CVI) est un concept de navigation en immersion interactive dans une scène virtuelle 3D. La CVI définit l'espace de travail de l'opérateur dans la scène virtuelle, embarque des outils virtuels ou réels et s'adapte aux éventuels facteurs d'échelle. Nous montrons que la navigation ne se résume pas à déplacer le point de vue sur la scène mais qu'elle déplace tout un espace d'interaction, et que la notion de navigation est différente de la notion de regard.

INTRODUCTION

Cet article tente de poser et d'explorer le concept permettant, de façon générique et coordonnée, de naviguer dans un univers virtuel et d'interagir avec les objets de cet univers tout en pouvant s'adapter aux contraintes de taille de cet univers, et ce dans un contexte immersif. De nombreux réalisateurs d'applications virtuelles ont programmé et exploité la navigation et l'interaction mais sans jamais expliciter comment fédérer ces deux techniques.

Plusieurs modes d'interaction sont proposés dans la littérature, contrairement à la navigation qui se limite souvent à déplacer un point de vue. Ces différentes tâches d'interaction 3D sont fréquemment couplées, mais leurs concepteurs explicitent rarement comment ces tâches sont adaptées afin de se compléter efficacement dans un cadre donné, et comment il est possible de généraliser ces couplages pour les utiliser dans d'autres contextes. En particulier, rien n'explique comment naviguer en transportant avec soi les outils que l'on utilisera pour interagir le moment venu. Rien n'est expliqué non plus sur la façon de s'adapter à l'échelle du monde, ni sur la façon d'adapter la portée des outils physiques à l'espace virtuel d'interaction dans lequel on les utilisera. C'est pour pallier ces manques que nous proposons de formaliser les relations entre l'utilisateur, les différents outils d'interaction et l'univers virtuel, à l'aide d'une cabine virtuelle d'immersion (CVI). La CVI est un concept de navigation en immersion interactive dans une scène virtuelle 3D. Elle définit l'espace de travail de l'opérateur dans la scène virtuelle, permet d'embarquer des outils virtuels ou réels et s'adapte aux éventuels facteurs d'échelle.

ETAT DE L'ART SUR L'INTERACTION 3D

Il existe de nombreuses façons d'interagir au sein des environnements virtuels 3D, il est donc presque toujours possible de trouver une technique d'interaction adaptée à un besoin précis d'interaction. Ces techniques d'interaction ont fait l'objet de nombreuses tentatives de catégorisation, parmi lesquelles celle de Hand [START_REF] Hand | A survey of 3d interaction techniques[END_REF] qui recense trois catégories d'interactions 3D : l'interaction avec l'environnement, la navigation, et le contrôle de l'application elle-même.

L'interaction avec l'environnement

Cette technique est souvent vue comme l'association d'une tâche de sélection suivie d'une tâche de manipulation, cette dernière consiste en général à positionner et/ou orienter le ou les objets sélectionnés. Bowman et al. [START_REF] Bowman | 3D User Interfaces, Therory and Practice[END_REF] distinguent le cas où l'objet avec lequel on veut interagir est hors de portée de l'utilisateur, et où il faut donc utiliser une technique de sélection permettant de désigner l'objet à distance, par exemple à l'aide d'un rayon 3D virtuel [START_REF] Bowman | An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments[END_REF]. Ces techniques de sélection à distance (télé-manipulation) s'avèrent peu efficace pour le positionnement si la distance entre le dispositif d'interaction et l'objet manipulé est importante. L'objet sélectionné ainsi doit donc être généralement manipulé d'une autre façon, par exemple en l'amenant près de l'utilisateur, dans son espace de travail, à l'aide d'une canne à pêche virtuelle [START_REF] Bowman | An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments[END_REF] pour le manipuler localement (par exemple pour l'orienter) puis pour le renvoyer ensuite au loin. Les manipulations locales peuvent alors se faire à l'aide de techniques plus classiques telles que celle de la main virtuelle [START_REF] Poupyrev | The Go-Go Interaction Technique: Non-linear Mapping for Direct Manipulation in VR[END_REF] ou du curseur 3D [START_REF] Zhai | The "Silk Cursor": Investigating Transparency for 3D Target Acquisition[END_REF], éventuellement avec de la colocalisation.

La navigation

Cette technique consiste à modifier le point de vue offert sur le monde pour le percevoir différemment. La navigation est une interaction avec un objet particulier de l'univers : le point de vue de l'utilisateur, c'est probablement l'interaction la plus utilisée dans les univers virtuels. Parfois même, on rencontre des cas où la navigation est quasiment la seule interaction proposée, comme pour les courses de voitures par exemple. Mais la plupart du temps, la navigation est une tâche préliminaire, et néanmoins très importante, permettant de se mettre dans une position qui permettra une interaction satisfaisante avec l'univers. Il y a de nombreuses façons de naviguer, qui peuvent s'appuyer sur l'expression par l'utilisateur d'une direction, d'une vitesse, d'une façon de démarrer et de s'arrêter. Plusieurs taxonomies des moyens de navigation sont présentées dans le chapitre 6 de [START_REF] Bowman | 3D User Interfaces, Therory and Practice[END_REF].

Le contr ôle de l'application

Cette technique permet de modifier indirectement (via des interfaces adaptées) certaines propriétés du monde virtuel exploré par l'utilisateur. C'est un peu le parallèle en 3D des widgets offerts par les IHM 2D classiques. Il s'agit ici d'utiliser des métaphores adaptées permettant de modifier le paramétrage des applications 3D, comme offrir un mode de sélection d'un outil d'interaction grâce à un menu 3D comme celui présenté dans [START_REF] Grosjean | Evaluation of the Command and Control Cube[END_REF]. On peut aussi trouver de nouvelles métaphores 3D comme la palette virtuelle [START_REF] Coquillart | The Virtual Palette and the Virtual Remote Control Panel: A Device and Interaction Paradigm for the Responsive Workbench[END_REF] qui permettent de lister et déclencher, de façon immersive, des actions globales à l'application. Encore une fois, Bowman et al. présentent de nombreux exemples de contrôle d'application dans [START_REF] Bowman | 3D User Interfaces, Therory and Practice[END_REF].

Comment adapter et fusionner ces techniques ?

Chaque exemple de technique de chacune de ces trois catégories a généralement été implémenté et validé, mais dans un contexte applicatif bien défini plutôt que dans des situations diverses et variées. Non seulement les auteurs de ces techniques ne font pas ressortir les éléments génériques de ces techniques, mais de plus l'implémentation qu'ils en ont faite dans un contexte particulier ne les amène pas à formaliser ces techniques dans un but de réutilisation maximale.

De tels exemples ne manquent pas, nous pensons à des techniques développées à l'intention de dispositifs de visualisation particuliers (workbench) qu'il faut modifier pour pouvoir être utilisées dans d'autres contextes, car elles ont été prévues pour être placées près du centre de l'univers où l'on a déposé les objets à étudier grâce au workbench, et avec lesquelles on n'a pas prévu de pouvoir se déplacer. Les acquisitions de positions et d'orientations via des capteurs 3D se font donc généralement en absolu (à une étape de calibration près) et les outils pilotés voyagent alors assez difficilement.

Nous pensons également aux problèmes de changements d'échelles, statiques et/ou dynamiques : comment réutiliser des techniques de navigation et/ou d'interaction, prévues pour être utilisées dans un certain contexte, dans un nouveau contexte où les objets à manipuler sont d'une taille très différente (beaucoup plus petits ou beaucoup plus grands) ? En effet bien souvent ces techniques utilisent des outils dont la taille a été déterminée en fonction de la taille des objets à manipuler, ou bien les objets à manipuler ont été mis à l'échelle des outils.

Enfin, il peut y avoir des problèmes d'adaptation des zones de déplacement physique de l'utilisateur lorsque l'on veut utiliser un périphérique d'entrée à rayon d'action différent de celui pour lequel la technique a été développée, ou encore lorsque l'on veut changer d'environnement physique de visualisation : on n'a pas les mêmes possibilités de déplacement selon que l'on se trouve assis devant un écran standard, debout devant un workbench, ou encore immergé dans un reality center ou bien dans un cave.

On peut donc avoir des difficultés à interfacer ces différentes catégories de techniques entre elles, ou même tout simplement à les réutiliser dans un nouveau contexte applicatif ou impliquant des périphériques d'entrée ou de sortie différents. C'est pour ces raisons que nous proposons une approche plus générique de la navigation immersive interactive facilitant le transport des outils d'interaction 3D et leur adaptation à différentes échelles.

LA CABINE VIRTUELLE D'IMMERSION (CVI)

Le but de la Cabine Virtuelle d'Immersion est de faciliter la définition et l'utilisation de techniques d'interaction 3D, tout en permettant leur usage dans des contextes variés, que ce soit au niveau applicatif ou encore pour des raisons d'utilisation de périphériques physiques particuliers. Nous avons implémenté un premier prototype de CVI sous forme de modules réutilisables au sein de l'environnement de développement et d'exécution d'applications 3D coopératives OpenMASK1 . Les concepts fondamentaux de cette CVI visent à pouvoir maîtriser l'espace de travail de l'utilisateur en lui permettant :

• de naviguer dans l'environnement, quelle que soit sa taille, avec des fonctionnalités de navigation les plus riches possibles, • d'emporter ses outils (virtuels et réels) avec soi, et de pouvoir les manipuler dans le référentiel de l'utilisateur (c'est-à-dire le référentiel attaché à la cabine) et éventuellement dans d'autres référentiels (le référentiel absolu de l'univers, celui d'un objet manipulé, ou encore celui d'un autre utilisateur),

• de s'adapter dynamiquement à des changements d'échelle afin de pouvoir explorer différents aspects de l'univers (pouvoir accéder à des niveaux microscopiques ou macroscopiques de l'univers), • d'intégrer les différents espaces physiques de travail liés à la liberté de déplacement offerte par les périphériques d'entrée ou de sortie.

Le champ d'action naturel de l'utilisateur est la CVI : il peut se déplacer à l'intérieur de la CVI, mais de cet endroit il ne pourra pas tout atteindre, il devra donc pouvoir déplacer la cabine ou utiliser des outils de plus longue portée comme présenté dans l'état de l'art. En effet, on a vu que le champ d'action de certains outils n'est pas limité à la cabine virtuelle : il existe des outils à longue portée comme les rayons virtuels, les curseurs "téléopérés", les bras extensibles, à condition de voir au delà de la CVI.

Naviguer avec la CVI

Nous avons défini la notion de cabine qui offre à l'utilisateur la possibilité d'aller en n'importe quel lieu de l'univers en transportant avec lui tout son espace de travail (dont ses outils d'interaction). Nous proposons les deux types de déplacements les plus classiques : les déplacements incrémentaux relatifs au repère de l'utilisateur (de type : avancer, reculer, tourner à gauche, tourner à droite, etc. ; le déplacement "générique" d'un delta (6D) indiqué) ; et le déplacement absolu en une position de l'univers virtuel. Ces deux types de déplacement ne sont pas les seuls possibles car nous avons aussi réalisé des outils permettant de tourner autour d'un objet, ou d'agir de l'extérieur sur une CVI pour la déplacer ou pour la faire regarder dans une direction particulière. Notre cabine diffuse de façon publique sa position et son orientation, ce qui permet aux outils de l'utilisateur de se positionner relativement au repère de la cabine plutôt que par rapport au centre de l'univers : il seront alors attachés à la cabine, se déplaceront avec elle dans l'univers virtuel, et resteront accessibles à tout moment par l'utilisateur présent dans la cabine. Ces outils pourront servir à l'interaction 3D directe avec les objets de l'univers (cf vue gauche de la figure 1), ou bien à la détermination de certains paramètres à l'aide de widgets 3D dédiés permettant par exemple de déterminer l'apparence d'un objet (cf vue droite de la figure 1).

S'adapter à l' échelle de l'univers virtuel

La CVI est définie à l'échelle de l'opérateur, c'està-dire celle du monde réel : c'est le seul moyen de pouvoir réaliser, si nécessaire, des colocalisations entre les outils réels et leur représentation dans la scène virtuelle. Il faut ensuite plonger la CVI dans l'univers virtuel. En modifiant son facteur d'échelle, on peut alors facilement lui donner la taille qui lui permettra de s'insérer correctement dans cet univers. Ce facteur d'échelle est un attribut public de la CVI, de même que sa position et son orientation. Les outils embarqués vont donc pouvoir eux aussi hériter de l'échelle de la CVI, et l'appliquer à la fois à leur géométrie et à leur position relativement à la cabine.

(a1) (a2) (b1) (b2) (c1) (c2)

Int égrer les espaces physiques de travail

Ici les problèmes qui se posent sont :

• faire correspondre la cabine virtuelle avec les contraintes des dispositifs de visualisation (workbench, reality center, cave),

•

Figure 1 .

 1 Figure 1. Outils embarqués : à gauche pour la manipulation de maquettes virtuelles, à droite pour définir la couleur d'un objet

Figure 2 .

 2 Figure 2. changements d'échelle de la CVI verte qui grandit lors des étapes a, b et c. Nous avons choisi d'adapter la cabine à l'échelle du monde virtuel plutôt que l'inverse car cela offre l'avantage de ne pas modifier l'univers virtuel dans sa globalité, ce qui est primordial pour des univers virtuels multi-utilisateurs, dans lesquels on pourra trouver une CVI par utilisateur. Cela permet également à plusieurs utilisateurs d'interagir dans un même univers virtuel simultanément et à des échelles différentes. Un exemple d'adaptation dynamique de

  PlusieursCVI peuvent cohabiter dans un même univers virtuel partagé, et de plus elles peuvent avoir des échelles différentes, ce qui est particulièrement intéressant pour un travail coopératif multi-échelles. Enfin, la CVI nous a permis de formaliser la différence entre la notion de navigation et la notion de regard en montrant que la navigation doit se faire en emmenant son espace de travail.A priori, les changements d'échelle, tout comme les changements de position et d'orientation de la CVI, peuvent avoir lieu en cours d'interaction avec des objets virtuels. Les outils virtuels de la CVI exercent alors une action sur les objets en interaction. Dans le cas d'une interaction haptique où l'objet en interaction "résisterait", la sensation perçue serait équivalente à un mouvement inverse de cet objet, et cela pourrait provoquer en retour un mouvement réel d'un périphérique physique à retour d'effort.Cette première implémentation de CVI intègre une partie des dispositifs d'interaction déjà disponibles dans la plate-forme OpenMASK depuis 2001. Cette CVI doit maintenant être évaluée afin de nous permettre de finaliser ce concept pour élaborer une boîte à outils plus complète. Une telle boîte à outils permettra de décharger les programmeurs des problèmes d'intégration et de transport des outils d'interaction 3D dans les univers virtuels.

	prendre en compte les contraintes des dispositifs
	d'interaction comme des bras à retour d'effort aux
	amplitudes limitées, des capteurs magnétiques ou
	ultrasoniques, . . .
	Une règle de conception de la CVI est que les out-
	ils virtuels placés dans la CVI doivent être à l'échelle
	de l'utilisateur et des outils physiques. Nous pou-
	vons éventuellement y faire apparaître la notion
	de frontières (pas forcément visualisées) qui car-
	actérisent certaines des limites de la cabine virtuelle. Dans le cas du workbench, du reality center ou	BIBLIOGRAPHIE
	du cave, dans lesquels l'utilisateur est immergé, ces
	frontières pourront représenter les limites physiques
	des écrans de projection aux travers desquels l'uti-
	lisateur ne peut pas passer. Elles délimitent ainsi
	l'espace à partir duquel l'utilisateur manipule ses out-
	ils d'interaction. Cette notion peut aussi être étendue
	à la portée des outils d'interaction : la frontière d'un
	capteur magnétique serait alors une demi-sphère.
	Dans le cas d'un utilisateur d'une simple station de
	travail la notion de cabine virtuelle reste tout à fait
	pertinente, même si c'est un cas simplifié où la nav-
	igation et le regard se confondent (un seul objectif
	et en général pas de suivi du positionnement de la
	tête), car elle permet d'harmoniser la gestion des out-
	ils d'interaction par raport à l'utilisateur. Contraire-
	ment à ce qui se passe dans un cave, où l'on peut voir
	la totalité de la CVI, ici la partie visible de la CVI
	se limite à son intersection avec la pyramide de vue.
	Le positionnement virtuel des périphériques d'entrée
	doit ici encore se faire relativement au centre de la
	cabine (ici le point de vue) mais rien n'empêche de
	les placer ailleurs dans le monde réel car il n'y a pas
	de colocalisation envisagée dans ce contexte.
	CONCLUSION
	La Cabine Virtuelle Immersive (CVI) est notre so-
	lution pour gérer de manière cohérente et simplifiée
	les outils virtuels d'interaction 3D et la navigation

de l'utilisateur dans l'univers virtuel : elle permet de naviguer en embarquant ses outils. La CVI permet également de s'adapter dynamiquement à l'échelle de l'univers en appliquant des facteurs d'échelle à la fois au champ de vision et aux outils embarqués. De plus, la CVI permet d'intégrer l'espace physique de travail, c'est-à-dire la configuration matérielle des dispositifs de visualisation et d'interaction utilisés.

www.openmask.org


Thierry Duval
email: thierry.duval@irisa.fr

Cédric Fleury
email: cedric.fleury@irisa.fr

Bernard Nouailhas
email: bernard.nouailhas@edf.fr

Laurent Aguerreche
email: laurent.aguerreche@irisa.fr

Collaborative Exploration of 3D Scientific Data

Keywords: CR Categories: H.5.1 [Information Interfaces and Presentation (e.g. HCI)]: Multimedia Information Systems-Artificial, augmented, and virtual realities; I.3.6 [Computer Graphics]: Methodology and Techniques-Interaction techniques 3D Interaction, Collaborative Virtual Environments, 3D Scientific Visualization

This demonstration introduces new ways for exploring Collabora-Virtual Environments (CVE) that contain 3D scientific data sets obtained by simulation. In order to make decisions accordingly to their collective knowledge and understanding of the simulation, the users must collaborate and share experiences and comments. We provide tools to enable a good coordination between the users, and to make each user aware of the activity of others. Each user can navigate within the CVE: change her own position, orientation and scale. Each user can also add annotations within the virtual universe. We propose several 3D layouts for the presentation of the data, associated with different 3D navigation tools. Consequently, the user can explore the data accoording to various parameters such as time or temperature. Last we propose a new 3D interaction tool, called 2D Cursor / 3D Pointer, dedicated to selection and manipulation of 3D objects, and application control. This 2D cursor is associated with a 3D geometry in order to make people aware of the activity of the users who are using this tool.

Context and goals

We want to use Virtual Reality techniques for collaborative visualization of 3D scientific data sets, to propose new ways to explore Collaborative Virtual Environments. Our goal is to allow several experts to examine scientific data sets. These experts must collaborate in order to make decisions thanks to the sharing of their knowledge, experience and understanding of the simulation. The data sets used for this demonstration are 3D temperature isosurfaces or 3D temperature axial slice cuts, given at several simulation steps, and timestamped from the beginning of the simulation to a time when the simulation does not evolve any longer.

Exploration

Awareness of other users

People sharing a CVE need to be aware of the activity of other users [START_REF] Fraser | Supporting Awareness and Interaction through Collaborative Virtual Interfaces[END_REF], in order to help them to understand the evolution of the CVE and to collaborate more efficiently with the other users. Showing the activity of the other users is a crucial factor for an efficient collaboration. This is why each user of our CVE is associated with a 3D avatar: a worker with a colored helmet (for the snapshots, the users are also wearing a real colored helmet). In the same way, we visualize the tools each user is manipulating for 3D interaction within the CVE.

Navigation and coordination

We allow each user to navigate within the CVE: each user can change her position, her orientation and her scale. We provide tools to allow a good coordination between the users, and to make each user aware of the activity of other users: a user can watch other users work, a user can also join another user, ask for help, or grab another user to show her something. As described in [START_REF] Mine | Moving objects in space: exploiting proprioception in virtual-environment interaction[END_REF]] and [START_REF] Duval | La cabine virtuelle d'immersion (cvi): un mode de transport des outils d'interaction dans les univers 3d[END_REF], the interaction tools are used within the frame of the user (her position, orientation and scale), they navigate and change scale with her. Each user owns a 3D map of the world, which is a scalable World In Miniature [START_REF] Stoakley | Virtual reality on a wim: Interactive worlds in miniature[END_REF] that can be useful for navigation, and within which the user can see the evolution of the virtual world, including the navigation and interactions of other users.

3D layouts for 3D scientific data sets

We propose several 3D layouts for the presentation of the scientific data sets, allowing to merge all the data at the same point or dispatching the data all over the 3D virtual world. Each kind of layout is associated with dedicated 3D widgets allowing to explore the data sets either according to a temporal dimension, or to a temperature dimension, or to a geometrical dimension (for the data axial slice cuts along the X, Y and Z axis).

When the data sets are merged in a multidimensional "switch", the navigation tools can be implemented for example as 3D sliders allowing to change the value of a parameter (time, temperature, X, Y or value). In this case, all the users share the same view of the merged data sets: the currently selected data set.

When the data sets are dispatched all over the world, the navigation tools can be some landmarks associated with each data, leading to another data linked to it according to the variation of a parameter (time, temperature, X, Y or Z value). In the latter case, each user can navigate freely and independently from other users all over the world. This is the situation where coordination tools are the most useful when some users want to meet somewhere.

3D data annotation

Each user can create annotations, global to the world or associated with a particular data, in order to share information with other users. These annotations can be used by other users when they explore the world to see what is interesting in some point of the universe. The annotations are currently implemented as interactive viewpoints: a user can switch from a viewpoint to another one, choosing either to explore only the viewpoints created by a particular user or to explore all viewpoints.

A new tool for collaborative interaction

The "3D Interaction Techniques for 3D Manipulation" chapter of [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF]] presents a lot of metaphors dedicated to 3D manipulations. One of them, the ray-casting, is very interesting because it is simple and efficient, especially when used at close range. We have adapted this metaphor in order to drive it with a device providing only a 2D position, like a classical 2D mouse, a gamepad or a Nintendo wiimote remote controller.

Description of the 2D Cursor / 3D Pointer

The 3D geometry of this cursor is a virtual ray, so other users are made aware of the moves of this 3D pointer in the same way they are made aware of the evolution of classical 3D interaction metaphors. The user only controls the 2D position of the closest ray end, and we compute the orientation of the ray so that its projection on the screen is always a point. This way, the user always feels that she is using a 2D Cursor while other users will see a 3D virtual ray (figure 2). So it is quite easy to use for the user, and quite easy to understand for the other users.

Adequation of this tool to 3D interaction

This tool is efficient for 3D object selection (for 3D annotation) and for viewpoint selection (for navigation) or for data sets exploration according to some parameters (by selecting a dedicated landmark). It is also efficient for 3D manipulation like moving a 3D slider cursor, here again for data sets exploration. As the tool moves with the viewpoint when the user navigates, like in 3DM [START_REF] Butterworth | 3dm: a three dimensional modeler using a headmounted display[END_REF]], an object that has been grabbed by this tool navigates also within the VE, which is another way to provide a new position and orientation to this object. Last, this tool can simply be used as a classical 2D cursor to trigger some elements of a 3D GUI carried by the user, in order to control the state of the application.

Hand [START_REF] Hand | A survey of 3d interaction techniques[END_REF]] separates virtual interactions into 3 categories: 3D interaction (selection and manipulation), navigation and appli-cation control. Here we see that this new tool, carried by the user, is well suited for these three kinds of interactions.

Figure 2: On the left we see the "Blue" user's viewpoint: "Blue" user is ready to select a 3D slider with her blue 2D Cursor, she sees the green 3D Pointer of "Green" user ready to select another slider. On the right we see the "Green" user's viewpoint, which is quite symmetric to "Blue" user's viewpoint.

Acknowledgments

work has been realized in a joint of the ANR N • 06 TLOG 031 Part@ge project and of the ANR N • 06 TLOG 029 SCOS project.


M Hirose

D Schmalstieg

C A Wingrave

Editors K Nishimura

Laurent Aguerreche

Thierry Duval

Anatole Lécuyer

Joint Virtual Reality Conference of EGVE -ICAT -EuroVR (2009)

Keywords: (according to ACM CCS): H.5.1 [Information Interfaces and Presentation (e.g. HCI)]: Multimedia Information Systems-Artificial, augmented, and virtual realities; I.3.6 [Computer Graphics]: Methodology and Techniques-Interaction techniques

In this paper we introduce a new 3D interaction technique called "3-Hand Manipulation", for multi-user collaborative manipulation of 3D objects. The 3-Hand Manipulation relies on the use of three manipulation points that can be used simultaneously by three different "hands" of two or three users. Interestingly, the three translation motions of the manipulation points can fully determine the resulting 6 degrees of freedom motion of the manipulated object. We describe the implementation of the 3-Hand Manipulation, its graphical representations and an illustration of its use by two or three users on an insertion task in immersive virtual environment. This technique could be used in various applications of collaborative VR such as for virtual prototyping, training simulations, assembly and maintenance simulations.

Introduction and Related Work

Object manipulation is one of the most fundamental tasks of 3D interaction in Virtual Reality (VR) [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF]. Due to new 3D input devices becoming widely available even for the general public, research in new 3D user interfaces is more relevant than ever [BCF * 08]. Furthermore, the collaborative manipulation of virtual objects by multiple users is a very promising area for Collaborative Virtual Environments (CVE) [START_REF] Benford | Collaborative Virtual Environments[END_REF]. Collaborative manipulation of objects seems indeed necessary in many different applications of VR such as virtual prototyping, training simulations or assembly and maintenance simulations [START_REF] Ruddle R | Symmetric and Asymmetric Action Integration during Cooperative Object Manipulation in Virtual Environments[END_REF]. In such virtual collaborative tasks, all the users should participate naturally and efficiently to the motion applied to the object manipulated in the VE.

Although most collaborative systems support simultaneous manipulation of different objects by different users, generally only one user at a time can manipulate a virtual object. Interaction metaphors that are usually used for single-user 3D interaction, such as virtual hands, virtual rays or virtual 3D cursors, have to be adapted for collaborative 3D virtual manipulations.

Two-Hand Object Manipulation

Some 3D interaction techniques have been proposed in the field of manipulation of virtual objects with the two hands of a single user [START_REF] Hinckley | Two-handed Virtual Manipulation[END_REF], but only a few of them such as "grab-and-carry", "grab-and-twirl" and "trackball" techniques [START_REF] Cutler | Two-Handed Direct Manipulation on the Responsive Workbench[END_REF] allow users to position and rotate at the same time virtual objects.

The "grab-and-carry" technique [CFH97] is a 5 Degrees Of Freedom (DOF) bimanual symmetric tool that allows users to carry and turn an object around with both hands. Object roll lacks since it is not possible to determine rotation around the axis defined by positions of the two hands. The "grab-and-twirl" technique extends the "graband-carry" technique, adding the sixth DOF using either the left hand's roll, the right hand's roll, or a combination of both. The "trackball" technique is a bimanual asymmetric tool that allows users to use the non-dominant hand to position a virtual object while the dominant hand rotates this object around its center. actions. Second, it is likely that they would not be used to simulate real interactions with large or cumbersome objects a user cannot manipulate alone.

Multi-User Object Manipulation

Several approaches are suitable to combine two users' movements to obtain virtual object's final movements [START_REF] Ruddle R | Symmetric and Asymmetric Action Integration during Cooperative Object Manipulation in Virtual Environments[END_REF]. A first approach consists in adding the two motions (asymmetric integration of movements). A second approach is to average the two motions. A third approach aims at keeping only the common part (intersection) of the two motions (symmetric integration of movements). But none of these combinations is ideal. Indeed, the intersection technique is the more relevant when the two users have to perform a similar action, whilst the average technique is preferred when users have to perform different tasks.

The Bent Pick Ray [START_REF] Riege | The Bent Pick Ray: An Extended Pointing Technique for Multi-User Interaction[END_REF] metaphor allows several users to co-manipulate simultaneously a virtual object. This technique merges users' inputs according to the amount of hand movement a user does with her input device. Rotations are calculated with a spherical linear interpolation, while the translations are interpolated using only offset transformations, in order to move the object incrementally. Results may be close to those of the average technique.

The SkeweR technique lets multiple users simultaneously grab any part of a virtual object [START_REF] Duval | SkeweR: a 3D Interaction Technique for 2-User Collaborative Manipulation of Objects in Virtual Environments[END_REF]. To determine the grabbed object's translation and rotation, SkeweR considers positions of those "crushing points". A problem remains for determining the rotation along the axis determined by the two crushing points, and even with more clues, these points cannot be used in a very natural or realistic way. A similar technique seems to be used to construct a virtual gazebo [START_REF] Roberts | Constructing a Gazebo: Supporting Teamwork in a Tightly Coupled, Distributed Task in Virtual Reality[END_REF]. Two users manipulate a beam by grabbing its extremities but no solution is proposed for the sixth DOF.

Another kind of collaborative manipulation consists in splitting the DOF task among users [START_REF] Pinho | Cooperative Object Manipulation in Immersive Virtual Environments: Framework and Techniques[END_REF]. In this case, number of DOF that each user can access and control is limited: one user controls object's rotations while the other one controls object's translations. This can be compared to the Two-Hand "trackball" technique.

Conclusion

Separate motions of several users' inputs (from several hands or users) can be used to define the final motion of a virtual object. However, due to the complexity of current VR interfaces, no universal collaborative solution has already been proposed to naturally apply a motion to a co-manipulated object. Therefore, we propose to extend these techniques by adding a third manipulation point for the collaborative manipulation.

The 3-Hand Manipulation Technique

Concept

We propose a new 3D interaction technique for 6 DOF multiuser collaborative manipulation of 3D objects. Our technique enables the determination of virtual object position and orientation through only positions of three non-aligned manipulation points on the surface of this object. These manipulation points can be used naturally, in a realistic way, by three different "hands" of two or three users.

Manipulation and Graphical Feedback

Hands are represented by pointers. When a hand is close enough to the object to manipulate, ray-casting from the hand gives an intersection point with the object. This point is called a manipulation point. A manipulation point differs from a SkeweR "crushing point" and should rather be considered as a hand position. If a user starts a manipulation, a virtual ball is added to display the location of the manipulation point. In addition, a rubber band is added between the virtual ball and the hand to avoid any ambiguity concerning its owner and to display the distance between the hand and the manipulation point (Figure 3). A screenshot of the virtual c The Eurographics Association 2009. environment with three hands manipulating a hood is given in Figure 2.

The rubber band drawn between a hand and its manipulation point is elastic and its color varies according to the distance between the hand and the manipulation point. The rubber band uses a green-yellow-red code: the farther a hand is from its associated manipulation point, the more the rubber band becomes redder. With such a feedback, users' hands are expected to remain close to their manipulation point to avoid instabilities during the manipulation.

Computation of Manipulated Object's Motion

The manipulated object motion can be computed in different ways using input motions of the three users' hands. One solution consists in making the manipulation points stay as close as possible to the hands. At the beginning of the manipulation, hands positions H 1 , H 2 , H 3 correspond to positions of their contact points P 1 , P 2 , P 3 with the manipulated object. These contact points are the manipulation points and are illustrated in Figure 4.

When users move their hands to H ′

1 , H ′ 2 , H ′ 3 , the T transto apply to the initial position Pc of the manipulated object is computed as follows:

H 0 = H 1 + H 2 + H 3 3 ; H ′ 0 = H ′ 1 + H ′ 2 + H ′ 3 3 ; T = ---→ H 0 H ′ 0 .
To compute the rotation difference with the initial plane orientation, the first step consists in computing the rotation ( R 1 , α 1 ) that transforms i into i ′ (see Figure 5): This rotation transforms j into an intermediate vector j ′′ :

i = ---→ H 0 H 2 ; j = ---→ H 0 H 3 ; i ′ = ---→ H ′ 0 H ′ 2 ; j ′ = ---→ H ′ 0 H ′ 3 R 1 = i × i ′ ; α 1 = arccos ( i • i ′ i * i ′ ).
j ′′ = ( R 1 , α 1 ) * j.
The second step consists in computing the rotation ( i ′ , α 2 ) that transforms j ′′ into j ′ . After constructing a new 3D orthogonal basis ( u, v, i ′ ), this angle is calculated as follows:

k ′′ = ( j ′′ • u) * u + ( j ′′ • v) * v k ′ = ( j ′ • u) * u + ( j ′ • v) * v α 2 = arccos ( k ′′ • k ′ k ′′ * k ′ ).
The difference between the current and the initial orientation of the object is then obtained by combining these two rotations: ( i ′ , α 2 ) * ( R 1 , α 1 ).

c The Eurographics Association 2009.

If the first triangle, defined by the three hands' positions, and the second triangle, defined by the three initial positions of the manipulation points, do not keep the same shape then the roll angle (rotation around the axis orthogonal to these triangles) is the best possible approximation, otherwise this roll angle value is exact.

Another solution for implementation is to use three "point-to-point" constraints of a physics engine like Bullet [Bul] or PhysX [Phy]. A constraint is dynamically added between a hand and a manipulation point. Here, if the triangles do not keep the same shape, there can be some small inconsistencies for the roll angle.

In both cases, the use of a colored rubber band can help users to keep their hands to the manipulation points.

Implementation

The 3-Hand Manipulation technique was implemented in a virtual reality center involving ARTracking markers [ART]. The five ART infrared cameras placed around a large screen were tracking hand positions in 3D space. Users were located in front of this large screen.

Two or three people could manipulate simultaneously a virtual hood to place it on a support. This hood had holes that users had to align with the support. This task is inspired by an assembly task faced in industry.

Figure 6: Three users manipulating a virtual hood with the 3-Hand Manipulation technique.

Physics and collisions in the virtual environments were implemented using the Bullet physics engine. Hands were manipulating objects through Bullet constraints.

First observations. During the manipulation, we observed that people needed to communicate a lot: to start or end the action, and to decide where to move the hood. All users found the 3-Hand Manipulation technique natural. During a 2-user manipulation, one user had notably the impression that "he was moving his hands in the air as he would have done with a real object".

General Conclusion and Perspectives

We have presented a new 3D interaction technique for multiuser collaborative manipulation of virtual objects called "3-Hand Manipulation". This technique relies on the use of three manipulation points that can be used simultaneously by three different "hands" of two or three users. The three translation motions of the manipulation points fully determine the resulting 6 DOF motion of the manipulated object.

The considered task consisted in assembling a virtual hood on a support. Different configurations were tested with two or three users. First user feedback suggests that the technique is suitable for collaborative manipulation, and further work is now necessary to evaluate and compare our approach to other existing techniques involving many users. This work was supported by the grant ANR 06-TLOG-031 in the context of the Part@ge project.

  These techniques seem to have two main restrictions. First, they are not very representative of real world interc The Eurographics Association 2009.

Figure 1 :

 1 Figure 1: 3-Hand Manipulation of a virtual hood by two users.

Figure 2 :

 2 Figure 2: Screenshot of a 3-Hand Manipulation of a virtual hood.

Figure 3 :

 3 Figure 3: A rubber band between a pointer and a manipulation point.

Figure 4 :

 4 Figure 4: Computation of virtual object's motion ( T ) given the three hand's motions.

Figure 5 :

 5 Figure 5: Rotation ( R 1 , α 1 ) and rotation ( i ′ , α 2 ).

c The Eurographics Association 2009.