Tijmen Klein

Florimond Guéniat

Luc Pastur

Frédéric Vernier

Tobias Isenberg

A Design Study of Direct-Touch Interaction for Exploratory 3D Scientific Visualization

Keywords: (according to ACM CCS): I.3.m [Computer Graphics]: Miscellaneous-Scientific visualization, 3D interaction, direct-touch interaction, large displays, flow visualization

We present an interaction design study of several non-overlapping direct-touch interaction widgets, postures, and bi-manual techniques to support the needs of scientists who are exploring a dataset. The final interaction design supports navigation/zoom, cutting plane interaction, a drilling exploration, the placement of seed particles in 3D space, and the exploration of temporal data evolution. To ground our design, we conducted a requirements analysis and used a participatory design approach throughout development. We chose simulations in the field of fluid mechanics as our example domain and, in the paper, discuss our choice of techniques, their adaptation to our target domain, and discuss how they facilitate the necessary combination of visualization control and data exploration. We evaluated our resulting interactive data exploration system with seven fluid mechanics experts and report on their qualitative feedback. While we use flow visualization as our application domain, the developed techniques were designed with generalizability in mind and we discuss several implications of our work on further development of direct-touch data exploration techniques for scientific visualization in general.

Introduction

Due to its many benefits (e. g., [START_REF]The Importance of the Sense of Touch in Virtual and Real Environments[END_REF][START_REF] Hornecker | Collaboration and Interference: Awareness with Mice or Touch Input[END_REF][START_REF] Agrawala | Determining the Benefits of Direct-Touch, Bimanual, and Multifinger Input on a Multitouch Workstation[END_REF]) and the increasing availability of supporting display hardware, direct-touch interaction rightfully receives a lot of attention in many domains. In particular in the field of human-computer interaction, researchers develop and evaluate novel interaction techniques for multi-touch surfaces. In the field of visualization interest has largely been concentrated in information visualization and visual analytics (e. g., [FHD09, IF09, NDL * 09, TIC09]). Scientific visualization, however, has many specific constraints [START_REF] Isenberg | Position Paper: Touch Interaction in Scientific Visualization[END_REF]. It focuses in many cases on spatially explicit three-dimensional data for which the mapping from 2D touch input to 3D manipulations is not obvious. Moreover, scientific visualization often requires multiple different exploration techniques for the same type of dataset. For example, one may need to navigate, place cutting planes, seed particles, and probe data values. These interactions often even have to share the same interaction space on the screen, which can be difficult to achieve with any single 3D interaction technique developed for manipulating 3D objects. It is thus imperative to investigate interaction techniques specifically tailored to the needs of scientific visualization [START_REF] Keefe | Integrating Visualization and Interaction Research to Improve Scientific Workflows[END_REF].

To address the mentioned challenges, we present a design study of an encompassing direct-touch exploration system in our example domain of fluid flow visualization (see the resulting interface in Fig. 1). We tackle the challenges of combining general 7 DOF navigation, 3 DOF cutting plane interaction, additional 2 DOF drilling exploration, the use of both volumetric visualization and iso-surface specification, the placement of seed particles in 3D space (5 DOF or more), and the exploration of temporal data aspects. Several of the included interaction techniques share the same 2D input space by employing dedicated postures [START_REF] Isenberg | Gestures vs. Postures: 'Gestural' Touch Interaction in 3D Environments[END_REF] or by making use of bi-manual interaction. In addition, we report on observations from interaction sessions with a representative number of domain experts, with consistent results. We chose this study methodology because qualitative observations are particularly appropriate to provide holistic insight into complex processes such as exploratory visualization [START_REF] Carpendale | Evaluating Information Visualizations[END_REF][START_REF] Greenberg | Usability Evaluation Considered Harmful (Some of the Time)[END_REF].

One contribution of our work is thus an encompassing direct-touch system for exploring data in the domain of fluid flow visualization. More importantly, however, we contribute a better understanding of how different 3D touchbased interaction techniques can be integrated with each other and the insight that different touch-based interaction techniques (3D and 2D ones) can and need to be combined in a single visualization system. Moreover, we show that touchbased visualization can hugely benefit from its support of direct manipulation as well as from the fluidity of interaction. Finally, our observations indicate that collaborative visualization can not only be facilitated by multi-touch settings on large displays but that it is also desired by experts, even when using vertical display configurations. The results of our work inform the creation of direct-touch interfaces for other domains since many of our supported exploration techniques are core to scientific visualization in general.

The remainder of the paper is structured as follows. We start by discussing related work in Section 2. Next, we describe our participatory design process in Section 3 and explain our design decisions in detail. In Section 4 we describe our qualitative evaluation with five domain experts and describe and discuss our observations. We conclude the paper in Section 5 and mention possibilities for future work.

Related Work

Work related to our own can be found in two major fields: existing solutions that employ direct-touch interaction in the context of scientific visualization and interaction with 3D data or objects using direct-touch input in general.

Direct-Touch Interaction with 3D Data or Objects

In this latter group, several approaches have been published. While a more complete list of approaches can be found elsewhere (e. g., [START_REF] Isenberg | Position Paper: Touch Interaction in Scientific Visualization[END_REF]), here we only mention a few selected techniques that are particularly relevant to our own work. Generally, touch-based interaction (3D manipulation/3D navigation) depends on the nature of the controlled objects. For example, for manipulating individual and medium-sized objects, techniques such as Hancock et al.'s three-finger [START_REF] Hancock | Shallow-Depth 3D Interaction: Design and Evaluation of One-, Two-and Three-Touch Techniques[END_REF] or sticky tools [START_REF] Hancock | Sticky Tools: Full 6DOF Force-Based Interaction for Multi-Touch Tables[END_REF] can be employed, or a set of dedicated gestures as in Eden [KMB * 11].

Using an constrained energy minimization approach, Reisman et al. [START_REF] Reisman | A Screen-Space Formulation for 2D and 3D Direct Manipulation[END_REF] facilitate intuitive rotate-scale-translate (RST) interaction-well suited, in particular, for manipulating planar surfaces in 3D space. In all these cases, however, objects that can be touched and which constrain the interaction are necessary. If these are not available, e. g., in the case of particle clouds or volumetric data, navigation of the displayed space by means of widgets such as Yu et al.'s FI3D technique [YSI * 10] or Cohe et al.'s tBox [START_REF] Cohé | tBox: A 3D Transformation Widget Designed for Touch-Screens[END_REF] can be used, as well as techniques to control the camera [START_REF] Hachet | Navidget for Easy 3D Camera Positioning from 2D Inputs[END_REF].

Direct-Touch Interaction in Scientific Visualization

Despite these general techniques, some research has also specifically addressed touch-based interaction with scientific visualization. In addition to some 2D techniques [START_REF] Forlines | DTLens: Multi-User Tabletop Spatial Data Exploration[END_REF][START_REF] Isenberg | Interactive Exploratory Visualization of 2D Vector Fields[END_REF], we are specifically interested in those approaches that support interaction with 3D scientific data. Here we can find two general approaches: those that develop techniques for a general type of data and those which develop or integrate techniques for a specific application domain. An examples for the former group is the previously mentioned FI3D technique [YSI * 10] that facilitates interaction with the visualization space, for example for particle cloud data. A second example is Coffey et al.'s Slice WIM [CML * 11] that supports the exploration of datasets in a virtual reality context, using touch interaction with 2D projections of a stereoscopically projected miniature version of the dataset (i. e., avoiding the conflict between touch input and stereoscopic projection) to control specific objects and the visualization in general. Finally, Fu et al.'s powers-of-10 ladder [START_REF] Fu C.-W | Multi-Touch Techniques for Exploring Large-Scale 3D Astrophysical Simulations[END_REF] facilitates the interaction with a dataset's scale rather than space and is applicable for most 3D visualizations.

Other work investigates very specific visualization domains. For example, Sultanum et al. [SSS * 10, SSSS11] describe techniques to support the exploration of potential oil deposits. They employ, beyond 'standard' touch-based navigation, posture-based techniques including dedicated local "probing," axis-aligned "cutting," sub-object distortion ("peeling"), and (assisted by tangibles) focus+context views that remove parts of the dataset that lie between the focal point and the camera. In an application for orthopedic surgery planning, Lundström et al. [LRF * 11] developed a multi-touch table setup that provides-due to their specific application domain-a constrained set of 6 DOF general 3D navigation techniques. They combine these techniques with specific exploration tools including "movable alternator pucks" to specify exploration modalities and "natural size zoom" for maintaining similarity to the clinical reality. Finally, Butkiewicz and Ware [START_REF] Butkiewicz | Exploratory Analysis of Ocean Flow Models with Stereoscopic Multi-Touch[END_REF] describe an application for stereoscopic oceanographic flow visualization to analyze ocean currents. Unique about their setup is that the dataset is relatively shallow and that it is projected using a tilted configuration on a similarly tilted 3D display. This makes the interaction with the stereoscopic data intuitive because there is an obvious touch surface (the ocean's surface) which is simi-larly arranged in physical space (due to the tilted display) as in virtual space (due to the tilted projection). Based on this setup, they also employ precise 3D positioning techniques using a two-finger pantograph selection.

Both the domain-specific setups and also the data-driven approaches pay attention to the ability to combine different interaction techniques in such a way that they can be employed in the same interaction space, an aspect that is also essential in our approach. Mode switching is often enabled using postures, while Sultanum et al. [START_REF] Sultanum | Point it, Split it, Peel it, View it": Techniques for Interactive Reservoir Visualization on Tabletops[END_REF] also use tangibles and Lundström et al. [LRF * 11] virtual objects for the same purpose. Some techniques essential in visualizationbeyond our 'normal' navigation and zooming [YSI * 10]have been studied before. For example, cutting [START_REF] Sultanum | Point it, Split it, Peel it, View it": Techniques for Interactive Reservoir Visualization on Tabletops[END_REF] and cutting planes [CML * 11], probing [START_REF] Sultanum | Point it, Split it, Peel it, View it": Techniques for Interactive Reservoir Visualization on Tabletops[END_REF], particle placement [START_REF] Butkiewicz | Exploratory Analysis of Ocean Flow Models with Stereoscopic Multi-Touch[END_REF], and combinations of 2D and 3D interaction [CML * 11] have previously been discussed individually. Our application domain of fluid mechanics, however, like many other visualization fields requires all of these and other techniques to be accessible, while some (e. g., cutting planes) also need to be controllable with more DOF. We address this problem with our design described next.

Multi-Touch Interaction with 3D Visualizations

Based on this understanding of the related work, we set out to design a tool for exploratory visualization that facilitates more than a simple 3D data navigation. We started with a detailed analysis of the requirements of our chosen application domain, flow visualization, but most apply to 3D scientific visualization in general. The resulting constraints were derived from several conversations with two fluid mechanics experts (co-authors of this paper) at our research lab to understand their current work practices as well as to extract needs for a new touch-driven interface. Based on this input (Section 3.1), we followed a participatory design process [START_REF]Participatory Design: Principles and Practices[END_REF] in which we closely involved the two experts in the new system's development. During the five-month development, we met with them once every one to two weeks for a total of approximately 15 meetings. We describe this process in Sections 3.2-3.6, and outline how the requirements as well as other decisions influenced the interaction design process.

Flow Visualization Interaction Requirements

From initial interactions with two fluid mechanics experts we derived a list of requirements to guide our development. Additional requirements stem from our own past experience developing direct-touch interactions and scientific visualizations. We now describe how requirements R1-R11 were derived and list a summary at the end of this section.

Data and Temporal Exploration

The specific application case we used for our design (data provided by the experts) is the exploration of time-dependent (R1) fluid flow simulations that consist of a scalar 3D1C field and a vector 3D3C field (R2) [START_REF] Haller | Distinguished Material Surfaces and Coherent Structures in Three-Dimensional Fluid Flows[END_REF]. While the vectors encode the flow's direction and velocity, the scalar field (Fig. 2) is a Finite Time Lyapunov Exponent (FTLE) field that represents the rate of divergence of neighboring material particles. The peaks of such a FTLE field represent Lagrangian Coherent Structures (LCS) which act as a material frontier in the flow. LCS frontiers separate areas of particles with different behavior because the particles cannot cross the frontiers.

Scalar Field Exploration

Visualizations based on these scalar fields provide information, for example, about how the mixing in a physical flow happens-a problem relevant in practical applications. Other kinds of important scalar fields in mechanics such as temperature or concentration could also easily be used in the application through trivial modifications, as would any vectors fields such as vorticity or the magnetic field. As means for exploring the scalar field, the experts required both iso-surface adjustment and a drilling interaction (R3, R8).

Exploration Using Cutting Planes

Our collaborating fluid mechanics experts reported that they traditionally explore their simulations using twodimensional cuts through a three-dimensional dataset, for example to analyze the flow behavior in and around cavities. Therefore, they requested the ability to use 2D visualization techniques (R5) in the new interface, in addition to 3D interaction (R4). This 2D visualization should be supported through freely positionable cutting planes (R6)-this positioning should be free in 3D space, axis-aligned cutting planes are not sufficient (R6).

Vector Data Exploration

Moreover, it needs to be possible to also explore the vector aspects of the dataset. A way to achieve this traditionally is to seed the 3D3C field at any given timestep and to look at the interaction between the particles and the field such as by following their trajectories. This interaction should also be possible for the new 3D visualization (R1, R7). For this purpose it needs to be possible to flexibly define regions of various sizes in the 3D space where particles are placed and from which streamlines emerge (R7).

Control and Modes

From our own experience with touch-based scientific visualization and related work in HCI, we chose a design which focused on interaction techniques which avoid semaphoric gestures (R9) to aid memory of supported interactions. The needed modalities should be specified in the form of spring-loaded, user-controlled modes [Bux86, SKB92] that let people actively invoke certain interactions without risking modal errors. For example, one can use bi-manual interaction, hand postures [START_REF] Isenberg | Gestures vs. Postures: 'Gestural' Touch Interaction in 3D Environments[END_REF], and non-obtrusive widgets. These widgets, however, should never cover the center of the exploration (R10) to always be able to focus on the data.

Hardware Setup

Our physical display setup was inspired by our own past experiences as well as by setups described in the literature, specifically Lundström et al.'s Medical Visualization Table (MVT) [LRF * 11]. Similar to the MVT, we based out design on a 1920 × 1080 tiltable display, but used a larger one (55 inch diagonal) to support more than one person comfortably (R11). In contrast to the MVT, our setup (Fig. 8) is heightadjustable and is primarily used in a vertical or close-tovertical orientation, guided by the preferences of the fluid mechanics experts. The multi-touch input is provided by a PQLabs Multi-Touch G 3 Plus overlay which provides the touch events via a TUIO [START_REF] Kaltenbrunner | TUIO: A Protocol for Table-Top Tangible User Interfaces[END_REF] interface.

In summary, we used the following eleven requirements to guide the development process: R1: Temporal exploration. R2: Visualization and exploration of a combination of vector 3D3C and scalar 3D1C data, using volumetric and iso-surface modalities for the scalar field. R3: Creation, adjustment, and removal of iso-surfaces. R4: 3D navigation and zoom. R5: 2D visualization and interaction. R6: Cutting planes: free placement, orientation, translation. R7: Definition of seed regions in 3D space from which streamlines emerge to show the vector field. R8: Drilling to explore the scalar values along a 3D line. R9: Using the interface's visual appearance to indicate function, i. e. use of bi-manual, postural, and widgetbased methods. R10: The data not obstructed by interaction widgets, touches on the data limited to essential ones. R11: The physical setup of the touch-based visualization supporting at least two people comfortably and flexibly configurable and adjustable.

Data Visualization and Navigation

Before we commenced with the project we decided on a scientific visualization environment on which to base the development. Our collaborators were most familiar with VTK For the interactive exploration of the 3D dataset at least 7 degrees of freedom (DOF) need to be supported: translation along three axes, orientation/rotation with respect to three axes, and uniform zoom (R4). To realize this interaction with direct-touch input several options exist as outlined in the related work. The fundamental decision we needed to make was whether to treat the data volume as a dedicated object that can be used to constrain the interaction or whether we rather wanted to support interaction with the data space instead. Techniques that focus on 3D interaction with smaller objects (e. g., [CDH11,HCC07,HtCC09]) were less suitable for our purpose because we wanted to support zooming into the dataset which leads to the data volume becoming large. Therefore, based on the same motivation as presented by Yu et al. [YSI * 10] for their interaction with particle data, we chose their FI3D widget as our most basic interaction paradigm. FI3D also only uses at maximum two simultaneous touch points and, thus, configurations with more than two fingers can be mapped to additional interactions. Specifically, we chose their second interaction mappingone finger for rotation, perpendicular-to-frame interaction for translation-because in our case rotations were expected to be much more common interactions than translations, and since in most cases the whole dataset (with an inherent center) would be shown. As in the initial FI3D technique, the frame interaction was combined with a two-finger RST interaction [START_REF] Reisman | A Screen-Space Formulation for 2D and 3D Direct Manipulation[END_REF] to support intuitive integrated rotation-scaletranslate interactions.

To show both the volumetric and the iso-surface visualizations at the same time (R2), we split the display into two main regions, each with an FI3D widget to control the regular 3D navigation (Fig. 3). Both views can be used independently or in a connected manner in which the view (projection settings) of one widget is mirrored in the other widget, while the selection of visualization techniques (volume rendering, iso-surfaces, both, or none of them) can be changed independently. In addition, the right side of the interface contains a view of the Jet color scale in which iso-values (represented as dots) can be added (by tapping on an empty spot), adjusted (by dragging a dot up or down), or removed (by dragging a dot out of the scale) to satisfy R3. of the dots produces a new iso-surface rendering in real-time.

Cutting Plane & Drilling Interaction

In addition to the regular 3D interaction with the dataset we needed to also support the manipulation of cutting planes as a fundamental exploration technique (R6). This means had to integrate the 3DOF cutting plane interaction with 7DOF interaction with the dataset/data space-ideally both being invokable in the same interaction space. With the FI3D widget in use for the latter, three or more finger techniques can be used for reorienting or translating cutting planes. This makes it possible to employ, e. g., Reisman et al.'s screenspace technique [START_REF] Reisman | A Screen-Space Formulation for 2D and 3D Direct Manipulation[END_REF] for touch interaction with planes. For our specific application, we used Reisman et al.'s approach as an inspiration-in an abstracted form. We wanted to enable precise control and, using a similar interaction, be able to translate the cutting plane along its normal. We decided to use a bi-manual technique and to specify an axis on the cutting plane using two fingers of the non-dominant hand and use the motion of a third finger to specify an angular rotation around this axis (Fig. 4). The symmetry of the chosen Figure 5: Drilling interaction (left) with bi-manual control; the 'drilling core' with its iso-values is shown both in the 3D view as well as in an undistorted way on its left side.

bi-manual technique makes it always possible for the user to choose the non-dominant hand to specify the axis. This configuration by itself can still invoke RST manipulations if the fingers are moved. To enable translation, we use the same two-finger posture to specify the interaction with the cutting plane, but use one of the FI3D widget's frames to specify a translation. Here, the frame is used as an auxiliary control.

To minimize obstruction of the data during both interactions (R10) the non-dominant hand can be released while the previously enabled interaction is continued as long as the third finger maintains contact. Our experts initially asked for nonplanar cutting surfaces to be integrated. Together with the experts, however, we began with an interaction design for planar ones to restrict interaction complexity. Non-planar cutting planes would require the specification of arbitrary cutting surfaces and an interaction paradigm of its own. We also implemented a drilling operation (R8) that is invoked bi-manually using a dedicated activation region on the left (non-dominant) frame of the FI3D widget. The dominant hand, in this interaction, specifies a point on the cutting plane, perpendicular to which a 'drilling core' is shown which displays the scalar values sampled along its path (Fig. 5). This is shown both as the 'drilling core' in the 3D view and as a 2D representation on the left side of the 3D view. Initially we had implemented a drilling interaction along the view ray defined by the dominant hand's touch, but after seeing it in action the experts suggested our current approach with the core perpendicular to the cutting plane.

2D Visualization and Seed Point Interaction

Based on the traditional use of 2D visualizations by our collaborating experts we also wanted to support twodimensional data exploration strategies (R5). We based these techniques on the cutting planes already specified in the interface, but added an additional 2D view that shows the undistorted intersection of the currently defined cutting plane (Fig. 6). In this 2D cut view we implemented techniques to place, in particular, seeds to be able to integrate streamlines in the 3D view (R7) as an essential and fundamental way to explore the characteristics of the dataset's vector field (R2), such as investigating the divergence of the particle trajectories when searching for LCS' in our specific application. The combination of 3D cutting plane specification and 2D seeding allowed us to solve the otherwise difficult problem of specifying a seeding region in 3D space, despite only being able to interact on a 2D surface.

By extending previous 2D, posture-based particle placement techniques [START_REF] Isenberg | Interactive Exploratory Visualization of 2D Vector Fields[END_REF], we decided to employ onefinger, two-finger, and three-or-more-finger techniques to define regions in which to place seed particles that are used to integrate streamlines. The specific design of these techniques, however, was heavily guided by discussions with the fluid mechanics experts. For the one-finger interaction we first considered to place a single seed, but based on the feedback we received we later decided to place seeds in a small spherical region centered around the touch point to be able to see minute changes of the vector field in a small region. Similarly, for two fingers we initially only placed one seed each for the touch points, but based on feedback changed this to place seeds along the line connecting the two touch points. Three or more fingers can be used to define a larger sphere whose size can be adjusted. Initially we used this larger sphere to only seed on its surface, but based on feedback decided to seed evenly inside its volume.

In all three cases the experts finally reported that they require control over how densely seeds are placed, so we added such a control by means of using the 2D view's frame as an auxiliary slider control: dragging a finger of the nondominant hand along this frame (depending on the dragging direction) increases or decreases the number of seeds (Fig. 7). This techniques lets people transition, for example, between our initially envisioned single seed per touch point in the one-and two-finger interactions to a dense seeding pattern. Moreover, we also change from 'normal' streamlines to a ribbon representation (which require more space) when only few seed points are employed because these provide more information about the dataset's characteristics. The orientation of these ribbons is derived locally from vorticity of the flow vectors. To provide even more information during the seeding exploration we also provide the numeric value of the scalar field for single-finger seeding interaction. Finally, as a seeding technique related to the two-finger technique, we also generate seed points along the drilling core when the drilling interaction is being used.

Temporal Exploration

We provide typical timeline-based interaction to support exploration of the dataset's temporal behavior with a timeline placed at the bottom of the interface (Fig. 3). The time step displayed in the interface can be controlled by dragging the timeline's time cursor. Playing the whole simulation is started by flip-dragging the cursor or briefly tapping it, and stopping by dragging the cursor out of the timeline. The temporal interaction takes the previously mentioned streamline seeding into account for both the posture-based and the drilling-based seed points. We store their position in space and time and based on this information show their motion over time. Finally, when other interactions are being performed, we stop a possibly running temporal animation due to performance reasons, and continue the animation when the interactions are completed.

Support for Co-located Collaboration

While we did not intentionally develop our interface to support it, the presence of two configurable views and the support of multi-touch interaction with up to 32 independent points facilitates the simultaneous collaborative interaction by at least two people. In particular, we allowed each of the two FI3D-based widgets to be switched independently between 3D view and 2D cut view, and in a 3D view the type of visualization (volumetric and iso-surfaces) can independently be enabled or disabled. Moreover, if both widgets show the 3D visualization, their views can be used independently or in a coordinated fashion. In the latter case, navigation interactions are replicated in the respective opposite widget so that both always show the same view. work, groups of two people can work independently, each with their own view by switching between 2D-and 3D-based exploration. Alternatively, they can also choose closely coupled work by using the whole interface in a joint fashion (e. g., Fig. 8), for example one person controlling seed point placement in the 2D view while the other manipulates the 3D view on the dataset. While we did receive some insight on such work practices in our evaluation session on which we report next, more dedicated design for collaborative use of the visualizations would be necessary.

This setup facilitates a range of different individual or collaborative exploration scenarios. Aside from individual

Evaluation and Feedback

In addition to the feedback we received during the participatory design process we also wanted to get a more thorough understanding of the interface toward the end of our development. For this purpose we asked our collaborating experts as well as five additional fluid mechanics researchers (who were previously not involved) to participate in longer observational evaluation sessions. We specifically chose a qualitative, observational study methodology [START_REF] Carpendale | Evaluating Information Visualizations[END_REF] because we were interested in studying the integration of several directtouch interaction techniques for exploratory visualizationnot in whether one specific technique outperforms another one which could be studied with a quantitative controlled experiment. After all, from a single technique's performance we cannot learn how people approach a whole system and how they use the techniques in combination, in particular when studying a complex process such as exploratory visualization [START_REF] Carpendale | Evaluating Information Visualizations[END_REF][START_REF] Greenberg | Usability Evaluation Considered Harmful (Some of the Time)[END_REF]. Moreover, specific performance and feedback benefits of direct-touch interaction techniques have been established in the past [START_REF]The Importance of the Sense of Touch in Virtual and Real Environments[END_REF][START_REF] Agrawala | Determining the Benefits of Direct-Touch, Bimanual, and Multifinger Input on a Multitouch Workstation[END_REF].

We thus split the seven participants (6 male, 1 female, ages 23-57 years, median 39 years) into four groups, three groups with two participants (G1, G2, G4) and one person by himself (G3). One group (G1) comprised the two researchers who also participated in the design process and who are coauthors of this paper, thus we specifically point out and separate their feedback if it differs from the other three groups.

A complete evaluation session took about 1.5-2 hours and all sessions followed the same procedure. After having been informed about the topic of the evaluation, participants were first given a tutorial of the visualization and interaction tech-niques using a simple dataset. Participants were allowed to try out the interactions themselves as well as to ask questions. Next, a larger dataset (used throughout this paper) was loaded and participants were asked to explore it, to look for interesting or unexpected aspects, or to confirm aspects they were familiar with. All participants were familiar with the used data type, with groups G1 & G3 and one person in G2 being most familiar with the specific data, while group G4 and the other person in G2 were familiar with similar fluid simulations. During this exploration phase, participants were asked to think-aloud, could ask questions to the experimenter, and were sometimes asked about their actions or were reminded of functionality (to reduce learning effects). Afterward, we conducted a semi-structured interview to discuss their experience in more detail, including differences between their traditional analysis process and the touch-based one and specific feedback about the direct-touch exploration tool. Finally, we asked the participants to fill out a questionnaire to inquire about demographics, background, and numeric ratings for their opinions about specific aspects (using 5-point Likert scales). Results were captured using videotaping as well as note-taking by the experimenter.

The findings from this expert evaluation can roughly be categorized into four groups: usability issues, interaction concepts, extensions of the tool, and implications for the design of other direct-touch scientific data exploration settings.

Usability Issues

We naturally found a number of smaller usability issues with the current implementation of our interface. These include, e. g., some inconsistencies with the implementation of the timeline interaction, the lack of remembering the view settings when toggling between 2D and 3D views, the lack of indication of the streamlines' direction, etc. These are all problems that can easily be addressed through small changes in the application. We also found usability issues which stemmed from the tested interaction design. These, in particular, relate to the interplay of 3D navigation and interaction with cutting planes in the same interaction space (i. e., interaction specified with postures). While all participants in G1 & G4 were able to deal with the interplay of techniques, one person in G2 and the person in G3 mentioned that it may require some learning to be used effectively. The other person in G2, however, found the interaction design for moving the cutting plane "unintuitive" and evidently had trouble to obtain the view settings and cutting plane configurations he desired. The observation that both experts in G1 were better able to use it than the other participants seems to further indicate that learning may help people better use the technique. The cutting plane orientation problem was, nevertheless, considered to be the most severe usability issue by all.

Interaction Concepts

The challenges of cutting plane interaction were indicative of more fundamental requirements of touch interaction with three-dimensional scientific visualizations: the need for pre-cise interaction techniques and the need to isolate interactions. The fact that initiating a cutting plane interaction always led to small changes in the view led to some visible frustration during interaction. One person suggested to use explicit system-controlled modes which he was familiar with in his traditional tools. However, problems with moded interfaces are well known and we believe it to be possible to find interaction techniques with improved usability that do not rely on system-controlled modes. Related to this interaction isolation issue, participants mentioned several times that they need to be able to achieve specific configurations of the cutting plane-such as planes parallel to the sides of the data box-to be able to make precise measurements. Nevertheless, the interface as a whole was not considered to be imprecise as may be assumed from the 'fat finger problem.' Reasons for this reaction that were mentioned by the participants include that the specific type of data had a lower resolution than the size of pixels on the screen, that other interactions (e. g., streamline exploration) were considered to be precise, and that the supported exploratory visualization was mentioned to have a more "qualitative" character in which pixel precision was less important.

Extensions

Despite the above mentioned usability challenges, participants viewed the application as very useful for exploratory visualization and asked for a number of additional features to make the tool even more useful in practice. Numerous ideas were mentioned and we, thus, highlight a selection of features participants asked for-some of them easy to realize while others are more challenging. For example, participants asked the ability to apply transformations to the 2D view, not only panning and zooming but also flipping to reduce mental mapping problems if the 3D view and its cutting plane are rotated by more than 90°around some axis. Also, related to the issue of precise interaction, participants asked for default views that are frequently used in exploration and which would allow them to easily return to a well-known configuration. Similarly, to ease navigation, participants asked for more specific information about the displayed data in form of detailed numeric read-outs, axis labels, small 3D (and projected 2D) coordinate system axes, etc. To be able to explore the vector properties more easily, the experts suggested placing small vectors (rather than streamlines) on the cutting plane or along a line (like our drilling core), placing several instances of streamlines to be able to compare cyclic behavior, and specifying locations that continuously emit particles. Naturally, they also requested the ability to toggle between several different scalar properties as well as to use iso-surfaces with a transparency. A very interesting suggestion, yet more difficult to realize, is to capture a history of interactions that would allow researchers to reproduce the exploration, to get images of specific encountered views or configurations, or to run similar explorations with other data.

Implications for Design

Finally, we learned from our experts about direct-touch inter-action with 3D scientific data in general. In particular, three main aspects were mentioned to be the most important advantages of the presented interaction combined with directtouch input: direct manipulation and fluidity of interaction, the combination of 2D with 3D exploration, and the support of collaboration. Participants mentioned that the first of these aspects-the ability to directly manipulate the data, the rapid access to exploration facilities, the fluidity of the interaction, and the immediate feedback-was essential for their effective exploration of the dataset. One person mentioned that the latency both from the touch processing and the demanding rendering of the visualization may start to become an issue if it became greater, but that the feedback that was provided helped to alleviate the problem. Compared to their traditional tools, the experts also named as an advantage that it was not necessary to build dedicated visualization pipelines. For touch-based scientific visualization we can thus learn that the ability to directly manipulate and the interaction fluidity are essential and should be maintained.

However, as we approach more realistic visualization settings, the interfaces will necessarily become more complex. This means that touch-based interaction cannot always remain completely intuitive but in some aspects may have to be learned. In our observations and in the experts' responses we saw that this fact is accepted by the people. They mentioned that interactions can be learned by exploring the interface while exploring the data at the same time, one hence needs to provide an explorable and exploratory interface.

The second aspect concerns the combination of 2D and 3D exploration techniques. It was evident that providing 2D exploration tools that are inter-connected with the 3D visualization is essential, in particular since several of the experts traditionally use 2D exploration tools for fluid flow research. While the placement of seed points by means of a cutting plane may be less direct than with, e. g., 3D tracking in VR, the technique at least allowed a reasonably flexible 3D seeding of streamlines with control over the shape and characteristics of the seed region. This worked well for 5 of the 7 participants (2 × 'liked it' and 3 × 'definitely liked it'), the remaining 2 rated it 'neutral.' In fact, one of the participants (G3) reported a previous experience of fluid flow visualization in a VR environment using a Phantom for specifying seed point placement directly in 3D space. He stated that he liked the direct-touch interaction much better than the VR setting and that he wants to use our tool in the future to explore his own data. In this context he also mentioned that stereoscopic displays should be investigated but that he is not sure that it would be better than a monoscopic one.

The final aspect relates to the ability to collaboratively exploring a visualization with an interface based on directtouch control. In our (interaction and evaluation) setting we specifically supported and observed the collaboration between two colleagues discussing a common problem. This type of collaboration was liked by G1, G2, and G4 (working together worked 'well' 3 × and 'definitely well' 3 ×) and G3 mentioned that he would really want to collaborate with others using the tool (without being asked about this fact specifically). All mentioned that 2-3 people would be best, with 2 people being the ideal configuration. The experts specifically mentioned the issue of awareness of the other's actions being a benefit, that "collaboration came naturally," and that they "complemented each other" in their work. However, G1, G2, and G4 exhibited very different collaboration styles. G1 closely collaborated, at many times worked simultaneously, and did not notice any interference while doing that. G2, in contrast, were explicit about taking turns and mentioned that usability issues kept them from working at the same time due to fears that an additional person interacting would cause (more) problems with the work of the first person. G4's style was somewhat in-between that of G1 and G2, with G4's participants taking turns without being explicit about it but discussing while one person was interacting. This preference for collaboration using our vertical display setting to some degree contradicts Rogers and Lindley's findings [START_REF] Lindley | Collaborating Around Vertical and Horizontal Large Interactive Displays: Which Way Is Best?[END_REF] that vertical displays are "difficult and awkward to collaborate around." We hypothesize that this is due to the specific type of data we used: 3D data that is thought to have an inherent orientation (with an up and a down). We specifically asked our participants after the session whether they would have preferred a horizontal setup (and demonstrated that the display can be turned), and all reported that the vertical or a slightly tilted setting would have been best. Of course, we cannot derive statistically significant findings from these few observations and a dedicated study would need to investigate the suitability of both horizontal and vertical settings for 3D data analysis, but we saw indications that vertical or slightly tilted setups are not as bad for scientific visualization (for small groups of 2-3 people, e. g., discussions of colleagues) as some literature [RL04] suggests.

Conclusion and Future Work

We presented a design study for supporting exploratory visualization on a direct-touch platform, using fluid mechanics as our example domain. We combined several interaction techniques that permit both the use of different exploration techniques in the same interaction space (the 3D view) as well as the combination of 3D and 2D exploration techniques. As part of both, we developed a technique to be able to place seed points into the 3D visualization space in various configurations, despite only using input from a 2D touch surface.

We closely worked with two domain experts to build a tool that is useful for them in practice for their work. Its usefulness and power was validated in an observational study with five additional domain experts-one of which, on his own initiative, expressed that he wants to continue and explore interaction with his own data. While many possibilities exist to improve the current tool, we showed that systems entirely controlled using multi-touch interaction can be very useful for interactive exploratory scientific visualization. The expert participants in our observational study said that tools like ours are good to get a first impression about an unknown dataset and to identify interesting aspects to investigate in detail, before proceeding with other, more specialized tools. Participants suggested that it would be good to alternate between traditional and touch setups to test ideas.

Our study also revealed that our initial approach did not yield a 100% ideal solution. Ways to alternate between view control and cutting plane manipulation, as well as the specific technique for interacting with the cutting plane, need further improvement. However, our study also showed that with learning people can master our technique and that it is possible to integrate several different 3D interaction techniques (view navigation, cutting plane interaction, drilling interaction) in the same interaction space (3D view widget).

The most important findings from our evaluation and thus this paper's conclusions, however, are those that relate to touch interaction with 3D data in general. We saw that direct manipulation and interaction fluidity are essential to permit rapid idea exploration, that combinations of 3D and 2D visualization and exploration techniques are useful, in particular, if 2D techniques are frequently used traditionally, and that collaboration using vertical display settings can be suitable and desired in exploratory scientific visualization. In addition to the mentioned improvements to the specific tool and the exploration of other domains, it will thus be important to study different display orientations for scientific visualization, to come up with guidelines on how and when vertical or tilted setups support collaboration in small groups, and to understand how collaboration can be further encouraged and supported. Moreover, it would be necessary to develop an integrated interaction toolkit that better supports the effective and intuitive combinations of multiple different 3D interaction techniques with different visualization tools [START_REF] Isenberg | Position Paper: Touch Interaction in Scientific Visualization[END_REF].

Figure 1 :

 1 Figure 1: Our direct-touch fluid flow exploration tool in use.

Figure 2 :

 2 Figure 2: Slice of a scalar FTLE field in Matlab, showing material divergence between close-by particles. The flow moves from the left to the right over the cavity; red lines indicate material frontiers through which mass fluxes are null.

Figure 3 :

 3 Figure 3: General interface design with two data views, showing the iso-surface and volumetric visualizations of the scalar FTLE field using the domain-specific Jet colormap.

Figure 4 :

 4 Figure 4: Cutting plane interaction with bi-manual control.

Figure 6 :

 6 Figure 6: Specifying seeds in the 2D cut view (two touches/ hands, right) to interactively place 3D streamlines (left).

Figure 7 :

 7 Figure 7: Bi-manual control of the seeding density.

Figure 8 :

 8 Figure 8: Co-located collaboration between two experts.

© 2012 The Author(s) © 2012 The Eurographics Association and Blackwell Publishing Ltd.

Acknowledgments

We thank all study participants for their input as well as Dorine Caqueret and Jean-Marc Vézien for building the display and Petra Isenberg for her feedback on the paper.


Matthew Van Der Zwan

Alexandru C Telea

Tobias Isenberg

Continuous Navigation of Nested Abstraction Levels

Keywords: Descriptors (according to ACM CCS): I.3.m [Computer Graphics]: Miscellaneous-Scientific visualization, volumetric flow visualization, illustrative visualization, dedicated abstraction control

HAL is

Introduction

Abstraction is a-if not even the-fundamental principle employed in virtually all areas of visualization because it allows us to uncover and understand principles about the subject matter that we visualize, rather than just seeing the raw data. As Rautek et al. [START_REF] Rautek | Illustrative Visualization: New Technology or Useless Tautology[END_REF] note, abstraction can be introduced in a visualization either implicitly by selecting a certain style of depiction ("low-level visual abstractions") or explicitly by employing means such as focus+context or distortion ("high-level visual abstractions"). The latter group of high-level abstractions are of particular interest because they are created to emphasize specific chosen aspects of interest to the viewers. Often, however, there exist many different means to achieve explicit or high-level abstractions, all of which are valid and show different important aspects of the same dataset. Therefore it is essential that we can link these different types of abstraction with each other [START_REF] Duke | Linking Representation with Meaning[END_REF] to allow viewers to understand the relationship between them.

In this paper we focus on addressing this issue of combining multiple different layers of abstraction of the same dataset specifically for 3D datasets whose abstract representations are spatially and semantically nested. By spatially nested we mean that the abstractions are defined in the same spatial embedding but each uses a different amount of screen space such that more abstract representations are, generally, 'inside' less abstract ones. The nesting property allows us to use two-dimensional techniques to generate real-time halos which appear volumetric and visually separate the different visualizations. This allows us to create transitions between different abstractions which do not allow seamless geometrical transitions as demonstrated using different representations of fluid flow. We also incorporate lens-based navigation into the defined abstraction space allowing investigation of a different level of abstraction. Taken together, these techniques facilitate an intuitive continuous navigation of a set of nested abstractions of a given 3D visualization.

Related Work

Abstraction is a core principle in visualization and takes many forms, depending on the visualized data. Dedicated, controlled abstraction has been investigated not only in nonphotorealistic rendering (e. g., [START_REF] Decarlo | Stylization and Abstraction of Photographs[END_REF][START_REF] Decarlo | Abstraction of 2D Shapes in Terms of Parts[END_REF][START_REF] Winkenbach | Computer-Generated Pen-and-Ink Illustration[END_REF]) but also in visualization. In the field of information visualization many forms of intentional abstraction are used such as edge bundling (e. g., [START_REF] Holten | Hierarchical Edge Bundles: Visualization of Adjacency Relations in Hierarchical Data[END_REF]) or focus+context (e. g., [START_REF] Carpendale | A Framework for Unifying Presentation Space[END_REF]). In scientific and specifically illustrative visualization many "high-level visual abstractions" [START_REF] Rautek | Illustrative Visualization: New Technology or Useless Tautology[END_REF] are used.

Relevant for our own work are those high-level visual abstractions that not only show more or less relevant parts of a dataset in more or less detail but which can relate different visual representations (i. e., different abstraction levels) to each other. Duke [START_REF] Duke | Linking Representation with Meaning[END_REF] describes this problem nicely and suggests linking different categories or representations to each other to uncover and understand the structure in a dataset, naming molecular visualization as one example. Van der Zwan et al. [START_REF] Van Der | Illustrative Molecular Visualization with Continuous Abstraction[END_REF] demonstrated such seamless transition between molecular abstraction levels in an interactive [START_REF] Van Der | Illustrative Molecular Visualization with Continuous Abstraction[END_REF] and a spatially explicit [LVvdZ * 11] manner. However, van der Zwan et al.'s [START_REF] Van Der | Illustrative Molecular Visualization with Continuous Abstraction[END_REF] realization of abstraction level transitions requires that meaningful intermediate stages exist which is not the case for many forms of abstraction-a problem that we address in our own work.

We employ halos as one visual technique to visually separate layered elements and thus to enhance spatial perception. While halos enhance the spatial perception of the depicted objects in volume [START_REF] Bruckner | Enhancing Depth-Perception with Flexible Volumetric Halos[END_REF][START_REF] Interrante | Visualizing 3D Flow[END_REF] and other visualization domains [EBRI09,TCM06], we employ them to support visual layering and thus related to their function of showing occlusion relationships in line-based techniques [START_REF] Appel | The Haloed Line Effect for Hidden Line Elimination[END_REF][START_REF] Elber | Line Illustrations ∈ Computer Graphics[END_REF]. In addition, we use interactive lenses to locally explore our layered abstractions. Lenses are not only frequently used to support focus+context techniques [START_REF] Carpendale | A Framework for Unifying Presentation Space[END_REF] but also to interactively reveal otherwise hidden information, an approach pioneered by Bier et al. [BSP * 93, BSP97]. Their Magic Lenses locally affect a 2D screen region using a user-selected operator. While lenses can be used in a 3D context to distort the projection [START_REF] Yang | Nonlinear Perspective Projections and Magic Lenses: 3D View Deformation[END_REF], they can also be used to specify non-view changes for a 3D scene in a separate 2D layer [START_REF] Hurter | MoleView: An Attribute and Structure-Based Semantic Lens for Large Element-Based Plots[END_REF][START_REF] Neumann | NPR Lenses: Interactive Tools for Non-Photorealistic Line Drawings[END_REF]. Our lenses have a similar function as we use them in a 2D layer over the 3D model to locally reveal relationships between abstraction layers, thus also relating to a number of smart visibility methods in visualization [START_REF] Viola | Smart Visibility in Visualization[END_REF].

Visualization Model

We start with a dataset d ∈ D and consider several visualizations of d, modeled as images V 1≤i≤N : D → R 2 and each producing a 2D image A i = V i (d). We call these images abstractions of d if they represent the information in d on different levels of detail. We distinguish two abstraction types: Semantic abstractions A i simplify the information in d by showing varying amounts of the information present in d using different visual representations. For example, a fluid flow volume d ⊂ R 3 can be rendered as an entire flow volume using LIC [START_REF] Cabral | Imaging Vector Fields using Line Integral Convolution[END_REF][START_REF] Stalling | Fast and Resolution Independent Line Integral Convolution[END_REF], as stream LIC structures for a set of given streamlines [START_REF] Helgeland | Visualization of Vector Fields using Seed LIC and Volume Rendering[END_REF], and as flow topology [START_REF] Theisel | Saddle Connectors -An Approach to Visualizing the Topological Skeleton of Complex 3D Vector Fields[END_REF]; these are increasingly simplified semantic representations. Sampling abstractions reduce the amount of points produced by a given semantic abstraction A i using data sampling. Rendering different numbers of streamlines, for example, are samplings of the streamline abstraction. We denote all S i samplings of a semantic abstraction A i by A j i , 1 ≤ j ≤ S i with A 1 i = A i the most detailed and A S i i the coarsest sampling. To be useful in an exploration scenario, abstractions must be described in terms of the amount of simplification they produce on some input dataset. In our model we assume that, for a given application domain (e. g., flow visualization), the abstraction set A = {A i } can be ordered in decreasing amount of provided simplification from the densest abstrac-tion A 1 to the sparsest one A N . We also require that simpler abstractions are visually nested within less simple ones, i. e. A j ⊂ A i , ∀i < j. This is often the case in scientific visualization where abstraction reduces the size and/or spatial dimensionality of the depicted visual elements while keeping them aligned in the space D. Our flow visualization scenario is such a case of nested abstractions: the topology is a part of the streamlines, these are nested within stream LIC representation, which in turn is a part of a LIC volume.

Navigating the Abstraction Space

Given an abstraction set A as just described, one typically wants to navigate A to get different types of insight which are best visible at different abstraction levels. One navigation option is to start with A N (most abstract) and browse through A i until A 1 (most detailed), optionally using spatial sampling to restrict the dense-data areas to zones of interest using, e. g., focus+context techniques. One can also start with the most detailed level A 1 and simplify the visualization to the coarsest level A N is reached. During both navigation types, we call the level of the highest abstraction A f being visualized the focus of the visualization: Given a user-selected f , we aim to produce a visualization combining all context abstractions A i , i < f and A f in a single visualization such that all abstractions and their spatial nestings are shown. This will permit smooth navigation in the combined space of semantic and sampling abstractions (as introduced in Section 3).

Such navigations are typically realized by toggling the rendering of the elements A i on and off. However, this creates sharp visual discontinuities in the transition, especially if the abstractions differ visually. Continuity can be added by smoothly interpolating the transparency or shape of consecutive A i using fading or morphing while navigating through A. However, blending blurs the spatial nesting insight and can result in too high opacities when too many abstractions are blended. Morphing is not trivial for any pair of (nested) shapes, works only for shape pairs, and requires 3D shape representations rather than their 2D visualization results A i .

We propose to create a continuous navigation function Nav : A × [0, 1] → R 2 to help navigation in the abstraction space. Given our set A of ordered, nested abstractions and a focus abstraction level f ∈ [0, 1], we combine all abstractions A to build a visualization V . As the user changes the focus level f , V continuously changes to show only A 1 (at f = 0), next show the focus abstraction A f nested within lower abstractions as context, and finally A N (at f = 1). The design of Nav should be such that it can be computed on any set of nested 2D or 3D abstractions, is continuous in f , clearly emphasizes the focus-context relation of nested abstractions, and is computed using only 2D image information instead of 3D shape information to achieve maximal performance.

We use an additive blending of the abstractions A i in nesting levels (decreasing i) and compute the navigation function as Nav(A, f ) = ∑ n i=1 α i ( f )A i (see Fig. 1). The design of navigation parameter f 0 1 the blending factors

f in i f full i f out i f gone i ψ i φ i sampling u A i is subsampled user-selected focus f A i fades in A i fades out f in i+1 f full i+1 f out i+1 f gone i+1 ψ i+1 φ i+1 sampling u A i+1 is subsampled A i+1 fades in A i+1 fades out semantic abstractions A i A i+1 LIC (A 1 ) seed LIC (A 2 )
α i : [0, 1] → [0, 1] is essential, we use α i ( f ) = φ i ( f ) • ψ i ( f ) • h i ( f ).
Here, φ i : [0, 1] → [0, 1] is the function used to fade in an abstraction A i , ψ j : [0, 1] → [0, 1] is the function responsible for fading out a (context) abstraction A j , j < i, and h i : [0, 1] → [0, 1] is the halo function used to create a halo around the selected abstractions. As fade-in function we use φ i = max 0, min 1,

f -f (i) in f (i) f ull -f (i) in where f (i) in
is the focus value from which we start fading in abstraction A i and f

(i)
f ull is the value at which A i is completely visible. In practice, we want to start fading in the next abstraction A i+1 when the current abstraction A i is fully visible. Hence, we choose f

(i+1) in = f (i)
f ull . Similarly, we define the fade-out function ψ i = min 1, max 0,

f (i) gone -f f (i) gone -f (i) out . Here, f (i)
out is the value for which we start fading out the context abstraction A i and f (i) gone is the value for which abstraction A i is no longer visible. Similar to fading in, we start fading out abstraction A i+1 when the A i is no longer visible and, thus, set f

(i) gone = f (i+1)
out . Also, we constrain f i out > f i+1 f ull such that A i does not start fading out before A i+1 is fully in focus. Finally, we set f 1 f ull = 0, f N out = 1 so that we start with a fully focused A 1 and end with a fully focused A N .

While combining φ i and ψ i allows us to fade abstractions in and out of view continuously, the resulting image will be unable to clearly show the nesting structure of the abstraction space: Depending on the specific abstraction image shapes and colors, it may be hard to see which result pixels belong to a (thin) abstraction being nested within a (larger) context abstraction, especially if both have similar colors. We therefore use the halo function h i to generate halos around abstractions:

h i ( f ) = min DT A i+1 /δ k i ( f ) , 1 .
In this function, DT Ω : R 2 → R + is the distance transform of a 2D binary shape Ω, which gives, for any points x ∈ R 2 , its distance to Ω [CC00]. DT is zero inside Ω and smoothly increases outside the shape. In our case, we construct such shapes by simply thresholding the rendered abstractions A i into foreground (rendered) and background (not rendered) pixels. Having DT A i+1 , we compute a halo around A i+1 by modulating the distance transform with a power function k i ( f ). The halo's width is limited to a maximal value of δ > 0 pixels. The effect of the power function is to create a smoother transition from the context A i of A i+1 than if linear distance functions were used. Finally, we set k i = φ i to increase the halo around the fading-in abstraction A i+1 , thus making it more prominent in its context A i where it is nested. Perspective-like halos can easily be obtained by modulating the value of δ with the depth of A i+1 at each pixel.

The above process describes how semantic abstractions A i are combined into a single image. However, as defined in Section 3, our input may contain sampled versions thereof. We integrate these smoothly in the above process by replacing, in the navigation function Nav(A, f ), each semantic abstraction A i with its sampled version A j i , the sampling parameter j being controlled by the distance from the userset focus f to the full-visibility f i f ull as j =

f -f i f ull f i out -f i f ull S i .
In other words, as the user increases f , the full-visibility abstraction is progressively simplified from A 1 i to A S i i (coarsest variant). During this process, all remaining visualization elements stay the same (halo sizes, overall abstraction transparency). When f reaches f i out , the coarse abstraction A S i i is further faded out. For abstractions which have no levelof-detail representations, the process simply uses the unique representation A i . This directly accommodates any number of semantic abstractions with any number of sampling representations thereof, effectively intertwining the navigation in the semantic and sampled spaces of abstractions.

Interactive Local Exploration

While this navigation facilitates an effective global abstraction space exploration, in many cases we are interested in getting local detail information. We thus also provide contextsensitive local lenses, whose goal it is to allow parts of an abstraction A l> f located inside the lens to become visible even when otherwise hidden due to the global abstraction level f . While a naïve implementation would change the blending factors α i ( f ) close to the lens center, this would interfere with our transparent distance-based halos. In multi-layer visualizations such as ours one also wants to see 'deeper' within the abstraction stack inside the lens and locate the parts of deeper-nested abstractions closest to to the lens.

For this, we use a focus-guided lens. Given a global abstraction level f , we combine revealing deeper-nested information at the lens center with revealing higher-abstraction structures A l> f close to it. We first locate the closest point A l (c) of abstraction A l to the lens center c. The point c can be directly computed as A l (c) = FT A l (c). Here, FT A l : R 2 → R 2 is the feature transform of the shape A l [START_REF]Shape Analysis and Classification: Theory and Practice[END_REF]. The feature transform of a shape Ω ⊂ R 2 is defined as FT Ω (x) = {y ∈ Ω|DT Ω (x) = xy }, i. e. the closest point y ∈ Ω to a given target point x, restricting ourselves to a one-point feature transform [EHP * 11]. With the lens center c and closest abstraction point A l (c), we construct our focus-guided lens by multiplying the halo functions h l> f with the distance transform of a beam-like shape created by two circles connected by a trapezium (light blue in Fig. 2). As the lens is moved, it behaves similarly to a light beam that shows the shortest spatial path from the lens center to the desired A l . This is useful as one does not need to fully remove (make transparent) all abstractions A k<l in order to discover A l . Hence, one can stay at a desired semantic focus level f and use the lens to search for another desired A l> f in the vicinity of any point.

Implementation and Results

For the realization we only require a set of N -1 2D images depicting the different context abstraction levels and one image depicting the abstraction in focus at the selected abstraction level, as our method works entirely in image space. These images are either generated on-the-fly or are precomputed if they do not change during the exploration.

From these images we compute the soft halos for our nesting (within 10 ms on a modern graphics card) by employing a recent CUDA-based implementation [START_REF] Telea | CUDA Skeletonization and Distance Transform Toolkit[END_REF] of exact Euclidean distance transforms and feature transforms [START_REF] Tang | Parallel Banding Algorithm to Compute Exact Distance Transform with the GPU[END_REF]. Finally, blending is implemented via OpenGL alpha blending. The entire process, including rendering LIC, seed LIC, streamlines, and precomputed topology, works at 5 frames per second on a MacBook with 2 GB RAM and an NVIDIA GeForce 320M graphics card with 256 MB RAM.

Conclusion

In this paper, we have presented an abstraction technique for continuous navigation of nested abstraction levels. This navigation can be achieved on both global and local levels, using lenses for the latter. For both types of navigation, we use two-dimensional distance transforms to create smooth halos in the image domain which look like three-dimensional volumetric halos. As an example application, we used our technique to navigate different semantic and sampling abstractions of fluid flow visualization.

  topology (focus) nested in streamlines, seed LIC, and LIC

Figure 1 :Figure 2 :

 12 Figure 1: Continuous navigation in a flow visualization abstraction space with four abstractions A 1 -A 4 .

Figure 3 :

 3 Figure 3: Navigating the flow visualization abstraction space: (a)-(c) Introducing seed LIC as focus in the LIC volume, (d) has the flow topology as focus abstraction and the previous abstractions as context from which the LIC volume is removed in (e).

Figure 4 :

 4 Figure 4: Fluid flow abstracted (a)-(b) locally with a guided lens and (c)-(d) with sampling abstractions (streamline filtering).

Fig. 3

 3 Fig. 3 shows our global abstraction applied to a direct incompressible Navier-Stokes flow simulation around a cylinder. Fig. 3(a)-(c) show three transition steps of stream LIC blended with the LIC volume separated by the halo around the new focus, i.e. a visualization with the streamline LIC as focus and the remainder (LIC) as context. A further abstraction (Fig. 3(d)) adds topology as the new focus. Finally, we remove the lowest abstraction (Fig. 3(e)). Our second example (Fig. 4) shows the local exploration of a different Navier-Stokes simulation, with Fig. 4(a) and (b) showing two lens locations. Fig. 4(c) and (d), finally demonstrate the inclusion of several sampling abstraction levels into the treatment.

© The Eurographics Association 2012.


Matthew Van Der Zwan
email: m.a.t.van.der.zwan@rug.nl

Alexandru Telea
email: a.c.telea@rug.nl

Tobias Isenberg
email: tobias.isenberg@inria.fr

Smooth Navigation Between Nested Spatial Representations

We investigate the dedicated control of multiple levels of semantic and sampling-based abstraction in 3D datasets, i. e., different types of data abstractions as opposed to samplingbased abstraction which shows more or less data. This dedicated navigation in the abstraction space facilitates the mental integration of different existing visualization techniques in many application areas including our example domain of fluid simulation. We realize the continuous abstraction control by interpolating between the levels while being able to simultaneously show multiple abstractions. We employ a halolike shading technique based on distance fields to blend between several levels while continuously navigating between focus and context abstractions. We further add a semantic lens to find focus abstractions close to a user-defined context abstraction. Our entire implementation uses 2D imagebased techniques to enable real-time performance, which seamlessly integrates within a 3D visualization tool.

Introduction

Abstraction is a-if not even the-fundamental principle employed in virtually all areas of visualization because it allows us to uncover and understand principles about the subject matter that we visualize, rather than just seeing the raw data. As Rautek et al. [START_REF] Rautek | Illustrative Visualization: New Technology or Useless Tautology[END_REF] note, abstraction can be introduced in a visualization either implicitly by selecting a certain style of depiction ("low-level visual abstractions") or explicitly by employing means such as focus+context or distortion ("highlevel visual abstractions"). The latter group of high-level abstractions are of particular interest because they are created to emphasize specific chosen aspects of interest to the viewers. Often, however, there exist many different means to achieve explicit or high-level abstractions, all of which are valid and show different important aspects of the same dataset. Therefore it is essential that we can link these different types of abstraction with each other [START_REF] Duke | Linking Representation with Meaning[END_REF] to allow viewers to understand the relationship between them.

In this paper we focus on addressing this issue of combining multiple different layers of abstraction of the same This paper has been submitted for presentation at ICT.OPEN 2012. This is the national Dutch ICT conference and serves, for this paper, the role of training the presentation skills rather than to push scientific limits. ICT.OPEN does not claim copyright. For this reason, ICT.OPEN 2012 encourages authors to submit mostly papers that have been sent to or have recently been presented at international conferences. dataset specifically for 3D datasets whose abstract representations are spatially and semantically nested. By spatially nested we mean that the abstractions are defined in the same spatial embedding but each uses a different amount of screen space such that more abstract representations are, generally, 'inside' less abstract ones. The nesting property allows us to use two-dimensional techniques to generate real-time halos which appear volumetric and visually separate the different visualizations. This allows us to create transitions between different abstractions which do not allow seamless geometrical transitions as demonstrated using different representations of fluid flow. We also incorporate lens-based navigation into the defined abstraction space allowing investigation of a different level of abstraction. Taken together, these techniques facilitate an intuitive continuous navigation of a set of nested abstractions of a given 3D visualization.

Related Work

Abstraction is a core principle in visualization and takes many forms, depending on the visualized data. Dedicated, controlled abstraction has been investigated not only in nonphotorealistic rendering (e. g., [START_REF] Decarlo | Stylization and Abstraction of Photographs[END_REF][START_REF] Mi | Abstraction of 2D Shapes in Terms of Parts[END_REF][START_REF] Winkenbach | Computer-Generated Pen-and-Ink Illustration[END_REF]) but also in visualization. In the field of information visualization many forms of intentional abstraction are used such as edge bundling (e. g., [START_REF] Holten | Hierarchical Edge Bundles: Visualization of Adjacency Relations in Hierarchical Data[END_REF]) or focus+context (e. g., [START_REF] Carpendale | A Framework for Unifying Presentation Space[END_REF]). In scientific and specifically illustrative visualization many "high-level visual abstractions" [START_REF] Rautek | Illustrative Visualization: New Technology or Useless Tautology[END_REF] are used.

Relevant for our own work are those high-level visual abstractions that not only show more or less relevant parts of a dataset in more or less detail but which can relate different visual representations (i. e., different abstraction levels) to each other. Duke [START_REF] Duke | Linking Representation with Meaning[END_REF] describes this problem nicely and suggests linking different categories or representations to each other to uncover and understand the structure in a dataset, naming molecular visualization as one example. Van der Zwan et al. [START_REF] Van Der Zwan | Illustrative Molecular Visualization with Continuous Abstraction[END_REF] demonstrated such seamless transition between molecular abstraction levels in an interactive [START_REF] Van Der Zwan | Illustrative Molecular Visualization with Continuous Abstraction[END_REF] and a spatially explicit [START_REF] Lueks | Spatially Continuous Change of Abstraction in Molecular Visualization[END_REF] manner. However, van der Zwan et al.'s [START_REF] Van Der Zwan | Illustrative Molecular Visualization with Continuous Abstraction[END_REF] realization of abstraction level transitions requires that meaningful intermediate stages exist which is not the case for many forms of abstraction-a problem that we address in our own work.

We employ halos as one visual technique to visually separate layered elements and thus to enhance spatial perception. While halos enhance the spatial perception of the depicted objects in volume [START_REF] Bruckner | Enhancing Depth-Perception with Flexible Volumetric Halos[END_REF][START_REF] Interrante | Visualizing 3D Flow[END_REF] and other visualization domains [START_REF] Everts | Depth-Dependent Halos: Illustrative Rendering of Dense Line Data[END_REF][START_REF] Tarini | Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization[END_REF], we employ them to support visual layering and thus related to their function of showing occlusion relationships in line-based techniques [START_REF] Appel | The Haloed Line Effect for Hidden Line Elimination[END_REF][START_REF] Elber | Line Illustrations ∈ Computer Graphics[END_REF]. In addition, we use interactive lenses to locally explore our layered abstractions. Lenses are not only frequently used to support focus+context techniques [START_REF] Carpendale | A Framework for Unifying Presentation Space[END_REF] but also to interactively reveal otherwise hidden information, an approach pioneered by Bier et al. [START_REF] Bier | Enhanced Illustration Using Magic Lens Filters[END_REF][START_REF] Bier | Toolglass and Magic Lenses: The See-Through Interface[END_REF]. Their Magic Lenses locally affect a 2D screen region using a user-selected operator. While lenses can be used in a 3D context to distort the projection [START_REF] Yang | Nonlinear Perspective Projections and Magic Lenses: 3D View Deformation[END_REF], they can also be used to specify non-view changes for a 3D scene in a separate 2D layer [START_REF] Hurter | MoleView: An Attribute and Structure-Based Semantic Lens for Large Element-Based Plots[END_REF][START_REF] Neumann | NPR Lenses: Interactive Tools for Non-Photorealistic Line Drawings[END_REF]. Our lenses have a similar function as we use them in a 2D layer over the 3D model to locally reveal relationships between abstraction layers, thus also relating to a number of smart visibility methods in visualization [START_REF] Viola | Smart Visibility in Visualization[END_REF].

Visualization Model

We start with a dataset d ∈ D and consider several visualizations of d, modeled as images V 1≤i≤N : D → R 2 and each producing a 2D image A i = V i (d). We call these images abstractions of d if they represent the information in d on different levels of detail. We distinguish two abstraction types: Semantic abstractions A i simplify the information in d by showing varying amounts of the information present in d using different visual representations. For example, a fluid flow volume d ⊂ R 3 can be rendered as an entire flow volume using LIC [START_REF] Cabral | Imaging Vector Fields using Line Integral Convolution[END_REF][START_REF] Stalling | Fast and Resolution Independent Line Integral Convolution[END_REF], as stream LIC structures for a set of given streamlines [START_REF] Helgeland | Visualization of Vector Fields using Seed LIC and Volume Rendering[END_REF], and as flow topology [START_REF] Theisel | Saddle Connectors -An Approach to Visualizing the Topological Skeleton of Complex 3D Vector Fields[END_REF]; these are increasingly simplified semantic representations. Sampling abstractions reduce the amount of points produced by a given semantic abstraction A i using data sampling. Rendering different numbers of streamlines, for example, are samplings of the streamline abstraction. We denote all S i samplings of a semantic abstraction A i by A j i , 1 ≤ j ≤ S i with A 1 i = A i the most detailed and A S i i the coarsest sampling.

To be useful in an exploration scenario, abstractions must be described in terms of the amount of simplification they produce on some input dataset. In our model we assume that, for a given application domain (e. g., flow visualization), the abstraction set A = {A i } can be ordered in increasing amount of provided simplification from the densest abstraction A 1 to the sparsest one A N . We also require that simpler abstractions are visually nested within less simple ones, i. e. A j ⊂ A i , ∀i < j. This is often the case in scientific visualization where abstraction reduces the size and/or spatial dimensionality of the depicted visual elements while keeping them aligned in the space D. Our flow visualization scenario is such a case of nested abstractions: the topology is a part of the streamlines, these are nested within stream LIC representation, which in turn is a part of a LIC volume.

Navigating the Abstraction Space

Given an abstraction set A as just described, one typically wants to navigate A to get different types of insight which are best visible at different abstraction levels. One navigation op-tion is to start with A N (most abstract) and browse through A i until A 1 (most detailed), optionally using spatial sampling to restrict the dense-data areas to zones of interest using, e. g., focus+context techniques. One can also start with the most detailed level A 1 and simplify the visualization to the coarsest level A N is reached. During both navigation types, we call the level of the highest abstraction A f being visualized the focus of the visualization: Given a user-selected f , we aim to produce a visualization combining all context abstractions A i , i < f and A f in a single visualization such that all abstractions and their spatial nestings are shown. This will permit smooth navigation in the combined space of semantic and sampling abstractions (as introduced in Section 3).

Such navigations are typically realized by toggling the rendering of the elements A i on and off. However, this creates sharp visual discontinuities in the transition, especially if the abstractions differ visually. Continuity can be added by smoothly interpolating the transparency or shape of consecutive A i using fading or morphing while navigating through A. However, blending blurs the spatial nesting insight and can result in too high opacities when too many abstractions are blended. Morphing is not trivial for any pair of (nested) shapes, works only for shape pairs, and requires 3D shape representations rather than their 2D visualization results A i .

We propose to create a continuous navigation function Nav : A × [0, 1] → R 2 to help navigation in the abstraction space. Given our set A of ordered, nested abstractions and a focus abstraction level f ∈ [0, 1], we combine all abstractions A to build a visualization V . As the user changes the focus level f , V continuously changes to show only A 1 (at f = 0), next show the focus abstraction A f nested within lower abstractions as context, and finally A N (at f = 1). The design of Nav should be such that it can be computed on any set of nested 2D or 3D abstractions, is continuous in f , clearly emphasizes the focus-context relation of nested abstractions, and is computed using only 2D image information instead of 3D shape information to achieve maximal performance.

We use an additive blending of the abstractions A i in nesting levels (decreasing i) and compute the navigation function as Nav(A, f ) = ∑ n i=1 α i ( f )A i (see Fig. 1). The design of the blending factors

α i : [0, 1] → [0, 1] is essential, we use α i ( f ) = φ i ( f ) • ψ i ( f ) • h i ( f ). Here, φ i : [0, 1] → [0, 1] is the function used to fade in an abstraction A i , ψ j : [0, 1] → [0, 1]
is the function responsible for fading out a (context) abstraction A j , j < i, and h i : [0, 1] → [0, 1] is the halo function used to create a halo around the selected abstractions. As fade-in function we use φ i = max 0, min 1,

f -f (i) in f (i) f ull -f (i) in where f (i) in
is the focus value from which we start fading in abstraction A i and f (i) f ull is the value at which A i is completely visible. In practice, we want to start fading in the next abstraction A i+1 when the current abstraction A i is fully visible. Hence, we choose f

(i+1) in = f (i)
f ull . Similarly, we define the fade-out function ψ i = min 1, max 0, for which we start fading out the context abstraction A i and f

f (i) gone -f f (i) gone -f (i) out . Here, f (i) out is the value navigation parameter f 0 1 f in i f full i f out i f gone i ψ i φ i sampling u A i is subsampled user-selected focus f A i fades in A i fades out f in i+1 f full i+1 f out i+1 f gone i+1 ψ i+1 φ i+1 sampling u A i+1 is subsampled A i+1 fades in A i+1 fades out semantic abstractions A i A i+1 LIC (A 1 ) seed LIC (A 2 )
(i)
gone is the value for which abstraction A i is no longer visible. Similar to fading in, we start fading out abstraction A i+1 when the A i is no longer visible and, thus, set f

(i) gone = f (i+1)
out . Also, we constrain f i out > f i+1 f ull such that A i does not start fading out before A i+1 is fully in focus. Finally, we set f 1 f ull = 0, f N out = 1 so that we start with a fully focused A 1 and end with a fully focused A N .

While combining φ i and ψ i allows us to fade abstractions in and out of view continuously, the resulting image will be unable to clearly show the nesting structure of the abstraction space: Depending on the specific abstraction image shapes and colors, it may be hard to see which result pixels belong to a (thin) abstraction being nested within a (larger) context abstraction, especially if both have similar colors. We therefore use the halo function h i to generate halos around abstractions:

h i ( f ) = min DT A i+1 /δ k i ( f ) , 1 . In this function,
DT Ω : R 2 → R + is the distance transform of a 2D binary shape Ω, which gives, for any points x ∈ R 2 , its distance to Ω [START_REF] Costa | Shape Analysis and Classification: Theory and Practice[END_REF]. DT is zero inside Ω and smoothly increases outside the shape. In our case, we construct such shapes by simply thresholding the rendered abstractions A i into foreground (rendered) and background (not rendered) pixels. Having DT A i+1 , we compute a halo around A i+1 by modulating the distance transform with a power function k i ( f ). The halo's width is limited to a maximal value of δ > 0 pixels. The effect of the power function is to create a smoother transition from the context A i of A i+1 than if linear distance functions were used. Finally, we set k i = φ i to increase the halo around the fading-in abstraction A i+1 , thus making it more prominent in its context A i where it is nested. Perspective-like halos can easily be obtained by modulating the value of δ with the depth of A i+1 at each pixel. The above process describes how semantic abstractions A i are combined into a single image. However, as defined in Section 3, our input may contain sampled versions thereof. We integrate these smoothly in the above process by replacing, in the navigation function Nav(A, f ), each semantic abstraction A i with its sampled version A j i , the sampling parameter j being controlled by the distance from the user-set focus

f to the full-visibility f i f ull as j = f -f i f ull f i out -f i f ull S i .
In other words, as the user increases f , the full-visibility abstraction is progressively simplified from A 1 i to A S i i (coarsest variant). During this process, all remaining visualization elements stay the same (halo sizes, overall abstraction transparency). When f reaches f i out , the coarse abstraction A S i i is further faded out. For abstractions which have no level-of-detail representations, the process simply uses the unique representation A i . This directly accommodates any number of semantic abstractions with any number of sampling representations thereof, effectively intertwining the navigation in the semantic and sampled spaces of abstractions.

Interactive Local Exploration

While this navigation facilitates an effective global abstraction space exploration, in many cases we are interested in getting local detail information. We thus also provide contextsensitive local lenses, whose goal it is to allow parts of an abstraction A l> f located inside the lens to become visible even when otherwise hidden due to the global abstraction level f . While a naïve implementation would change the blending factors α i ( f ) close to the lens center, this would interfere with our transparent distance-based halos. In multi-layer visualizations such as ours one also wants to see 'deeper' within the abstraction stack inside the lens and locate the parts of deeper-nested abstractions closest to to the lens.

For this, we use a focus-guided lens. Given a global abstraction level f , we combine revealing deeper-nested information at the lens center with revealing higher-abstraction structures A l> f close to it. We first locate the closest point A l (c) of abstraction A l to the lens center c. The point c can be directly computed as A l (c) = FT A l (c). Here, FT A l : R 2 → R 2 is the feature transform of the shape A l [START_REF] Costa | Shape Analysis and Classification: Theory and Practice[END_REF]. The feature transform of a shape Ω ⊂ R 2 is defined as FT Ω (x) = {y ∈ Ω|DT Ω (x) = xy }, i. e. the closest point y ∈ Ω to a given target point x, restricting ourselves to a one-point feature transform [START_REF] Ersoy | Skeleton-Based Edge Bundling for Graph Visualization[END_REF]. With the lens center c and closest abstraction point A l (c), we construct our focus-guided lens by multiplying the halo functions h l> f with the distance transform of a beam-like shape created by two circles connected by a trapezium (light blue in Fig. 2). As the lens is moved, it behaves similarly to a light beam that shows the shortest spatial path from the lens center to the desired A l . This is useful as one does not need to fully remove (make transparent) all abstractions A k<l in order to discover A l . Hence, one can stay at a desired semantic focus level f and use the lens to search for another desired A l> f in the vicinity of any point.

Implementation and Results

For the realization we only require a set of N -1 2D images depicting the different context abstraction levels and one image depicting the abstraction in focus at the selected abstraction level, as our method works entirely in image space. These images are either generated on-the-fly or are precomputed if they do not change during the exploration. From these images we compute the soft halos for our nesting (within 10 ms on a modern graphics card) by employing a recent CUDA-based implementation [START_REF] Telea | CUDA Skeletonization and Distance Transform Toolkit[END_REF] of exact Euclidean distance transforms and feature transforms [START_REF] Cao | Parallel Banding Algorithm to Compute Exact Distance Transform with the GPU[END_REF]. Finally, blending is implemented via OpenGL alpha blending. The entire process, including rendering LIC, seed LIC, streamlines, and precomputed topology, works at 5 frames per second on a MacBook with 2 GB RAM and an NVIDIA GeForce 320M graphics card with 256 MB RAM. A video illustrating our method is available at http://tobias. isenberg.cc/VideosAndDemos/Zwan2012CNN. Fig. 3 shows our global abstraction applied to a direct incompressible Navier-Stokes flow simulation around a cylinder. Fig. 3

Conclusion

In this paper, we have presented an abstraction technique for continuous navigation of nested abstraction levels. This navigation can be achieved on both global and local levels, using lenses for the latter. For both types of navigation, we use two-dimensional distance transforms to create smooth halos in the image domain which look like three-dimensional volumetric halos. As an example application, we used our technique to navigate different semantic and sampling abstractions of fluid flow visualization.

  topology (focus) nested in streamlines, seed LIC, and LIC

Figure 1 :Figure 2 :

 12 Figure 1: Continuous navigation in a flow visualization abstraction space with four abstractions A 1 -A 4 .

Figure 3 :

 3 Figure 3: Navigating the flow visualization abstraction space: (a)-(c) Introducing seed LIC as focus in the LIC volume, (d) has the flow topology as focus abstraction and the previous abstractions as context from which the LIC volume is removed in (e).

Figure 4 :

 4 Figure 4: Fluid flow abstracted (a)-(b) locally with a guided lens and (c)-(d) with sampling abstractions (streamline filtering).

  (a)-(c) show three transition steps of stream LIC blended with the LIC volume separated by the halo around the new focus, i.e. a visualization with the streamline LIC as focus and the remainder (LIC) as context. A further abstraction (Fig. 3(d)) adds topology as the new focus. Finally, we remove the lowest abstraction (Fig. 3(e)). Our second exam-ple (Fig. 4) shows the local exploration of a different Navier-Stokes simulation, with Fig. 4(a) and (b) showing two lens locations. Fig. 4(c) and (d), finally demonstrate the inclusion of several sampling abstraction levels into the treatment.