Jian Zhao
email: jianzhao@dgp.toronto.edu

Fanny Chevalier
email: fchevalier@ocad.ca.

Emmanuel Pietriga
email: emmanuel.pietriga@inria.fr.

Ravin Balakrishnan

Dgp

Exploratory Analysis of Time-series with ChronoLenses

Keywords: Time-series Data, Exploratory Visualization, Focus+Context, Lens, Interaction Techniques

Exploring a live P2P streaming dataset with ChronoLenses. The main interface includes (A) a charts panel showing the timeseries; (B) a lens creation toolbar; (C) a lens analysis pipeline view of (D) the currently selected lens; (E) a property panel showing details of the currently selected lens. A context menu (F) can be invoked to perform lens-based operations, and (G) the lens toolbar associated with each lens allows quick access to a lens' parameters.

Abstract-Visual representations of time-series are useful for tasks such as identifying trends, patterns and anomalies in the data. Many techniques have been devised to make these visual representations more scalable, enabling the simultaneous display of multiple variables, as well as the multi-scale display of time-series of very high resolution or that span long time periods. There has been comparatively little research on how to support the more elaborate tasks associated with the exploratory visual analysis of timeseries, e.g., visualizing derived values, identifying correlations, or discovering anomalies beyond obvious outliers. Such tasks typically require deriving new time-series from the original data, trying different functions and parameters in an iterative manner. We introduce a novel visualization technique called ChronoLenses, aimed at supporting users in such exploratory tasks. ChronoLenses perform on-the-fly transformation of the data points in their focus area, tightly integrating visual analysis with user actions, and enabling the progressive construction of advanced visual analysis pipelines.

INTRODUCTION

Time-series data are found in almost every domain, ranging from finance to many engineering and scientific disciplines. Time has inherently unique characteristics: for most purposes outside the theory of relativity, things evolve over time, but time does not depend on other variables. Time is considered uniform and absolute. It is thus often treated as a special variable, in terms of both how the data is structured and how it is presented to users. Tasks associated with the analysis of temporal datasets typically focus on the evolution of other (dependent) variables with respect to time: identifying trends and re-curring patterns, establishing correlations and possibly predicting the future based on past and current behavior.

Visual representations of time-series take advantage of people's innate perceptual abilities to process information and detect structure [START_REF] Müller | Visualization methods for time-dependant data-an overview[END_REF], making it significantly easier for users to discover trends and patterns at different scales, but also to identify anomalies in the data [START_REF] Buono | Interactive Pattern Search in Time Series[END_REF][START_REF] Mclachlan | LiveRAC: Interactive Visual Exploration of System Management Time-series Data[END_REF]. However, basic time-series visualizations using line plots do not scale well; and as time-series data are often very large, featuring multiple, possibly heterogeneous, dependent variables measured for long periods of time and/or at high sampling rates, visualization of real-world time-series data poses significant challenges and has been an active area of research for many years. Many interactive visualization tools have been developed to address this scalability problem, offering innovative alternatives to the common line plot visualizations or enhancing the visualization with advanced interactive features.

There has been comparatively little research on how to support the more elaborate tasks typically associated with the exploratory visual analysis of time-series, e.g., visualizing derived values, identifying correlations, or identifying anomalies beyond obvious outliers. Such tasks typically require deriving new time-series from the data, visualizing those time-series and relating them to the original data plots.

Visual exploration techniques take advantage of human abilities to drive the data exploration process and are especially useful for undirected searches, when users know little about the data or have only vague exploration goals [START_REF] Keim | Information Visualization and Visual Data Mining[END_REF][START_REF] Shneiderman | The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations[END_REF]. As emphasized by Keim's visual analytics mantra -"Analyze first, Show the important, Zoom, Filter and analyze further, Details on demand" [START_REF] Keim | Challenges in Visual Data Analysis[END_REF] -this process is iterative. For it to be efficient, visual representations, that support human judgment, and interactions, that re-parameterize the visual representation, should be tightly integrated, enabling users to quickly choose and refine parameter values that best suit the task at hand [START_REF] Aigner | Visual Methods for Analyzing Time-Oriented Data[END_REF]. New plots derived from the original data should be put in context and made easy to relate both to the original data and to other plots that have been derived as part of the exploratory process.

We introduce a novel, domain-independent time-series visualization technique called ChronoLenses, aimed at supporting users in exploratory visual analysis tasks. ChronoLenses, as many other types of lenses, delimit a region of interest in the data to be put in focus through magnification [START_REF] Cockburn | A Review of Overview+detail, Zooming, and Focus+context Interfaces[END_REF][START_REF] Pietriga | Representation-Independent In-Place Magnification with Sigma Lenses[END_REF], visual filtering [START_REF] Furnas | Generalized Fisheye Views[END_REF] or other arbitrary transformations of the underlying content [START_REF] Bier | Toolglass and Magic Lenses: the See-through Interface[END_REF]. Based on the metaphor of direct manipulation, ChronoLenses perform on-the-fly transformation of the data points in their focus area, tightly integrating visual analysis with interaction. Users can build pipelines composed of lenses performing various transformations on the data (e.g., remove mean, compute 1st derivative, auto-correlation), effectively creating flexible and reusable time-series visual analysis interfaces. At any moment, users can change the parameters of already created lenses, with the modifications instantaneously propagating down through the pipeline, providing immediate visual feedback that supports the iterative exploration process. Figure 1 gives an overview of the technique.

After a review of related work, we introduce tasks and requirements associated with the exploratory analysis of time-series, gathered from expert users in varied domains including operations monitoring and control for a radio-observatory, environmental research related to weather forecast, as well as financial and network streaming data analysis. We then describe the general concept of ChronoLenses, followed by its implementation in a research prototype application that provides rich interaction and analytical support for time-series. Example analysis pipelines follow with usage scenarios based on our work with network analysts, and astronomers of the ALMA radio-observatory, demonstrating how ChronoLenses can be used for the monitoring and offline analysis of complex time-series data. We conclude the paper with a discussion of the current implementation's limitations.

RELATED WORK 2.1 Time-series Visualization

Visual representations of time-series date back to the 18th century with seminal work using line charts by Playfair [START_REF] Tufte | The Visual Display of Quantitative Information[END_REF]. Many new static and dynamic techniques have been proposed since then. See [START_REF] Aigner | Visualizing Time-oriented Data -a Systematic View[END_REF][START_REF] Müller | Visualization methods for time-dependant data-an overview[END_REF][START_REF] Silva | Visualization of Linear Time-Oriented Data: A Survey[END_REF] for relevant surveys. Some innovative techniques focus on helping users identify periodic patterns in the data. For instance, van Wijk [START_REF] Van Wijk | Cluster and Calendar based Visualization of Time Series Data[END_REF] uses a calendar-based display to visualize time-oriented series aggregated on a daily, weekly or monthly basis via similarity clustering. Other visualizations, such as SpiralGraph [START_REF] Weber | Visualizing Time-Series on Spirals[END_REF] and SpiralView [START_REF] Bertini | SpiralView: Towards Security Policies Assessment through Visual Correlation of Network Resources with Evolution of Alarms[END_REF], lay out the data on a spiral-shaped time axis to reveal cycles. These techniques can yield meaningful representations of data that feature periodicity, making recurring patterns easy to spot. VizTree [START_REF] Lin | Visually mining and monitoring massive time series[END_REF] takes a different approach, computing a symbolic representation of time-series and then representing the sequence of symbols using a suffix tree. The resulting visualization is radically different from other temporal visualizations and can be disconcerting at first, but represents a potentially powerful alternative to other techniques for identifying patterns in large datasets.

Most of the work on time-series visualization has focused on the scalability of the more conventional line plot and bar chart representations, either by proposing variations on the original plotting techniques or by enhancing them with advanced interactive visual filtering techniques. Variations on conventional line plots include Small multiples and sparklines [START_REF] Tufte | The Visual Display of Quantitative Information[END_REF], Horizon graphs [START_REF] Heer | Sizing the Horizon: the Effects of Chart Size and Layering on the Graphical Perception of Time Series Visualizations[END_REF], and Braided graphs [START_REF] Javed | Graphical Perception of Multiple Time Series[END_REF]. These techniques focus on the issue of optimal space management, enabling the display of an increased number of time-series compared to regular line plots. Lopez-Hernandez et al. [START_REF] Lopez-Hernandez | A Layer-oriented Interface for Visualizing Time-series Data from Oscilloscopes[END_REF] address the same problem for a specific type of time-series -univariate oscilloscope digital signals -by wrapping the signal in time, representing the different traces on layers that can be brushed by users.

Multi-scale Representations

Multi-scale visualization techniques adapt the representation depending on available screen real-estate. Time-series can be represented at several levels of detail and abstraction using different qualitative scales, or hybrids combining qualitative and quantitative information [START_REF] Bade | Connecting Time-oriented Data and Information to a Coherent Interactive Visualization[END_REF], gradually revealing more information as more space gets allocated to the chart, or as the user zooms in a region of interest [START_REF] Lam | Overview Use in Multiple Visual Information Resolution Interfaces[END_REF]. Line Graph Explorer [START_REF] Kincaid | Line Graph Explorer: Scalable Display of Line Graphs using Focus+context[END_REF] provides an overview+detail interface for the exploration of large collections of time-series. It displays a compact overview of the entire collection by encoding the y-dimension of individual line graphs using color instead of space and viewing selected graphs in detail as standard line graphs. The technique is also interesting as the overview visualization lends itself well to sorting and clustering of the graphs using various similarity distances.

Hao et al. [START_REF] Hao | Multi-Resolution Techniques for Visual Exploration of Large Time-Series Data[END_REF] present a space-filling, multi-resolution matrix representation of time-series where the color of a cell encodes the magnitude of the corresponding value in the data. The same authors propose another space-filling visualization capable of displaying multiple time-series, where the various charts are organized based on their importance in a treemap-like structure [START_REF] Hao | Importance-Driven Visualization Layouts for Large Time Series Data[END_REF]. Stack zooming [START_REF] Javed | Stack Zooming for Multi-focus Interaction in Time-series Data Visualization[END_REF] uses a relatively similar technique, taking advantage of the one-dimensional nature of time-series to present them as hierarchies of strips, where each strip contains magnified versions of one or more region(s) of interest delimited in the above strip. Stack zooming can also be seen as a specialization of the DragMag [START_REF] Ware | The DragMag image magnifier[END_REF] that supports recursive magnification and automatic arrangement of magnification windows. The technique enables efficient exploration of time-series at multiple levels of detail. LiveRAC [START_REF] Mclachlan | LiveRAC: Interactive Visual Exploration of System Management Time-series Data[END_REF] offers another variation on this approach, organizing charts in a reorderable spreadsheet-like matrix that allows side-by-side visual comparisons at multiple levels of detail. The technique supports semantic zooming [START_REF] Cockburn | A Review of Overview+detail, Zooming, and Focus+context Interfaces[END_REF], providing visual representations adapted (cell-wise) to the allocated screen space.

Lens-based Interaction for Time-series Visualizations

Timeboxes [START_REF] Hochheiser | Dynamic Query Tools for Time Series Data sets: Timebox Widgets for Interactive Exploration[END_REF] take a different approach to the scalability problem. Timeboxes act like visual filters that specify constraints on what timeseries to display, only showing plots that intersect the one or more boxes that form conjunctive queries, enabling the dynamic exploration of large data sets by direct manipulation of the rectangular boxes. In that sense, timeboxes can be seen as data filtering lenses, that have an impact on the visualization beyond the region of interest.

SignalLenses [START_REF] Kincaid | SignalLens: Focus+Context Applied to Electronic Time Series[END_REF] provide another type of lens, conceptually closer to the usual lens-based focus+context visualization techniques [START_REF] Cockburn | A Review of Overview+detail, Zooming, and Focus+context Interfaces[END_REF]. SignalLenses are used for the visual analysis of large electronic timeseries and help perform tasks such as anomaly detection and motif discovery. They provide in-place magnification of the signal, achieving a smooth transition between the focus and context regions through 1D distortion [START_REF] Carpendale | A Framework for Unifying Presentation Space[END_REF][START_REF] Pietriga | Representation-Independent In-Place Magnification with Sigma Lenses[END_REF]. Though they can provide additional measurement tracks to assist the exploration of time-series data by computing time-aligned properties, SignalLenses are limited to the magnification of plots in the region of interest. The MagicAnalytics lenses of Kro-noMiner [START_REF] Zhao | KronoMiner: Using Multi-Foci Navigation for the Visual Exploration of Time-Series Data[END_REF] go beyond simple magnification, building on the concept of Magic Lenses [START_REF] Bier | Toolglass and Magic Lenses: the See-through Interface[END_REF] originally designed for arbitrary 2D graphics. MagicAnalytics lenses compute and display the result of a function involving two time-series. They represent one of the basic building blocks of ChronoLenses, though limited to single step transformations.

ChronoLenses go beyond these simple transformations, offering multi-step transformations of one or more time-series. These can be purely visual transformations, or transformations that change the value of the data (1st derivative, point-wise maximum, etc.). ChronoLenses enable users to construct elaborate visual analysis pipelines, with modifications automatically propagating downstream, providing immediate visual feedback that supports the iterative exploration process.

To better understand the needs for interactive visualization when exploring time-series data, we conducted informal interviews with expert users from various domains, including: astronomers, experts doing research in network streaming data analysis, finance, and weather forecasting. Based on their feedback and on the literature [START_REF] Andrienko | Exploratory Analysis of Spatial and Temporal Data[END_REF], we identified low-level tasks that users typically perform to carry out time-series analyses. From these tasks, we derived design requirements for the ChronoLenses technique, as introduced in this section.

The analysis of time-series datasets typically implies feature extraction and data comparison by transforming one or more time-series into another. Such computations usually require performing one, or a combination of, the following low-level tasks: However, exploring time-series often implies going through a more elaborate analysis pipeline that combines various tasks [START_REF] Keim | Challenges in Visual Data Analysis[END_REF]. Practically speaking, this implies that users should be able to make compound queries on the data by iteratively performing sequences of lowlevel tasks T1 and T2, combining and modifying the transformation parameters as part of the visual exploration process. From these observations we derive the following requirements: R1 Dynamic transformations. Low-level tasks T1 and T2 are the core components of the visual analysis process. The transformations that support them should be easy to perform through operators applied to the input time-series data. Visual representations and interactions that re-parameterize these transformations should be tightly integrated so as to facilitate exploratory analysis.

(a) Dynamic selection of input region of interest: users should be able to dynamically select and modify what timespan(s) in the input data are to be processed through the operators. (b) Dynamic transformation parameters: users should be able to easily configure and edit transformation parameters. (c) Immediate visual feedback: the system should provide instant visual update to reflect these changes.

R2 Dynamic analysis pipeline. Enabling the easy combination of operators makes it possible to progressively build and refine the analysis pipeline, thereby helping formulate complex queries [START_REF] Keim | Challenges in Visual Data Analysis[END_REF].

(a) Dynamic composition: users should be able to build the analysis pipeline iteratively, combining basic transformations through the incremental composition of operators that take as input arbitrary combinations of time-series from the original dataset and time-series that result from earlier transformation steps upstream in the pipeline. (b) Reuse of intermediate results and easy backtracking: The above requirement entails that all intermediate time-series transformation steps in the pipeline should be reusable as input to downstream operators, enabling users to branch out and explore, and possibly compare, alternatives at any point while sharing the data transformation steps that come earlier in the analysis process. (c) Visual representation of the pipeline: the system should provide an overview of the analysis pipeline, helping users maintain a mental map of the transformation steps and keep track of the exploration history.

We also rely on general principles for visual exploration as listed in [START_REF] Zhao | KronoMiner: Using Multi-Foci Navigation for the Visual Exploration of Time-Series Data[END_REF], including: direct manipulation; overview first, zoom & filter, details on demand; and support for dynamic multi-focus exploration.

CHRONOLENSES FRAMEWORK

ChronoLenses is an interactive visualization technique for the exploratory analysis of time-series data, whose design was driven by the above requirements. It computes on-the-fly transformations of data points and displays the result of those transformations in place, using the metaphor of lenses. The MagicAnalyticsLens technique introduced in [START_REF] Zhao | KronoMiner: Using Multi-Foci Navigation for the Visual Exploration of Time-Series Data[END_REF] relies on the Magic Lens concept [START_REF] Bier | Toolglass and Magic Lenses: the See-through Interface[END_REF] and forms one of the basic building blocks for ChronoLenses. While MagicAnalytics lenses enabled users to apply four single-step basic transformations to exactly two input time-series, ChronoLenses extends the concept, allowing for an arbitrary number of input time-series and introducing several additional operators. But most importantly ChronoLenses enable multi-step transformations to be specified, where the result of time-series transformed through a given lens can serve as input to another lens, and where multiple pipelines can be branched out to explore multiple alternative visualizations simultaneously, easing the formulation of complex queries.

Lens Parameters

We define a lens L as the transformation of an input time-series in the focus region of that lens into a resulting time-series. Time-series can be seen as streams of data that get transformed through the lenses that constitute an analysis pipeline structured as a dataflow. Transformations are computed on-the-fly according to a set of parameters that can be dynamically adjusted1 . To support tasks T1 and T2, L can either transform a single data stream, or perform cross-data stream operations. Data streams can be univariate or multivariate. Each lens L is defined by four transformations, all optional:

Unary Operator: L unary (•) defines the transformation that applies to a single data stream in the focus region of the lens (T1);

Binary Operator: L binary (•, •) defines the transformation that takes the data in the focus region of the lens (processed by L unary if set to anything else than the identity transform) as the first operand, and the output data stream resulting from the parent lens in the hierarchy (detailed later), if any, as the second2 operand (T2);

Filter: L f ilter (•, θ ) defines visual filters that hide time-series specified by parameter θ (applies to multivariate data streams only);

Scaling: L scale (•, s) determines the magnification factor s applied to the data rendered in the lens' focus.

Operators L unary and L binary perform actual computations on the input data and can take some data points outside the lens' time span, such as when computing the 1st derivative. On the contrary, L f ilter and L scale only affect the visual representation of the processed data within that time span. They do not need to access data outside it.

Pipelines of Lenses

As mentioned earlier, the output of a lens, i.e., the time-series resulting from the transformation of an input time-series, can be fed to one or more lenses. In other words, lenses can be piped, effectively creating a lens-based data flow pipeline that can be used to progressively build elaborate visual analysis interfaces. The hierarchical combination of lenses relies both on a layering system and on cross-data stream parent-child relationships between lenses. The tree structure not only helps users keep track of the sequence of exploration steps, but also makes it possible to backtrack, that is, adjust the intermediate processing stages iteratively, the system providing immediate visual feedback of the consequences of these parameter adjustments by propagating them downwards in the hierarchy. In the following, we describe the steps involved in rendering data seen through a lens, and then describe how lenses can be combined to build a full analysis pipeline.

Rendering Pipeline

Figure 2 illustrates the steps of the rendering pipeline applied to some multivariate input data, eventually leading to the visualization of the resulting time-series through lens L , that gets part of its input from a parent lens. More formally, the rendering pipeline can be described as follows. Let D = (d 1 , d 2 , . . . , d m ) T be the m-dimensional set of data points defined by the focus region of lens L . We note L op ( D i ) the result of applying one of the transformations steps to the data, where op ∈ {unary, binary, f ilter, scale}. The final result L ( D) is obtained by performing the following four computational steps: Any of the four operators can be set to null, in which case the associated step is equivalent to an identity transform. As mentioned earlier, operators L unary and L binary perform actual computations on the input data while L f ilter and L scale only affect the visual representation of the data. Thus, a simple magnifying lens can be obtained by only setting L scale . And a lens showing only one time-series in a multivariate plot can be obtained by only setting L f ilter with θ limited to that particular series. The following section focuses on the more complex computational steps C1 and C2, and describes complete analysis pipelines involving multiple lenses.

C1

Analysis pipeline

Users can specify elaborate transformations by combining multiple lenses. Two lenses L 1 and L 2 can be combined either by overlaying them in the chart, in which case the operators defining the lenses get applied sequentially to the input time-series data points (first those of L 1 , then those of L 2 ); or by declaring L 1 as the parent lens of L 2 , the transformed output of L 1 being an operand of L 2 's binary operator. Figure 3.I gives a schematic representation of these two combination mechanisms. Basic transformations that apply to a single data stream only require a lens with a unary operator (Figure 3-a) as defined in C1. Cross-data-stream computations require the definition of a child lens (Figure 3-b), that takes as input a data stream resulting from a parent lens and the data in the child lens' focus. The two streams are processed through the child lens' binary operator (C2), with the unary operator (C1) pre-processing the data points in the child lens' focus, if set. Figure 3.II illustrates all possible elementary lens combinations:

E1 Simple transformation (Figure 3-c): the most basic lens, where L unary is set and L binary is null. It is conceptually similar to a Magic Lens [START_REF] Bier | Toolglass and Magic Lenses: the See-through Interface[END_REF], with the unary operator (noted α) transforming the data in the lens' focus region;

E2 Series of transformations (Figure 3-d): two lenses that are piled up (they observe the same timespan) on separate layers as in traditional graphics editors. The first lens processes the input timeseries data points that fall into its focus through unary operator α 1 .

The resulting data is fed to the second lens and gets processed by unary operator α 2 . It is possible to pile up an arbitrary number of lenses, as with Fishkin and Stone's Movable Filters [START_REF] Fishkin | Enhanced dynamic queries via movable filters[END_REF].

E3 Cross-data computation (Figure 3-e): cross-data computations are achieved by creating a parent-child relationship between two lenses. Output data from the parent lens and time-series data points in the child lens' focus region are the operands given to binary operator β . The unary operator of the child lens is set to null (identity transform).

E4 Simple transformation and cross-data computation (Figure 3-f): both the unary operator α and the binary operator β are set. The lens first applies transformation α to the time-series data points in its focus. The result of this transformation, D 1 , is then fed to the binary operator β along with the output of the parent lens' transformation, D P .

More elaborate analysis pipelines can be built by creating hierarchies consisting of the above elementary combinations (see Section 6).

CHRONOLENSES INTERFACE

We developed a proof-of-concept, full-featured interactive visualization tool implementing the ChronoLenses technique for the visual exploration of multivariate time-series. The interface (Figure 1) consists of five main components: (A) the chart panel displaying the different time-series in a classical stacked view; (B) a lens creation toolbar; (C) a lens analysis pipeline view displaying all ancestors of the currently selected lens (D); and (E) a property panel showing the latter lens' settings. Following the design requirements listed in Section 3, the interactive visualization was designed to allow rapid exploration of time-series and construction of analytical pipelines. This section describes the interactive visual interface of our prototype. Concrete examples of use are introduced in the following section.

Figure 4-I shows the user interface of a lens L . The region of interest (time span D in the lens focus) is visually represented as a blue focus bar on the time axis (Figure 4-c). The time-series L ( D) is displayed inside the lens' frame (Figure 4-b). A toolbar located on the top border of the lens provides quick access to the lens' main parameters (Figure 4-a). This toolbar is visible only when the cursor is hovering over the lens, so as to minimize visual clutter. Detailed information of a lens is also provided in a tooltip when hovering.

The user can modify the visual representation of plots, offering different perspectives on the data. Our prototype supports classical plots such as line charts and dot plots, but also statistical plots including histograms and Q-Q normal plots for comparing data distributions.

Creating an Analysis Pipeline

Advanced analysis pipelines are typically built by creating and progressively combining multiple lenses corresponding to the desired elementary operations E1-E4 defined in Section 4.2.2.

To create a basic lens, the user first specifies its parameters through the creation toolbar, for instance choosing a unary operator such as remove means, and setting the binary operator to null. She then selects the data to be processed through the lens by initiating a rubber-band selection on any time-series loaded in the charts panel. This selection, achieved thanks to a simple mouse drag (Figure 4-e), defines the time span of interest, that will become the lens' focus.

Lenses that enable cross-data stream analyses obviously accept two input streams 3 . One stream is defined by the lens' focus, set as explained above. The second stream is provided as the output of another lens. This lens is considered as the parent lens; it feeds data to the child lens, that performs the cross-data stream analysis. The parent lens must exist in order to create a child lens.

The user can derive a child lens from the currently selected lens, by clicking on the Create Child Lens button in the creation toolbar or on the equivalent contextual menu item (Figure 1-f), after having set the desired parameters for the unary operator (if relevant) and binary operator (e.g. cross-correlation). The time span of a child lens must match that of the parent lens. This is to guarantee that input streams contain the same number of data points for binary operations, and thus make sure that cross-data stream operations are consistent. However, it would be possible to support unequal time spans between parent and child lenses, using time-series transformations that would make the number of data points between both streams consistent using, e.g., linear interpolation methods for time-series.

The created lenses are z-ordered using layers and can be piled up according to this ordering. Piling up lenses entails piping their operators as described in Section 4.2.2. In the current implementation, two lenses get actually piped when the upper lens' time span is fully contained within that of the lens underneath: the result of the latter then becomes the input of the former, as when composing Movable Filters by overlapping them [START_REF] Fishkin | Enhanced dynamic queries via movable filters[END_REF].

To help the user keep awareness of the analysis pipeline, a pipeline view is provided, depicting the currently selected lens' ancestors (Figure 1-C). The tree structure displays dynamic miniatures of the different lens branches, alongside labels detailing the operators associated with each lens in the pipeline. The current lens is visually emphasized with a thick blue border, and always corresponds to the top-left element in the pipeline view. By hovering over a lens either in the pipeline view or in the main chart view, the user can get an overall preview of all its ancestors. All lenses upstream are visually identified using an outer-glowing effect varying from green (closest ancestor) to yellow (furthest ancestor) in both views. The hovered lens is also emphasized with a blue glow effect (Figure 1-D).

Visual Exploration through Direct Manipulation

Visual exploration, the process of interactively browsing through different regions of a dataset to gain a better understanding of it, is not only useful as a quick and effective technique for hypothesis confirmation. It is also, and perhaps more importantly, essential for discovering the unexpected and raising new questions [START_REF] Tukey | Exploratory Data Analysis[END_REF]. Direct manipulation has been successfully applied to time-series interactive visualizations as a means to facilitate visual exploration in conjunction with immediate visual feedback; see, e.g., PatternFinder [START_REF] Buono | Interactive Pattern Search in Time Series[END_REF] and KronoMiner [START_REF] Zhao | KronoMiner: Using Multi-Foci Navigation for the Visual Exploration of Time-Series Data[END_REF]. ChronoLenses, by virtue of their progressive analysis pipeline building process, are meant to facilitate step-by-step exploration through direct manipulation, while giving users freedom to edit intermediate steps.

Changes to a lens in the hierarchy are automatically propagated to its descendants, and all affected plots are visually updated accordingly.

Modifying Lens Parameters

To facilitate opportunistic discovery of regions of interest, we make it possible to redefine the focus region of a lens by simply dragging the lens frame (Figure 4-g) or the associated focus bar (Figure 4-h) left or right. The immediate redisplay of the newly computed result in both the chart view and the pipeline view enables quick access to different regions of the data stream, and thereby effectively supports visual exploration. The red arrow controllers placed on the focus bar (Figure 4-f) make it possible to adjust the lens' time span. The time span of all lenses downstream is adjusted accordingly so as to keep the number of data points consistent, as explained earlier. The user can also decide to move a lens to a different chart to further her analysis on another dataset, as illustrated in Figure 4-k.

The lens frame's width can be increased using the mouse wheel. The time span considered does not change, which means that resizing affects the L scale parameter: the lens behaves as a magnifying glass 4 . Resizing on the vertical axis works similarly, except that the lens frame's height never changes and is always equal to the underlying chart's bounding box height. The y-axis origin can be adjusted by dragging the plot up and down inside the frame. An auto-adjust function is also provided, that automatically adjusts the y-axis' origin and scale so as to make the best use of available screen real-estate within the lens' frame as the user drags it.

Groups of Lenses

As mentioned earlier, the z-ordering of lenses that belong to the same analysis pipeline implicitly defines in what order transformation operations are applied to the data. The z-ordering and relative position of lenses that make a given analysis pipeline is important and should be kept constant during visual exploration via direct manipulation (when brushing the time-series plots with the lenses). Having to move each lens would be extremely cumbersome and would greatly impede the interactive visual exploration process. To avoid this, lenses can be grouped. All lenses that belong to the same group move synchronously as the user drags any one of them. The user can therefore create a compound lens set in which lenses are piled up in a specific order, thus creating a reusable analysis pipeline.

Grouping lenses can also be useful when analyzing multiple charts at a time, or when looking for seasonality. Such tasks require the interval between lenses (time lag) in the data stream to remain constant. By defining a group consisting of the different lenses to be moved synchronously, the user can keep the time lag between these constant.

A lens can be added to the currently selected group through the contextual menu (Figure 1-F). All lenses belonging to the same group as the currently selected one are outlined with a green stroke, giving feedback about the time-lag constraints that exist between those lenses. Group membership is also reflected in the tree view, where the group number is explicitly specified alongside the lens miniatures.

Dealing with Visual Clutter

The instantiation of many lenses can quickly lead to visual clutter. Lenses that are part of the same analysis pipeline will often overlay each other, either partially or fully occluding one another. This can be a problem when the user wants to visualize intermediate computational steps of the pipeline. Occlusion problems can also arise when magnification lenses (any lens with L scale > 1) look at time spans that are disjoint but close to one another.

To address these issues, we introduce a mechanism that enables the user to detach the lens frame from its associated focus bar, thereby giving her control over where to display the lens' output, independently of the actual input region of interest (focus bar location). Figures 4-g to 4-j illustrate this mechanism. To detach a lens from its focus bar, the user simply clicks on the lock icon (Figure 4-d) to unlock the lens from the focus bar. In that mode, dragging the lens frame does not affect the focus interval (Figure 4-i), making it possible to space out lenses or, on the contrary, move them closer to facilitate comparison. In the same manner, dragging the focus bar only affects the input time span, the lens frame remaining in place (Figure 4-j). Wherever a lens and its focus bar are positioned, locking them again will keep the position offset constant (Figure 4-g-h the lens, the latter hides the lens' frame, but not its focus bar. The bar remains visible so as to keep the user aware of its existence. At all times, double-clicking it makes the lens' frame visible again.

Multi-foci Exploration

As many other types of datasets, time-series are amenable to multiscale navigation. For a given series, the user might be interested in behaviors and event patterns that occur at various time scales, from months or years down to fractions of a second. Magnification lenses support the process of interactively drilling down into the data, and enable the comparison of different time spans (a common practice when analyzing time-series data [START_REF] Elmqvist | Color Lens: Adaptive Color Scale Optimization for Visual Exploration[END_REF][START_REF] Javed | Stack Zooming for Multi-focus Interaction in Time-series Data Visualization[END_REF][START_REF] Zhao | KronoMiner: Using Multi-Foci Navigation for the Visual Exploration of Time-Series Data[END_REF]) simply by instantiating multiple lenses. Magnification lenses can easily be instantiated in ChronoLenses: any lens with L scale > 1 is a magnification lens that enables drilling down into the data. However, magnification lenses pose some problems from an interaction perspective [START_REF] Appert | High-precision Magnification Lenses[END_REF][START_REF] Pietriga | Representation-Independent In-Place Magnification with Sigma Lenses[END_REF]. In-place magnification of the focus region means that the immediate surroundings of the region of interest will be occluded if the lens is simply overlaid on the original data, hiding potentially valuable information and hindering navigation. Smooth transition between the magnified focus and surrounding context can be achieved using spatial distortion [START_REF] Kincaid | SignalLens: Focus+Context Applied to Electronic Time Series[END_REF], but the introduced deformation has a cost in terms of legibility and interpretation of the visualization.

An alternative drill-down method called Stack Zooming was recently proposed by Javed et al. [START_REF] Javed | Stack Zooming for Multi-focus Interaction in Time-series Data Visualization[END_REF]. The technique consists in stacking multiple views as the user drills down into the data. The content of a given view is a magnification of a region in the view imme- diately above, delimited by a lens-like viewfinder similar to a (onedimensional) DragMag [START_REF] Ware | The DragMag image magnifier[END_REF]. The technique can seamlessly be integrated with ChronoLenses and is implemented as follows.

The user creates a new empty chart panel in the main view. Then, dragging-and-dropping an existing lens into this empty panel duplicates the data within the span of the lens and displays it in this new panel. The visual representation is stretched horizontally to fill the panel, thus providing a magnified view of the lens' content that is dynamically updated whenever that lens moves. Figure 5 illustrates this technique, with chart (b) providing a magnified version of the region seen through the lens in chart (a). Lenses can be instantiated on this new panel, enabling users to drill-down recursively and build a truly multi-scale view hierarchy in which lenses at any level can be freely adjusted. Lenses that form this hierarchy are of course not limited to simple magnification and can apply to the data any of the elaborate transformations enabled by the ChronoLenses operators.

Multivariate Time-Series

In ChronoLenses, multivariate datasets can be visualized either overplotted in the same chart (Figure 1, top chart) or separately, stacked on top of one another (Figure 1, bottom chart). When applying a lens to a multivariate data stream, each series gets processed independently, except for some operators that aggregate the data, such as point-wise minimum or maximum. In that case, the output is a univariate stream, that can be duplicated and overlaid in all charts. We distinguish aggregated operators from other operators by color-coding in red all the replicas of the unique plot resulting from data aggregation (Figure 6-b) as opposed to the one-to-one color mapping used when the different variables are processed separately (Figure 6-a).

To focus on a subset of variables, the user can filter out series that she does not want to be considered in the computational pipeline by setting the filter operator L f ilter accordingly. To do so, she can either select all the series to be taken into account in the corresponding menu accessible from the lens toolbar, or specify the list in the property panel (Figure 1-E), by entering the corresponding textual expression using a very simple syntax. When filtered out, streams are neither computed nor rendered, as the first and fourth streams in Figure 6-a. Note that when a lens defines an aggregated operator, filtering out a stream has an impact on the result, as the stream is no longer considered as an operand. Again, we offer the user full control over what data gets processed, making the analysis process highly flexible and customizable.

USE CASE SCENARIOS

In this section, we illustrate how ChronoLenses can be used with two use cases involving real datasets.

ALMA Observatory Usage Scenario

The Atacama Large Millimeter/submillimeter Array (ALMA, [START_REF] Alma | Atacama large millimeter/submillimeter array[END_REF][START_REF] Schwarz | Goodbye to WIMPs: A Scalable Interface for ALMA Operations[END_REF]) is a single telescope (under construction in the Chilean Andes) that will eventually be composed of 66 high-precision antennas. Observations are based on the principle of interferometry: a source in the sky is observed by at least two antennas; the signals (radio waves) captured by each antenna are then combined by a central computer called the correlator to form images suitable for performing scientific analysis. Time-series visualization are used by both operators and astronomers for a variety of tasks, ranging from checking some of the thousands of monitor points in the system to performing scientific data quality assurance during observations. In the following, we focus on one example where ChronoLenses can help users in their daily task.

When combining the signals coming from the different antennas taking part in a given observation, the correlator must know the length of the path traveled by the signal through the fiber optics cables with an accuracy of hundredths of a millimeter. A round-trip laser signal gets sent to all antennas in order to continuously monitor the length of the optical fibers, as the latter can expand and contract due to temperature variations. These changes in path length must be compensated in real- time. This is achieved by the Line Length Correction (LLC) system, which ensures that path lengths are all stabilized to about 1 micron. Operators are interested in monitoring the LLC stretcher voltage for each antenna, and observing potential deviation of an antenna's LLC stretcher voltage compared to that of the other antennas.

From a time-series visualization perspective, different things can be happening at different time scales: from several days to a few minutes. Efficient multi-scale and multi-focus visualization, as enabled by the L scale operator and the variation on stack zooming [START_REF] Javed | Stack Zooming for Multi-focus Interaction in Time-series Data Visualization[END_REF] implemented in ChronoLenses, is thus an essential feature. Figure 5-a plots voltage against time for 2 days at a sampling rate of 1 second.

Starting from this 2-day overview, the operator is first interested in finding out whether all antennas are behaving normally or if one or more are somehow deviating. All antennas are expected to behave more or less similarly. Direct visual comparison between the plots, overlaid or stacked, is sufficient to see the deviation of one antenna during the second day (Figure 5-a).

To better see the smaller and shorter fluctuations, the operator creates a lens spanning a 5-hour period, with L unary operator remove mean (Figure 5-a). The lens is dragged and dropped to create panel 5-b. The content of that panel is a stretched version of what is seen through 5-a's lens. The operator can clearly see that the fluctuations are in phase, which implies that they all come from a single source. She could then plot various monitoring points simultaneously based on system or environmental components likely to cause these fluctuations and create lenses defining a cross-correlation L binary operator (not shown here), eventually tracing the source to temperature fluctuations in the local oscillator room.

Finally, zooming in further to display just 50 minutes (Figure 5-c), the operator can see some odd features in the signal for one of the antennas. She filters out antennas that behave normally using a lens defining the appropriate L f ilter . She also switches from a scatterplot to a line-plot rendering inside the lens focus (Figure 5-d), revealing fast oscillations at a much lower scale that occur on top of the slow ones observed earlier for this particular antenna. This antenna-specific issue eventually gets traced to a device repeatedly turning on and off in the antenna.

Peer-to-peer Network Fluctuation Analysis

We consider a network system manager who is trying to optimize bandwidth usage on a centralized peer-to-peer system. She uses line graphs of the visiting population of two different video-on-demand channels. The relative evolution through time of the two channels, temporal distances, and the activity load before and after peak events are important clues that the analyst can rely upon, assisting her in the decision making process. For instance, identifying regular activity patterns helps better predict future fluctuations and therefore better pinpoint the needs for server pool optimization. Being able to spot anomalies such as overloads in their context also helps identify the potential causes, and plan for technical solutions. Left: the analysis pipeline, right: the corresponding lens hierarchy.

Analyzing the Evolution of Correlations

Our dataset consists of the visiting population of two different channels over a month, sampled from the server every 10 minutes (Figure 7). The overall yearly or monthly trends of the channels might be different. For instance, one might be rising much faster than the other, or one might fall while the other is rising a bit. In this first scenario, the user is rather interested in the smaller scale fluctuations, comparing evolution and identifying potential correlations at this scale. The user first focuses on a single channel, looking for repeating patterns. To do so, she compares differentiated data at different times by performing the following steps: 1) remove mean, to make the data more stationary and thus more amenable to comparison for our purposes; 2) compute the first derivative (or differencing) of those; and 3) perform the cross-correlation between the two for comparison. The pipeline that supports this type of analysis, described below, is composed of four lenses, as shown in Figure 8.

The user creates lenses L1 and L2, with the L unary operator set to remove mean (step 1). She then creates L3 (L unary operator set to 1st derivative) and places it on top of L1 (step 2). Its child lens L4 gets positioned on top of L2 with operators L unary and L binary set to 1st derivative and cross-correlation respectively (steps 2-3). In the end, L4 outputs the correlation of the data below it (1st derivative of L2) and its parent's output (L3). The discovery of recurring patterns usually requires performing computations on data with parameterized-yet-fixed lags in time. Here, we use lens groups -that make all member lenses move synchronously -to build dynamically-parameterized operators enabling efficient exploration of the effect of different time lags and time spans via direct manipulation.

Grouping L1 and L3 on the one hand (L1-3), and L2 and L4 on the other hand (L2-4) makes it possible to drag one or the other focused intervals along time and look for strong correlation by observing the result of L4. When such a correlation is identified, as in Figure 7-a, the user can group L1-3 and L2-4 altogether and drag them. If L4 exhibits a stable plot while dragging, then a recurrent trend is identified. If on the contrary the plot varies, direct manipulation with immediate feedback helps better understand the correlation variation before and after peaks of interest, whereas it might be more difficult to apprehend when computing automatic pattern detection.

When the analyst has located a pattern of interest, she can focus on the second channel to look for correlations. She creates pipeline L5-6 similar to group L2-4 using the data from the second channel as input: L5 (remove mean) and L6 (L3's child placed on top of L5 with 1st derivative and cross-correlation operators), as depicted in Figure 7-a.

Different analysis tasks can easily be performed with the current set of lenses: (1) drag the group L5-6 until the plot is like, or the opposite of, L2-4 to identify correlations between the two series (see Figure 7a); (2) group L2-4 and L5-6 so that they preserve their relative distance while dragging one or the other in order to compare the correlation fluctuations by simultaneously looking at the results of lenses L4 and L6. If those behave the same, then a similar evolution is identified and the relative lag is known. If they always behave the opposite, there is an inverse causality; and (3) by extending the group to L1-3 and dragging the whole, the analyst can also explore if a pattern is preserved in time. The user can also change the lenses' operators at any time during exploration, as for example changing L3, L4 and L6's L unary operator to 2nd derivative and looking for insights when more lag is involved (20 minutes in our example).

Comparing different Time Spans in the same Series

Discovering seasonality is another important task in time-series data analysis. It helps better predict future fluctuations. In this second part of the scenario, our user is interested in finding seasonal differencing on the two channels simultaneously, in order to gain a better understanding of what characterizes the typical data streaming cycle, if such a cycle exists. Gaining this knowledge will eventually help develop strategies to optimize bandwidth usage, but also assist in spotting abnormal behavior when it occurs.

Box-Cox Transforms [START_REF] Box | An Analysis of Transformations[END_REF] can be used to perform a variance stabilizing transformation. Seasonal differencing parameterized with a time lag can then be applied to find out if the non-stationarity of the original time-series is removed [START_REF] Madsen | Time Series Analysis[END_REF], but this requires to know the lag value in advance. The identification of an appropriate lag value to perform seasonal differencing can be made easier by building the following pipeline: an auto-correlation lens (L1) on top of a Box-Cox transform lens (L2). A strong correlation is observed at a delay of approximately 1440 minutes = 24 hours (see Figure 7-b).

The analyst deletes L2 and furthers her exploration as she creates L2's child lens L3 with Box-Cox transform and subtraction, synchronized so that L3 is always 24 hours ahead of its parent (Figure 7-c). An auto-correlation lens is then applied on top (after the differencing operation) to evaluate stability of the series. At this position in the original time-series (Figure 7-d), there does not seem to be any significant seasonal pattern, as high correlation values exist only at zero-delay. But moving the lens group along the timeline, one pattern is eventually discovered (Figure 7-e), calling for further exploration.

DISCUSSION

ChronoLenses aim at facilitating the creation of customized analysis pipelines for easy exploration and navigation through time-series datasets. Our initial prototype, although already offering a large set of functionalities, still has limitations. It could be improved in several ways, as discussed in this section.

Layering and Treeview Limitations

In its current implementation, the ChronoLenses interface does not support effective layer management for the z-ordering of lenses that pile up. Although the grouping of lenses keeps the z-ordering constant, when not grouped, selecting a lens puts it on the topmost layer. This might be annoying when dealing with complex pipelines as clicking on a lens only to get information about it or even by mistake can alter the pipeline. Making layer management more stable and more easily configurable through a dedicated synchronized view would reduce the need for advanced planning when building the pipeline and would aid keep an accurate mental map of the data flow.

Similarly, the lens hierarchy treeview could be enhanced. As is, the hierarchical view dynamically changes as the user selects a different lens. This might be distracting and difficult to interpret. Moreover it is not interactive. There is an opportunity for improvement here, as an additional and complementary representation of the analysis pipeline could be used as an alternative means of performing exploration tasks if enriched with interactive capabilities. In particular, allowing for the creation of lenses, duplication of analysis branches, or other changes to the analysis pipeline from this alternate view would make it easier to build up and manage complex pipelines.

Integration of SigmaLens Focus+Context Techniques

Another limitation of the current implementation is related to the positioning of lenses. When displaying an overview of large time-series, ChronoLenses only draws a subset of all points (subsampling). Magnifying the data through a lens enables the display of more detail in context. However, as the lens' magnification factor increases, small movements of the lens translate to larger jumps in terms of time span observed in the lens' focus. For instance, given a magnification factor of 10x, moving the lens by 10 pixels will move the time span observed through the lens by 100 equivalent pixels at that zoom factor (see [START_REF] Appert | High-precision Magnification Lenses[END_REF] for a detailed description of this problem of quantization). The problem also exists when controlling the time span seen in a plot through a lens observing a lower scale version of that plot à la Stack zooming. Adapting one of the high-precision magnification lens techniques introduced in [START_REF] Appert | High-precision Magnification Lenses[END_REF] would solve this issue, enabling both fast repositioning and precise selection within the lens focus.

A related issue is that our lenses occlude the area immediately surrounding the region of interest, as basic magnifying lenses do. Sig-nalLenses [START_REF] Kincaid | SignalLens: Focus+Context Applied to Electronic Time Series[END_REF] address this issue by distorting the representation in a bounded transition region between the lens focus and the context. We chose not to rely on spatial distortion because of the issues it raises in terms of making accurate analytical comparison and interpretation. However, there are other solutions to this problem, such as speedcoupled Sigma Lens focus targeting techniques [START_REF] Pietriga | Representation-Independent In-Place Magnification with Sigma Lenses[END_REF]. For instance, the Speed-coupled Blending Lens technique consists in coupling the lens focus' opacity to its speed, smoothly fading out the lens as speed increases and smoothly fading it back in when it comes to a stop, having reached its target position. The technique addresses the problems of visual occlusion and would be relatively straightforward to implement in ChronoLenses as we already support translucency in lens renderings. However, potential issues related to visual interference between the layers during lens repositioning would have to be investigated.

CONCLUSIONS AND FUTURE WORK

We have presented ChronoLenses, a novel domain-independent visualization technique for the visual exploration of time-series data. ChronoLenses relies on the metaphor of lenses, that compute onthe-fly data transformations in place. ChronoLenses allow users to progressively build elaborate analysis pipelines by interactively compounding elementary operations, thus supporting complex userdefined exploratory analysis tasks.

The concept of ChronoLenses can be extended to other types of data than time-series, e.g., image analysis and image processing techniques for exploring temporal media such as video, where contentaware operators (feature detection algorithms, sharpening and color correction filters) would complement more generic operators (magnification, track filtering).

In addition to exploring new applications of the framework, we plan to improve our implementation by addressing the limitations discussed in the previous section, and by exploiting analytical mechanisms to guide exploration. For instance, lenses could be made to snap to local optima in the visualized time-series as the user drags them, using measures such as, e.g., the strongest local cross-correlation.

Fig. 3 .

 3 Fig. 3. Schematic representation of the ChronoLenses combination mechanism (I), and all possible elementary combinations (II).

Fig. 2 .

 2 Fig. 2. ChronoLenses rendering pipeline for lens L . (P) processing pipeline in parent lens; (1) applying Unary Operator L unary ; (2) applying Binary Operator L binary ; (3) applying Filter Operator L f ilter ; (4) applying Scaling Operator L scale .

  Apply unary operator L unary , such that D 1 = L unary ( D) (Figure 2-1); C2 Apply binary operator L binary , such that D 2 = L binary ( D 1 , D p ) (Figure 2-2), where D p is the set of data points resulting from applying the parent lens to the same or to another set of input data points (Figure 2-P); C3 Apply visual filter operator L f ilter such that D 3 = L f ilter ( D 2 , θ ), where θ defines the subset of time-dependent variables to be rendered in case of multivariate input (Figure 2-3); C4 Apply scaling operator L scale such that D 4 = L scale ( D 3 , s), where s defines the magnification factor (Figure 2-4). We note L ( D) = D 4 the final result.

Fig. 4 .

 4 Fig. 4. (I) UI of a ChronoLens: (a) toolbar enabling adjustment of the lens' main parameters, (b) tooltip (pops-up when hovering), (c) focus bar with resizing handles, (d) lock icon for detaching/reattaching the lens' frame from/to the focus bar; (II) associated interactions.

Fig. 5 .

 5 Fig. 5. Lenses applied to multivariate data: (a) auto-correlation for three series out of five; (b) point-wise aggregated maximum showing the resulting plot (common to all series) overlaid on top of each variable's plot.

Fig. 6 .

 6 Fig. 6. Exploring ALMA Line Length Correction Stretcher Voltage plots for four antennas: (a) two-day overview at a sampling rate of one second; (b) magnification of the 5 hours seen through the remove mean lens applied in (a); magnification of the 50 minutes for a single antenna seen through a filtering lens, rendered in scatterplot mode (c) and line-plot mode (d).

Fig. 7 .

 7 Fig. 7. Analysis of a month of the visiting population of two P2P video-on-demand channels: a) analysis of the evolution of the correlation within and between the channels (see Section 6.2.1); b) looking for seasonal pattern lag value, the blue arrows show a strong correlation at a delay of 1440s (=24h) and c-d) comparing the delayed time spans (see Section 6.2.2).

2 RFig. 8 .

 28 Fig. 8. Comparing the evolution of two time-series using ChronoLenses. Left: the analysis pipeline, right: the corresponding lens hierarchy.

ChronoLenses support the representation of stacked multivariate timeseries. In that case, each individual series is processed separately using the L transformation, and is then stacked in the lens' frame.

[START_REF] Aigner | Visual Methods for Analyzing Time-Oriented Data[END_REF] Most of the operations that consider more than two input data streams can usually be simulated by cascading a series of binary operations.

As explained in Section

4.2.1, these streams can contain multivariate timeseries. The binary nature of the transformation does not mean that input timeseries are restricted to two variables. See Section 5.3 for more information about handling multivariate time-series.

The above-mentioned blue bar still shows the original region of interest.

ACKNOWLEDGMENTS

We wish to thank all expert users that participated in our interviews and gave us feedback about the ChronoLenses tool, including: Denis Barkats from the Joint ALMA Office, National Radio Astronomy Observatory; Lindsey Davis from the National Radio Astronomy Observatory, Norbert Driedger and Brian Greaves from Environment Canada; Yashar Ganjali, from the Computer Systems and Networks Group at University of Toronto; Igor Jurisica from the Ontario Cancer Institute; Di Niu, from the Electrical and Computer Engineering department at University of Toronto; and Joseph Schwarz from the European Southern Observatory. We also thank Theophanis Tsandilas for helpful comments about early drafts of this paper.


Michel Beaudouin-Lafon

Stéphane Huot
email: stephane.huot@lri.fr.

Mathieu Nancel
email: mathieu.nancel@lri.fr.

Wendy Mackay
email: wendy.mackay@lri.fr.

Emmanuel Pietriga
email: emmanuel.pietriga@inria.fr.

Romain Primet
email: romain.primet@inria.fr.

Julie Wagner
email: julie.wagner@lri.fr.

Olivier Chapuis
email: olivier.chapuis@lri.fr.

Clément Pillias
email: clement.pillias@lri.fr.

James R Eagan
email: james.eagan@telecom-paristech.fr.

Clemens Klokmose
email: clemens@cs.au.dk.

Multisurface Interaction in the WILD Room

The WILD room (wall-sized interaction with large datasets) serves as a testbed for exploring the next generation of interactive systems by distributing interaction across diverse computing devices, enabling multiple users to easily and seamlessly create, share, and manipulate digital content.

Ubiquitous computing offers a vision in which each person owns multiple computers that work together seamlessly, embedded into the fabric of everyday life [START_REF] Weiser | The computer for the 21st century[END_REF]. Part of this vision has arrived: interactive surfaces are everywhere, from smartphones, tablets, and laptops to large-screen televisions and smart boards; from car navigation systems to fitness monitoring devices. Their integration, however, is hardly seamless: data is often trapped in individual applications or services, and interaction is usually limited to a single device at a time.

As the "The WILD Platform" sidebar describes, the WILD room (wall-sized interaction with large datasets) is a multisurface environment featuring a wall-sized display, a multitouch table, and various mobile devices that we designed to help scientists collaborate on the analysis of large and complex datasets. We combine empirical studies, participatory design, and fundamental research on basic interaction tasks to explore the design and engineering of the next generation of interactive systems. The key to this approach is to distribute interaction, not just content, across a variety of interactive surfaces.

Designing with extreme users

Our research strategy involves designing an extreme environment that pushes the limits of technology-both hardware and software. To ground the design process, we needed extreme users-people whose daily work both inspires and stress-tests the environment. We chose scientists who use a variety of techniques to understand exceptionally large and complex datasets. We invited researchers from the Paris-Saclay campus in astrophysics, particle physics, chemistry, molecular biology, neuroscience, mechanical engineering, and applied mathematics to an initial "show-and-tell" workshop. Scientists from each lab presented specific examples of the challenges they faced at that time, along with their data analysis processes and tools. We discussed the similarities and differences among their approaches, seeking to identify both universal needs and unique opportunities.

For example, a group of microbiologists might arrive in the WILD room with their laptops and analysis tools to study how one molecule docks with another. One might bring up a large molecular model downloaded from the research lab's server, another might add interactive 3D models of related molecules, and others might access online databases, websites, and research articles. They could shift smoothly among different representations of each molecule and transfer them from one interactive display to another, working together in the same room or collaborating with remote colleagues.

We identified four common strategies for managing complex scientific data where the WILD multisurface environment could significantly improve and even completely change work practices:

1. navigation through a single, very large object, such as a simulation of a molecule with tens or hundreds of thousands of atoms or a gigapixel image of deep space containing thousands of galaxies; 2. comparison of a large number of related images, such as pathological brain scans or observations of regions of the sky at different wavelengths; 3. juxtaposition of a variety of heterogeneous forms of data from different sources, such as a mix of research articles, raw data tables, formulas, graphs, photographs, and video clips; 4. communication with remote colleagues about all of the above to facilitate collaborative exploration.

We then used the WILD room as a working laboratory for exploring advanced multisurface interaction techniques.

Sidebar: The WILD platform

The WILD room (Wall-size Interaction with Large Datasets) features a large wall display (top, left) powered by a 16-computer cluster (top, right) and two front-end computers, a motion tracking system (bottom, left), and an interactive table (bottom, right).

The wall display consists of 32 off-the-shelf 30-inch monitors organized in an 8 x 4 grid, for a total resolution of 131 million pixels (20480 x 6400). The high pixel density (about 100 dpi), a defining characteristic of WILD, is rare on wall displays. The monitors are mounted on four movable carts, letting users test different configurations such as the triptych shown in The interactive table uses FTIR (frustrated total internal reflection) technology to track up to 32 simultaneous contact points with a 1920 x 1080 resolution. Because it has only half the pixel density of the wall display, we are adding a second table with higher pixel density and a flat screen. Smartphones, PDAs, tablets, and laptops provide additional, personal interactive surfaces. We also use input devices such as gyroscopic and wireless mice and custom devices.

Exploring multi-surface interaction

We employ two complementary strategies for generating and testing ideas: participatory design, which focuses on qualitative understanding and external validity, and controlled experiments, which focus on quantitative evaluations and internal validity.

Participatory design actively involves users throughout the design process. We visited several labs to observe their current research procedures and conducted participatory design workshops in the WILD room with the astrophysicists and neuroanatomists, who face interestingly different analysis challenges.

One of the most effective techniques was the Wizard of Oz, in which scientists acted out ideas for manipulating their data, using paper images, laptops, and other props. A member of the group, identified as the wizard, would operate the WILD wall so that it reacted to the users' actions, creating a compelling shared experience of a possible future. This often sparked additional ideas and provided insights as to which techniques were most worth pursuing. For example, the scientists spontaneously experimented with midair hand gestures and using external props to manage their data. One neuroscientist brought along a 3D physical model of his own brain from an MRI scan. He had the idea of using it to control the orientation of all 64 normal and pathological brains displayed on the wall. He had dreamed of doing this in his lab, where he was limited to using a mouse to compare at most four brain scans on a single screen.

Scientists also explored relationships among mobile and stationary devices. For example, one astrophysicist was examining a large image of the Milky Way galaxy, accompanied by a series of smaller images at different wavelengths. He suddenly grabbed an iPad tablet, held it up to the primary image, and simulated how he would like to treat it as a physical, interactive filter. Figure 1 shows how he envisioned moving the tablet around, maintainng an overview of the whole image while flipping through different filters to focus on specific wavelengths.

Participatory design helped us to delve deeply into the problem space and generate specific innovative ideas. However, we also needed a more systematic approach for characterizing the design space of interaction techniques and making informed choices. For example, the astrophysicists showed us a 400,000-pixel-wide image of the center of the galaxy. While they could see it on WILD much better than in their lab, the image was still 20 times larger than the display capabilities of our wall.

These and other gigapixel images highlighted the need for powerful panning and zooming techniques that could be operated from any location in front of the wall. This suggested midair interaction, using the hands to point to the locus of the zoom within an image and to control its expansion and contraction from there. Based on the participatory design results and our own explorations, we identified three important dimensions, illustrated in Figure 2, that characterize the design space for pan-and-zoom on a wall display.

We ran a controlled experiment to evaluate our hypotheses about which factors increase performance, accuracy, and comfort [START_REF] Nancel | Mid-air pan--and--zoom on wall--sized displays[END_REF]. Our goal was not necessarily to determine the single "best" technique, but rather to understand the tradeoffs and help users and designers decide which to use under what circumstances.

We found that, in general, two hands are better than one; linear gestures are faster than circular ones, despite the need to "clutch;" and greater guidance (or fewer degrees of freedom) significantly increases performance. Most midair freehand gestures are tiring and inefficient. The only exception is the two-handed linear gestures in free space shown in Figure 2f-an appealing technique that requires no additional device.

These and other experiments, together with the results of the participatory design sessions, have led to an effective set of techniques that we now use routinely in WILD.

Developing multi-surface applications

Developing software for multisurface environments raises several challenges. First, applications are inherently distributed and the environment is dynamic: 20 to 30 computers are involved in a typical session, including the cluster running the wall, the computers running the table and motion tracking system, the handheld devices, and the users' laptops. Second, input devices can be combined in various ways to interact with the various surfaces, and multiple users must be able to interact in parallel. Finally, content comes from a variety of sources, including static documents brought by users and live windows from legacy applications.

Our goal was to simplify the development of applications in this context without sacrificing the flexibility and openness required by our users. This led to a modular approach that separates user interaction, graphical rendering, and content sources.

Distributed interaction

Our concept of ubiquitous instrumental interaction separates interaction from the rest of the application [START_REF] Klokmose | VIGO: Instrumental interaction in multi-surface environments[END_REF]. An interaction instrument mediates interaction between a user and the objects of interest. For example, users can designate objects with a pointing instrument, move them with a drag-and-drop instrument, and change their color with a color selection instrument. Instruments are independent of the objects they operate on: they need only know that the object implements a given protocol, such as selecting, changing position, or setting a color. Multiple instruments can be used in parallel. Instruments can also be embodied in portable devices-for example, a smartphone used as a laser pointer. In this case, the instrument runs on the device and interacts with objects located on other surfaces.

We have created generic instruments for selecting, moving, organizing, and annotating objects, as well as more specific ones, such as the brain prop shown in Figure 3, which is used to control the orientation of brain scans on the wall. These interaction instruments have proven very flexible since they can be customized to the users' needs without modifying the application: instruments discover which objects they can interact with based on the protocols that the objects implement.

(left) A jBricks application manages a scene of 2D objects laid out on an infinite canvas. On the cluster, render servers replicate the scene and display only the objects that lie in their viewing frustum. (right) A configuration of the WILD Input Server for a virtual device combining a VICON position-tracking component and an iPod handheld device. The configuration can be tested outside the WILD room by replacing the VICON component with those in gray and using a mouse for position input. The pan-zoom component on the right sends highlevel events to the application.

At a lower level, input in a multisurface environment can come from a variety of sources, including standard devices such as mice and keyboards, multitouch devices such as interactive tables and tablets, and systems such as motion trackers. Rather than sending this raw input directly to applications or instruments, we have created an intermediate layer called the WILD Input Server [START_REF] Pietriga | Rapid development of user interfaces on cluster--driven wall displays with jBricks[END_REF].

The WILD Input Server uses the ICon visual editor [START_REF] Dragicevic | Support for input adaptability in the ICon toolkit[END_REF] to create and edit input configurations. Figure 4 shows how a configuration transforms low-level input from physical devices into higherlevel events sent to client applications. The WILD Input Server supports standard protocols such as USB-HID, OSC, TUIO, and VRPN as well as devices such as LiveScribe interactive pens or the VICON motion tracker. The server sends events to applications through various protocols (primarily OSC; http://opensoundcontrol.org) or plug-ins. Applications can also remotely control the server to start, stop, or change a configuration or to load a plug-in.

Developers can easily create and modify configurations by assembling components such as filters, adapters, and flow controllers, even during a prototyping session. Configurations typically define virtual devices that aggregate input from multiple sources. For example, the application sees a multitouch handheld device whose 3D position is provided by the motion tracking system as a single device.

Our implementation of the pan-and-zoom techniques from Figure 2 illustrate the flexibility of this approach. We developed the techniques outside the WILD room, substituting a mouse or a Wiimote for the motion tracking system, and created a set of virtual devices that we could modify and fine-tune in the WILD room, without relaunching the application.

Distributed rendering

Displaying graphics in a multisurface environment is challenging because users want to organize their data onto a virtual canvas that spans multiple surfaces. Depending on the configuration and the task at hand, different surfaces display either the same part or different parts of the canvas. Tiled displays require particularly high performance to create the illusion of a single, continuous surface with no tearing.

Existing cluster-based systems for distributed rendering do not fit our requirements. For example, Equalizer and CGLX require adapting or rewriting applications using OpenGL, while SAGE [START_REF] Jeong | Ultrascale collaborative visualization using a display--rich global cyberinfrastructure[END_REF] uses pixel streaming and therefore cannot take full advantage of ultra-high resolution wall displays. Our approach uses replication: each machine driving a display runs a replica of the complete application or a rendering client that holds a copy of the scene. Each replica knows which part of the scene to display; a master application synchronizes changes to the scene and the viewing camera.

We created two frameworks to develop multisurface applications based on this model. The first, jBricks [START_REF] Pietriga | Rapid development of user interfaces on cluster--driven wall displays with jBricks[END_REF], is based on a 2D scene graph that describes the canvas's content and a set of reactions that describe how to respond to user actions, similar to traditional user-interface toolkits. Scene graph objects include geometric shapes, text, images, and Java Swing widgets laid out on an infinite canvas and observed through one or more cameras.

jBricks uses a replicated approach to render the scene graph on a cluster-driven tiled display. The toolkit supports smooth real-time panning and zooming of very large information spaces, including gigapixel images, as well as interactive visual effects such as magnifying lenses. By making distribution transparent to the application, jBricks greatly lowers the barrier to developing multisurface applications.

Our second framework, Shared Substance, takes a different approach by making distribution explicit [START_REF] Gjerlufsen | Shared Substance: developing flexible multi--surface applications[END_REF]. A Shared Substance application is a collection of processes called environments that run on different machines. The application discovers environments dynamically, and they can appear and disappear at any time. Each environment contains a hierarchical data structure that it can share, in whole or in part, with other environments.

An environment accesses a shared subtree either by replicating it and accessing the local copy or by mounting it and accessing the original through remote procedure calls. Environments can use facets to dynamically add functionality to a shared subtree. For example, Figure 5 shows how our Substance Canvas application uses facets to display the canvas, modify its content, and support interaction. Shared Substance provides great flexibility and makes it possible to create applications that dynamically adapt to their use context and are reconfigurable at runtime.

Distributed content sources

In a multisurface environment, users need to juxtapose content from multiple sources, as if the various surfaces were extensions of their laptops. Sources include passive documents such as PDF files and images, active documents such as webpages, and live applications such as data analysis and visualization programs. The challenge lies in integrating such heterogeneous sources into a unified environment.

We began with simple but effective solutions based on conventional tools: a user can e-mail a document to WILD to display it on the wall or "print to the wall" by sending a document to a printer queue that WILD monitors. Users can also fill out a simple Web form or use a bookmarklet to display webpages on the wall. Even so, scientists must be able to use existing applications. Since porting them to our frameworks is not practical, at least in the short term, both jBricks and Shared Substance support the display of live applications running on a different computer, typically a user's laptop. For Linux, we use Metisse [START_REF] Chapuis | Metisse is not a 3D desktop![END_REF] to send pixel-based representations of the windows. For Mac OS, we use Scotty [START_REF] Eagan | Cracking the cocoa nut: user interface programming at runtime[END_REF] to send vector-based representations of the windows, resulting in smooth scaling when displayed on the wall. In both cases, the scientists can use an instrument that simulates a mouse to interact with the teleported applications.

An alternative with better performance is to run the legacy application on the WILD cluster itself. Using Shared Substance, we wrapped the BrainVISA 3D visualization application (http://brainvisa.info) into an environment that shares the address of the scan being displayed and the position of the virtual camera controlling its orientation. Figure 3 shows the cluster running 64 such environments, each displaying a different brain scan. The table runs an instrument for organizing the brain scans, while the brain prop controls the orientation of a master camera, which is shared by the 64 environments that display the individual brain scans.

The resulting application was created in a few days, providing neuroanatomists with a unique tool to study the brain. We used a similar approach with the PyMol molecule viewer. We can display a single molecule on the full wall by having each replica display its part. Rotating it in real time shows no visible tearing.

By distributed content, rendering, and interaction we have created a modular architecture that simplifies the development of multisurface applications while supporting flexible interaction as well as legacy content and applications. Even without optimization, performance is good: users can interact with full-wall images in real-time with little perceivable lag. The ability to change configurations and components on the fly during a design session makes these tools an excellent platform for rapid prototyping.

Sidebar: Recommended Reading

Researchers have long been interested in room-scale interaction. An early project was the Stanford iRoom [START_REF] Borchers | Stanford interactive workspaces: a framework for physical and graphical user interface prototyping[END_REF], an infrastructure that enabled the devices in a room to communicate with each other. Lucia Terrenghi and colleagues provided a comprehensive taxonomy of different scales of multisurface environments [START_REF] Terrenghi | A taxonomy for and analysis of multi--person-display ecosystems[END_REF], from wristwatches and phones to the side of a building. These environments support users interacting in isolation or simultaneously, in parallel or collaboratively.

At the room-sized scale of this spectrum, much work has focused on creating large highresolution displays such as wall-sized tiled displays and CAVEs. These projects often focus on high-performance distributed rendering and data-sharing rather than on interaction. Tao Ni and colleagues surveyed the technologies and application for such environments and emphasized the need for better interaction techniques [START_REF] Ni | A survey of large high--resolution display technologies, techniques, and applications[END_REF]. Our work addresses these issues by introducing concepts and techniques for distributed, multisurface interaction [START_REF] Klokmose | VIGO: Instrumental interaction in multi-surface environments[END_REF][START_REF] Pietriga | Rapid development of user interfaces on cluster--driven wall displays with jBricks[END_REF][START_REF] Gjerlufsen | Shared Substance: developing flexible multi--surface applications[END_REF].

Conclusion

Realizing the vision of ubiquitous computing requires creating interaction architectures and paradigms that harness the power of combining devices and services into integrated environments. Today's smartphones, tablets, multitouch tables, and wall displays bring little more than the sum of their parts. In contrast, the WILD room's multisurface interaction paradigm illustrates how interaction, not just content, can be distributed across multiple devices.

The scientists we have worked with are eager to use WILD for their daily work. By involving them in the design process, we have been able to focus on their real needs and identify the real technological challenges. We have learned the following lessons in the process:

• decouple tools from one another and use simple protocols to facilitate their integration; • focus on interaction rather than rendering, and assume that hardware will provide sufficient performance; • leverage existing tools when possible, but also develop from scratch when needed; and • explore alternative designs to gain deeper understanding of their respective advantages and disadvantages.

However, this is just the beginning. We must work with additional user groups to gain new insights and expand the scope of multisurface interaction, extend our interaction vocabulary to match the richness of desktop interfaces, and scale our software architectures to test them with other applications.

One important requirement not currently addressed by WILD and unanimously requested by our users is support for collaboration among remote colleagues. While the multisurface interaction paradigm naturally scales to remote groups, additional technology is needed to support face-toface communication. The WILD room is now part of Digiscope (http://digiscope.fr), a larger project that will create a network of interactive visualization rooms specifically designed to address these issues.

In the long run, platforms such as WILD will become increasingly affordable. Wall-sized displays will combine high-definition and multitouch surfaces without borders, and motion tracking will become more reliable, without the need for markers. These advances will reduce the constraints on users and support a wider range of multisurface interactions.

  Figure A. Each computer has two graphics cards driving one screen each. Displaying wall-sized images requires distributed software that runs across the cluster. The motion tracking system uses 10 infrared cameras to detect the position of passive markers attached to different devices, such as the T-shaped tool shown at the lower left in Figure A. The system has very low latency and a precision of less than one millimeter across the room. We typically use it to precisely track each device's position and to support advanced interaction techniques.

Figure 1 :

 1 Figure 1: Using the Wizard of Oz technique to prototype how a tablet can serve as a mobile, physical filter atop a wall-sized image.

Figure 2 :

 2 Figure 2: A design space for midair pan and zoom techniques with three dimensions: interaction with one hand (top row) or both hands (bottom row); gestures that are constrained to one dimension (left column), to a 2D surface (center column), or free in 3D space (right column); linear or circular gestures (insets in each cell). For example, (d) corresponds to using the dominant hand as a laser pointer to indicate the focus point and the nondominant hand to control zooming with linear or circular gestures on a handheld device. In (c), both tasks are carried out with the dominant hand.

Figure 3 :

 3 Figure 3: Interaction instruments. (left) An interaction instrument sorts the 64 displayed brain scans, (center) a brain prop controls the scan orientation, and (right) a digital pen annotates content on the wall. (Source: Photothèque CNRS, Cyril Fresillon.).

Figure 4 :

 4 Figure 4: jBricks and the WILD Input Server. (left) A jBricks application manages a scene of 2D objects laid out on an infinite canvas. On the cluster, render servers replicate the scene and display only the objects that lie in their viewing frustum. (right) A configuration of the WILD Input Server for a virtual device combining a VICON position-tracking component and an iPod handheld device. The configuration can be tested outside the WILD room by replacing the VICON component with those in gray and using a mouse for position input. The pan-zoom component on the right sends highlevel events to the application.

Figure 5 :

 5 Figure 5: Substance Canvas application. (left) Two users share content between the wall, the table and a laptop. (right) A master environment shares a scene graph representing a canvas. Rendering environments replicate the scene graph to add local rendering capabilities, while interaction instruments mount the scene graph to add editing functions. Content providers then mount the scene graph to modify its content, for example, through a webservice. (Source: INRIA.)

We anticipate that this technology will become prevalent in the workplace, first in meeting rooms and design studios, then in offices, and later in the home, offering families new ways to play, study, communicate, and enjoy entertainment. Only then will multisurface interaction become truly integrated into the fabric of our everyday lives.

Acknowledgments

We thank our partner laboratories, in particular IAS (astrophysics), LAL (particle physics), IGM (biology) and Neurospin (neuroscience) for their participation. WILD is supported by a Région Île-de-France/Digiteo grant and by Université Paris-Sud, INRIA, CNRS, ANR and the INRIA-Microsoft joint laboratory.