Jiali Liu Ltci
email: jiali.liu@telecom-paristech.fr

Nadia Boukhelifa
email: nadia.boukhelifa@inra.fr

James R Eagan Ltci

Understanding Alternatives in Data Analysis Activities

Keywords: Data science, sense-making, visual analytics

Data workers are non-professional data scientists who engage in data analysis activities as part of their daily work. In this position paper, we share our past work and our multidisciplinary approaches on understanding data workers' sense-making practices and the human-tool partnerships. We introduce our current research ideas on the role of alternatives in data analysis activities. Finally, we conclude with open questions and research directions.

Introduction

Data analysis shares similar characteristics with experimental [START_REF] Tukey | Data Analysis and Statistics: An Expository Overview[END_REF] and design processes [START_REF] Peng | The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis[END_REF]: both are open-ended, highly interactive, and iterative-where a broad space of possible solutions are explored until a satisfactory result emerges [START_REF] Hartmann | Design as exploration: Creating interface alternatives through parallel authoring and runtime tuning[END_REF][START_REF] Beth Kery | Variolite: Supporting Exploratory Programming by Data Scientists[END_REF]. As such, data workers generate and compare multiple schemas [13], formulate solutions by combining parts from different exploration paths [START_REF] Beth Kery | Variolite: Supporting Exploratory Programming by Data Scientists[END_REF], and deal with uncertainty that could arise at different layers of the analysis [START_REF] Boukhelifa | How Data Workers Cope with Uncertainty: A Task Characterisation Study[END_REF]. Moreover, multiple types of data workers need to collaborate together to perform more complex analytic tasks [START_REF] Boukhelifa | An Exploratory Study on Visual Exploration of Model Simulations by Multiple Types of Experts[END_REF][START_REF] Kandel | Enterprise Data Analysis and Visualization: An Interview Study[END_REF].

While real-world analyses tend to be messy and complex, today's tools still largely rely on single-state models involving one user at a time. This disconnect contributes to making analytic practices cumbersome and increases cognitive load [START_REF] Lunzer | Subjunctive Interfaces: Extending Applications to Support Parallel Setup, Viewing and Control of Alternative Scenarios[END_REF][START_REF] Beth Kery | Variolite: Supporting Exploratory Programming by Data Scientists[END_REF]. Worse still, a lack of support for exploration of alternatives, explicit management of uncertainty in analysis, and support for collaboration in sense-making can lead to bad problem solving and decision-making [START_REF] Mclellan | Presidential Decisionmaking in Foreign Policy: The Effective Use of Information and Advice[END_REF].

Our research more generally focuses on understanding how data workers make sense of data and on providing them with rich, flexible tools that are adapted to the highly situated nature of such interactions. Our current work focuses more specifically on understanding the role of alternatives in data workers' analysis practices.

In this position paper, we describe our past and on-going work on understanding data workers' practices. We describe our background and our general approach , and close with open research questions drawn from our experience.

Background & Approach

We are researchers in information visualisation and more generally human-computer interaction, with a background in computer science. One co-author is a researcher in an applied laboratory for agronomics research, while the other two are in an engineering school.

Our work focuses on data workers: professionals who engage in data analysis activities as part of their daily work but who would generally not characterize themselves as a data analyst or data scientist. Our approach tends to combine different qualitative methods to understand these users' needs and practices, tool-building to address these needs, and other qualitative and quantitative evaluation methods.

In prior work, for example, we conducted interviews and walk-through scenarios with data workers to understand how they think about and manage the various kinds of uncertainty that arise in their work [START_REF] Boukhelifa | How Data Workers Cope with Uncertainty: A Task Characterisation Study[END_REF]. In a recent study, we examined how multiple types of data workers collaborate to explore complex data sets (e. g., model simulations) using a visualization tool [START_REF] Boukhelifa | An Exploratory Study on Visual Exploration of Model Simulations by Multiple Types of Experts[END_REF].

Beyond analyzing the data workers' general sense-making strategies, we also looked at how they explore large search spaces and how they reconcile conflicting optimisation criteria. The results of this study revealed an iterative analysis approach adopted by our data workers, that interleaves different types of analysis scenarios.

We found exploration scenarios where data workers examine together new and refined research questions and hypotheses, and other scenarios where they learn to appropriate the exploration tool and setup, and attempt to recap and establish common ground (storytelling). This type of study can help improve our understanding of the role of human expertise and its inter-play with visual analytics in reaching new insights and building common ground during collaborative data work.

Other work focuses more on building better tools for analysts-usually within a specific context. For example, the EvoGraphDice prototype [START_REF] Cancino | Evographdice: Interactive evolution for visual analytics[END_REF] combines the automatic detection of visual features with human interpretation to aid in the exploration of multidimensional datasets. Our recent work on Codestrates [START_REF] Rädle | Codestrates: Literate Computing with Webstrates[END_REF] builds literate computing capabilitiessimilar in concept to Jupyter notebooks-on top of Webstrates [START_REF] Klokmose | Webstrates: Shareable Dynamic Media[END_REF] to provide a malleable, collaborative environment in which users can not only collaboratively write code but can also dynamically extend the environment from within the environment. Such systems aim to reduce the barriers for users to adapt the tool to their contextual needs and further a co-adaptive relationship between tool and user [START_REF] Mackay | Users and Customizable Software: A Co-Adaptive Phenomenon[END_REF].

Alternatives in Data Analysis Activities

In our current study, we take our first step at trying to gain a deeper understanding of the nuance and complexity of how data workers explore alternatives. For example, in the early stage of data analysis, alternative ideas are explored to better define the problem [START_REF] Kandel | Enterprise Data Analysis and Visualization: An Interview Study[END_REF]; in the implementation stage, alternative data features, models, and evaluation methods are explored for different reasons [START_REF] Boukhelifa | An Exploratory Study on Visual Exploration of Model Simulations by Multiple Types of Experts[END_REF][START_REF] Hill | Trials and tribulations of developers of intelligent systems: A field study[END_REF]. However, it remains unclear what kinds of alternatives exist within the data sensemaking loop-what characteristics do they share and how do these considerations of alternatives influence sensemaking.

We focus on understanding: (1) When do data workers consider and try alternatives (if at all)? (2) What kinds of alternatives do they consider? (3) What strategies do data workers adopt to explore and manage alternatives? (4) What are the triggers and barriers to "alternative exploration"? (5) How do existing tools support the exploration of alternatives?

We conducted semi-structured interviews with 12 participants (4 from an enterprise setting and 8 from the research domain). We collected 827 minutes of recordings yielding 585 unique observations. We are analyzing these data with a combination of affinity diagramming and workflow diagram methods with the goal of characterizing the role of alternatives in these kinds of sense-making activities.

Our aim is to provide a richer understanding of these practices and to identify specific gaps that call out the need for new tools or new ways of designing such tools.(nb: I wonder if we have a results teaser to add here, even if very high level, couple of sentences or so ...)

Open questions & research directions

Data work involves a rich body of disciplines and methods. While significant work has identified and formalized various sense-making processes, there is still much to learn about these practices in impromptu or less-formal settings.

Moreover, much analysis work takes place in the head of the user. How can we build better tools and processes to help get such tacit learning out of the head of the individual and into a form that can be more easily shared within collaborative contexts.

(jl: Data workers often need to explore a large scale of alternatives. How do we design tools to enable better focus on exploring and comparing alternatives, as opposed to just managing them?) Alternatives can be generated, updated, revisited, and recombined in different stages along the analysis. How do we design tools to actively support and to record this process in a fluid manner.? Data workers need to revisit past and on-going data analysis projects, both to share key insights with others, but also to reflect upon and to adapt their sense-making strategies. How do we facilitate reflective and collaborative sensemaking in a team of data workers having differing skills and expertise?