Nadia Boukhelifa
email: nadia.boukhelifa@inrae.fr

Christopher R Johnson

Kristi Potter
email: kristi.potter@nrel.gov

Hans-Christian Hege

Christoph Heinzl

Matthew Kay

Miriah Meyer

Stefan Hagen Weber

Daniela Oelke

Stefan H Weber Main

Thomas Wischgoll

Christina Gillmann

Robin Maack

Matthew Marangoni

Michael Correll

Stephanie Deitrick

Paul Rosen

Regina Maria

Veronika Schuster

Visualization and Decision Making Design Under Uncertainty

Keywords: and phrases Decision making, Uncertainty visualization, Visual Analytics, Visualization Digital Object Identifier 10.4230/DagRep.12.8.1

This report documents the program and the outcomes of Dagstuhl Seminar 22331 "Visualization and Decision Making Design Under Uncertainty". The seminar brought together 33 researchers and practitioners from different domains concerned with visualization and decision making under uncertainty including visualization, visual analytics, human-computer interaction, artificial intelligence, climate research, geography and geology. The programme was organized in two parts: In the first part which lasted two days, participants gave short talks where they discussed current practices and the uncertainty visualization challenges they encountered in their own research. At the end of day two, participants brainstormed collectively around the main uncertainty visualization research challenges across domains and applications. In the second part, participants voted for the following three main challenges they wished to discuss for the remainder of the seminar (one and a half days): applications, human-centered uncertainty visualization, a design process for uncertainty visualization. Thus three break-out groups were formed to discuss these challenges. Abstracts for the individual talks and the break-out group activities are included in this report.

Uncertainty is an important aspect to data understanding. Without awareness of the variability, error, or reliability of a data set, the ability to make decisions on that data is limited. However, practices around uncertainty visualization remain domain-specific, rooted in convention, and in many instances, absent entirely. Part of the reason for this may be a lack of established guidelines for navigating difficult choices of when uncertainty should be added, how to visualize uncertainty, and how to evaluate its effectiveness. Unsurprisingly, the inclusion of uncertainty into visualizations is a major challenge to visualization [1]. As work concerned with uncertainty visualization grows, it has become clear that simple visual additions of uncertainty information to traditional visualization methods do not appropriately convey the meaning of the uncertainty, pose many perceptual challenges, and, in the worst case, can lead a viewer to a completely wrong understanding of the data.

The goal of this Dagstuhl Seminar was to bring together experts with diverse knowledge of uncertainty visualization and comprehension toward building a foundation of accessible, practical knowledge that practitioners and researchers alike can rely on in addressing challenges related to uncertainty. Specifically, this seminar brought together leaders in the field of uncertainty visualization and communication, along with experts on quantification and practitioners and domain experts dealing with uncertainty on a daily basis. Drawing on the knowledge of the participants, the seminar worked toward goals of synthesizing disparate findings and approaches from across computer science and related literature, noting current practices surrounding uncertainty, and identifying unsolved problems in common workflows, and areas needing further study.

As a major result from the seminar, the following challenges and research topics in visualization and decision making under uncertainty have been identified:

Applications, Human-centered uncertainty visualization (including how to support "feeling uncertain"), A design process for uncertainty visualization, Defining terms related to uncertainty, Algorithms and uncertainty quantification, Software dissemination, User studies, Ethics of uncertainty (when to include uncertainty information), Surveys of uncertainty-aware visual analytics, and Teaching uncertainty visualization.

The top three challenges were discussed in depth during this intensive three and a half days Dagstuhl Seminar as part of the break-out groups, and are further discussed in this report. In particular, the break-out groups examined uncertainty visualization research challenges from three complementary perspectives: from an application viewpoint looking at how uncertainty visualization and assessment are used in many domains; from a human-centered perspective considering the needs and information of the viewer; and from a more theoretical stand focusing on the problem space for designing uncertainty visualization.

The seminar ended with a presentation from each group and discussions on the next steps. Interesting research questions and potential solutions were identified during the discussions, and plans were made to continue the collaboration. Details of the individual talks and break-out group discussions are provided in this report.

References 1

Chris R. Data visualization has become indispensable for efficient interpretation of complex data generated across diverse scientific domains, such as biomedical imaging and meteorology. Many critical decisions directly rely on the quality of data visualizations. Inaccuracies in visualizations cannot be averted due to uncertainties inherent in underlying data and non-linear transformations of data caused by the stages of the visualization pipeline. The uncertainty in the final visualizations can adversely impact the decision-making process. The accurate quantification of uncertainties in data visualizations has, therefore, been recognized as the top research challenge for minimizing risks associated with scientific decisions.

In this talk, I will present the abstract statistical methods for uncertainty visualization and a few uncertainty visualization applications. My main topics of discussion are as follows: 1) Need for uncertainty visualizations, 2) abstract statistical methods for uncertainty quantification, 3) a few applications of uncertainty visualization to key scientific visualization techniques, such as fiber surfaces and Morse complexes, and domain-specific data, e.g., biomedical imaging, 4) open research challenges in uncertainty visualization. Our experimental results relevant to uncertainty visualizations confirm the significance or need for incorporating statistical error analysis into computational models for visualization applications.

A Tentative List of Uncertainty Visualization Research Challenges

Nadia Boukhelifa

License

Creative Commons BY 4.0 International license © Nadia Boukhelifa Uncertainty visualization research has made considerable progress leading to a variety of techniques, algorithms, systems, frameworks and user studies. The goal of this talk is to provide a preliminary list of open problems and challenges that our visualization community has been focused on in the last 20 years. I present findings from a literature survey of 17 papers from 2002 -2022, covering multiple domains including scientific visualisation, information visualization and visual analytics. I focus on surveys, state-of-the-art reports, viewpoint articles and position papers rather than on papers on specific techniques, algorithms, systems or user studies.

The results of this survey shows eight main areas of open challenges related to conceptualisation, evaluation, formalisation and theory, quantification, representation, training and dissemination, uncertainty-aware tools, and user Interaction. Some of the found challenges may have already been solved, and new ones may not yet have been fully documented. There is a need to review progress of the field of uncertainty visualization across domains, and to highlight success stories, long-standing problems as well as emerging and new ones. Climate models simulate the most important processes governing the climate system, i.e. the coupled system of atmosphere, ocean, sea ice, land-biosphere and ocean-biogeochemistry. Simulations result in 3D time-dependent multivariate data sets, characterized by high variability at various time scales. Internal variability of the coupled climate system additionally contributes to this noise. However, the high variability reduces the signal-to-noise ratio, thus makes it hard to detect climate change signals. Analyzing and visualizing climate change in the presence of noise is challenging, but with ensemble simulations, the signal-to-noise ratio can be enhanced and the internal climate variability assessed. I present examples from climate change research that show the visualization of robustness in the presence of a highly variable field. However, with respect to the climate change to 2100, the largest uncertainty is in the range of possible evolutions of the socio-economic system. Furthermore, I show visualizations of the CMIP6 multi model ensemble of simulations conducted globally with regard to the 6th IPCC report that capture this range through a range of scenarios describing different socio-economic development pathways. Finally, I briefly present recent collaborative work with Gerik Scheuermann's group to highlight the challenges in the visualization of uncertain topology-based features for highly variable complex phenomena such as the North Atlantic Oscillation and its evolution in a changing climate.

The Impossibility of Zero: Effects of Individual Differences in Medical Decision Making

Remco Chang (Tufts University -Medford, US)

License

Creative Commons BY 4.0 International license © Remco Chang Making decisions that might affect a person's long-term physical wellbeing can be difficult and stressful. As most medical diagnosis contains some amount of uncertainty (including type I and type II errors), it is often up to a patient to assess their own comfort level with different treatment options. In this talk, I present three challenges relating to medical decision making through the perspective of the patients, namely risk communication, reasoning with conditional probability, and visualization design for decision making.

First, I present a design study of a visualization tool for communicating a patient's prostate cancer risk. After interviewing 6 prostate cancer patients and two urologists, we iteratively designed the visualization based on the participants' feedback. Our takeaways from this design study include: (1) prostate cancer patients (who tend to be older men) have trouble using even basic visualizations (e.g. bar chart, stacked area chart, scatterplot, etc.). Text explanations that accompany the visualizations are a must. (2) Emotion and stress can affect a patient's ability to reason about their diagnosis. After receiving a positive diagnosis, a patient often has limited cognitive capacity to think through the diagnosis rationally. (3) Most Patients' first question after receiving a positive diagnosis is "how much time do I have left," suggesting that there's an order to the presenting of information that can best meet the patients' decision-making needs.

Second, I present an experimental study on people's ability to reason about their diagnosis as conditional probabilities. Most screening tests contain some amount of uncertainty, in particular as type I and type II errors. When a patient is told that they have a positive diagnosis for a disease, it is often up to the patient to reason through these probabilities to assess what their "true risks" are. In our experiment, we tested 6 visualization designs that were accompanied by text explanations. Our initial analysis of the results found no statistical significance between the effectiveness of the 6 visualizations. However, when the participants were stratified based on their spatial ability scores (as measured using the paper-folding test), we found that some of the visualizations are very effective (near 100% accuracy) for the participants with high spatial abilities. Unfortunately, we found no visualization that was helpful for the participants with low spatial abilities.

Lastly, I discuss the challenges in designing visualizations for helping patients make difficult medical decisions. For example, a patient might not perceive any difference between a diagnosis with 30% or 31% of having a disease. However, when the difference is between 0% and 1% chance of having a disease, the same difference of 1% becomes more significant to a patient as it represents "not having a disease" versus "possibly having a disease." A visualization will need to incorporate individuals' risk perception and risk tolerance utility curves to best support their decision making process.

Underthinking Uncertainty Visualization

Michael Correll (Tableau Software -Seattle, US)

License

Creative Commons BY 4.0 International license © Michael Correll Uncertainty visualization is viewed as a hard problem. Sources of these difficulties include complexity and disagreement around how uncertainty is modeled or quantified and clashes between between idealized forms of decision-making and the actual behavior of human beings. It is true that these are problems. But we can't wait for statisticians and psychologists to settle all of their internal disputes on these topics; people have decisions to make today. What we can do, however, is find solutions that are likely to be generally good enough for many practical purposes.

In this talk I will introduce a framework for ways to address uncertainty without having to think too hard, specifically around leveraging the ability of people to estimate statistical properties in visualizations without additional scaffolding, and the ability of visualization designers to "nudge" these estimates to align with statistical models of decision-making without being dogmatic or domineering. This is good news for uncertainty visualization as a discipline in that it does not require either designers or viewers of visualizations to be perfectly rational statistical deities to get their work done, but perhaps bad news in that we now have to do much more work as a field to build a deeper understanding of graphical perception for "fuzzier" tasks, take stronger stances around desired behavior from viewers of visualizations, and to better integrate statistical models, models of inference, and rhetorical goals into our design thinking.

Uncertainty in Public Policy Decision Making

Stephanie Deitrick ( Public policy decision makers leverage both qualitative and quantitative data as part of their decision-making processes. With increased interests in science-based information and leveraging data for their decisions, agencies are often expanding their workforce to include more data scientists and partnering with researcher on a variety of topics. While policy makers understand that data are uncertain at some level, that may not be something they explicitly consider as part of how they currently leverage data.

Since data are often communicated through visualization, such as maps and charts, should uncertainty be part of that communication? Would it produce better or more informed decisions?

Uncertainty-aware Visual Analytics

Christina Gillmann (Universität Leipzig, DE)

License

Creative Commons BY 4.0 International license © Christina Gillmann Visual analytics has been successfully applied to a variety of applications also in terms of uncertainty analysis. Unfortunately, the visual analytics process does not include a mechanism to systematically handle uncertainty. In order to solve this issue, we developed the concept of uncertainty-aware visual analytics. Therefore, an extension of the classic visual analytics cycle is achieved that includes the quantification of uncertainty in each component, the exchange of analysis and visualization approaches in general by uncertainty-aware options and the introduction of provenance to monitor the accumulation and propagation of uncertainty throughout the visual analytics cycle. In order to create uncertainty-aware visual analytics cycles for particular applications, we determined a workflow that consists of 5 steps that constructs an uncertainty-aware visual analytics cycle starting from the classic approach. The procedure is based on a developed taxonomy of uncertainties that allow to understand the nature of different uncertainty events and their effect on the visual analytics cycle.

Visualizing uncertainty in digital geologic map databases

Amy Gilmer (USGS -Denver, US) and Kathleen Warrell (UCAR -Boulder, US)

License

Creative Commons BY 4.0 International license © Amy Gilmer and Kathleen Warrell

The geologic map remains the primary tool geologists use to model and communicate what we know about Earth's surface. All geologic models contain some level of uncertainty, but this uncertainty is rarely incorporated in traditional geologic maps, potentially limiting application by decision makers. Even as our geological depictions have migrated to digital geologic map databases, our map symbology has largely remained the same as that used on traditional paper maps. While varying dash length for contacts and faults may convey a relative sense of uncertainty to experienced users, it does not convey meaning to the nonexpert user. Cartographic uncertainty visualizations are an effective way to communicate how well we know what and where something is.

The adoption of the Geologic Map Schema (GeMS) standard for geologic maps has enabled geologists to capture feature-level metadata, including location uncertainty, as well as feature identity and existence confidence. To visually communicate the underlying locational uncertainty in the USGS Intermountain West geological framework database, we have developed an ArcGIS Python toolbox that extracts existing location confidence data from feature attributes, and then buffers and aggregates the uncertainty across a tessellation grid. The tessellation grid can then be visualized by any of the statistical fields generated. This toolbox can be applied to any geologic map database adhering to the GeMS format to produce visualizations summarizing uncertainty. While there is still much we can do to refine how we quantify uncertainty in mapping geologic features, this type of visualization, when provided alongside the geologic map data, summarizes the uncertainty without requiring the user to understand the nuances of traditional map cartography. Additionally, this quantitative approach can help identify areas characterized by high levels of uncertainty, potentially a result of low-resolution map data, that can be used for geologic mapping needs assessments and to better inform end users to limit improper use of the map data.

Summarization, Uncertainty, Estimation. . . : Models as a basis for visualization

Michael Gleicher (University of Wisconsin-Madison, US)

License

Creative Commons BY 4.0 International license © Michael Gleicher Models are part of most (if not all) use of data. However, they are often hidden or implicit. We often expect viewers to figure out what they are, estimate their parameters, and apply them correctly to achieve their goals. I argue that models should be a first class citizen in how we help people work with data. Many problems, including uncertainty, seem to be made worse because the models are hidden. Many concepts, such as summarization, estimation, and uncertainty are often conflated, especially when models are hidden. My conjecture is that by having a better way to include models in our thinking and by de-conflating the key terms, we can better discuss, design, and evaluate tools to help people work with data.

Actionable Uncertainty Visualization

Kristi Potter (NREL -Golden, US)

License

Creative Commons BY 4.0 International license © Kristi Potter Ensemble simulations capture the variability present in the predictions of future states by combining multiple runs of a computational model with different parameter settings. Datasets derived from ensemble simulations are often quite large and complex, making it hard to create visualizations that facilitate decisions, particularly for people not intimately involved with the scientific domain or the creation of the dataset. An example comes from the renewable energy space, where improvements to the electrical grid to facilitate the large-scale reduction of carbon emissions involves making decisions on highly complex systems. Traditional methods for uncertainty visualizations primarily focus on the challenge of visually presenting largescale, high-dimensional datasets in an exploratory manner. However those approaches do not facilitate decision making by non-experts, such as policy-makers, who may not know enough about the computational system to appropriately choose appropriate parameter settings to achieve a desirable outcome. In this talk I will discuss ideas for distilling down the parameter space by importance, annotating contextual information needed for better understanding, and designing a visualization tool that is streamlined for decision-making.

A design theory for uncertainty visualization?

Maria Riveiro (Jönköping University, SE)

License

Creative Commons BY 4.0 International license © Maria Riveiro Despite the large volume of research on uncertainty visualization, we do not fully understand the impact of uncertainty visualization on decision-making. There is evidence of both positive and negative effects of visually depicted uncertainty on decision-making.

This talk presents examples of evaluations carried out with practitioners in various application areas, including autonomous driving, air traffic risk assessment and maritime surveillance. I summarise the effects of the uncertainty visualizations provided on the users and their decision-making processes in these evaluations.

Finally, we discuss the need for a design theory/space of uncertainty visualization and elaborate on the multiple dimensions/variables that such a design space should have. 4 Kinkeldey, C., MacEachren, A. M., Riveiro, M., & Schiewe, J. (2017). Evaluating the effect of visually represented geodata uncertainty on decision-making: systematic review, lessons learned, and recommendations. Cartography and Geographic Information Science, 44(1), 1-21.

Critical Points of an uncertain Scalar field

Gerik Scheuermann (Universität Leipzig, DE)

License

Creative Commons BY 4.0 International license © Gerik Scheuermann Critical points like extrema or saddles are a well established concept in (deterministic) scalar field visualization. There is strong practical interest, a clear mathematical concept in continuous and discrete settings, and corresponding algorithms, including implementations in commercial systems. Looking at uncertain scalar fields, formally described as smooth stochastic processes, practically often given as ensembles over a common grid, the situation changes. The concept of a "critical point" is not exactly defined. Two major definitions are "critical point of the (deterministic) mean field" or "probability distribution of critical points in a sample from the stochastic process/ensemble". The choice of concept definition has effects on the visualization and its interpretation, like "a maximum being multiple (significant) maxima in one sample and no significant maximum in another case". Also, the sampling quality of the ensemble should somehow be integrated into the visualization. The talk concern these issues. Looking at the distribution definition, I show how to infer critical point distributions from ensembles using Bayesian Inference. Looking at the mean field definition, I will discuss how bootstrapping allows to reason about the sampling quality to derive significant results. Finally, the talk shows how this allows to decide two practical questions regarding the future of the north atlantic oscillation (NAO) depending on Climate Change. (NAO describes they interplay between Iceland Low and Azore High -which is the most dominant factor in European winter weather.) We derive that the centers of action of both pressure systems move substantially depending on the global warming, and that the IceLand Low will most likely see a split into two centers of action in the extreme scenarios. The work was done with Dominik Vietinghoff, Christian Heine, and Michael Böttinger.

Uncertainty in Time Series and Geographic Data

Johanna Schmidt (VRVis -Wien, AT)

License

Creative Commons BY 4.0 International license © Johanna Schmidt URL https://www.digi-hydro.com/ Design decisions must be made to make data visual, and modifications to the data are needed. Data modification includes reconstruction, resampling, filtering, and aggregation. In one of our projects, we have to deal with time series data recorded from sensors installed in hydropower machines. The project's purpose is to better understand which sensors can give information about the current state of the hydropower machine. This needs to be done with exploratory data analysis. It is not yet known to the mechanical engineers which sensors will be descriptive for detecting certain stages during machine operation. However, the data is large (approximately 30 TB of data), and it is impossible to analyze the raw data in this case. We, therefore, need to apply resampling and filtering to the data, which introduces uncertainty in the analysis the mechanical engineers should be informed about. In the case of geological data, reconstruction (of point cloud data) and 3D rendering introduce uncertainty in the data representation. When performing analyses, methods like plane fitting are also not wholly accurate. This uncertainty in the data and how it is presented to the users needs to be communicated, as both user groups (mechanical engineers and geologists) highly depend on detailed analysis results.

Quantifying and Visualizing Uncertainty in Medical Image Segmentation

Thomas Schultz (Universität Bonn, DE)

License

Creative Commons BY 4.0 International license © Thomas Schultz Joint work of Shekoufeh Gorgi Zadeh, Thomas Schultz Main reference Shekoufeh Gorgi Zadeh, Maximilian W. M. Wintergerst, Thomas Schultz: "Intelligent interaction and uncertainty visualization for efficient drusen and retinal layer segmentation in Optical Coherence Tomography", Comput. Graph., Vol. 83, pp. 51-61, 2019. URL http://dx.doi.org/10.1016/j.cag.2019.07.001

Neural networks have greatly increased the accuracy in many medical image segmentation tasks, and have been successfully deployed for large-scale image analysis. However, fully automated results are still not reliable enough to be trusted blindly in applications where segmentation quality might be critical to the well-being of individuals. Using an application example in ophthalmology, we demonstrate that visualizing the uncertainty in neural network based segmentations, and providing uncertainty-aware tools for segmentation editing, can make it more time efficient to identify and correct remaining segmentation errors. We also discuss the important open question of reliable uncertainty quantification in an out-ofdistribution setting, for example when processing images that have been acquired with a different scanner, and we mention strategies for approaching that problem.

Visual representations of density under uncertainty have been explored for geographic data, scatterplots, line charts or parallel coordinates. Ensemble forecasting is a widely known application for uncertainty visualization. Often end users have specific requirements and tasks for the visualization, e.g. each forecast (line) should be visible and interactable, the resulting chart should not be overcrowded the uncertain space between ensembles should be filled by upsampling the uncertainty should be made visible Identifying outliers is as important as spotting the main trend All these requirements can be realized by a novel technique for generating density representations for line charts that is visually and computationally scalable with respect to the number of lines that are shown. In contrast to alternative kernel-based density representations, it also keeps the course of the lines visible unless the local density is very high. Points are on top of each other (or crossing lines) represent the (un)certainty (density surface) and are mapped to color. A smoothed representation with an upsampling effect is done by adding a "glow" around the lines. This glow is implemented by decreasing the alpha value with increasing distance from the line. The amount of glow at a certain distance is determined by the shape of the specific kernel function. The kernel width determines the extension of the glow around the line. Some considerable effort was spent to design and implement the visualization and integrate it into a commercial system (linking & brushing), to fulfill the end user's requirements. The result was evaluated together with the end user who wanted to gain more insight into their ensemble forecasts to answer the question "When is the best time to buy oil?". The end user first inspected the overall distribution pattern in time over all ensemble members. They used the median to separate the higher half of the distribution from the lower. They got immediate insights regarding the trend and distribution. The visualization was more explored, and the end user provided very positive feedback. Our expectation was of course that the result will be used from now on. However, the opposite happened. The end user so far was not sure if he can trust the ensemble forecast method. With our visualization he gained trust in the AI after a few hours. From that point Uncertainties are difficult if not impossible to avoid. Capturing data from the analog world almost always results in some form of uncertainty. The amount of uncertainty depends on the method of measurement and its accuracy. When visualizing data that has some associated uncertainty, it is essential to properly process and convey such uncertainty and especially the amount of uncertainty keeping in mind that additional processing steps can amplify the uncertainty. There are various sources of uncertainty, such as numerical limitations or limitations of the capture device. However, there are other sources of uncertainty. Some of these uncertainties stem from model assumptions or limitations of how we translate natural specimens to 3D representations. Molecular structures are one example of this. This talk will illustrate this further and point to some of the solutions.

Uncertainty Visualization of Health Data

Liang Zhou (Peking University, CN)

License

Creative Commons BY 4.0 International license © Liang Zhou Health science relies on a wide range of different types of data. There, uncertainty is ubiquitous and is aware by health science experts. Uncertainty visualization is, therefore, important and could potentially aid decision making. In this talk, I will introduce my own research work on new visualization techniques for representative health data. These examples focus on uncertainty visualization of ensemble medical imaging data, local correlation and subspace visualization for multidimensional data, and perceptual enhancement for visualization images. I will also discuss works on visual analytics of health data with uncertainties from missing data. Finally, I will discuss uncertainty challenges that I identified in the various types of health data. Uncertainty visualization and assessment are used in many domains, including medical applications, non-destructive testing, industrial AI, geology, renewable energies, and climate research. The way uncertainty is used by users in these domains differs depending on the required tasks and the data used. As an outcome of this working group, we identified success stories of published or successfully applied in seven domains. Based on these success stories, we identified common open challenges and research questions that will be worth working on: (i) Uncertainty could be viewed from a mathematical point of view, looking at stochastic processes, statistics, correlations, and similar. This would also enable the quantification of uncertainty in different domains. (ii) The different sources of uncertainty need to be discussed -whether they are similar in different domains and to which degree they depend on tasks and data. Also, the terminology used in different domains to describe uncertainty differs. (iii) An interesting question is to differentiate between visualization applications where uncertainty visualization is needed and where not. It might depend on the task, and the types of questions users have, whether it makes sense to include uncertainty in a visual representation or not. (iv) Visualizing uncertainty also relates to perceptual issues, describing how well uncertainty can be perceived using different encodings. (v) As a wrap-up, it would be interesting to find out how far uncertainty visualization is already used in commercial software.

22331 -Visualization and Decision Making Design Under Uncertainty

decision-making aligns with that of some normative model of utility or decision quality. This model-driven rather than human-driven perspective introduces several key limitations when designing or evaluating uncertainty visualizations. For one, it elides many aspects of decision-making under uncertainty that are not amenable to tidy quantification, such as situated or implicit knowledge. For another, it ignores psychological, sociological, rhetorical, or ethical aspects of presenting uncertainty information. We propose a human-centered view of uncertainty visualization in which the needs and information of the viewer, rather than backing statistical or inferential models, are given precedence.

In the human-centered view of uncertainty visualization, viewers are neither rote reciters of p-values, nor conditioned to mimic the actions of a statistical test. Rather, they have many goals, including being able to audit or justify their decisions, build appropriate trust in the data source and designers, integrate their own mental models and domain knowledge with existing data, or even just walk away satisfied that they made a reasonable decision given the information they had to hand. In this paper, we show how existing frames around uncertainty visualization may fail to result in designs that accomplish these goals, and present both existing strategies for better integrating the human in the uncertainty visualization design process as well as open problems in visualization research. Visualization researchers seek appropriate abstractions to help us design, analyze, organize, and evaluate the things we create. Information visualization literature has many task structures (taxonomies, typologies, etc.), design spaces, and related frameworks. In this working group, we discussed current frameworks for designing visualizations, and we considered developing a new problem space that complements the existing ones by focusing on the needs that a visualization is meant to solve. Briefly, the proposed problem space is based on the earlier work by [1], considering the 5Ws and H (who, why, what, where, when and how). We believe that this problem space provides a valuable conceptual tool for designing and discussing visualizations, including uncertainty visualisations.

A Problem Space for Designing (Uncertainty) Visualizations

3

 3 Visualization of Climate Simulation Data and related Uncertainty Michael Böttinger (DKRZ Hamburg, DE) License Creative Commons BY 4.0 International license © Michael Böttinger

References 1

 1 Vietinghoff, D., Heine, C., Böttinger, M., Maher, N., Jungclaus J., and Scheuermann, G. Visual Analysis of Spatio-Temporal Trends in Time-Dependent Ensemble Data Sets on the Example of the North Atlantic Oscillation. 2021 IEEE 14th Pacific Visualization Symposium (PacificVis), 2021, pp. 71-80, doi:10.1109/PacificVis52677.2021.00017 2 Vietinghoff, D., Böttinger, M., Scheuermann, G. and Heine, C Detecting Critical Points in 2D Scalar Field Ensembles Using Bayesian Inference IEEE 15th Pacific Visualization Symposium (PacificVis), 2022, pp. 1-10, doi:10.1109/PacificVis53943.2022.00009.

14 22331 -

 22331 Visualization and Decision Making Design Under Uncertainty

Maria Riveiro (

 Riveiro Jönköping University, SE), Remco Chang (Tufts University -Medford, US), Oliver Deussen (Universität Konstanz, DE), Christina Gillmann (Universität Leipzig, DE), Michael Gleicher (University of Wisconsin-Madison, US), and Tatiana von Landesberger (Universität Köln, DE) License Creative Commons BY 4.0 International license © Maria Riveiro, Remco Chang, Oliver Deussen, Christina Gillmann, Michael Gleicher, and Tatiana von Landesberger Main reference Hans-Jorg Schulz, Thomas Nocke, Magnus Heitzler, and Heidrun Schumann. 2013. A Design Space of Visualization Tasks. IEEE Transactions on Visualization and Computer Graphics, 19(12):2366-2375.

of Talks 3.1 Statistical Analysis for Uncertainty Quantification and Visualization of Scientific Data

  Johnson and Allen R. Sanderson. A next step: Visualizing errors and uncertainty. IEEE Computer Graphics and Applications, 2003, vol. 23, no 5, p. 6-10. Quantifying and Visualizing Uncertainty in Medical Image Segmentation Thomas Schultz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Visualizing the Uncertainty in Image Analysis -Previous work and new opportunities Brian Summa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Uncertainty and Trustworthy AI Stefan Hagen Weber . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Overcoming Uncertainties in Molecular Visualization Thomas Wischgoll . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Uncertainty Visualization of Health Data Liang Zhou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	License Working Tushar Athawale (Oak Ridge National Laboratory, US) Creative Commons BY 4.0 International license © Tushar Athawale Joint work of Tushar Athawale, Chris R. Johnson

groups Applications Tushar Athawale, Michael Böttinger, Amy Gilmer, Hans-Christian Hege, Christoph Heinzl, Christopher R. Johnson, Gerik Scheuermann, Johanna Schmidt, Thomas Schultz, Jarke J. van Wijk, Stefan Hagen Weber, and Xiaoru Yuan . . . . . . . . . What's the point?: Focusing on the human in uncertainty vis Nadia Boukhelifa, Michael Correll, Stephanie Deitrick, Matthew Kay, Miriah Meyer, Kristi Potter, Paul Rosen, and Regina Maria Veronika Schuster . . . . . . . . . . . A Problem Space for Designing (Uncertainty) Visualizations Maria Riveiro, Remco Chang, Oliver Deussen, Christina Gillmann, Michael Gleicher, and Tatiana von Landesberger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Remote Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 Overview

  Arizona State University -Tempe, US)

	License	Creative Commons BY 4.0 International license
		© Stephanie Deitrick

Working groups 4.1 Applications

  Tushar Athawale, Michael Böttinger, Amy Gilmer, Hans-Christian Hege, Christoph Heinzl, Christopher R. Johnson, Gerik Scheuermann, Johanna Schmidt, Thomas Schultz, Jarke J. van Wijk, Stefan Hagen Weber, and Xiaoru Yuan

	Tushar Athawale (Oak Ridge National Laboratory, US), Michael Böttinger (DKRZ Hamburg,
	DE), Amy Gilmer (USGS -Denver, US), Hans-Christian Hege (Zuse-Institute Berlin, DE),
	Christoph Heinzl (Universität Passau, DE), Christopher R. Johnson (University of Utah -
	Salt Lake City, US), Gerik Scheuermann (Universität Leipzig, DE), Johanna Schmidt (VRVis
	-Wien, AT), Thomas Schultz (Universität Bonn, DE), Jarke J. van Wijk (TU Eindhoven,
	NL), Stefan Hagen Weber (Siemens -München, DE), and Xiaoru Yuan (Peking University,
	CN)
	License	Creative Commons BY 4.0 International license
	©

What's the point?: Focusing on the human in uncertainty vis

Nadia Boukhelifa (INRAE -Palaiseau, FR), Michael Correll (Tableau Software -Seattle, US), Stephanie Deitrick (Arizona State University -Tempe, US), Matthew Kay (Northwestern University -Evanston, US), Miriah Meyer (Linköping University, SE), Kristi Potter (NREL -Golden, US), Paul Rosen (University of Utah -Salt Lake City, US), and Regina Maria Veronika Schuster (Universität Wien, AT)

References 1

Vietinghoff, Heine, Böttinger, Maher, Jungclaus, Scheuermann. Visual analysis of spatiotemporal trends in time-dependent ensemble data sets on the example of the north atlantic oscillation.


Margot Bouhon
email: bouhonmargot@gmail.com

Claire Ferreira

Sandy Bahuon

Barbara Tillmann

Nathalie Bedoin

Improving non-native duration contrast with dichotic training in

Introduction

Late second language (L2) learners typically misperceive non-native phonemes as instances of the closest native phonemes of their first language (L1) and produce them accordingly [START_REF] Flege | Effects of experience on non-native speakers' production and perception of English vowels[END_REF]. For instance, the English short/long vowel contrast is irrelevant in French and rarely perceived and produced by French speakers. However, L2 vowel perception and sometimes vowel production can improve with perceptual training [START_REF] Lengeris | The effect of native vowel processing ability and frequency discrimination acuity on the phonetic training of English vowels for native speakers of Greek[END_REF][START_REF] Wong | The effects of perceptual and/or productive training on the perception and production of English vowels /I/ and /i:/ by cantonese ESL learners[END_REF]. Our study addressed this issue with a new training program based on assumptions of the Asymmetric Sampling in Time model (AST, Poeppel, 2003) according to which speech is decoded in the auditory cortex through a large temporal integration window (125-300 ms) relevant to syllables in the right hemisphere (RH), and a narrow integration window (20-80 ms) relevant to phonemes in the left hemisphere (LH) [START_REF] Poeppel | Speech perception at the interface of neurobiology and linguistics[END_REF]. This difference may be subserved by the dominance of slow theta oscillations (4-8 Hz) in the right auditory cortex and faster gamma cerebral oscillations (> 30 Hz) in the left auditory cortex,. In contrast to consonants, results about hemispheric asymmetry for vowels are mixed.

For instance, bilateral MMN emerged for long deviant vowels in EEG measurements [START_REF] Asano | Temporal integration of segmentedspeech sounds probed with mismatch negativity[END_REF], and right-dominant activation occurred in the anterior superior temporal gyrus in fMRI for perception of long German vowel perception (duration 333 ms [START_REF] Obleser | Vowel sound extraction in anterior superior temporal cortex[END_REF]. This result pattern might be explained by (1) the rich spectral content of vowels and the spectral fine-tuning of the right auditory cortex [START_REF] Warren | Analysis of the spectral envelope of sounds by the human brain[END_REF][START_REF] Zatorre | Structure and function of auditory cortex: Music and speech[END_REF], and

(2) the long duration of many vowels (> 150 ms). Indeed, RH dominance of cerebral activation has been reported for vowel discrimination specifically in case of long vowel pairs [START_REF] Britton | The role of spectral and durational properties on hemispheric asymmetries in vowel perception[END_REF], which reconciles the RH involvement for some phoneme perception with the assumptions of the AST model. The present study proposed a training that builds on the potential difference in hemispheric asymmetry related to the processing of short and long vowels.

According to the Temporal Sampling Framework (TSF) [START_REF] Goswami | A temporal sampling framework for developmental dyslexia[END_REF], the different cerebral modulation frequency ranges (cerebral oscillations) are not unimpaired in dyslexia, resulting in impaired temporal sampling of the acoustic speech signal, leading to phonological deficits. Consistent with this framework, an atypical absence of significant rightward lateralization for low frequencies and a rightward lateralization for high frequencies have been showed in children and adults with dyslexia [START_REF] Lizarazu | Developmental evaluation of atypical auditory sampling in dyslexia: Functional and structural evidence[END_REF]. The TSF specifically postulates that a difficulty with slower temporal modulations -including the theta range -takes a large part in the origin of speech processing impairments in dyslexia. This is in agreement with atypical right hemisphere activity during speech processing in children and adolescents with dyslexia [START_REF] Heim | Altered hemispheric asymmetry of auditory P100m in dyslexia[END_REF]. For instance, abnormal activity in the right supra-marginal gyrus during 2 Hz modulation processing has been shown in dyslexic children [START_REF] Crombie | The effects of specific learning difficulties (dyslexia) on the learning of a foreign language in school[END_REF]. Additionally, Di Liberto et al. (2018) reported impaired low-frequency cortical tracking to phonetic features, and a correlation between this disorder and phonological skills in case of dyslexia. Despite accumulated evidence for atypical processing of slow rate changes during speech processing in dyslexia, temporal processing in the highfrequency ranges has also been documented. For instance, a lack of LH-specialization for fast oscillations [START_REF] Lehongre | Impaired auditory sampling in dyslexia: Further evidence from combined fMRI and EEG[END_REF], and reduced leftward lateralization for high-frequency entrainment [START_REF] Dushanova | Speech-brain frequency entrainment of dyslexia with and without phonological deficits[END_REF] have been reported in developmental dyslexia. Dichotic stimulation might be an appropriate method to improve the English long/short vowel contrast (e.g., /i:/-/ɪ/) processing in French speakers, notably by promoting hemispheric complementarity for duration processing. Dichotic listening provides participants simultaneously with two different sounds, one to each ear [START_REF] Kimura | Functional asymmetry of the brain in dichotic listening[END_REF]. This method was designed to increase the dominance of contralateral hemisphere projections. Similar dichotic stimulation has been used successfully in [START_REF] Bouhon | Stimulation dichotique ou musicale pour l'aide aux apprenants de l'anglais L2, dyslexiques ou non : conception et validation d'un programme d'entraînement informatisé[END_REF] to help French dyslexic and typicalreading adults to improve identification, discrimination and pre-attentional sensitivity to another phonemic contrast based on temporal acoustic cues: the voicing of English stop consonants. In contrast to French (i.e., L1 of the participants), voiced stops have a short Voice Onset Time (VOT) and voiceless stops have a long VOT in English [START_REF] Bouhon | Stimulation dichotique ou musicale pour l'aide aux apprenants de l'anglais L2, dyslexiques ou non : conception et validation d'un programme d'entraînement informatisé[END_REF]. In the present training targeting vowel perception, words containing /i:/ (i.e., long vowel) were presented to the left ear with white noise in the right ear, whereas words containing /ɪ/ (i.e., short vowel) were presented to the right ear concurrently with white noise in the left ear. The aim was to specifically involve the most efficient hemisphere for vowel duration processing.

The training effect was evaluated with two pre-post word production tasks (i.e., reading aloud tasks with or without auditory model) in French adults with typical development or with dyslexia. Dyslexic individuals have been reported to frequently demonstrate a lower level of oral proficiency in L2 [START_REF] Crombie | The effects of specific learning difficulties (dyslexia) on the learning of a foreign language in school[END_REF][START_REF] Sparks | Native language predictors of foreign language proficiency and foreign language aptitude[END_REF].

Method

Participants

Forty monolingual French native speakers (Table 1) from Lyon University with English level (L2) inferior to B1 (Common European Framework of Reference for Languages, CECRL) participated: 20 adults who suffered from dyslexia in childhood (DYS) (16 females; M = 22.80 years; SD = 3.62) and 20 controls (CTR; 18 females; M = 21.28 years; SD = 2.57), with more participants in the latter group having a slightly higher English level. Thirty-four were right-handed (≥ 80% "right hand" responses in the Edinburgh test, 16 DYS and 18 CTR).

They had to exhibit a Right Ear Advantage (REA) to process short phonetic features according to a dichotic listening test in French introduced by [START_REF] Bedoin | Hemispheric asymmetries depend on the phonetic feature: A dichotic study of place of articulation and voicing in French stops[END_REF]. In one of the conditions, two words differing by the place of articulation of their initial plosive consonant (i.e., short acoustic difference in formant transition, e.g., /pas/-/tas/) were simultaneously presented to the right ear and the left ear, respectively. Better identification of stimuli presented to the right ear (i.e., REA) was assumed to reflect better processing of short phonetic features by the left hemisphere. Exclusion criteria were uncorrected auditory or visual disorders, known neurological or neurodevelopmental disorders other than dyslexia (e.g., ADHD). The experiment was approved by the French Ethics Committee (Comité de Protection des Personnes); participants provided written informed consent.

Materials

Thirty-four English minimal pairs of monosyllabic words opposed by /i:/-/ɪ/ (e.g., sheep-ship)

were

Procedure

A pre-test (T1) -training -post-test (T2) design was used. Two speech production tests have been designed to assess the duration of /i:/ and /ɪ/ in monosyllabic words at T1 and T2. In the first one (reading test), the participant had to read a series of words without auditory model. In the second one (repetition test), the printed word and an auditory model were presented, and the participant had to imitate the word.

The training was presented as a lexical enrichment program over three consecutive days (20 min/day). Words containing /i:/ were presented to the left ear concurrently with white noise to the right ear to specifically involve the RH in /i:/ processing; the reverse lateralization was used for words containing /ɪ/. Participants did not receive explicit instructions about vowel durations. Exercises A and B were done on the first two days;

Exercises B and C on the third day. In Exercise A (word-learning), participants listened twice to an English word illustrated by a picture and English and French spellings, to promote learning of both lexical representations (specific phonological-orthographic-semantic associations) and general grapheme-phoneme correspondences. In Exercise B (orthographic judgment), participants saw a new picture, listened to an English word, and had to select its orthographic form out of a minimal pair. Although the task can be performed based on lexical knowledge, the two printed words were designed to repetitively drive attention to focus on reliable grapheme-phoneme relations (1 letter = /ɪ/, short; 2 letters = /i:/, long), favoring the acquisition of these rules. In Exercise C, participants heard an English spoken word without spelling information, then they selected one of two novel pictures to illustrate it. Feedback was provided after each response in Exercises B and C. Exercise B underlined graphemephoneme relations between spelling and vowel duration, while Exercise C encouraged the acquisition of lexical phonological representations.

Data analysis

Produced vowel durations were measured using Praat (Boersma & Weenink, 2021), and /i:/ minus /ɪ/ differences at T1 and T2, respectively, as well as /i:/ lengthening and /ɪ/ shortening from T1 to T2 were calculated. As residuals were not normally distributed for T1-T2 differences, they were compared using Wilcoxon signed rank tests. Vargha-Deleney (A) effect sizes were reported for significant effects. This improvement occurred whether the auditory model was available (i.e., repetition task) or not (i.e., reading aloud) in DYS, but not in the repetition test in CTR, probably due to ceiling performance. This progress was generalized to novel words in both groups, although L2 learning has been reported to be challenging for dyslexics [START_REF] Crombie | The effects of specific learning difficulties (dyslexia) on the learning of a foreign language in school[END_REF][START_REF] Sparks | Native language predictors of foreign language proficiency and foreign language aptitude[END_REF]. Our findings suggest that improvement can transfer from perception to production, and new phonological contrasts can be learned by adults without explicit teaching. As training with the dichotic device has been previously shown to improve learning of consonant voicing based on VOT duration in English [START_REF] Bouhon | Stimulation dichotique ou musicale pour l'aide aux apprenants de l'anglais L2, dyslexiques ou non : conception et validation d'un programme d'entraînement informatisé[END_REF], future research could test its adaptation to other languages with duration contrasts for vowels (e.g. German). The tasks used in the present study suit particularly well for languages with grapheme-phoneme rules for vowel duration.

Results

Repetition

For the reading test, /i:/ lengthening and /ɪ/ shortening contributed to increase the duration difference in both groups after training. This result pattern is consistent with the assumption that dichotic training could reinforce associations between right-lateralized slow oscillations [START_REF] Poeppel | Speech perception at the interface of neurobiology and linguistics[END_REF] and long vowel processing, on the one hand, and fast leftlateralized oscillations and short vowel processing, on the other hand.

Detailed analyses have suggested long vowel lengthening to be the default mechanism after training to produce vowel duration contrast, as 1) in the repetition test, it was the source of /i:/-/ɪ/ difference increase, and 2) in the reading test, the effect size of /i:/ lengthening exceeded that of /ɪ/ shortening in CTR.

In the DYS group, the large effect size of /ɪ/ shortening in the reading test after dichotic training may seem surprising in light of the previously reported atypical reduced leftward lateralization of fast oscillations in dyslexia [START_REF] Dushanova | Speech-brain frequency entrainment of dyslexia with and without phonological deficits[END_REF][START_REF] Lehongre | Impaired auditory sampling in dyslexia: Further evidence from combined fMRI and EEG[END_REF]. However, typical beta powera rhythm which is well-suited for short vowel processing -has been reported in the LH for dyslexic adults [START_REF] Chang | Atypical beta power fluctuation while listening to an isochronous sequence in dyslexia[END_REF] Two main limitations have to be acknowledged for this pilot study. First, the potential test-retest effect will need be evaluated with an untrained group of normal-reading students at T1 and T2. Second, the effect of dichotic presentation has to be disentangled from the classical effect of perceptual training, by for example adding a control condition using binaural presentation of the stimuli during training. Higher increase in performance from T1 to T2 is hypothesized in case of dichotic training than in binaural training, as previously reported for consonants [START_REF] Bouhon | Stimulation dichotique ou musicale pour l'aide aux apprenants de l'anglais L2, dyslexiques ou non : conception et validation d'un programme d'entraînement informatisé[END_REF].

Conclusion.

In sum, short perceptual training using dichotic listening can improve new duration contrast production in late foreign-language learners with or without dyslexia. Due to the lack of binaural training condition, the extent to which dichotic presentation contributed to training effectiveness cannot be evaluated here, but the triggering of lengthening and shortening mechanisms suggests that lateralized complementary skills have been enhanced by dichotic stimulation (see also [START_REF] Bouhon | Stimulation dichotique ou musicale pour l'aide aux apprenants de l'anglais L2, dyslexiques ou non : conception et validation d'un programme d'entraînement informatisé[END_REF]. The findings are consistent with multi-time resolution models [START_REF] Poeppel | Speech perception at the interface of neurobiology and linguistics[END_REF] and, if replicated with larger participant groups, they could provide promising training perspectives to address L2 learning difficulties.

  recorded by a native English speaker. Thirty pairs were used for training and testing; four additional pairs were included in the production tests. The mean durations were 150 ms for /i:/ and 60 ms for /ɪ/. Pictures under creative commons license were used for training. Due to COVID-19, training and tests were performed remotely under the supervision of the experimenter, using OpenSesame (Mathôt et al., 2012); participants used their headphones, dictaphones, and computers. Participants' set up was tested with a specialized video with left and right stereo channel isolation (#Stereo: Left and Right Stereo Sound Test, 2018).

  test. The /i:/-/ɪ/ duration difference tended to be smaller in DYS than in CTR at T1 (W = 269, p = .064), but not at T2 (W = 222, p = .561). It increased from T1 to T2 only in DYS (V = 40, p = .016, A = .36), due to /i:/ lengthening (V = 49, p = .036, A = .60) (Fig. 1, left).

Figure 1 .

 1 Figure 1. /i:/-/ɪ/ duration difference produced at T1 (white) and T2 (black) by the control group (CTR, left side) and the dyslexic group (DYS, right side), for the repetition test (panel A), and for the reading aloud test (panel B). Error bars represent the standard error. * = significant differences (p < .05) between T1 and T2.

Figure 2 .

 2 Figure 2. Scatterplot showing the correlation between the REA and the reduction of /ɪ/ duration from

Table 1 .

 1 Characteristics of the control (CTL) and dyslexic (DYS) participants.

	Group	Mean Age	Age (SD)	Parity	Edinburgh	Mean REA	English
		(years)			laterality test	(SD)	level
	CTL	21.28	2.57	18 females	18 right-handed	0.50 (0.47)	4 A2
				2 males	2 ambidextrous		16 B1
	DYS	22.80	3.62	16 females	16 right-handed	0.80 (0.92)	12 A2
				4 males	4 ambidextrous		8 B1

  , which could allow for processing the /ɪ/ brevity after appropriate stimulation by dichotic training, at least as well as for control adults. The correlation between the REA and the shortening of /ɪ/ after training, observed only in the DYS group, is consistent with the hypothesis that improved processing of short phonetic events by the LH might be a key progress triggered by dichotic training in the DYS group.The correlation between /ɪ/ shortening and REA in dyslexic adults suggests that the larger the typical dominance of fast oscillations in the LH was, the higher was the compensatory mechanism based on /ɪ/ shortening in this group. This correlation has to be investigated in a larger scale study, but replicating this result would encourage further perspectives for practitioners to take into consideration the REA for short acoustic cues, as measured by dichotic listening tests (e.g., in English,Rimol et al., 2006; in French, Bedoin et al., 2010) before using a training program with dichotic stimulation. Individual differences in hemispheric asymmetry could potentially help to predict whether dichotic training for duration contrast learning could lead to both appropriate long vowel lengthening and short vowel shortening.

AM Business-Contentsquare and the National Agency for Research and Technology (Doctoral Grant)