Anderson Augusma

Dominique Vaufreydaz

Frédérique Letué

MULTIMODAL GROUP EMOTION RECOGNITION IN-THE-WILD USING PRIVACY-COMPLIANT FEATURES AUTHOR VERSION

Keywords: Multimodal, Privacy safe, Transformer networks, Group emotion recognition in-the-wild

This paper explores privacy-compliant group-level emotion recognition "in-the-wild" within the EmotiW Challenge 2023. Group-level emotion recognition can be useful in many fields including social robotics, conversational agents, e-coaching and learning analytics. This research imposes itself using only global features avoiding individual ones, i.e. all features that can be used to identify or track people in videos (facial landmarks, body poses, audio diarization, etc.). The proposed multimodal model is composed of a video and an audio branches with a cross-attention between modalities. The video branch is based on a fine-tuned ViT architecture. The audio branch extracts Mel-spectrograms and feed them through CNN blocks into a transformer encoder. Our training paradigm includes a generated synthetic dataset to increase the sensitivity of our model on facial expression within the image in a data-driven way. The extensive experiments show the significance of our methodology. Our privacy-compliant proposal performs fairly on the EmotiW challenge, with 79.24% and 75.13% of accuracy respectively on validation and test set for the best models. Noticeably, our findings highlight that it is possible to reach this accuracy level with privacy-compliant features using only 5 frames uniformly distributed on the video.

Introduction

Emotion recognition research is of interest in multimodal interaction for numerous applications like social robotics, conversational agents, e-coaching, or learning analytics. Among others, one challenge is the automatic recognition of grouplevel emotions in ecological or "in-the-wild" scenarios, which involves considering group dynamics, individual behavior, emotional expressions, postures, environmental elements, and different kinds of activities [START_REF] Sharma | Automatic group level affect and cohesion prediction in videos[END_REF]. This task is further complicated by cultural and ethnic factors and some technical issues like the quality of recording point of view. Adding concerns about ethics and privacy limits some choices in the data that can be employed as input and outputs of the machine learning algorithms.

This article presents a privacy-compliant architecture to classify audio-visual data labeled with group-level emotions: the VGAF dataset [START_REF] Sharma | Automatic group level affect and cohesion prediction in videos[END_REF]. This machine learning model materializes our participation in the EmotiW 2023 [START_REF] Dhall | Emotiw 2023: Emotion recognition in the wild challenge[END_REF] challenge. As our former research, Petrova et al. [START_REF] Petrova | Group-Level Emotion Recognition Using a Unimodal Privacy-Safe Non-Individual Approach[END_REF], we impose ourselves some privacy rules on the input of our machine learning model. Unlike other approaches [START_REF] Liu | Group level audio-video emotion recognition using hybrid networks[END_REF][START_REF] Andrey | Neural network model for video-based analysis of student's emotions in e-learning[END_REF][START_REF] Sharma | Audiovisual automatic group affect analysis[END_REF][START_REF] Sun | Multi-modal fusion using spatio-temporal and static features for group emotion recognition[END_REF], our privacy-compliant model must not use any information that can be helpful to identify or track any person on the videos. It avoids individual data like postures or facial landmarks, for instance, focusing only on global features computed on images and sound.

The paper is structured as follows. After the presentation of related work (section 2) and the VGAF dataset (section 3), section 4 introduces our methodology. Section 5 details our extensive experiments while section 6 discusses the results in regard to the state-of-the-art.

Progresses in machine learning in the last decade permit to address emotion recognition in real-world or "in the wild" scenarios, and more recently group level emotion prediction in videos. While some research remains monomodal [START_REF] Ottl | Group-level speech emotion recognition utilising deep spectrum features[END_REF][START_REF] Petrova | Group-Level Emotion Recognition Using a Unimodal Privacy-Safe Non-Individual Approach[END_REF][START_REF] Andrey | Neural network model for video-based analysis of student's emotions in e-learning[END_REF], most of them use multimodal inputs [START_REF] Belova | Group-level affect recognition in video using deviation of frame features[END_REF][START_REF] Evtodienko | Multimodal end-to-end group emotion recognition using cross-modal attention[END_REF][START_REF] Pinto | Audiovisual classification of group emotion valence using activity recognition networks[END_REF]. Surrace et al. [START_REF] Surace | Emotion recognition in the wild using deep neural networks and bayesian classifiers[END_REF] used a method that examines both general and specific information. They focused on the overall scene in a top-down strategy. They also extracted personal information in a bottomup strategy, isolating faces from wider images. Other studies [1, 8-12, 17, 19, 23, 25] also used individuals' information for emotion recognition. For example, Liu et al. [START_REF] Liu | Group level audio-video emotion recognition using hybrid networks[END_REF] used a network for recognizing facial emotions and body poses, while Fisher et al [START_REF] Yu | Group emotion recognition based on global and local features[END_REF] developed a method to detect facial expressions. This research goes in a different direction by focusing on global features to avoid harming as far as possible privacy.

The use of synthetic data for model training is a wellestablished practice in machine learning research. This datadriven method is often employed by researchers to enhance the generalizability or, a contrario, the specificity of their models. For instance, Dwibedi et al. [START_REF] Dwibedi | Counting out time: Class agnostic video repetition counting in the wild[END_REF] harnessed real videos to construct synthetic repetition videos for training their "Counting Out Time" model. Marín-Jiménez et al. [START_REF] Manuel | Laeo-net: revisiting people looking at each other in videos[END_REF] altered the relative head positions of individuals in images to comply with a real scenario. Their aim was to improve the generalizability of their Look At Each Other (LAEO) model. In research specifically related to the EmotiW challenge, Petrova et al. [START_REF] Petrova | Group-Level Emotion Recognition Using a Unimodal Privacy-Safe Non-Individual Approach[END_REF] employed monomodal synthetic static images to enable their model to concentrate on individuals' faces while disregarding background. Their method creates static images to train their feature extraction. This research extends this approach to generate coherent video sequences coupled with class-related sound.

VGAF Dataset

The VGAF dataset [START_REF] Sharma | Automatic group level affect and cohesion prediction in videos[END_REF] is a collection of web videos that include a wide variety of genders, ethnicities, event types, numbers of people, and poses. These videos are grouped into three categories of group level emotions: Positive, Neutral, and Negative (as shown in Figure 1). The classification of each video was determined by a voting process involving several annotators, with a majority vote deciding the final classification. The dataset is divided into a Training set, a Validation set, and a Test set, with 2661, 766, and 756 videos in each set respectively. The Validation set is unevenly distributed, with more videos in the Positive and Neutral categories compared to the Negative category.

The videos include groups of at least two people and feature a range of events like interviews, festivals, parties, and protests, among others. They vary in resolution, and are split up into 5-second labeled clips. The frame rate varies from 13 to 30 frames per second. The camera focuses on different positions in each video, resulting in frames that highlight different group organisations. Thus, the technical and content varieties of the videos, the association of images and sounds to identify emotions, make the classification complexity of the VGAF dataset.

One of the dataset's complexities is that some videos feature children playing and hitting tables or chairs, generating sounds that may be confused with protestation noises. However, these two events are different because playing with children creates a positive atmosphere, while protestations generate a negative atmosphere.

Methodology

Model architecture

The proposed neural architecture is illustrated in Figure 2. This model encompasses two monomodal branches producing embedding for video and audio frames. The video branch exploits a pre-trained, then fine tuned, vision transformer (ViT-Large, 14 patches) [START_REF] Dosovitskiy | An image is worth 16x16 words: Transformers for image recognition at scale[END_REF] ending by a custom linear layer with an output dimension of 1024, i.e. the embedding dimension of the model (d model ). The audio branch consumes a sequence of Mel-Spectrograms as input through four specialized CNN blocks. As depicted, these blocks are composed of 2D convolutions stack with Relu and a final max pooling. The final section of the audio branch integrates a Transformer Encoder layer, as outlined in [START_REF] Vaswani | Attention is all you need[END_REF] with fixed positional-encoding and four self-attention heads. The feed-forward hidden-size is set at twice the value of d model , while output dimension remains d model . To integrate interaction between audio and video modalities, a late fusion mechanism concatenates embeddings from the monomodal branches jointly with a Cross Attention one. In cross-attention, the audio modality serves as query (Q), while the video modality serves as key (K) and value (V ). The resulting attention weights are then multiplied with the video embeddings to obtain the final transformed representation. An average operation is then applied to the concatenated sequence averaging all frame embeddings to one of size 3 * d model . The classification task is carried out using a fully-connected layer followed by a Softmax activation, which yields to the final classification result in 3 classes.

Privacy compliant features

As stated, one self-imposed constraint is to remain as privacy compliant as possible. To do so, our approach carefully avoids all individual features, i.e. features that could identify a person directly or within a group, focusing on global features.

For the video branch, our system does not employ body pose, shape, height, nor body language features like gesture or agitation. Computed facial information like FACS, emotion or person id are excluded, like counting and tracking individuals within the scene. The video input is a set of n still video frames uniformly taken over the video and resized to 224 × 224. n is chosen among two extrema: 1 frame per second (fps) and 15 frames per second on the 5-second videos of VGAF. From our experience, 1 fps is a lower band to get a non-random classification. As 15 fps is the lowest fps on the training videos, we set it as max fps to train on actual frames without repeating some of them. Thus, n is taken in {5,75} for each video in this research. For the audio processing, we avoid speaker identification, speaker diarization or any speech-to-text processing. The audio of all videos is standardized by resampling it at 16 Khz and converting it to mono channel. By compliance with video branch, we extract 5 or 75 audio frames per video. With 5 frames, the audio frame corresponds to 1 second with no overlap. In the case of 75 frames, a sliding window is set to 67 milliseconds to get the right number of frames. Last, each audio frame is converted into Mel-Spectrograms using 128 Mel filters, to produce an input image of 128 × 251 for the CNN blocks.

Synthetic video dataset

Inspired by former research [START_REF] Petrova | Group-Level Emotion Recognition Using a Unimodal Privacy-Safe Non-Individual Approach[END_REF], we augmented our training set with synthetic data. The purpose is to guide the neural network to concentrate on faces while searching for positive, neutral or negative contexts, ignoring the background. The original generation process involves placing real faces expressing basic emotions [START_REF] Ebner | Faces-a database of facial expressions in young, middle-aged, and older women and men: Development and validation[END_REF] onto random backgrounds [START_REF] Yu | LSUN: construction of a large-scale image dataset using deep learning with humans in the loop[END_REF] to create still images conveying emotions. To create synthetic videos, we take from 3 to 9 faces expressing the right emotions, for instance positive emotions to generate positive video, and randomly move them on a fixed background. To mitigate the impact of occlusions, we introduce masks in the generation process, which allows a maximum of ten percent of occlusion on faces. As generating audio is a very complex task in our context, we associated randomly an audio with the same class from the VGAF training set to the generated video sequence. The synthetic video dataset has the same size and class balance as the VGAF training dataset: 802 (30,16%), 923 (34,71%), 934 (35,13%) of positive, neutral and negative videos respectively.

Experiments

Optimal synthetic video ratio

We conducted experiments to measure the impact of synthetic data ratio on performance using 5 frames per video. The synthetic ratio was gradually increased from 10% to 50% with a 10% step of the total training data. The accuracies on the val- Following this result, in all subsequent experiments comprising standard and synthetic data, the synthetic ratio is set to 30%.

Multimodal ablation Study

To evaluate relevance of each of our training inputs, (VGAF video and audio, synthetic data), experiments of all their combinations with 5 and 75 frames were conducted. Models were trained using cross-entropy loss and the SGD optimizer with a fixed learning rate of 10 -5 . We used the VGAF (audio and/or video) and the synthetic datasets in monomodal and multimodal variants architecture. The ViT-Large model consists of 24 pre-trained transformer encoder blocks on 14 patches. Its weights are frozen for 10 epochs, and then released to be finetuned on our data. Cross attention is present only when both video and audio branches of the network are active. Results on the VGAF validation set are summarized in Table 2.

Training on audio data only leads to average performance up to 56.4% with 5 frames per video. Using 75 audio frames, the performance decreases. This may be explained by the averaging effect of the sliding window applied in this case. Using only synthetic videos performs better with arround 60% of accuracy, showing interest of this approach. The best mono input performance is 74.15% using 75 video frames. As in many computer vision tasks, pre-trained vision transformers prove to be efficient to capture relevant features on the VGAF dataset. Combining audio and videos inputs produces various performance in term of validation accuracy. Joining synthetic video and audio does not improve much the performance of individual inputs. The best multimodal performance (78.07%) is obtained by combining all inputs using 5 frames. Our best accuracy concerns a monomodal combination of VGAF and synthetic video dataset during training phase. This model outperforms all other models with a validation accuracy of 79.29%.

Five monomodal and multimodal versions of our model participated in the EmotiW challenge 2023. Performances on the test set are reported in Table 3. As anticipated, audio and synthetic data models (v1 and v2) remain around 55% of test accuracy. Unlike validation performance, video plus synthetic model (v3) is less accurate than the 2 multimodal ones (v4 and v5). Noticeably, both multimodal systems, using 5 (v4) or 75 frames (v5) per video, have the same performance even if their predictions differ: their prediction agreement, i.e. when the same class is predicted for the same video, is 88%. As expected, prediction agreements of v5 versus v1 and v3 are respectively 53% and 90%, highlighting that, for our architecture, audio provides less information than video.

Discussion

Excluding systems from the EmotiW 2023 [START_REF] Dhall | Emotiw 2023: Emotion recognition in the wild challenge[END_REF] challenge as results were not available at the writing time of this article, one can compare the 5 versions of our model with the stateof-the-art. Accuracies on validation and test sets are reported in Table 4. Using only audio, as we limited input to Melspectrograms, our v1 model do not reach score of Ottl et al. [START_REF] Ottl | Group-level speech emotion recognition utilising deep spectrum features[END_REF] proposal, due to their more complex usage of OpenS-MILE features [START_REF] Eyben | Opensmile: the munich versatile and fast open-source audio feature extractor[END_REF] coupled with Deep Spectrum analysis. On [START_REF] Sharma | Audiovisual automatic group affect analysis[END_REF] 61.61% 66.00% Pinto et al. [START_REF] Pinto | Audiovisual classification of group emotion valence using activity recognition networks[END_REF] 65.74% -Wang et al. [START_REF] Wang | Implicit knowledge injectable cross attention audiovisual model for group emotion recognition[END_REF] 66.19% 66.40% Sun et al. [START_REF] Sun | Multi-modal fusion using spatio-temporal and static features for group emotion recognition[END_REF] 71.93% -Belova et al. [START_REF] Belova | Group-level affect recognition in video using deviation of frame features[END_REF] 71.95% -Liu et al. [START_REF] Liu | Group level audio-video emotion recognition using hybrid networks[END_REF] 74.28% 76.85% Ours v4 78.07% 75.13% Ours v5 78.72% 75.13%

video only system, the v3 model have the best performance even if, as said, its test accuracy is lower than its validation counterpart. On multimodal systems, v4 and v5 remain under the proposal of Liu et al. [START_REF] Liu | Group level audio-video emotion recognition using hybrid networks[END_REF]. Nevertheless, our approach exposes fair performance while using only global thus privacycompliant features.

Looking at intrinsic analysis of our models, they benefit from the pre-trained ViT network fine-tuned on the VGAF data and from our synthetic video approach but lack in the audio branch.

Increasing the number of parameters like number of attention heads wages unconditionally to overfitting, showing that such architectures require a lot of training data. Gathering more group emotion data, (transfer) learning on close or generated data can be part of the solution. Our synthetic generation process is straightforward and can be enhanced in several ways. Generative Adversarial Networks can be trained to create new data. But all these approaches are limited by the complexity and diversity of group emotion videos in terms of audio and video content.

As anticipated, avoiding individual features leads to set a thread-off between performance and privacy. In this research, our aim was to investigate the maximum performance that could be obtained without contravening this rule. Depending on the target application context, some additional information could be added to improve performance.

Conclusion

This research introduces a privacy-compliant yet efficient method for recognizing group emotions in uncontrolled environments, proposing to eliminate the need for individual feature extraction. The privacy compliance is achieved through a selection of non-individual characteristics in signal inputs, providing only global information on the scene. The proposed model is composed of a video and an audio branches. The video branch is based on a ViT architecture fine tuned to compute relevant embedding on image sequences. The audio branch extracts Mel-spectrograms and feed them through CNN blocks into a transformer encoder. A cross-attention mechanism is added before the late multimodal fusion and the final classification. Our training paradigm includes a generated synthetic dataset to increase the sensitivity of our model on facial expression within the image in a data-driven way.

The extensive experiments show the relevance of our methodology. Our proposal performs fairly on the EmotiW challenge, with 79.24% and 75.13% of accuracy respectively on validation and test set for the best models. Contributions of each modality differ, the audio branch having rooms for larger improvement. Noticeably, our findings highlight that it is possible to reach 75% of accuracy with privacy-compliant features using only 5 frames uniformly distributed per video.

To conclude, the targeted privacy compliance of this research comes at a cost in performance. The accuracy remains slightly lower than other methods employing individual features. The next research challenge is thus to validate whatever it is possible, with individual features, to minimize the performance gap of privacy-compliant models, allowing to use them in more application contexts.

Figure 2 :

 2 Figure 2: At left, the proposed model is a combination of two monomodal branches, a Cross-Attention and a late fusion paradigm. The video branch uses a pre-trained vision transformer (ViT) model [3]. The audio branch encompasses 4 CNN blocks followed by a transformer encoder. At right, description of one CNN block used in the audio branch.

Table 1 :

 1 Evaluation of synthetic video ratio on the validation accuracy.

	Synthetic ratio Synthetic videos Total videos	Acc.
	10%	297	2958	74.80%
	20%	666	3327	74.93%
	30%	1140	3801	75.07%
	40%	1773	4434	74.41%
	50%	2661	5322	74.28%

idation set of these variants can be found in Table

1

. The best accuracy (75.07%) is reached using 30% of synthetic data in the training. Less or more data leads to lower performance.

Table 2 :

 2 Ablation study investigating the impact of different inputs on validation accuracy using 5 and 75 frames per video. The ablation considers all combinations of (VGAF) video, (VGAF) audio, and generated synthetic videos.

		Input Data	5 Frames 75 Frames
		audio	56.40%	54.96%
	Video	synt video video video + synt video	57.44% 73.62% 75.98%	60.05% 74.15% 79.24%
	Audio + video	synt video + audio video + audio video + audio + synt video	62.14% 76.11% 78.07%	58.75% 77.42% 78.72%

Table 3 :

 3 Test Set Accuracy of 5 versions of our model.

	Vers.	Input data	Nb Frames	Acc.
	v1	audio	5	55.29%
	v2	synt video	75	54.23%
	v3	video + synt video	75	74.73%
	v4	video + synt video + audio	5	75.13%
	v5	video + synt video + audio	75	75.13%

Table 4 :

 4 Comparison with SOTA systems on the VGAF dataset. Columns detail usage of individual features, accuracy on official validation set and, when reported, on the test set. The table is ordered by input modalities and then by validation accuracy.

			Ind. feat. Val. Acc. Test Acc.
	Audio	Ours v1 Ottl et al. [14]	56.40% 59.40%	55.29% 62.30%
		Petrova et al. [15]	52.36%	59.13%
	Video	Ours v2 Savchneko et al. [17]	60,05% 70.23%	54.23% -
		Ours v3	79.24%	74.73%
		Evtodienko et al [6]	60.37%	-
	Audio + Video	Sharma et al

ACKNOWLEDGMENTS

This work was supported by the PERSYVAL Labex (ANR-11-LABX-0025). This work was granted access to the HPC resources of IDRIS under the allocation 2023-AD010614233 made by GENCI. We express our gratitude to Garance Dupont-Ciabrini for her contribution to the first step of the synthetic video preparation.


St. Hsiang-Yun Wu

TAT: Toughening the Foundation of Abstraction in Visualization Techniques Organizers

In this meeting we discussed the concept of visual abstraction in visualization. Abstraction is essential to the field yet we have been using the term "abstraction" largely by intuition. Based on presentations from representatives from the field that cover a large variety of backgrounds and research focuses, in the meeting we worked toward a formalized understanding of the concept to better clarify its theory, its potential, and its uses. 1

Background and Introduction

Abstraction is considered as a conceptual process, whose outcome stands for the corresponding subordinate concepts as a whole. This representative notion allows users to facilitate comprehensive understanding and better memorability of the relevant complex knowledge in a more hierarchical or structured fashion. The concept of abstraction is especially significant when the information space is large and when it comes to real-world applications. The Tube Map of London Underground, for instance, is a classical map that removes unnecessary detail but retains sufficient information for better usability. The map successfully simplifies the geometry of the transportation structure, and facilitates users to effectively perform their tasks, which often includes [START_REF] Bock | OpenSpace: A system for astrographics[END_REF] which route provides the optimal path in terms of distance or price, (2) how many stops still remain until the destination, [START_REF] Halladjian | ScaleTrotter: Illustrative visual travels across negative scales[END_REF] where to transfer to another line, and so on. Other than cartography, abstraction is powerful in many application domains, such as mathematics, biology, physics, etc., because experts in these fields can focus on vital elements of their data and think through the problem on a more abstract level.

Similarly, in data visualization, abstraction plays an important role because the outcome of the abstraction process needs to reflect what an expert has in mind and the transition process between a set of different visual representations should support the comprehension of the high-level concept in various aspects. To visualize essential aspects of the data, however, we need dedicated mechanisms that abstract the unnecessary detail to allow the viewer of a visualization to focus on the important elements, depending on the given task. Viola and Isenberg [START_REF] Viola | Visual abstraction[END_REF][START_REF] Viola | Pondering the concept of abstraction in (illustrative) visualization[END_REF] have done an initial investigation on this topic and confirmed that the crucial problem in this context is that it is impossible to know what is important and what is not in a general way-importance changes based on the research question, on the application domain, on the data size, on the user, on the specific situation or task, etc. Visualization techniques, therefore, are expected to support a dynamic change of data's visual abstraction to reflect these contextual changes.

Beyond 2D representations such as metro maps, Rautek et al. [START_REF] Rautek | Illustrative visualization: New technology or useless tautology[END_REF] and other researchers have established an understanding of visual abstraction for spatial 3D data to classify low-level and high-level visual abstraction techniques, while the support of reasoning and insight communication for abstraction techniques is still a fundamental challenge that remains today. Recently, Viola and Isenberg [START_REF] Viola | Visual abstraction[END_REF][START_REF] Viola | Pondering the concept of abstraction in (illustrative) visualization[END_REF] have intensively explored the concept of abstraction as it is used in visualization. They realized that researchers so far have used the concept of abstraction largely by intuition without a precise meaning, and thus initialized the pioneering discussion on theoretical foundations of abstraction in visualization. This lack of specificity left questions on the characteristics of abstraction, its variants, its control, or its ultimate potential for visualization and, in particular, illustrative visualization mostly unanswered. After this investigation, several open questions are still awaiting to be answered, while it requires not only visualization researchers, but also domain experts in order to formulate a research agenda for the practical usage of abstraction for the visual representation and exploration of data.

With the growth of data complexity, the need for abstraction techniques is increasing. Although there have been a few approaches studied along this line, the theory, technology, and applications have so far integrated less well in terms of the stability and usability of abstraction models for visual analytics. These challenges concern approach scalability, the applicability of the models in application areas, as well as the technology of the environment in which the abstraction is performed. In summary, the theory and application of abstraction in visualization poses a research challenge not only due to the complex nature of the data, but also its dynamics and semantics. In this pioneering Shonan meeting we thus gathered researchers from visualization, information theory, and applied science to address the topic.

We have invited every participant to suggest any topic they would like to focus on during the meeting. All topics suggested by the participants, detailed through abstracts in the next section, extend the visual abstraction formalism in the four distinct categories:

• Theoretical Formalization of the Abstraction Concept as it relates to Visualization: The term abstract is an antonym of concrete or tangible, resulting in an inherent difficulty to describe it. Viola and Isenberg have initiated the definition of abstraction as a transformation which preserves one or more key concepts and removes detail. Together with Chen [START_REF] Viola | Visual abstraction[END_REF], they have linked the initial definitions with abstraction in philosophy of science and have proposed to quantify it by means of the information theory. During our Shonan meeting we added a new essential component of the formalism by introducing the notion of meaningfulness.

• Abstraction Techniques: During the Shonan meeting, several presentations introduced abstraction in the context of specific aspects of the visualization pipeline. Some abstraction techniques were centered on specific type of data, such as the networks or graphs for example. Other presentations showcased visual abstractions from a historical perspective, or by using specific visual encoding strategy, such as visualization sequence or glyphs. We discussed multiple axes of visual abstraction, and the role of abstraction in user interaction.

• Visual Abstraction Evaluation: A key theme of the meeting was related to abstraction evaluation and quantification. We have built upon the concept of faithfulness [START_REF] Nguyen | On the faithfulness of graph visualizations[END_REF] which measures how a visualization faithfully represents ground truth information of the abstract data.

Inspired from this existing concept we define the abstraction meaningfulness. Furthermore, we have discussed how the information theory toolbox can quantify the degree of visual abstraction.

• Visual Abstraction Applications: Visual abstraction can serve as a means to reduce various costs, such as costs of computing, costs of drawing, costs of interpreting, or costs of communicating. This implies opportunities where abstraction can be highly beneficial: in characterization, standardization and exchange formats of visualization design, in data and visualization provenance, in dealing with contextual and cultural biases, or missing data.

Overview of Talks

Thoughts on (Visual) Abstraction

Ivan Viola; KAUST, Saudi Arabia

In my talk I introduce basic definitions of abstraction and terminology that relates abstraction to visualization. First, I clarify its use in other disciplines, such as fine arts, geography, mathematics, programming paradigms, and I present how abstraction is rooted in the philosophy of science.

Axes of (Visual) Abstraction

Tobias Isenberg; Inria, France I talk about the notion of abstraction axes that emerge in many visualization papers/contributions that look at different levels of scale of a given subject matter. I will base this discussion on our earlier formalization attempts of visual abstraction and will use examples from the visualization literature to illustrate my discussion.

Visual Abstraction from a Historical View

Xiaoru Yuan; Peking University, China In this talk, I discuss the broad spectrum of visual abstraction from the perspective of historical visualizations. More specifically, I also share a few examples of earlier stage visualizations from China.

A Formal Definition of Abstraction

Min Chen, University of Oxford, United Kingdom In my talk, I will examine a general definition of "abstraction" (including computational and cognitive abstraction) from an information-theoretic perspective. In particular, I will discuss the ability for information theory to help explain, in abstraction, various phenomena in visualization and visual analytics. Explanation is the first step in any theoretical development. Hopefully, this work will stimulate further theoretical developments in Measurement and Prediction.

Abstraction and Bias

Guido Reina, University of Stuttgart, Germany Visualization and any subsequent abstraction need to perform some kind of reduction. I am interested in the fine line we walk on our way to insights: between preservation of features, guidance, story-telling, and the potential biasing of the user. I hypothesize we need to better define and capture the intent or task correspondence for visualizations and abstractions, given the loss of generality inherent to these aspects. This is compounded when we take into account focus and context techniques: the context moves on a continuum between informer and distractor, but the coupling to the focus and its perception and interpretation make these two very hard to disentangle and set up appropriately in a general way.

Exploring Abstraction in Immersive Environments: Navigating Multiscale Visualizations and User Interaction Lingyun Yu, Xi'an Jiaotong-Liverpool University, China Visual abstraction focuses on distinct features or essential concepts within complex structures. It aims to emphasize key information while simplifying details, thereby facilitating effective data analysis and communication. Immersive environments create captivating shared spaces, enabling multiple users to engage closely with data visualization. However, these environments also pose challenges when it comes to exploring multiscale data. This presentation will address fundamental questions concerning immersive multiscale visualizations and user interaction: 1) Navigation in Multiscale Environments: How do we allow people to navigate freely through a multiscale visualization space without confusing them? This is particularly relevant in the context of multiuser and multiscale visualizations within shared environments; 2) Adaptive Interaction for Multiscale Visualizations: What design strategies promote natural interaction across various levels of abstraction; 3) Designing Immersive Interaction Techniques for Multiscale Visualizations: Prioritizing User Needs and Enhancing Understanding.

Visualization Sequence as an Abstraction Technique

Puripant Ruchikachorn, Chulalongkorn Business School, Thailand A visualization sequence, especially in the form of an animation, is generally considered to be a visual representation of temporal data. However, nontemporal data, such as multidimensional data, is often shown in a sequence, such as a slideshow, in which each visualization represents a subset of the data. The process of transforming data into a visualization can occur gradually, and the intermediate representations can be shown in sequence as well. Even in a dashboard with multiple views, a viewer typically observes one visualization at a time.

Based on prior work, a visualization sequence may need a compositor similar to one used for multidimensional data abstraction. It should also avoid the same pitfalls as other abstraction techniques, such as being misinterpreted as a temporal sequence or having inconsistent visualization types across a sequence. Still, most research on visual abstraction has been for a single display, so the potential of visualization sequence as a tool of visual abstraction is unexplored. These are some questions that need to be addressed. When data is split into several chunks, are their permutations cognitively different? Are there any references in other fields on particular visualization sequences? Is there any information-theoretic cost or benefit of certain sequencing strategies?

Visual Abstraction for Network Representation

Karsten Klein, University of Konstanz, Germany Networks are used as a data model in many areas, in particular in the life sciences. A plethora of research has been conducted to advance network analysis and visualisation over several decades. However, the increasing scale, complexity, and heterogeneity of the underlying data as well as the developments in the hardware and software available for visual analysis, such as immersive environments, have rather increased the number of questions than to close the gap of data complexity and user capacity to perceive and understand it. Abstraction can support understanding but also introduce issues regarding confidence and faithfulness. I gave a short overview on potential contributions of abstraction research, some pointers to challenges and requirements, and discussed my viewpoint on several related aspects.

Faithful Abstraction for Big Complex Graphs

Seok-Hee Hong, University of Sydney, Australia In this talk, I will discuss several recent approaches and examples for Data Abstraction, Task Abstraction and Visual Abstraction for big complex graphs.

More specifically, Data abstraction methods include filtering, sampling, dimension reduction and clustering. Task abstraction examples include the metro map visualisation and clustered graph visualisation. Visual abstraction examples include edge bundling methods and graph map representations.

I will conclude my talk with faithful abstraction including Information faithfulness, Task faithfulness and Change faithfulness.

Towards measuring meaning in / meaningful abstractions

Torsten Möller, University of Vienna, Austria If we'd like to know whether abstractions are meaningful, we would need to ask 'For Who?' and then how to measure them. This is the connection to visual (data) literacy, which is slowly becoming of interest to the visualization community.

I want to demonstrate a recent representative survey of the Austrian population with a bar chart and a line chart as found in news articles. I will argue that we are far from properly assessing such a multi-dimensional concept as 'meaning.' Previous work has focused on task-based assessment. I argue to extend this assessment to several aspects of self-perception, including complexity and abstraction, graph familiarity, aesthetics, critical thinking, topic knowledge, and numeracy. There are lots of open questions, which hopefully stimulate a discussion.

Building Visual Metaphors for Visual Abstraction

Siming Chen, Fudan University, China Visual metaphors and glyphs play an important role in visualization for understanding complex data. However, how to create suitable and novel metaphors is a challenging research question. With my previous experiences in designing map and bridge metaphors, I would like to discuss the following issues: 1) What is the design space for using visual metaphors and glyphs for abstraction? How to make use of the metaphor for readers to understand complex information in a simplified and vivid way? 2) How to design and build visual metaphors and glyphs for visual abstraction based on the design space? Moreover, which visual channel and data mining methods can better be integrated for designing the metaphors? 3) How to leverage recent AI models to automatically/semiautomatically create the metaphors for visual abstraction? I will discuss my opinions through the above three points.

Abstract Visualizations in Life Sciences: Bridging Insights Across Complex Data Landscapes and Scientific Areas

Michael Schwärzler, Takeda -Pharmaceutical Sciences, Japan Abstraction in visualization techniques for life sciences offers a valuable approach for comprehending intricate biological systems and discovering candidates for new treatments. After a molecule has been selected as a candidate for a product, it enters the pharmaceutical process development stage, where stakeholders deal with large amounts of heterogeneous data sources generated over a decade or more until market entry. This data, spanning structured and unstructured formats, includes molecular structures, time series data from devices, sample results, image data, chromatography curves, experimental documentation, but also budgetary considerations, strategic objectives, and more.

While the involved personas apply a multitude of advanced analytics and abstract visualization techniques to solve their tasks in their respective area of responsibility, the variety of perspectives and the differing understandings pose a major challenge -even though they all have a common goal and work on the same data. Additionally, varying visual abstraction techniques, cultural influences, and the loss of knowledge over time (due to personnel turnover or outdated technology) play roles. While "data silos" have become less of a problem, "knowledge silos" have become the most prominent challenge today, and missing visual abstraction standards and frameworks play a role in that. In fact, exchange and decision making happens mostly on a rather poor level of abstraction: using short summaries in PowerPoint presentations, giving up data provenance and the possibility to getting a deeper understanding of the underlying data.

At the data level, the use of knowledge graphs generated with taxonomies and ontologies aids in defining structures and relationships. However, finding ways to harmonize insight-generating abstract visualizations so that the barriers for knowledge exchange can be lowered as much as possible could have the potential to really achieve a coherent overview. Proposing a scientific definition of abstraction in visualization could establish harmonized standards or "visual knowledge interfaces". These would guide crafting of easier interpretable, meaningful abstract visualizations for stakeholders in highly heterogenous data contexts generated over a long time span for decision making on all levels. In my talk, I give an overview on the area, its data and visualization techniques, and trigger a discussion regarding observed aspects that might have a relevance for the theoretical foundations of abstraction for visualization techniques.

Abstraction by Human Visual Intelligence

Jian Chen, The Ohio State University, USA I present how the human visual system simplifies information across different temporal scales to facilitate the processing of complex scenes. (1) At the scale of 200-500 milliseconds, global scene gist acts as a holistic feature representation. ( 2) Between 1 second and 3 seconds, memorable experiences often align with linguistic concepts. (3) Within a minute of viewing very large images, we can extract valuable signals to uncover meaningful patterns in the data. I provide empirical evidence demonstrating how we can harness these innate cognitive abilities to visualize complex simulation results, with the ultimate goal of understanding viewer behavior and developing a visual scene vocabulary.

Evaluation of Abstraction

Weidong Huang, University of Technology Sydney, Australia As Viola and Isenberg mentioned for visualization, "the abstraction serves the goal of facilitating the understanding of the subject matter". Abstraction can be implemented in different forms in different contexts for different purposes for a given dataset. Once it is done, the question is whether or not an abstraction has achieved its goal. How do we know that? Does abstraction filter unwanted information and include only the information that we want? Does it introduce new or misleading information? To answer these questions, we need evaluation. What are the appropriate metrics, how do we measure them and how do we evaluate them? In this talk, I will present some challenges and share my views on these.

Quantifying Visual Abstraction: Visual Complexity or Data Insight Fidelity?

Yong Wang, Singapore Management University, Singapore Visual abstraction is the process of transforming data into visual representations that can reveal data insights to viewers. For a given dataset, there can be multiple ways of visual abstraction, leading to different visual representations. But how abstract is a visual abstraction? Can we quantitatively compare the degree of abstraction of different visual abstractions? In my presentation, I will briefly discuss the two ways that seem able to quantify the abstraction degree of different visual abstractions: visual complexity and data insight fidelity. Further, I use concrete examples to illustrate that the measurement of abstraction degree of different visual abstractions essentially depends on the data insight(s) we intend to explore (or convey to viewers). Accordingly, I conjecture that visual complexity is probably NOT a good way to quantify the abstraction degree of visual abstractions, and data insight fidelity is a promising measurement to quantify visual abstraction. An overall framework for calculating the abstraction degree is also proposed by mathematically modeling the visual encoding process, the visualization decoding process and the data insight fidelity between insights existing in the original dataset and data insights kept in the decoded data. The visualization decoding process via human perception relies on 1) the visualization resulted from the visual abstraction process and 2) relevant factors of visualization viewers like their familiarity with data visualization, prior knowledge and/or culture background. Given that the relevant factors of human viewers are difficult to model at current stage, I would propose modeling human perception process by using image processing techniques (e.g., object detection and segmentation) to extract data from visualizations, which can lead to a simplified but practical framework for quantifying the abstraction degree of visual abstractions.

Meeting Schedule

Check-in day: September 10 (Sun)

• Welcome banquet

Summary of discussions

The following text reflects the state of our discussions at the meeting. The text is not meant to represent a final proposal of a new theory in the field of visualization, it is rather a draft that will be revised into a scientific article, where the nomenclature together with the relationships might be subjected to change.

Foundations

Before we can discuss particular abstraction techniques, we first need to establish the concept of abstraction. Figures 3 to 7 show several examples from related work that have already performed this process prior to an actual formalization of it.

Definition

The definition of abstraction, as given by Viola and Isenberg [START_REF] Viola | Visual abstraction[END_REF][START_REF] Viola | Pondering the concept of abstraction in (illustrative) visualization[END_REF], was that An abstraction is a process that transforms a source thing into a less concrete sign thing of the source thing. Abstraction uses a concept of point-of-view, which determines which aspects of the source thing should be preserved in its sign thing and which should be suppressed.

Three aspects related to this definition require clarity to avoid misunderstandings:

• Transformations: What is a transformation in the sense of the definition?

• Reduction: What is meant with suppressing?

• Meaning: The concept of a point-of-view is not clearly defined, but seems to be the essential ingredient for bringing in a cognitive aspect of the term abstraction.

Hence, we propose to properly define these three aspects and tie them into the existing discussion of the process of visualization from the visualization community.

Transformations T

Data Transformations are part of any data analysis process. Whether it is a change of basis function for metric spaces (such as Fourier-or Wavelet Transforms), a one-hot encoding for text data, or graph diffusion processes, these transformations can be expressed through a general (mathematical) mapping operation T :

T : A → A ′ (1)
Here, the spaces A and A ′ are seen in the most general way. Hence, this understanding is not constrained to Data transformations per se but includes visual encoding transformations just as well as transformations in the visual space. Hence, it properly ties into the wealth of work on the Data Visualization Process / Pipeline as already discussed by Card and Mackinlay [START_REF] Card | The structure of the information visualization design space[END_REF] or Heer [START_REF] Heer | Prefuse: A toolkit for interactive information visualization[END_REF] up to such recent treatments by Kindlmann and Scheidegger [START_REF] Kindlmann | An algebraic process for visualization design[END_REF] and van Wijk [START_REF] Van Wijk | The value of visualization[END_REF].

Reduction R

One crucial aspect of abstraction is that there will be aspects that are suppressed. Hence, some kind of reduction in data or information plays a central role. Therefore, we define a reduction operator R as a transformation, which is surjective. Hence, the space of all reduction operators is a proper sub-class of all transformation operators T , i.e., R ⊂ T .

Important examples of reduction operators include clustering operators, dimensionality reduction, as well as filtering.

As a part of reduction, new, transformed reduced data is generated from some original data. This new data set is strictly smaller than the original data set. However, while the reduction process compresses the data, if we consider original data together with the newly generated data, the overall data has actually increased. We highlight that the reduction only secures that the reduced data is a smaller set from the original data, while both of these data can be utilized in subsequent data processing. Nonetheless, as through abstraction, new data is created, the underlying amount of information I does not increase.

Meaningfulness M

An essential concept of abstraction, as used within the visualization community, is the fact that it carries meaning for an individual within the context of a task or embedded within a mental model or cognitive construct. This is expressed as the point-of-view aspect in the original definition of abstraction. In order to conceptually cast it as a formal concept, we need to introduce the knowledge space K that would be required to assign meaning to some data. Hence, we define the meaningfulness M as a mapping from space A to a scalar value between zero and one with respect to some knowledge K:

M : A K -→ T [0, 1] (2)
Hence, assigning meaningfulness clearly is a form of reduction to a very special space (the unit interval). Hence, it is a proper subset of all reduction operators M ⊂ R. Abstraction A is a proper subset of all reductions R, i.e., A ⊂ R : A K > 0. Some reductions R might not have any meaning (= 0) for a particular task T and knowledge K in mind. Such reductions are not abstractions. Abstraction A is a reduction that is at least somewhat meaningful (> 0).

Coupling Reduction and Meaning

A complementary approach to the above would be the coupling of both concepts. We start from the assumption that some data D contains a certain amount of information I, and with respect to a task, this information will carry some meaning M for the user such that M (I, T) > 0. Note that, as already outlined in section 1.3, meaning is undefined unless there is a specific task.

Given that an abstraction reduces information, A : I → I ′ +R,with I ′ ⊂ I, we can exactly quantify the reduced/hidden/lost information as R. In this light, the design of an appropriate abstraction has to optimize user effort (cognitive load) while retaining as much meaning as possible: arg maxM(I ′ , T). Conversely, we can say that an accurate abstraction for a task does not lose what we now can define as meaningful information, or that the reduced information should be irrelevant for the task: M(R, T) → 0.

Faithfulness F

Closely related to our meaningfulness quantity, is previously defined concept of faithfulness. The faithfulness metrics measure how a visual representation V accurately represents the ground truth information of an abstract data D [START_REF] Nguyen | On the faithfulness of graph visualizations[END_REF]. These metrics complement existing human perception based quality metrics (i.e., how human perceive and understand the visualizations), for example, the readability metrics such as edge crossings in network visualization.

Roughly speaking, the outcome V of visualization V is information faithful, if the ground truth information of D can be reconstructed from V. V is task faithful, if it displays sufficient information to perform a specific task T on D accurately. For example, the cluster faithful metrics measure how a visualization V accurately represents the ground truth cluster of an abstract data D as a geometric cluster in V [START_REF] Meidiana | A quality metric for visualization of clusters in graphs[END_REF].

For the purpose of abstraction, the faithfulness model can be extended to include abstraction faithfulness and perception faithfulness, see Figure 1.

The process of abstraction

Now that we have a stronger formal footing of abstraction, we embed it into the concept of the visualization pipeline. First, data about a particular phenomenon or system under investigation are collected, e.g. by measuring certain characteristics or by running a simulation code. When we abstract, we initially reduce our data D by first removing items or part of their attributes A 1 : D -→ T D 1 based on the knowledge (or assumption) that a specific task T at hand does not require the information contained within them at the available level of detail 1 .

1 Please note that data abstraction as defined by Munzner [START_REF] Munzner | Visualization Analysis and Design[END_REF], refers to various data types and data set types. In Munzner's nomenclature, these could have been termed data idioms, consistently with the visualization idioms. In our definition we refer to data abstraction as to a meaningful data-reducing transformation Figure 2: Data D can be abstracted to different visual representations V i given certain tasks T i . As shown by the vertical dashed arrow, there can be connections between the visual representations in the visual-abstraction space as well-typically in the form of (human-constructed!) visual explanations that were called abstraction axes or abstraction space in the past [START_REF] Viola | Visual abstraction[END_REF][START_REF] Viola | Pondering the concept of abstraction in (illustrative) visualization[END_REF]. The mental operations in these abstraction space can be supported by superposition, animation, or morphing through the use of a metaphor. This can be caused, for example, by the sheer volume of data, or the high frequencies contained within it. Then we perform some operation that transforms the reduced data (or part of it) into another form A 2 : D 1 -→ T D 2 . The union of all data resulting from abstractions can be expressed as D ′ = D 2 ∪ D 1 ∪ D and all this digital representation of information I can be used in the next steps of the analytical or visualization pipeline. When we perform visualization V of this new form D ′ , we end up with a visual representation V that has a reduced complexity in the sense that it is easier to interpret, faster to render or is more expressive for conveying information related to task T:

V : D ′ -→ T V (3)
In case, this visual transformation does not show the full set of information contained in the original data D ′ , and it meaningfully presents relevant information with respect to the task, we call this transformation a visual abstraction. This means, even if all information from D i is visually conveyed, and no reduction happens during the transformation

D i -→ T V, the composite meaningful reduction D -→ T D i • • • -→ T
V is a visual abstraction. As a part of this process, each time we visualize data, some form of abstraction A typically takes place. In each realistic scenario where visualization is utilized, abstraction, or several abstractions take place between observation of a given phenomenon up to the synthesis of the rendered representation. Also note that, strictly speaking, abstraction augments the data in that it creates additional abstracted representations D i or V that are needed for the task, similarly as a Level of Detail (LOD) scheme or a MIP map image pyramid. We can keep the original data to facilitate switching between visualizations, or remove the unnecessary data D\ {D i }.

Characterizing Meaningfulness

Depending on the task T, a task-dependent ground truth D T can be derived from D. There can be potentially many task-dependent ground truths D Ti depending on specific task T i . One can think of D T as the digital representation of the information I ′ that needs to be communicated to the user to accomplish the task T, i.e. the subset of the information that carries most of the meaning relevant for the task (as outlined in section 1.4): arg maxM (I ′ , T). The task-dependent ground truth is normally the last part of the visualization pipeline, somewhat close to the insight or to the take-home message. Task-dependent ground truth D T will serve the purpose of understanding and quantifying the meaningfulness of an abstraction. It represents a theoretical construct that contains only data highly relevant to a particular task and nothing else. It represents the entire information a user needs to comprehend from visualization, nothing more, nothing less. While several closely related concepts such as expressiveness [START_REF] Mackinlay | Automating the design of graphical presentations of relational information[END_REF], appropriateness [START_REF] Norman | Things That Make Us Smart: Defending Human Attributes in the Age of the Machine[END_REF], and faithfulness [START_REF] Nguyen | On the faithfulness of graph visualizations[END_REF] describe the relation of visually represented data and ground truth data, these concepts do not include human perception and interpretation of the visualization.

We say that a visualization V is meaningful if the visualization user gains at These can be seen as results from individual visualization pipelines that are finally composited into a single view. Images from [START_REF] Parulek | Continuous levels-of-detail and visual abstraction for seamless molecular visualization[END_REF], used with permission.

least some information contained in the task-dependent ground truth D T . This can be measured by any of the facets of understanding: explanation, interpretation, application, perspective, empathy, and self-knowledge [START_REF] Wiggins | Understanding by Design[END_REF]. For example, the meaningfullness can be measured by psychophysical studies that quantify the match between D T and human H reconstruction of D V,H after viewing V. D V,H is the reconstructed task-relevant data that the user verbally reported on, has sketched, or has correctly identified in the context of a multiple-choice test. Such measurement inherently includes perception and cognition and forms the feedback loop shown in Figure 8.

Abstraction Characteristics

Abstractions along the visualization pipeline affect the domain A of the attributes, for example by compressing, binning, or lifting an attribute. Therefore, we also need to define A : A -→ T A ′ . This effect can be observed in Figure 3, where the histogram is a binning of one of the columns in the table and the boxplot compresses all bins into a single one that represents the whole domain, using less screen space but a more complex mark (box, line, whiskers, outliers instead of one box per bin). As defined above, the reduction is a subjective transformation, which leads to the fact that the co-domain is smaller than the domain. Therefore, domain shrinking is a natural characteristic of an abstraction process.

In the exemplary visualizations depicted in Figures 34567we can see that abstractions can have multiple levels, and we can either have distinct processes that perform visualization directly from different abstractions A in an independent fashion, but can also have abstractions that operate on the result of previous ones A i : D i-1 -→ T D i that could result in different useful visual representations V on different levels, if applied partially. The latter gives us a sequence Figure 8: Data Processing and Visualisation Pipeline: Data from a phenomenon under investigation is collected and after pre-processing a ground truth data set D is derived. A non-empty part of this ground truth, D T , will be task-relevant. The result of data analysis and abstraction, D ′ , is mapped into the visualisation space where it is perceived by the human, resulting in the perceived data D V,H for which we can evaluate meaningfulness with respect to D T . of gradually increasing visual abstractions of the underlying data D. But even without an increasing abstraction or increasing reduction of data, the different representations may be placed (by a human) in a meaningful order such that we can transition between them, in a way to explain a subject matter to others or to record insights gained from some data exploration. This is indicated by the vertical dashed arrow in Figure 2-and which were called axes of abstraction in the past [START_REF] Viola | Visual abstraction[END_REF][START_REF] Viola | Pondering the concept of abstraction in (illustrative) visualization[END_REF].

One very interesting challenge that relates to this sequence of meaningful abstracted representations is the free exploration of abstraction space, meaning the direct transition from one abstracted visualization to another (e.g., [START_REF] Miao | DimSUM: Dimension and scale unifying maps for visual abstraction of DNA origami structures[END_REF][START_REF] Mohammed | Abstractocyte: A visual tool for exploring nanoscale astroglial cells[END_REF][START_REF] Van Der Zwan | Illustrative molecular visualization with continuous abstraction[END_REF]. This could be done along one sequence of abstractions as described above, or one can also imagine a transition across different visual abstraction pipelines [START_REF] Heer | Animated transitions in statistical data graphics[END_REF][START_REF] Kim | Gemini 2 : Generating keyframe-oriented animated transitions between statistical graphics[END_REF]. While in principle the transition from one chain of transformations to another can be formalized, the actual implementation in most cases is probably rather challenging. This exploration seems to have similar properties to the faceting operation as defined by Satyanarayan et al. [START_REF] Satyanarayan | Vega-Lite: A grammar of interactive graphics[END_REF].

We propose that given a task T and a target user H, we can design a tool or system that communicates some insight by offering an appropriate set of visual representations V i and a means to arrange them. Depending on their respective domain, this can be juxtaposition, superposition, animated interpolation, ... [START_REF] Munzner | Visualization Analysis and Design[END_REF]. Which of these variants are possible depends on the different domains A as described above: while juxtaposition is generally feasible, it can, however, be very expressive if there is an overlap in the domain, like in scatterplots combined with marginal histograms. Superposition, on the other hand, strictly requires a common domain.

We observe that in theory it is possible to move arbitrarily in abstraction space, meaning that all the visual representations in the images above could be applied to a single dataset, once the correct transformations have been applied. Some combinations might seem dubious at first glance, like a node-link diagram of data without connectivity, but this does not mean there is no possible transformation that can generate meaningful connectivity from the raw data. For example, we can generate hydrogen bonds for an atomistic dataset.

Finally, we need to consider the ground truth: generally, it would represent a data set as a whole, including all possible insights. In its raw form, this ground truth might be generally so large and complex to be unintelligible. The goal of domain research often is to condense this into a (sufficiently faithful) model of the data, such that it can be used interchangeably and allows for prediction of the real world, for example. In practice, given sufficiently complex data, we usually have to settle for a ground truth with respect to some specific task D T . We can provocatively call this a partial truth, and it can itself be seen as abstraction of the ground truth. As long as we keep in mind the task T, this serves its purpose, but there are risks: A visualization designed to communicate D T biases the user towards this partial truth. This is problematic since we want to allow for explorative analysis, but that will-by design-be constrained by the currently applied abstractions. Given that a reduction R has been performed, we lose generality with respect to the ground truth, since some insights (other tasks) are potentially coupled to the information contained in what has been removed: M (R, T). It is thus vital that we formally define and document the provenance of a visualization in order to avoid fostering erroneous deductions: one can mistake absence of a feature or a data item as information while it just represents the performed abstraction. This ties back into the observation that an abstraction is defined and meaningful exclusively with respect to a task.

Categorization of Abstraction Techniques

In this section we present categories of abstraction techniques, which, broadly speaking, encompass abstractions in the data domain, abstractions in visual representation, and abstractions in interaction.

First, regarding abstractions in the data domain, within the visualization process, these include data filtering (subset), abstractions in data scale, and data sampling. Through abstractions in the data domain, we can reduce the desired information to a meaningful subset, enabling further exploration and insight communication. In terms of visual exploration, a sequence of abstractions can traverse freely the abstraction space from one representation to another, or keep the representation fixed and explore inside a single representation.

Second, methods for abstracting from data to visual representations or between visual representations include aggregation, composition, and layout change. In aggregation methods, discrete numerical distributions can be converted into aggregated numerical distributions, such as transforming data from a table visualization into a bar chart or aggregating individual points on a map into a heat map. In composition methods, we can combine different visual forms, for instance, assembling multiple data dimensions into a form resembling parallel coordinates. We can also create visual metaphors for conveying vivid data patterns. In layout change, we can utilize superposition, juxtaposition, and implicit layout forms to arrange certain relevant visualizations.

Third, abstractions in interaction primarily support higher-level user interactions to select, highlight, and explore corresponding data subsets, making them visible within the visualization. The user interaction is typically designed such that with less of time spent on interaction more of a task can be achieved. We can also employ a focus-and-context approach to direct users' attention towards the focal area, providing varying levels of detail, whether it is less or more detailed information. For example, the Magic Scale Lens and DNA Untwister in Figure 9 are employed to embed different representations into one single view using a lens, enabling users to inspect local representations as needed. These two interaction concepts tackle challenges related to occlusion and multiscale exploration. The first concept uses a lens to transform the representation of a specific focusing region. The focus region conveys information on a particular level of detail, while a more abstract representation serves as the context. This approach facilitates the seamless combination of multiple scales within a single view, allowing users to choose the level of detail they wish to display on demand. The second interaction concept is applied when an abstract representation suffices to convey the data or message effectively. For example, an untwisted DNA depiction simplifies the overall visualization without sacrificing the essential two-strand structure when intricate details are unnecessary. Within the lens, the DNA helix is untwisted into parallel strands for clarity and simplicity.

However, there are unexplored dimensions concerning how users can finely control the level of detail within the focusing region. This concern is linked to our earlier discussion on transitioning between different visual abstractions and the methods by which users control this transition. Another interesting aspect relates to user interactions within these multiscale embedded visualizations. To effectively navigate such visualizations, which encompass diverse visual abstractions within a single view, it becomes crucial to consider adaptive interaction techniques. Given that users perceive varying levels of detail, especially when working with distinct representations, they may develop diverse mental models for data manipulation and exploration, as well as expectations regarding the outcomes of specific interactions. Thus, the effect of these interaction techniques should depend on the abstraction to which they are applied, and the visual abstractions perceived by users.

Other related techniques such as faceting and overview-and-details also similarly employ visual abstractions but in a different composition or arrangement. A data dashboard may be composed of more than two visual abstractions. For example, the histogram and line chart in Figure 3 are superimposed and can be shown to the left of a box plot to give a complete picture of the data. A dashboard user can go back and forth between visual abstractions or interact with one of them to gain data insights. Similarly, a visualization of quanti-ties through gauges can represents visual abstractions that aggregate the gauge visualization on multiple levels of detail [START_REF] Matkovic | Process visualization with levels of detail[END_REF].

In scientific visualizations, blending and masking may be used to create a continuous impression between visual abstractions as in examples in Figure 7. These visual abstractions often happen by cleverly arranging (super-positioning in 3D space, potentially also combined with animations between different representations) different visual representations V i such that the viewer mentally connects the different representations, as depicted as the vertical bar in Figure 2.

Given ever-increasing and more and more complex networks that analysts have to face in many application areas, it is natural that network abstraction is of increasing interest to support the network analysis process. One example of the successful application of abstraction is the use of powergraph analysis for protein network analysis. Royer et al. [START_REF] Royer | Unraveling protein networks with power graph analysis[END_REF] show that on average over 50% of network edges are redundant due to the typical patterns found in these networks. In addition to this compression effect, powergraph analysis facilitates visual representations that greatly improve the identification of functional subunits by visual separation. A typical example for abstracting from data to visual representation of networks are Graph Thumbnails [START_REF] Yoghourdjian | Graph thumbnails: Identifying and comparing multiple graphs at a glance[END_REF], small icon-like representation of networks that support quick browsing of large network sets and high-level comparison of networks. They abstract a network to the nesting structure of connected components and k-cores, which is represented in a nested circle packing.

An abstraction technique is often associated with a strategy for presenting data, typically in a linear sequence. The resulting visualizations can be arranged either temporally, as in animations, or spatially, as in dashboards, with the primary objective of conveying a singular message. When each visualization represents a distinct subset of a dataset, this exemplifies abstraction in the data domain (filtering). If each visualization shows a distinct representation of the same dataset, this corresponds to the second form of abstraction. And, when each visualization drills down on the data, it is not unlike abstraction in interaction. Previous work [START_REF] Lu | Interactive storyboard for overall time-varying data visualization[END_REF][START_REF] Radiano | Story albums: Creating fictional stories from personal photograph sets[END_REF][START_REF] Shi | Calliope: Automatic visual data story generation from a spreadsheet[END_REF] on storytelling has explored into how to shorten or lengthen this sequence of visualizations or to control the abstraction parameter. For these purposes, animation serves as an important means of facilitating intuitive comprehension.

The design of glyph is a highly abstract visual design that must strike a balance in the selection of the most important data features for mapping, and organically combine these features into a whole. The abstraction here entails three layers of meaning. Firstly, Glyph is a form of abstract representation of complex data, with its elements being concise and understandable. Secondly, the design of the glyph requires the abstraction of data, selection of important data dimensions, aggregation of data, and acquisition of more abstract data features mapped within the glyph. Thirdly, the design of the glyph necessitates the combination of important data features, which is the process of obtaining abstract data representation.

In the generation of glyph, we should consider and practice "abstraction" in the following ways: Firstly, based on the attributes of the data (whether ordinal, sequential, or quantitative), we select dimensions for visualization mapping, find suitable combinations, and make choices based on the characteristics of the graphic elements. This process requires selecting Glyph designs that are both representative and not overly complex. Secondly, we need to sort or explore the correlations of data dimensions, select representative dimension combinations, and minimize meaningless or redundant dimensions. Thirdly, we need to organically combine the selected data dimensions, which presents a significant design space that requires further research and exploration.

Opportunities and Applications

In this section, we discuss potential application and industry opportunities derived from our findings regarding abstraction in visualization.

Main Impact in Industrial Scenarios

A formalization of abstraction in visualization could lead to benefits in industrial settings and applications:

• Using abstract visualization layers could act as "knowledge interfaces" between diverse groups of users, who are not the primary target audience of a developed visualization. Higher levels of abstraction could be used for presentations, decision making, and knowledge sharing in the industry, replacing the current standard of using slides, which is completely decoupled in terms of data provenance and data exploration, error-prone, introduces the chances for bias and manipulation, and is moreover creating additional preparation work.

• Formalized definitions and guidelines for abstraction in visualization bring opportunities for automation and reuse of visualization-on a software development and design level, but also through easier interpretation of users from other domains. We could even see standardized abstraction layers that different visualization tools agree and rely on to exchange knowledge through visualizations between user groups that work on the same data with a completely different focus.

• Temporal consistency and re-usability are currently not considered enough. Both abstract visualizations as well as the underlying data have to be accessible AND understandable decades later. Standardization (not only on a data level, but particularly for abstraction in visualization) helps to ensure that visualizations could be considered reliable and future-proof, reducing the hesitation to apply visualizations in such settings.

Role and value

Through a formalization of abstraction in visualization techniques we see a chance to increase the value and acceptance of the fundamental work of visualization researchers by redefining their perceived roles or tasks:

• Formal definition, guidelines, standards, etc. lower the barriers for adapting visualization and reduce the chance of failures.

• This could lead to the creation of expert roles for the coordination of datavisualization workflows, considering all aspects from data requirements, visualization software and data architectures, abstraction layers, design guidelines, performance, data and software life cycle management, etc.

• Formalization also allows to continuously re-evaluate and adapt existing guidelines and their applicability to the problem to solve, particularly when it comes to changing cultural background, etc.

• All this could help to define and reveal the true value of visualizations [START_REF] Van Wijk | The value of visualization[END_REF]-not just in the research area, but also to quantify and justify visualizations, their application, their risks, etc.

During our discussion, we came up with multiple roles names (reaching from "Visualization Scientist" over "Visualization Data Scientist" to "Data Intelligence Workflow Design and Optimization expert" (which is not considering the the term visualization at all, and was therefore perceived with ambivalent feelings by the participants).

The visualization community is at the interface between (business) users and data scientists, so aiming for the establishment of a term that reflects the corresponding role's responsibilities could help to make these more graspable and to increase the perceived value of visualizations in the industry. By understanding visual abstraction and abstraction in general (i.e., including statistical abstraction and algorithmic abstraction), a new generation of visualization researchers can play a more significant role in designing and optimizing data intelligence workflows.

The main aspect is to apply visual abstraction principles to guide and model the processes of establishing visualizations-but there is the need to clarify that this task is not only about design. It involves the high-level coordination to establish overarching visualization environments that provide the benefits described in Section3.1. This could lead to a higher perceived value for this "profession".

Chances in the Industry

Establishing visualization techniques in the industry is often a difficult endeavor. The reasons are manifold, and include

• The (business) value of visualizations are hard to define. Formalizing and standardizing abstraction in visualization could make this task easier.

• Visualizations envisioned by visualization experts are hard to grasp for users before they exist-particularly abstractions. Relating to existing standards might have a positive impact on this process.

• Visualization tools introduce a significantly higher complexity in terms of required resources for maintenance, computation, training, etc. than non-visual analytics. In a business setting, justifying these investments and being able to understand the risks better is crucial. Standardization increases comparability of solutions and allows better estimations.

• Related to this, standardization also increases re-usability of visualization techniques, having a benefit on all the aforementioned aspects.

• A standardization of abstraction in visualization techniques could potentially lead to the design of corresponding Domain Specific Languages (DSL)

One of the biggest advantages of well-defined abstraction layers in visualizations in the industry is workload reduction. By integrating for example standardized presentation layers, i.e., the highest level of abstraction, into standard presentation workflows, removes the task of creating additional, completely de-coupled presentation material without any provenance.

Multi-level abstract visualizations should ultimately not just be a tool for a certain group (e.g., for expert users of a scientific area), but to support whole processes for knowledge discovery, knowledge generation, knowledge sharing, and decision making between multiple groups. Full traceability over years as well as relying on meaningful representations are extremely valuable aspects for any industry.

It seems important to point out that the current need is not to define tools and templates that provide an overarching solution, but to find applicable abstraction definitions, design rules, etc. that people can agree on and that can be implemented in various ways.

Another big chance for the industry lies in using an additional side effect of the establishment of visual abstraction standards: As soon as the value is understood and accepted, it could be a driving force for increasing the data quality and the underlying data capturing processes to allow the best possible facilitation. The intrinsic motivation of people to change their processes in industry can significantly rise with a clear purpose and value.

Day 1 :

 1 September 11 (Mon) • Talks and discussions • Group photo shooting Day 2: September 12 (Tue) • Talks and discussions • Break-out discussions Day 3: September 13 (Wed) • Break-out discussions • Initial report writing • Excursion and Main Banquet Day 4: September 14 (Thu) • Report writing • Wrap-up

Figure 1 :

 1 Figure 1: Faithfulness pipeline for abstraction.

Figure 3 :

 3 Figure 3: Series of different chart visualization of information, each shows a different purpose in communicating statistical results.

Figure 4 :

 4 Figure 4: Series of different views of structural abstraction of a molecule, each generated with a differently parameterized visualization pipeline that individually focused on different aspects of the molecular data. Images from [24], images are in the public domain c p.

Figure 5 :

 5 Figure 5: Series of views from OpenSpace [1], showing different levels of detail when zooming out from the earth over the milky way to the extents of the Sloan Digital Sky Survey.

Figure 6 :

 6 Figure 6: Series of different depictions of the DNA, from the cell nucleus to individual atoms. Each view shows a different level of detail and a different subset of the data. Images by Halladjian et al. [3], c b CC-BY 4.0.

Figure 7 :

 7 Figure 7: Examples of visual representations that transition between different visualization parameterizations depending on the distance from the viewer.These can be seen as results from individual visualization pipelines that are finally composited into a single view. Images from[START_REF] Parulek | Continuous levels-of-detail and visual abstraction for seamless molecular visualization[END_REF], used with permission.

2

 2

Figure 9 :

 9 Figure 9: The Magic Scale Lens: within the Magic Scale Lens, the single strand scale (Focus) is embedded into the double strand scale (Context). Images from [8], c b CC-BY 4.0.

One example could be the "ortho" rule in Google Sketchup software, where modeling in

3D assumes that lines connect under 90 degree angle, when interpreting positioning of line endpoint in 3D space from a view projection image

List of Participants