Micheline Elias

Marie-Aude Aufaure

Anastasia Bezerianos

Enhancing User Interaction with Business Intelligence Dashboards

Keywords: Visualization Dashboard, Business Intelligence, Annotation, Storytelling, User Experience

The goal of this thesis is to improve visualization dashboards in order to help decision making and Business Intelligence (BI) analysis, and make them accessible to a larger user audience. We identify current trends and use practices of visualization dashboards in the BI domain, and following a user-cantered design approach we provide enhancements and solutions. More specifically we address the following challenges: making dashboard construction and use accessible to visualization novices, enhancing dashboards with advanced annotation capabilities to help BI analysts in the analysis process, and add storytelling functionality to dashboards to help the communication between analysts and decision makers. Our user-centered approach consists of gathering user requirements (through background search, in-depth interviews or prototyping sessions with BI experts), iterative prototype development, and evaluation of our prototypes with representative users.

First, our work reports gathered insights regarding novice user practices on Exploration Views (EV), a system that allows novice visualization users to easily build and customize BI information dashboards, but also provides functionality needed by experts. We evaluated EV with both expert and novice visualization users, and found differences in how experts and novice users interact with the interface. Based on those observations we provide new guidelines that augment previous work on designing for visualization novices, in the context of interactive visualization systems in the form of dashboards. BI analysts using dashboards (experts or novices) need support in record-keeping of their analysis, their data discoveries and gathered insights. This support is lacking in most dashboard visualization systems. We conducted in-depth interviews with BI experts that led to new annotation needs for multi-chart visualization systems (such as dashboards), on which we based the design of a dashboard prototype that supports data and context aware annotations, shared across visualizations and datasets. We focused particularly on novel annotation aspects, such as multi-target annotations, annotation transparency across charts, and annotations that can be shared among different data-sources and BI applications. We identified and provided solutions to challenges in using context aware annotations, notably issues arising when the annotation's "context" changes (e.g. annotated data are deleted or changed). Our prototype was evaluated with BI experts that were able to easily perform several tasks that they deemed important in their work.

To increase the value of an analysis, recorded insights and knowledge need to be organized and communicated to others, and made available for future analysts to learn from. This communication step is very important, as often the analyst of data and the decision maker are two different people. To achieve this we need an easy insight organization and sharing mechanism to facilitate the transfer of knowledge, experiences, and stories, to decision makers and other analysts. We interviewed BI analysis experts and collected new requirements related to BI visual storytelling. Based on these requirements we designed and implemented a storytelling prototype that is integrated in a dashboard analysis tool, and allows easy transition from analysis to story creation and sharing. Our system was identified by experts as having great potential for training other analysts. Moreover, it can help BI analysis authors organize and communicate their findings faster to decision makers, and they can reach a broader audience since the results of a data analysis presented in the storytelling tool can be read with little to no training.

The main contributions of this work are:

(1) Report how novice users interact with the EV system and the differences in how novice and expert users react to EV dashboard system that targets both.

(2) Provide new design guidelines for practitioners building dashboard applications, on the needs of novice visualization users.

(3) Define a new set of requirements for annotations, targeting dashboards and other coordinated view visualization systems.

(4) The design and evaluation of a prototype dashboard that supports context-aware annotations.

(5) Identify and provide solutions to challenges when using context aware annotations, such as issues arising when the "context" changes (e.g. annotated data are deleted).

(6) Based on in-depth interviews with expert BI analysts, we derive and provide a better understanding of the actual practices in creating BI stories, and extract new requirements for enhancing BI visual analysis tools with storytelling capabilities.

(7) Following these requirements and using a user centered design approach, we design and implement a prototype storytelling tool that is fully incorporated in an existing visual analysis dashboard. We report on feedback from BI experts creating a story on the usefulness of the prototype as a communication and teaching medium, and from BI novices reading a story on its effectiveness for story communication.

Résumé

Dans la première étape, notre travail rapporte idées recueillies au sujet des pratiques des utilisateurs novices sur Exploation Views (EV), un système qui permet aux utilisateurs novices de visualisation à créer facilement des dashboards et les personnaliser, mais fournit également des fonctionnalités nécessaires aux les experts. Nous avons évalué EV à la fois d'experts et des utilisateurs novices de visualisation, et constaté des différences dans la fac . on dont les experts et les utilisateurs novices interagissent avec l'interface. En se basant sur ces observations nous proposons de nouvelles lignes directrices, qui augmentent les travaux antérieurs, sur la conception pour les novices, dans le contexte des systèmes de visualisation interactifs sous forme de dashboard.

Les analystes BI utilisant des dashboards (experts ou novices) ont besoin d'aide dans la tenue des dossiers de leur analyse, de leurs découvertes et des idées recueillies des données.

Ce soutien fait défaut dans la plupart des systèmes de visualisation de dashboard. Nous avons effectué des entretiens approfondis avec des experts en analyse BI, qui ont révélé de nouveaux besoins d'annotation pour les systèmes de visualisation multi-graphiques (tels que des dashboard), sur lesquels nous avons basé la conception d'un prototype de tableau de bord qui gère les données et les annotations contextuels, qui peuvent être partagées à travers visualisations et des ensembles duel données. Nous nous sommes concentrés en particulier sur les aspects d'annotation innovantes, tels que les annotations multi-cibles, la transparence des annotations à travers des graphiques, ainsi que les annotations qui peuvent être partagées entre les différentes sources de données et des applications de BI.

Nous avons identifié et fourni des solutions aux problèmes en utilisant les annotations contextuels, notamment les questions qui se posent lorsque le contexte de l'annotation change (par exemple les données annotées sont supprimés ou modifiés). Notre prototype a été évaluée avec des experts, utilisateurs BI, qui ont réussi d'effectuer facilement plusieurs tâches qu'ils estiment importants dans leur travail. Les contributions principales de ce travail sont:

(1) Rapport de la fac . on dont les utilisateurs novices interagir avec le système EV, et les différences entre la fac . on dont les utilisateurs novices et les experts réagissent au système de dashboard EV.

(2) Fournir des lignes directrices de conception des applications de dashboad, liées aux besoins des utilisateurs novices de visualisation.

(3) Définir des exigences pour les annotations, sur les dashboard et autres systèmes de visualisation de vues coordonnées .

(4) La conception et l'évaluation d'un dashboard qui supporte des annotations contextuels.

(5) Identifier et apporter des solutions aux problèmes de l'utilisation des annotations contextuels, comme les questions qui se posent lorsque le contexte change (par exemple données annotées sont supprimées ou modifiées).

(6) Basé sur des entretiens avec des experts analystes BI, nous dérivons une meilleure compréhension des pratiques réelles dans la création d'histoires BI, et nous extrairons de nouvelles exigences pour améliorer les outils BI d'analyse visuelle avec les capacités de narration visuelle. [START_REF] Foody | User-centered business intelligencet[END_REF] En se base sur ces exigences et en utilisant une approche de conception centrée utilisateur, nous concevons et mettons en oeuvre un outil de narration qui est entièrement incorporé dans un dashboard, pour transmettre fluidement entre l'analyse et la création d'une narration. Nous rapportons les commentaires des experts BI qui créent une narration, sur l'utilité du prototype comme une moyenne de communication et d'enseignement.

Nous rapportons aussi sur l'utilisation du prototype par de novices BI qui lisent une narration fait par notre prototype.

5.1

First page of a BI report given by an expert: a dashboard snapshot that presents the bug status in a specific development project. (Used with the experts' permission) .............................. 5.2 An example of a BI report given by an expert that was used in the evaluation of our system. This dashboard represents the "technical backlog" & "functional users stories" items status in different perspectives. [START_REF] Segel | Narrative visualization: Telling stories with data[END_REF] Green group, represents the status of the items per team. (2) Yellow group, shows the created items versus the completed "done" items, and the category (critical, major, etc) of each. (3) Pink visualization presents the remained items to be completed and its progression through time. .............. 5. [START_REF] Lopez | Early findings from the 2010 gartner ceo and business executive survey[END_REF]

Abstract

The explosive growth in the amount of data poses challenges in analyzing large data sets. Among the large number of available BI systems that help with large data set analysis, are BI dashboards. BI dashboards provide BI analysts with advanced visual analytics capabilities. But business users still spend too much time creating dashboards, analyzing their data, and preparing them for sharing findings with others. The motivation of our work is to improve the above processes. In this chapter we present the broad context of this work that flows between HCI, BI, and Data visualization, with a particular focus on dashboards, which are the main focus of this thesis. We present in this chapter the thesis motivation, approach, challenges that we address, as well as the content of the following chapters.

Research context

"There is nothing better than a picture for making you think of questions you had forgotten to ask (even mentally)." -John W. Tukey (1985) The amount of digital data in our world is growing at an incredibly quick rate according to research by the International Data Corporation (IDC) [2], and the increasing volume and detail of information captured by enterprises will fuel exponential growth in data for the foreseeable future. Therefore, analyzing large datasets is a key focus in business [START_REF] Lopez | Early findings from the 2010 gartner ceo and business executive survey[END_REF].

Extraction of hidden predictive information from large databases helps companies predict future trends and behaviors, and make decisions. This leads more people in every business to try to understand and exploit big data, not just managers that used to be the main target group of Business Intelligence systems. This emphasizes the need to reduce the cost of finding information as well as getting relevant answers to complex questions easily.

Business Intelligence (BI) deals with the collection of processes and software that supports organizations in understanding large business datasets, retrieving and analyzing information and making decisions. The value of visual presentation of data was identified early on in this field, and it usually takes the form of dashboards, interfaces resembling an automobile's dashboard that organize and present information in an easy to read manner [START_REF] Sorenson | Dictionary definition of dashboards, searchcio.com[END_REF]. BI visualization dashboards provide collections of multiple visual components, such as charts, on a single view so that information can be monitored at a glance [START_REF] Cutroni | Google Analytics[END_REF].

The current life-cycle of a BI dashboard involves multiple actors [START_REF] Chiang | Creating dashboards: The players and collaboration you need for a successful project[END_REF], including end-users and business analysts. In user-centered BI [START_REF] Foody | User-centered business intelligencet[END_REF] end-users intervene and provide feedback to the business analysts that create customized dashboards to meet the end-users needs.

This feedback comes at different stages of the dashboard design and setup, and involves a large amount of communication between business analysts and end-users. Thus there is an intrinsic delay introduced in any end-user requirement change or customization request.

When end-users needs are understood, the analysts conduct data analysis, and prepare their findings to present to their end-users or other analysts.

In this thesis we focus on enhancing and improving user interaction with BI dashboards in the different parts of a dashboard life cycle. Our work falls under the Human-Computer Interaction (HCI) field, that aims to improve the interactions between users and computers by making computers more usable and receptive to the user's needs. Following a popular HCI methodology, we adopted a user-centered design approach to understand the users' Chapter 1. Introduction 3 requirements toward dashboard creation and analysis, and propose solutions for each step in the life of a dashboard.

Successfully analyzing big data sets offers considerable benefits to companies and organizations (e.g. a competitive edge against competitors), but also to individual users (e.g. a better investment plan). Thus we have seen an increase of visualization platforms targeting visualization novices. Researchers are now focusing on gathering insights regarding novice user practices. But dashboards are still only created by experts due to a complex and long creation process, that often cannot be conducted by end-users, and that involves a large amount of communication between business analysts and end users. This long creation process that requires special expertise makes dashboards less popular with novice users, and less accessible to a broader audience; in our work we focused on making dashboard creation accessible to visualization novices, developing and testing a dashboard that is easy to create and customize.

While attempting to aid users in their analysis, we identified the need of annotation and organization of findings as of particular interest, as in other visual analysis domains.

Despite the growing use and research work on annotations for visual data analysis in general, related research on the domain of BI dashboards and other complex multi-chart visualization environments (coordinated views), has not equally progressed. Our work attempts to rectify that. Through user interviews we found out that users need more sophisticated solutions for annotating dashboards. Those requirements were implemented in a context-aware annotation dashboard prototype that was evaluated by BI experts.

In the last part of the dashboard's life, the experts' insights, knowledge and analysis stories need to be recorded in order to communicate them to others (e.g. end-users) more efficiently and make them available for future analysts to learn from. This communication step is very important, as often the analyst of data and the decision maker are two different people. Based on requirements from analysts we designed and implemented a storytelling prototype tool that is integrated in a dashboard analysis tool, and allows easy transition from analysis to story creation and sharing.

In the next sections we delve into each research field related to this thesis.

Business Intelligence Systems

"Business Intelligence is a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making." [START_REF] Evelson | Topic overview: Business intelligence[END_REF] This thesis focuses on Business intelligence systems (BIS). BIS are interactive systems intended to help decision makers use communication technologies, data, documents, knowledge, and analytical models to identify and solve problems, thus a BI system can be called a decision support system (DSS) [START_REF] Power | A brief history of decision support systems[END_REF]. We focus on Online Analytical Processing (OLAP) tools, that enable users to interactively analyze multidimensional data from multiple perspectives. The core of the OLAP technology is the data cube Figure 1.2, which is a multidimensional database model. The model consists of dimensions and measures. The measures are numerical data such as revenue, cost, sales and budget. Those are categorized by dimensions, which are used to group the data similar to the "group by" operator in relational databases. Typical dimensions are time, location and product, and they are often organized in hierarchies. A hierarchy is a structure that defines levels of granularity of a dimension and the relationship between those levels. A time dimension can for example have hours as the finest granularity, and higher up the hierarchy can contain days, months, quarters and years. In the past, business intelligence systems were used to answer OLAP queries in table based reports that answered multiple queries separately. The new generation of BIS has two salient features: integration of multiple query answers and visualization of these answers (e.g. using visualization dashboards).

Data Visualization and Visual Analytics

Friendly [START_REF] Friendly | Milestones in the history of thematic cartography, statistical graphics, and data visualization[END_REF] has identified data visualization as "the study of the visual representation of data, meaning -information that has been abstracted in some schematic form, including attributes or variables for the units of information-."

The main goal of data visualization is its ability to visualize data, communicating information clearly and effectively. It doesn't mean that data visualization needs to look boring to be functional or extremely sophisticated to look beautiful. To convey ideas effectively, both aesthetic form and functionality need to go hand in hand, providing insights into a rather sparse and complex data set by communicating its key-aspects in a more intuitive way.

-Vitaly Friedman [START_REF] Friedman | Data visualization and infographics[END_REF] Data visualization plays a major role in supporting data analysis and summarize the main characteristics of a data set in an easy-to-understand form, often with visual graphics or charts. It permits the analytical reasoning process to become faster and more focused [START_REF] Tufte | The visual display of quantitative information[END_REF].

Visualization research seeks to supplement human cognition by leveraging human visual capabilities to make sense of abstract information [START_REF] Mackinlay | Readings in information visualization: Using vision to think, volume Learning with animation[END_REF].

Visualizing multidimensional data is a challenging problem, because the human perceptual system is well equipped to process data presented in maximum three dimensions. Beyond this limit we cannot simply map the data into a graphical construction. Usually data is considered multidimensional if it has more than three dimensions. An important challenge in multidimensionality is the visualization of the relationships between data items across dimensions.

Nevertheless as explained by James et al. [START_REF] James | Illuminating the Path: The R&D Agenda for Visual Analytics[END_REF], specific types of visual representations and interaction techniques allow users to see, explore, and understand large amounts of information at once. In the domain of multidimensional visualization these include:

scatterplot matrices [START_REF] Hartigan | Printer graphics for clustering[END_REF], multidimensional glyphs (such as StarGlyphs [START_REF] Ribarsky | Glyphmaker: Creating customized visualizations of complex data[END_REF][ 17] [START_REF] Ward | Xmdvtool: integrating multiple methods for visualizing multivariate data[END_REF] and Chernoff Faces), parallel coordinates [START_REF] Dâăźocagne | Coordonnees paralléles et axiales: mèthode de transformation geométrique et procédé nouveau de calcul graphique déduits de la consideration des coordonnées paralléles[END_REF][ 20] [START_REF] Wegman | Hyperdimensional data analysis using parallel coordinates[END_REF] or starplots, multidimensional scaling [START_REF] Borg | Modern Multidimensional Scaling: Theory And Applications[END_REF], tabular visualization techniques [START_REF] Codd | A relational model of data for large shared data banks[END_REF][ 24] and coordinated and multiple views [START_REF] Roberts | State of the art: Coordinated & multiple views in exploratory visualization[END_REF].

This thesis focuses on coordinated and multiple views such as dashboards, described next, as they are the most popular within the BI domain.

Coordinated and Multiple Views represent a highly interactive visualization environment that enables the user to see diverse datasets in various forms, to manipulate the visual presentation in different ways, and also interact and coordinate the interaction between the different views [START_REF] Roberts | State of the art: Coordinated & multiple views in exploratory visualization[END_REF]. Coordinated and Multiple Views help users to generate new information and understand their data better by interacting with the presented information and viewing it through different representations, that are all visible at the same time.

Often the discussed visualization techniques are part of a larger set of visual analytics tools (VA). Visual Analytics has been identified as "the science of analytical reasoning supported by highly interactive visual interfaces" [START_REF]Ieee visual analytics science and technology (vast) symposium[END_REF]. VA combines automatic data analysis methods (e.g. data cleaning and data mining) and interactive visualizations. People use VA tools and techniques to derive insight from complex and large data and synthesize information into knowledge, that directly support assessment, planning, and decision making. There is a demand for VA support in different application areas where large information spaces have to be processed and analysed, such as monitoring climate and weather, biology, medicine and business. Our work focus is Business Visual Analytics, that aids in analysing huge business data sets and permits taking right decisions. Nevertheless our focus is mainly on the interaction and visualization challenges, and less on the automatic analysis methods.

Dashboards

A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance.

-Stephen Few [START_REF] Stephen | Dashboard Confusion[END_REF] In the domain of business intelligence (BI) the main visualization tool is BI dashboards [START_REF] Cutroni | Google analytics[END_REF]. Dashboards can provide an effective solution to the overwhelming amount of data that business users experience every day [START_REF] Stephen | Information Dashboard Design: The Effective Visual Communication of Data[END_REF]. Current market trends (seen in the Gartner Survey [START_REF] Lopez | Early findings from the 2010 gartner ceo and business executive survey[END_REF]), recognize dashboards and data analysis are one of the driving forces adding value to companies. This indicates a tendency for an expanding end-user population with diverse needs.

Dashboards can be classified according to their role. Few [START_REF] Stephen | Dashboard Design for Real-Time Situation Awareness[END_REF] has identified the following dashboard types: strategic, analytical, operational, or informational.

Most often, dashboards are used for strategic purposes [START_REF] Stephen | Information Dashboard Design: The Effective Visual Communication of Data[END_REF], they provide a general and quick overview of the organization's "health". They provide quick overviews and comparisons to target values as well as simple performance evaluations (e.g. good or bad).

The analytic dashboard, is designed to help in data analysis. This can include making comparisons, reviewing extensive histories, evaluating performance, and supports interactions with the data such as drilling down into different data levels.

Operational dashboards are for monitoring data, that needs real-time, and minute-byminute attention, and has no statistics or analysis functionalities. The provide alerts and clear presentation of the appropriate response in an emergency event. They are generally used by those on a departmental, rather than executive level [START_REF] Rasmussen | Business Dashboards, A Visual Catalogue for Design and Deployment[END_REF].

Dashboards are tailored to fit specific requirements, and a dashboard does not need to fit a single category in order to be successful. Rather, it needs to provide clear and immediate information in order to improve decision making and understanding of data [START_REF] Stephen | Dashboard Confusion[END_REF]. In this thesis we kept the dashboard purpose open to meet different requirements of BI domain analysts.

Methodology

"Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them."

-Hewett, Baecker, Card, Carey, Gasen, Mantei, Perlman, Strong and Verplank [START_REF] Hewett | ACM SIGCHI Curricula for Human-Computer Interaction[END_REF] The main goal of Human Computer Interaction HCI is to make computers more userfriendly by minimizing the barrier between the human's goal of what they want to accomplish and the computer's understanding of the user's task.

One of the most significant HCI design methodologies is User-centered design Figure 1.3, in which users take center-stage in the design of any computer system, and is the design methodology we followed to understand our users' needs and propose solutions in this work. (2) Specify requirements: Identify any business requirements or user goals that must be met for the product to be successful. (3) Create design solutions: This part of the process may be done in stages, building from a rough concept to a complete design. (4) Evaluate designs: The most important part of this process is that evaluation -ideally through usability testing with actual users -is as integral as quality testing is to good software development.

A central concept in HCI is that of usability [START_REF] Carroll | Human Computer Interaction (HCI). The Interaction-Design[END_REF], ie "The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use".

[ISO definition]

Many methods exist for studying usability, but the most basic and useful is user testing, which has 3 components as identified by Nielsen [START_REF] Nielsen | Usability 101: Introduction to usability[END_REF]:

-Get hold of some representative users, in our case Business Intelligence experts or novices.

-Ask the users to perform representative tasks with the design.

-Observe what the users do, where they succeed, and where they have difficulties with the user interface. This is the method we have followed to improve the usability of the different prototypes implemented during this thesis.

Thesis Focus

This work followed a user centred design approach to enhance user interaction and address the challenges when creating, customizing and using dashboards or coordinated view systems, in the BI domain for expert and novice users. We intervene in all aspects of life cycle of a dashboard: creation, analysis and communication.

Creation:

The tendency for expanding dashboard end-user population with diverse needs, requires users, who are often novices in information visualization, to get quick access to customizable and easy to build Business Intelligence information dashboards. This is reflected in advances in easily customizable dashboard visualization systems (e.g. [35][36]), it coincides with advances in easily accessible information visualization environments for novices (such as ManyEyes [START_REF] Viegas | Manyeyes: a site for visualization at internet scale[END_REF]), and is also reflected in the increasing focus of the InfoVis research community on how to support novice users [38][39]. Nevertheless, there are still many questions about how novice users interact with actual information visualization systems and more so with dashboards. In the BI analysis domain, software vendors focus usually on adding more features into the next dashboard release, with no time to do the research needed to discover what actually works, or even to step back and observe how their products are really being used.

To find out the answers to such questions we gathered insights regarding novice user practices on Exploration Views (EV), a system that allows novice visualization users to easily build and customize Business Intelligence information dashboards. EV provides an intuitive environment for dynamically creating, rearranging, searching and exploring multiple visual data representations from diverse data-sources. These aspects aid users Chapter 1. Introduction 10 to better retrieve, experiment and familiarize themselves with their data. We provide new guidelines that augment previous work on designing for visualization novices, in the context of dashboards. We expand on this aspects in Chapter 3.

Analysis: Just collections of visual representations do not speak for themselves; to become meaningful they require interpretation and explanation. But because of the limits of human memory [START_REF] Cowan | Attention and Memory: An Integrated Framework[END_REF] remembering all reasoning details and milestones in the analysis process becomes challenging. We report dashboard limitations in analysis record-keeping, data discovery and gathering insights.

We conducted in-depth interviews with dashboard experts and reported the need of making sense of their visualizations by annotating them with their findings. This led to new annotation needs for multi-chart visualization systems, on which we based the design of a dashboard prototype that supports data and context aware annotations that can be shared across visualizations and datasets. We focused particularly on novel annotation aspects, such as multi-target annotations, annotation transparency across charts, as well as attaching annotations to a data layer shared among different data-sources and BI applications, allowing cross application annotations.

This prototype was evaluated by BI experts by performing several important tasks in their work. We also identified and provided solutions to challenges in using context aware annotations, notably issues arising when the annotation's "context" changes (e.g. annotated data are deleted). We discuss this work in detail in Chapter 4.

Communication: Although the work practices of BI analysts' may differ, in the end they all construct stories that include: collections of visual components (dashboards, charts or tables) to support their stories, verbal explanations during presentation(s), or detailed textual annotations and descriptions in the form of written reports and external resources such as wikis. This process remains cumbersome and requires frequent switching between different software (dashboard analysis tools, screen captures, text editors, presentation or drawing tools, etc). BI analysts need an easier mechanism to share their analysis results

and tell stories with their data.

Our work aims to aid BI analysts with their storytelling needs, in order to develop a high analysis value. Our interviews indicates that their recorded insights and knowledge, need to be organized and transferred to others, as well as remain available for future users.

To achieve this we need a designing and sharing mechanism to facilitate the transfer of knowledge and experiences. We thus collected new requirements related to BI visual storytelling. Based on these requirements we designed and implemented a storytelling prototype tool that is integrated in a dashboard analysis tool and allows easy transition from analysis to story creation and sharing. Chapter 5 provides details on this work.

Thesis Contributions

The main contributions of this work are:

(1) Report of how novice users interact with the EV system and the differences in how novice and expert users react to EV dashboard system that targets both.

(2) New design guidelines for practitioners building dashboard applications, for novice visualization users.

(3) New set of requirements for annotations, targeting dashboards and other coordinated view visualization systems.

(4) Design and evaluation of a prototype dashboard that supports context-aware annotations. Using this system, we explain how annotations support advanced analysis functionality such as: "annotate once, see everywhere", annotations for multi-chart visualizations, annotation recommendations to promote knowledge discovery, and the use of the common data model layer present in BI systems to link annotations with external data sources and enable cross application annotations and information foraging.

(5) Identification of challenges and possible solutions when using context aware annotations, such as issues arising when the "context" changes (e.g. annotated data are deleted or changed at the data source).

(6) A better understanding of the actual practices in creating BI stories, based on in-depth interviews with expert BI analysts, and emphasize the need of BI storytelling support.

We extracted new requirements for enhancing BI visual analysis tools with storytelling capabilities.

(7) The design and implementation of a prototype storytelling tool that is fully incorporated in an existing visual analysis dashboard, to fluidly transition between analysis and story creation. We report on feedback from BI experts creating a story on the usefulness of the prototype as a communication and teaching medium, and from BI novices reading a story on its effectiveness for story communication. Chapter 2

Related Work

Abstract

In this thesis we focus on three aspects of interaction with BI dashboards, begining with (1) dashboard creation and customization. (3) Third we focus on communicating business analysis results to others, and how storytelling that has been used with other data visualization domains can inspire similar work in dashboards.

Introduction

This chapter reviews the background work related to this thesis, organized in the three main areas studied by this thesis: dashboard construction and customization, visualization annotation, and narrative visualizations (storytelling).

Dashboards

There are dozens of software packages that mine data repositories and present details on how a business is performing. Sometimes called analytics or business intelligence software, they aggregate data from disparate internal and external sources and display it in the form of customized views. The fashionable term for these views is digital dashboards.

-Jeffrey Schwartz [START_REF] Schwartz | New Digital Dashboards Help Drive Decisions[END_REF] In this section we will present related research in the fields of dashboard creation and visualization construction, especially in the context of novice visualization users.

Dashboards creation

Few [START_REF] Stephen | Dashboard Design for Real-Time Situation Awareness[END_REF] has identified the characteristics of a well-designed dashboard, that can be used to evaluate dashboards. Information should be 1. Well organized. Dashboards should help people to immediately recognize what needs their attention.

2. Condensed, primarily in the form of summaries and exceptions. Summaries represent a set of numbers (often a large set) as a single number such as sums and averages, while exceptions represent something unusual happening that could be a problem or an opportunity. Thus, viewers don't wade through hundreds of values when only one or two require attention.

3. Specific to, and customized for, the dashboard's audience and objectives.

4. Displayed using concise and often small media that communicate the data and its message in the clearest and most direct way possible.

The majority of the commonly used commercial BI dashboards (e.g. Dundas [START_REF]Dundas dashboard[END_REF], OracleBI 10g [START_REF]Oracle bi 10g dashboard[END_REF], SAP BusinessObjects dashboard [START_REF]SAP business objects xcelsius[END_REF], see (Figure 2.1)) assume that from the end-user perspective the dashboard design will remain consistent, and focus more on easy report generation.

To create new or customize existing dashboards, end users need to either contact trained dashboard designers, or install separately provided dashboard design components and spend time to train themselves. There are some notable exceptions. Tableau [START_REF] Tableau | [END_REF] incorporates the design functionality in the dashboard system. Spotfire [START_REF]Spotfire[END_REF] also accommodates novice users with a flexible drag-and-drop interface for customizing visualizations on the fly. Finally, the prototype Choosel environment [45] allows for easy visualization creation and further exploration of the visualized data, but does not yet provide advanced functionality (e.g. hierarchical data representation as in OLAP data models [START_REF]OLAP and olap server definitions[END_REF], with drillup/down functions), required in real life business data analysis. Thus with few exceptions, dashboards are hard to create and customize.

Visualization creation

A variety of visualization toolkits allow the creation of sophisticated visualization charts and combinations of them (e.g. Flare1, Silverlight2, JavaScript InfoVis tk3 [47], ivtk4 [48],

D3 [START_REF] Bostock | D3: Data-driven documents[END_REF]). These do not target novice visualization users, and usually require additional programming to create the visualizations, to synchronize them in dashboard views, and to connect them to data-sources.

Several environments allow the construction of highly customized, powerful and fully linked visualizations. For example Improvise [START_REF] Weaver | Building highly-coordinated visualizations in improvise[END_REF] (Figure 2.2) provides dynamic queries where users can interactively explore and change parameters of representations. Snap-Together

[51] (Figure 2.3) allows the user to mix and match visualizations, load relations into visualizations then coordinate them based on the relational joins between them to construct custom exploration interfaces. (For a comprehensive and more detailed list of work on coordinated views see [START_REF] Roberts | State of the art: Coordinated & multiple views in exploratory visualization[END_REF]). Nevertheless their setup cost and learning curve is usually high and they target visualization experts rather than novice users.

Visualization for novices

On the other hand, there is a set of visualization platforms that specifically target novice users. For instance, the Google analytics platform [START_REF] Cutroni | Google analytics[END_REF] provides a highly customizable dashboard with good default visualization charts for web-related data. Other web-based visualization platforms easily accessible to novice visualization users, like ManyEyes [START_REF] Viegas | Manyeyes: a site for visualization at internet scale[END_REF],

Sense.us [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF], or Polstar [START_REF] Cubrarnic | Polstar: Assisted navigation for exploring multi-dimensional information spaces[END_REF], usually restrict users to a viewing and sharing a single visible chart at a time. Thus creating a dashboard is a laborious process performed outside the platform, and linking of multiple charts is no longer possible.

Novices in visualization

With the increase of visualization platforms targeting novices, researchers have started gathering insights into their practices and behaviors. Heer et al. [START_REF] Van Ham | Creation and collaboration: Engaging new audiences for information visualization[END_REF] group different visualization systems by how skilled their users have to be, and note that tools supporting detailed data exploration target mostly experts, whereas it is mostly communication/reporting systems that accommodate novices.

They also suggest that visualization tools for novices should:

(G1) Allow user-friendly data input for common data formats.

(G2) Automatically provide visualizations or reasonable defaults based on data types.

(G3) Use contextual information to clarify the displayed data and encoding. Grammel et al. [START_REF] Storey | How information visualization novices construct visualizations[END_REF] investigate how novices create single charts, using a human mediator to interact with the creation interface. They found that the main activities performed by novices (apart from communicating their specifications to the mediator) are data attribute selections, visual template selections, and viewing and refinements of the above.

They observed that subjects often had partial specifications for their desired visualization, and faced three major barriers: selecting correct data attributes, choosing appropriate visual mappings, and interpreting the visual results.

They also verified suggestions from [START_REF] Van Ham | Creation and collaboration: Engaging new audiences for information visualization[END_REF], and provided additional guidelines, such as:

(G4) Facilitate searching of attributes.

(G5) Automatically create visualizations when possible.

(G6) Provide explanations.

(G7) Promote learning.

Our work follow the above seven guidelines from Grammel et al. [START_REF] Storey | How information visualization novices construct visualizations[END_REF] and Heer et al. [START_REF] Van Ham | Creation and collaboration: Engaging new audiences for information visualization[END_REF],

and extends those design guidelines for BI information dashboards, respecting Few's [START_REF] Stephen | Dashboard Design for Real-Time Situation Awareness[END_REF] Chapter 2. Related Work 20 dashboard creation guidelines:

-Fit a great deal of information into a small amount of space, resulting in a display that is easily and immediately understandable.

-Base efforts on a firm understanding of how people perceive and think, building interfaces, visual displays, and methods of interaction that fit seamlessly with human ability.

-The ability to see everything you need at once in one view instead of multiple views, without fragmenting data.

-Try to present all within eye span. Something critical is lost when loosing sight of some data by scrolling or switching to another screen to see other data.

We build a fully functional dashboard visualization system, and observed how novice users use the interface to create dashboards (without a human mediator [START_REF] Storey | How information visualization novices construct visualizations[END_REF]). In CHAPTER 3

we report our observations of novices and experts reactions, and derive additional design guidelines for visualization dashboards that target both user groups.

Annotations

An annotation is a commentary on an object, in order to add information. This information could be represented by text, link, photo, audio or a video. It is a common practice during long term human activities to take notes and annotate documents in order to indicate or highlight interesting items that need further investigation, mark items that have been completed or remain to be done or add explanations and insights to facts.

Current visualization and BI systems provide minimal (if at all) annotation capabilities in the form of simple text. As BI dashboards are highly visual in nature, there is a need to support annotations and comments on the visualizations.

In order to understand the challenges of annotating on dashboards, determine the best annotation suitable support, and ways to navigate through these annotations, we report on existing digital annotation scenarios in recent research.

Annotation of Textual data

An annotation is frequently used to support critical thinking and learning while reading, a process called "active reading" by Adler at al. [START_REF] Adler | How To Read A Book[END_REF]. Marshall [START_REF] Marshall | Annotation: from paper books to the digital library[END_REF] has examined the practice of annotation on textbooks to help designing annotation tools, specifically for readers working in a digital library tool. This study gave a clear identification of annotation usage, such as:

Chapter 2. Related Work

21

-Annotations as procedural signals: annotations are used in anticipation of future attention, to designate reading assignments, responsibility for "knowing", and desire to reread.

-Annotations as place-markings and aids to memory: a way of remembering or remembering to remember.

-Annotations as in situ locations for problem-working: markings were used to indicate action (or to help visualize action) to take at a later date.

-Annotations as a record of interpretive activity: as notes of interpretations of unfamiliar language, interpretation of structures, or adding insights.

-Annotations as a visible trace of reader's attention: when the material is difficult and in narrative form; attention is easier to maintain if the material is relatively accessible.

-Annotations as incidental reflections of the material circumstances: they reflect circumstances that are entirely outside of the realm of the text.

Moreover, text annotations can be used to support feedback exchange through discussion threads, that can benefit Information Retrieval systems [START_REF] Cabanac | Collective annotation: perspectives for information retrieval improvement[END_REF].

These important practices of text annotation for reading support, are translated to digital libraries using "annotation systems" such as Annotea [START_REF] Kahan | Annotea: an open rdf infrastructure for shared web annotations[END_REF]. Digital annotations can persist throughout the lifetime of a document, and can easily be shared. But the variation of reading platforms poses a unique challenge in the adaptation of freeform pen-and-paper annotation to the digital domain: Each time the content of a digital document reflows to a new layout, any digital ink annotations must also reflow to keep up with it as done in Bargeron et al. work [START_REF] Bargeron | Reflowing digital ink annotations[END_REF].

The annotated data in this context is text, so it will appear in the form of letters on lines only, while in chart visualization, data has different visual representations (visualization types) and different granularities.

Annotation of Image Data

Image annotation deals with annotating and commenting directly on an image. This means that the annotation is associated with a collection of pixels and is usually stored as a layer on top of images, such as in I2net [START_REF] Chronaki | I2net medical image annotation service[END_REF]. When accessing annotations on large image corpa (in the form of images), one of the most important tasks is the need to find the images that have specific objects or to find keywords that best describe the image content [START_REF] Duygulu | Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary[END_REF]. Hence, this raises the possibility of several interesting applications such as image annotation with the semantics of the picture, to help in retrieving images in response to textual queries that correspond to its annotations semantics [START_REF] Feng | Multiple bernoulli relevance models for image and video annotation[END_REF].

Chapter 2. Related Work

22

Libraries have traditionally used manual image annotation for indexing and then later retrieving their image collections. However, manual image annotation is an expensive and labor intensive procedure and hence there has been great interest in coming up with automatic ways to retrieve images based on content such as [START_REF] Jeon | Automatic image annotation and retrieval using cross-media relevance models[END_REF][63] [START_REF] Li | Real-time computerized annotation of pictures[END_REF].

Annotation of Visualizations

To help analysts track their analysis process and findings, researchers have suggested the use of history tools (e.g. Aruvi [START_REF] Shrinivasan | Supporting the analytical reasoning process in information visualization[END_REF], Tableau [START_REF] Heer | Graphical histories for visualization: Supporting analysis, communication, and evaluation[END_REF], CzSaw [START_REF] Kadivar | Capturing and supporting the analysis process[END_REF], Visage [START_REF] Derthick | Data exploration across temporal contexts[END_REF]) to review, re-visit and retrieve previous analysis steps [START_REF] Shneiderman | The eyes have it: A task by data type taxonomy for information visualizations[END_REF]. Augmenting visual representations with annotations to record analysis details and findings has also been recommended by Heer et al. [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF] and implemented in information visualization systems such as: Voyagers and voyeurs [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF], HARVEST [START_REF] Babu Shrinivasan | Connecting the dots in visual analysis[END_REF], and GeoTime [START_REF] Kapler | Geo time information visualization[END_REF], allowing analysts to re-visit their analysis and re-use previous findings [START_REF] Marshall | Work practice study: Analysts and notetaking[END_REF].

Marshall [START_REF] Marshall | Work practice study: Analysts and notetaking[END_REF] identified three main processing activities during analysis: searching, interpreting, and presenting. Pirolli and Card [START_REF] Pirolli | The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis[END_REF] refer to the first two activities as the information foraging and sense making loops. Annotations are traditionally used to support interpretation (sense making). Visual Analytics (VA) systems like ManyEyes [START_REF] Viegas | Manyeyes: a site for visualization at internet scale[END_REF], sense.us [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF], and CommentSpace [START_REF] Willett | Commentspace: Structured support for collaborative visual analysis[END_REF] allow multiple users to annotate their insights on a single visualization, and search through them, encouraging peer learning and social sense making [START_REF] Nelson | With a little help from my friends: examining the impact of social annotations in sensemaking tasks[END_REF]. Others, like Sandbox [START_REF] Wright | The sandbox for analysis: concepts and methods[END_REF] and VisualLinks [START_REF]Visual analytics inc[END_REF], aid users to collect and group multiple discoveries and insights on a single visualization, and record them in the form of annotations. These support the sense making process of framing relevant information together, clarifying connections and information importance [START_REF] Klein | A dataframe theory of sensemaking[END_REF]. During analysis, fragmented information pieced together often reveals connections and further opportunities for investigation [START_REF] Attfield | Sense-making in visual analytics: processes and challenges[END_REF], contributing to the searching (foraging) aspect of sense making. Thus annotation support is clearly important in visual analysis.

Context-Aware Annotations in Visualizations

Recent VA systems have moved beyond this simple information grouping. Annotations are no longer simply attached to visualizations or chart graphics, but to the data, moving towards more context aware annotations. These annotations can then be used explicitly to search (forage) for related visualizations or annotations. The sense.us [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF] system provides doubly linked discussions, allowing users to move from visualizations to annotations and vice versa. Aruvi [START_REF] Shrinivasan | Supporting the analytical reasoning process in information visualization[END_REF] allows users to arrange their annotations in a "mind map", a diagram linking related visualization views and their annotations. Users can access past analysis views using keyword searches and text similarity metrics. Shrinivasan et al. [START_REF] Babu Shrinivasan | Connecting the dots in visual analysis[END_REF] recommend related annotations based on the context of the current analysis. Finally, Chen et al. [START_REF] Chen | Click2annotate: Automated insight externalization with rich semantics[END_REF] use annotation context as a way to semi-automatically annotate facts belonging to the same analysis categories across visualizations.

Most existing VA work supports context aware annotations on a single visualization view.

Traditionally BI analysts conduct their analysis using complex visualization dashboards.

Dashboards visualize multi-dimensional datasets through a combination of linked charts [START_REF] Cutroni | Google Analytics[END_REF], and are very widespread in BI (e.g. [START_REF]Dundas dashboard[END_REF][43] [START_REF]SAP business objects xcelsius[END_REF][35] [START_REF] Tableau | [END_REF]). Annotations are supported in most dashboards, and are usually attached to data points (not the chart graphic).

Nevertheless, these systems do not adequately support several needs identified by BI experts (detailed later in CHAPTER 4), such as annotations spanning multiple charts, or annotation transparency across data dimensions and visualizations (annotate once, see everywhere, as is done in text [START_REF] Cheng | Context-based page unit recommendation for web-based sensemaking tasks[END_REF]). These advanced functionalities, such as multi-chart annotation, can be hard to achieve, as annotating multiple points is an involved process:

since only one visualization is shown at a time, an annotation can only be added to one query. To add more queries to an existing annotation, the user needs to continuously switch visualizations.

This body of research uses context aware annotations for different purposes. Our work extends it in two ways: based on expert users' comments we identify desired behavior and possible uses of context aware annotations on visualization dashboards; and we explore challenges in their design and use, for example due to the dynamic nature of "context", discussed in CHAPTER 4 .

Storytelling

Stories are series of events and facts, with an order and connections between them (Oxford English Dictionary). In the domain of intelligence analysis, it is furthermore an abstraction used by analysts to conceptualize threats and understand patterns in the analytical process [START_REF] Bier | Entitybased collaboration tools for intelligence analysis[END_REF], and communicate their findings to others [START_REF] Baber | Sensemaking as narrative: Visualization for collaboration[END_REF].

Stories in Business

In recent years, organizations and their leaders have identified the importance and value of narrative and anecdotal information conveyed in the form of stories [START_REF] Sole | Storytelling in organizations: The power and traps of using stories to share knowledge in organizations[END_REF], to present either complex ideas, the context and details of crucial information, or personal interpretation of that information [START_REF] Snowden | The art and science of story or 'are you sitting uncomfortably?[END_REF].

Research conducted to date has demonstrated the value of storytelling to improve organizational structure [START_REF] Mitroff | Stories managers tell: A new tool for organizational problem-solving[END_REF] and collaboration quality [START_REF] Mcwhinney | How remythologizing can revitalize organizations[END_REF][START_REF] Boyce | Collective centring and collective sense-making in the stories and storytelling of one organization[END_REF], socialization and adaptation of new employees [START_REF] Brown | That reminds me of a story: Speech action on organizational socialization[END_REF][START_REF] Louis | Surprise and sense making: What newcomers experience in entering unfamiliar organizational setting[END_REF][START_REF] Louis | Organizations as culture-bearing milieu. Organizational symbolism: Monographs in organizational and industrial relations[END_REF], organizational and financial success [START_REF] Boje | The storytelling organization: A study of story performance in an office-supply firm[END_REF][START_REF] Boje | Stories of the storytelling organization: A postmodern analysis of disney as "tamara-land[END_REF], innovation and new product development [START_REF] Mcwhinney | A health confluence[END_REF], and teaching and learning [START_REF] Helmer | Between horses: An ethnographic study of communication and organizational culture at a harness track[END_REF].

The majority of this work is a meta-analysis of the effect of storytelling within an organization, rather than the development of tools to enhance the storytelling process as is our case. Moreover, the stories themselves discussed in this work, relate to the transmission of information and knowledge within an organization, mostly in textual or verbal form, rather than in visual form. The widespread use of visualization dashboards in the domain is a more recent development [START_REF] Cutroni | Google Analytics[END_REF], and so is the transmission of knowledge between organizations (dedicated BI analysis organizations and their clients or end-users). Thus storytelling needs in the domain have evolved and in our work we take a step at characterizing them more precisely.

Stories in sense making

Baber et al. [START_REF] Baber | Sensemaking as narrative: Visualization for collaboration[END_REF] point out that contemporary theories of sense making rely on the idea of 'schema', of a structure to organize and represent factual information, as well as the knowledge, beliefs and expectations of the people who are engaged in sense making. They can thus be considered as a collection of narratives. They further discuss the formalism of stories in sense making, and how the most effective stories are organized around the actors in the stories, their actions and rationale, events, their context, and most importantly the relationships between these. As further argued by Bier et al. [START_REF] Bier | Entity-based collaboration tools for intelligence analysis[END_REF], for effective collaboration and communication we need to use less text, and organize knowledge around entities (people, places, things, times etc.) rather than free form text. Similarly, Pirolli & Russell [START_REF] Pirolli | Introduction to this special issue on sensemaking[END_REF] propose the mapping of intelligence facts and insights into frames, that can be expressed in a variety of forms including stories, maps, organizational diagrams or scripts. It is thus clear that conducting intelligence analysis, communicating findings, and organizing knowledge in stories, has a strong visual component that represents entities and their connections.

Stories in data visualization

Often in narratives text or audio transmit the main story, while visualizations or images support the story or provide details. Comics and flowcharts are special types of narratives relaying mostly on visual components rather than text. Recently, we have seen an increase in integrating complex visualizations into narratives in many news organizations [START_REF] Farhi | Cnn hits the wall for the election[END_REF],

journalism reports (e.g. New York Times, Washington Post, the Guardian), and television In the domain of business intelligence the main visualization tool is BI dashboards [START_REF] Cutroni | Google analytics[END_REF],

collections of multiple visual components (e.g. charts, tables) on a single view [START_REF] Few | Show me the numbers: Designing tables and graphs to enlighten[END_REF].

Nevertheless, as pointed out by Wojtkowski and Wojtkowski [START_REF] Wojtkowski | Storytelling: its role in information visualization[END_REF], dashboards and other visualization tools used to analyze complex content cannot simply be used to tell stories.

They need to be "tailored" to accommodate storytelling so as to better highlight items of importance within very large data resources [START_REF] Gershon | What storytelling can do for information visualization[END_REF], in a way that is not cumbersome for the storyteller and clear for the audience.

Some visualization systems began to integrate storytelling. For example GeoTime [START_REF] Eccles | Stories in geotime[END_REF],

a geo-temporal event visualization system, integrates a story system that shows events in space and time, hypertext linked visualizations, and visual annotations to create an environment for both analytic exploration, and story creation and communication Figure 2.6.

Storytelling tools in the business domain are not yet as advanced. Systems like Sense.us [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF] and Tableau [START_REF] Tableau | [END_REF] allow analysts to visualize their data, conduct their analysis, and store a history of the analysts exploration. This history can serve as a first step towards creating a story. Many Eyes [START_REF] Viegas | Manyeyes: a site for visualization at internet scale[END_REF], Tableau Public [104] and Sense.us [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF] allow publishing of interactive visualizations online, and permit collaborative analysis through comments on a single visualization, creating an evolving analysis. Again this collaborative annotation can be seen as are a first step towards making a collaborative knowledge narrative, where an analysis story could be extracted from the visualizations and comments. Nevertheless, these tools do not provide explicit means to indicate story progression, and to highlight or explain relationships between multiple visualizations (seen in dashboards) which is key to intelligence analysis and communication [START_REF] Baber | Sensemaking as narrative: Visualization for collaboration[END_REF].

Chapter 2. Related Work

Summary

In this chapter, we have presented the related work to our three main research foci: (1) dashboard creation and customization for novices. We saw few systems that provide novice users with an easy to use interface to create dashboards. Moreover no work has studied novice user interaction with dashboards specially in the BI domain.

(2) For enhancing dashboard analysis with annotations, we have presented the broad research done for annotating text, images and visualizations. The important functionalities in annotating text of "annotating once, seeing it everywhere", is not yet applied to visualization annotation. Rather, very often annotations were attached to images (visualization snapshots), and users loose their annotation if they change the data visualization type. Moreover, advanced functionalities such as multi-chart annotation is still missing or hard to achieve. ( 3) The third point we focused on, is how storytelling is important for presenting complex information in business, by organizing and representing factual information supported by strong visual component that represents entities and their connections. Then we presented some systems like GeoTime that supports storytelling, while we saw that storytelling tools in the business domain are not yet as advanced.

Chapter 3

Understanding Dashboard Creation and Customization for Visualization Novices

Abstract

With the increase of visualization platforms targeting novices, researchers are now focusing on gathering insights regarding novice user practices. This chapter focuses on novice and expert business users that require access to easily created and customized dashboards.We describe the design and evaluation of Exploration Views (EV), a system that allows novice visualization users to easily build and customize Business Intelligence information dashboards.

EV provides an intuitive environment for dynamically creating, rearranging, searching and exploring multiple visual data representations from diverse datasources. These aspects aid users to better retrieve, experiment and familiarize themselves with their data. We evaluated EV with both novice and expert dashboard designers and report here (i) how novice users interact with the system, (ii) differences in how novice and expert users react to a dashboard systems that targets both, and (iii) provide new design guidelines for practitioners building dashboard applications, on the needs of novice visualization users.

Introduction

Appropriate visual representations in dashboards, using colors, size and shape, are combined with interactive exploration [START_REF] Few | Show Me the Numbers: Designing Tables and Graphs to Enlighten[END_REF] to amplify human cognition and enhance information understanding [START_REF] Mackinlay | Readings in information visualization: Using vision to think, volume Learning with animation[END_REF]. The current life-cycle of a BI dashboard involves multiple actors [START_REF] Chiang | Creating dashboards: The players and collaboration you need for a successful project[END_REF], including end-users and business analysts. In user-centered BI [START_REF] Foody | User-centered business intelligencet[END_REF] end-users intervene and provide feedback to the business analysts that create customized dashboards to meet user needs. This feedback comes at different stages of the dashboard design and setup (Figure 3.1), and involves a large amount of communication between business analysts and end users, in order to define functional specifications and a positive user experience. Thus there is an intrinsic delay introduced in any end-user requirement change or customization request. Current market trends recognize dashboards and data analysis as one of the driving forces in business and are becoming more popular. This indicates an expanding end-user population with diverse needs, requiring quick access to customizable dashboard technology, ideally bypassing the cost of communication with dashboard designers. This is reflected in advances in easily customizable dashboard visualization systems (e.g. Spotfire [START_REF]Spotfire[END_REF], Tableau [START_REF] Tableau | [END_REF]), and can also be seen in related research in the InfoVis community [START_REF] Storey | How information visualization novices construct visualizations[END_REF], [START_REF] Van Ham | Creation and collaboration: Engaging new audiences for information visualization[END_REF].

Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 31

Nevertheless, there are still many questions about how novice users interact with actual information visualization systems. To better understand how novice users interact and build visualizations on their own (without dashboard designers or other human mediators) we created Exploration Views (EV) in collaboration with SAP engineers. EV is a dashboard prototype (Figure 3.2) build specifically to empower end users, following guidelines from previous work on visualization novices. To evaluate its effectiveness, we compared how dashboard experts and novices reacted towards EV when creating visualizations for realistic tasks.

Exploration Views (EV) Prototype

This section describes the design rationale behind Exploration Views (EV) and the functionality that supports novice users, based on guidelines from previous work Section 2.3.4.

We will first explain the main goals of EV: easy creation, customization and interaction with multiple visual representations in a unified dashboard environment.

Although dashboard users may have clear questions regarding their data, they can be novices when it comes to visualization, and naive as to how dashboard software can support their needs. An easy creation process is essential for novices to experiment with dashboard designs fast. As seen in Grammel et al. work [START_REF] Storey | How information visualization novices construct visualizations[END_REF], visualization novices often have partial mental specifications for their visualization needs and tend to refine and change their designs. To ensure a user-friendly dashboard creation, the sequence of steps needs to be simple, with continuous visual feedback on the final outcome of user choices.

Moreover, as novice users may have no previous knowledge of what visual templates and representations are possible for different data types, EV must provide chart suggestions and templates to choose from (also suggested by Grammel et al. [START_REF] Storey | How information visualization novices construct visualizations[END_REF], Heer et al. [START_REF] Van Ham | Creation and collaboration: Engaging new audiences for information visualization[END_REF]).

Finally, EV should support common data formats (recommendation by Heer et al. [START_REF] Van Ham | Creation and collaboration: Engaging new audiences for information visualization[END_REF]), and users should not be restricted by data storage architecture, but be able to use data from multiple data-sources (e.g. personal data in excel and OLAP data from their enterprise).

As novice users create dashboards, they may need to explore and try out alternative vi-

Easy Creation

When creating a new dashboard the system provides miniature icons of saved dashboards and several template dashboards to choose from (Figure 3. (e.g. sales, year of sale, etc), and match them to visual mapping (a visual property in the template). This is a challenging task for novice users [START_REF] Storey | How information visualization novices construct visualizations[END_REF] and EV aids them (as in After selecting one or more data attributes in the form of measures (data attribute, e.g.

sales prices Figure 3.5) and categories (possible groupings of the attribute, e.g. per state, year), the system presents appropriate analysis types for this data. For example for a dataset of sales per state over the period of 2 years, possible analysis types include comparisons (sales by state or year), contributions (the percentage of sales per state over all sales), or trends (the evolution of sales over time). If the user selects one of these analysis types, the system automatically recommends visual templates (charts) suitable to the specific analysis and to the nature of the attributes (e.g. how many measures and categories are selected, if they are categorical, ordinal, etc), using a default visual mapping. These recommendations are ranked from simple and more commonly used charts to more complex ones, and aid novices to explore reasonable alternatives. Available visual templates include charts, gauges and tables. The "explore" mode is a detailed view of a chart, where users can undo/redo actions, choose data categories, sort items, change data sources, switch chart templates, and create local data filters. To avoid switching to the "explore" mode, part of this functionality is also available as a drawer toolbar for each chart in the main dashboard, with added options to lock the chart.

Ease of Customization

During the life of a dashboard, user needs may change; new hypothesis may need to be explored and new data sources to be added. Especially given novice users' tendency for iterative visualization specifications [START_REF] Storey | How information visualization novices construct visualizations[END_REF], EV has several features for user friendly customization, such as actions to add/remove components, change their properties, duplicate their content, and explore sub-parts of the data using data filters.

Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 36

Users can add new components on the dashboard or drag-and-drop existing ones from the Clipboard. If the components are visualization templates, the user has to go through data, attribute and mapping selection as described before. If the components are saved in the Clipboard, we use the properties of the component as it was stored. Existing components on the dashboard can be refined and customized by clicking the "explore" icon of the component. This provides a detailed view of the component (Figure 3.6), together with choices for datasets, visual templates, attributes and mappings (as described in the Section 3.3.2). Changes are immediately reflected on the component in an automatic preview, so that users can experiment with different choices [START_REF] Storey | How information visualization novices construct visualizations[END_REF]. Users can also customize the dashboard layout itself through functionality for hiding/showing dashboard components, dragging to rearrange them, resizing them, etc.

EV intelligently resizes components to fit in the dashboard (e.g. Figure 3.7). At any point users can save content for later use (in this or other dashboards), by dragging them in the Clipboard Drawer. This allows exploration of alternatives without risking loosing existing components. To further ensure that novice users are not afraid to experiment with their dashboards, all actions are reversible through undo history functionality applied on the entire dashboard and locally on specific charts.

Supporting Visual Analytics Process

Apart from the creation and customization functionality, EV supports a series of functions that aid the visual analytics process. EV has text search support that follows the guidelines of dynamic queries [START_REF] Shneiderman | The dynamic homefinder: Evaluating dynamic queries in a real-estate information exploration system[END_REF]: as the user searches text the corresponding text components on the dashboard that match it (axis labels, categories, tags and any other textual information) get dynamically highlighted. All our visual representations are essentially linked coordinated views [START_REF] Roberts | State of the art: Coordinated & multiple views in exploratory visualization[END_REF], [START_REF] Weaver | Building highly-coordinated visualizations in improvise[END_REF] If the underlying data-sources follow the classic OLAP data organization, users have access to the traditional OLAP functionality [START_REF]OLAP and olap server definitions[END_REF], such as sorting, drill up/down, pivot, roll-up, etc. Users can also flip the chart axis to experiment with different chart layouts. This exploration functionality is necessary for expert users. It is present in the "explore" chart mode, but also in the main dashboard in the form of a widget drawer for each chart to avoid opening the "explore" view (similar to Tableau [START_REF] Tableau | [END_REF]). At any point users can undo/redo their actions on each chart.

Finally EV provides functions to support long term and collaborative visual analysis, such as save and share. To help users follow visual changes during data exploration, when using OLAP functions or filters, changes are smoothly animated to ensure visual momentum [START_REF] Woods | Visual momentum: a concept to improve the cognitive coupling of person and computer[END_REF].

Exploration Views Technical Platform

EV is not just a lab prototype, but a fully operational BI dashboard that can handle large amounts of data from different sources. Although this chapter focuses on the interaction aspect of EV, we briefly describe its architecture. EV has 3 layers. The first is a web client based on the Adobe Flex that contains the dashboard layout, the exploration and faceted navigation components, and the chart manager.

The chart manager CVOM (Common Visualization Object Model) provides a set of services for chart creation and recommendation based on the analysis users want to perform and on data measures. CVOM provides three categories of visualizations: (1) traditional business graphics (line, bar, pie, 3D charts, surface,scatter/bubble, dual, radar, etc), (2) geographic visualizations, (3) tree/network relationship graphics such as star tree. Those visualizations support hierarchical data dimension labelling, and labels are laid-out according to dimension hierarchy and available space. CVOM also supports multiple visualization templates, animated transition and visual effects.

The second layer is a server containing a user session manager (as in Polstar [START_REF] Cubrarnic | Polstar: Assisted navigation for exploring multi-dimensional information spaces[END_REF]), a dashboard state management, and a component that synchronizes different data sources to enable the multi-chart visualizations. The third layer consists of an in-memory database for quick data storage and retrieval [START_REF] Popovici | Simd-scan: Ultra fast in-memory table scan using on-chip vector processing units[END_REF], and a semantic layer that helps us to connect visual queries from multiple charts and have synchronized views.

Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 41

Study Design

In this section, we define the purpose of our study, its design and methodology. Our goal is to examine how novice visualization users create dashboards. We attempt to answer three questions: (i) is it easy for novices to discover visualization and dashboard functionality in an interface specifically designed using previous guidelines for novices; (ii) are the existing guidelines effective and what is missing from the literature, especially in the context of dashboard visualizations; and (iii) do novice and expert visualization users react similarly to such an interface.

Q1. Can novice users discover functionality and build dashboards? We created EV following guidelines by [START_REF] Storey | How information visualization novices construct visualizations[END_REF][39], and asked users to construct visualizations to answer questions, without the help of a human mediator (contrary to [START_REF] Storey | How information visualization novices construct visualizations[END_REF]). We wanted to see if novices can discover functionality, learn the interface and answer questions based on the created visualizations. Thus we provided no interface training and observed difficulties in interface discovery and visualization creation process.

Q2. Are the existing guidelines enough? Our users went through the entire process of dashboard design and customization, including data and visual template selection and attribute mapping, to answer multiple questions. We observed this process to verify existing guidelines and collect potentially new ones specifically for designing dashboard systems for novice users.

Q3. What are the differences between novice and expert users? Finally, we wanted to identify differences between the interaction needs and expectations of novices and BI visualization experts. Experts are knowledgeable in understanding which visualization templates and attributes mappings can answer specific questions, but not necessarily in how to interactively construct or customize a dashboard or visualization. In this respect all our users are novices in creation and customization of dashboards, but not in visualization choice and interpretation. We conducted a pilot study with two novice participants and found that both had trouble understanding the difference between a measure and a category, as well as some data exploration concepts such as data filtering and drill-down/up. We thus decided to include a graphical explanation of these aspects (Figure 3.11) as training in the actual study.

Participants and Pilot Studies

We also observed some interface problems: some icons were not clear enough, and were changed to more easily understandable icons and text descriptions.

Procedure and Apparatus

Participants performed the study in a usability lab using a 15.6" HD screen laptop. Sessions were audio/video recorded and screen captured. One observer was always present to give instructions, explain tasks and observe participants. Each session lasted between 1.5 to 2 hours. It started with a survey on the participant's background experience in computer usage, chart and dashboards, and specific data exploration functionalities such as filtering, drill down/up, etc. As one of our goals is to see how easy it is for novice users to discover functionality and create dashboards with EV, we provided graphical explanations and examples of concepts such as measures and categories, data filtering and drill-down/up, but no information about how to perform them in EV. Next, participants were introduced to a hypothetical situation and dataset (see Section 3.5.3).

We followed a think-aloud protocol, requesting users to vocalize their thoughts and actions while conducting their tasks. This phase lasted between 35-55 minutes. After finishing all the tasks in our scenario, users were asked to complete a Likert type questionnaire to elicit their opinion on specific aspects of EV, and were interviewed to clarify observations made by the experimenter, identify problems users faced, and suggest areas for improvement.

Tasks and Dataset

Our dataset is an e-fashion dataset (also used by Riche et al. [START_REF] Riche | Understanding interactive legends: a comparative study with standard widgets[END_REF]), containing 3860 records with 9 measures (e.g. sales revenue, quantity sold) and hierarchical categories (e.g. state/city /store, year/quarter/month, line/category). The dataset was new to both novice and experts so the impact of the dataset and domain on task performance is limited.

The dataset is also complex enough to allow complicated analysis questions.

Our tasks were presented within a larger scenario. Participants were told they work in the financial department of a global chain of garment sales, monitoring financial aspects using a dashboard tool. We encouraged participants to think aloud, and if they failed to do so we prompted them to vocalize their insights and expectations. Apart from the graphical explanations (Figure 3.11), we also gave them a brief explanation of the dataset (what measures and categories exist), and told them to use the dashboard to create and rearrange visualizations.

Five tasks were performed, ordered from simple to complex (Table 3 Tasks began by loading an existing dashboard and participants were given time to familiarize themselves with the dataset and dashboard. At the end of the session participants took part in an interview and filled out a questionnaire. The goal of the interview was to clarify observations made during the study, prompt users to identify difficulties/barriers they faced, and suggest improvements. The goal of questionnaire was to rate EV on a 7

point Likert scale on aspects such as its utility, ease of use, overall satisfaction of use, etc.

Data Analysis

We used four different data capturing and analysis methodologies: functionality check the study were verified, and expanded upon, in a second pass through the recorded material (video, audio and screen captures).

To answer Q1 on discoverability and usability, we kept a check list of functions in the interface and observed how easy it was for participants to discover and use them (results in Section 3.6.1). Questionnaire answers also contributed towards answering Q1.

To answer Q2 on guidelines for building dashboard systems targeting novice users, we noted success events and barriers faced during dashboard creation and customization.

These observations are augmented by user comments in the openended interview session.

Findings are summarized in Section 3.6.2, and the resulting new guidelines are presented in Section 3.7.

To answer Q3 on reactions from novices and experts, we report explicitly only the differences between the two.

Findings

We present our main findings on ease of use and discoverability of the EV interface (observed and user self-reporting, Section 3.6.1), as well as broader findings (based on observations), applicable to dashboard design for novice users in general (Section 3.6.2). We then provide design guidelines for creating dashboards for novices (Section 3.7). 3.6.1 Discoverability and Ease of Use of Interface In the follow-up interview both groups of users reported EV was easy to use and customize.

They appreciated that customization and data exploration actions became active immediately for quick feedback and verification of their choices. They stated that EV is intuitive and doesn't require training. Finally, mostly our expert users requested additional functionality, like extra customization (e.g. changing colors) and annotation features, and the ability to export dashboards in a printable version. Base views and experimentation. We often found that users (novice and experts) created basic charts with measures they thought important (e.g. one for sales over time, one for number of items sold per state, etc). These charts which we refer to as base views, were usually 2-3 in number. They were used as a starting point for customizing and refining further in order to answer different questions.

When customizing a chart, we observed that novices experimented frequently with filters, drill-up/down operations and measures, to "play with things and understand the different options". On the other hand experts attempted to create the appropriate visualization without much exploration.

Chart per question. We observed that both user groups created a single visualization per task question. When asked if they see a benefit in using a dashboard, since one chart Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 48 was the focus of the analysis at a time, they all mentioned that it "helps keep the context of all the data" (benefit identified in BI dashboard literature [START_REF] Few | Show me the numbers: Designing tables and graphs to enlighten[END_REF]).

Use of categories. Novice participants often did not recognize the effect of adding hierarchical categories on a chart. In EV when a refined category is added in a chart, it affects aspects such as sorting, aggregation (averages), etc. For example a user had created a chart with sales sorted by year. She then added the category quarter, which is a drill-down category for year. At this point sorting was applied by quarter, but the user still expected sorting to be done by year. Thus automatically calculating metrics based on the finer categories of a hierarchy was not understood by novice users.

Data filtering. As we mentioned EV allows both global dashboard data filtering, as well as local filtering on a single chart. All users found this functionality useful, nevertheless it was often hard for novice users to make correct use of it.

There are 2 creation options for global filters: select part of the data on a chart and then apply the option "filter" (Figure 3.14) which will filter data based on the selected categories and values; or select the option "filter" and then go over the process of selecting on which categories and values to filter. We found that novices generally preferred the second option because they want to make sure they "selected all correct values". Contrary, experts created global filters with selections that they then refined further.

Another aspect where expert and novice users differed was the presentation of possible filter categories. Experts wanted categories to be presented grouped thematically, e.g. grouping of time (year/quarter), or geographical categories (state/city), while novices preferred an alphabetic presentation.

Users found the existence of both global and local filters useful and relevant when analyzing data as it allows advanced analysis, for example comparison of different products sold by stores for several periods. All participants commented that presenting global and local filters consistently is important, but that local filters should be further highlighted in charts. The EV interface gave such detailed filter views (Figure 3.8), and users commented on the bulkiness of the filter panels. There is a thus a tradeoff between presenting filter details and space, novice participants want filter details to always be visible (both local and global), while experts found they take up too much space.

Finally, we found that sometimes users chose local filters that were in conflict with global ones. This caused problems for several novice users who could not understand why no data was displayed. Undo as exploration strategy. All participants were familiar with undo/redo, but we found that it was novice participants that made extensive use of undo per chart. Undo was not used as a correction mechanism when mistakes were made, but rather as an exploration aid. Novice users reported that if they wanted to try something and see if their predictions were correct (e.g. drill-down), they could use undo to quickly dismiss their tests (instead of reversing their actions, e.g. drilling-up). They also used undo as a way of returning to base views (charts used as starting points for further exploration): from base charts they would drill-down/up or filter briefly to answer a question, and then use undo to return to the original chart.

Clipboard. This feature allows users to create a palette of active charts that they can later reuse. All participants found it useful for storing, but they reported it is most useful for sharing chart snapshots with others. This remark, reinforced by requests for ways to email or export charts to presentations, indicates that users also saw the clipboard as a storytelling creation platform for sharing their findings with others.

Participants from both groups also requested features such as annotation, putting titles, adding comments, and assigning the current date to a chart. Experts in particular requested stored charts to also clearly and visibly describe context (e.g. dataset, measures, filters, etc). These requests indicate that participants want to use the clipboard to bookmark their exploration process: keep track of interesting charts they explored, comment them, and store the date they conducted their exploration. Some novice users mentioned they were unsure exactly what is captured when selection that is active on a chart is placed on the clipboard (i.e. does the capture only include the selected data or not). This is an interpretation problem related the effect one action has to the remaining visualizations and data that are seen in coordinated views.

Visualization locking. In our prototype we provided functionality for visualization locking (not allowing editing or application of global filters). Expert participants expected this locking to apply to filters, but also visual properties like size and position in the dashboard. Some novices felt that locking a visualization "creates confusion because it takes out this particular chart from the context of the dashboard". They requested very clear visual representation to indicate locked charts, such as depressed charts.

Terminology. Overall users were satisfied with the terminology and labels used in EV. Some novice users suggested more precise terminology even for some features that are familiar from other UIs. For example our original icon for editing a chart was confusing and novice users were not sure if they were going to edit the title, the chart or the entire dashboard. Thus they often requested tool-tips detailing the function of icons and their context. Finally, while experts were accustomed to English terms (in terms of functionality, Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 52

dataset measures and categories), this was not true for novices with English as a second language who requested support in their mother tongue.

Design Tradeoffs

The design of EV balances several tradeoffs to support both expert and novice users.

Functionality required by experts, such as chart locking, axis flipping, and extended chart recommendations, often confuses novices. In our work, we attempted to always provide the simpler and most used functionality in a more prominent way. For example options like sorting, filtering, and visualization changing are first in the chart widget drawer, while axis flipping or locking come last. In the chart recommendation list, most common charts always appear first. Moreover, when possible we use reasonable defaults, for example in assigning color and other visual variables to categories. Nevertheless, we often found that functionality required by expert users still confused novices occasionally (e.g. the locking option).

Discussion and Design Guidelines

All novice participants in our study were able to successfully perform the required tasks in EV, a system built using guidelines for novice user interaction. Based on our observations we have additional guidelines than the ones proposed by Grammel et al. [START_REF] Storey | How information visualization novices construct visualizations[END_REF] and Heer et al. [START_REF] Van Ham | Creation and collaboration: Engaging new audiences for information visualization[END_REF], when designing visualization systems for novice users, especially in the context of dashboards.

Basic training on concepts such as categories: Our novice users were successful in creating visualizations and solving their tasks. But, based on our pilots, they were given small tutorials on the differences between measures, categories, and the nature of basic OLAP and data exploration functionality (drill-down/up, filtering, etc). Thus dashboard interfaces for novices should provide small tutorials of these concepts. This training is particularly important for hierarchical categories. Novice users need to know if metrics (averages, sorted lists, etc) are recalculated in charts when adding finer hierarchical categories. Based on user comments we recommend either not recalculating metrics unless users request it, or recalculating per category on demand.

Considerations in coordinated views: When creating a dashboard, novice users assume that charts are linked by default, and expect brushing and linking functionality. Its implementation for single data sources where charts share measures and categories is fairly easy. Nevertheless, if measures and categories need to be matched across data sources, then matching approaches need to be provided [START_REF] North | Snap-together visualization: A user interface for coordinating visualizations via relational schemata[END_REF][50]. Extensive exploration aids: Novice users created a small number of base view charts, most often of familiar visualization templates, that they later copied and refined to answer questions. When possible, such basic views should be provided.

Novices also made extensive use of chart undo/redo as an exploration aid, thus this functionality should be provided on a per chart basis.

Furthermore, clipboards were seen as good ways to share data for storytelling and communication, as well as bookmarks of the exploration process. For clipboard snapshots to be useful this way, they require functionality such as annotations and labeling, as well as ways to report their metadata, such as creation date and details of the context of the chart (i.e. dataset, active filters, etc).

Finally, although locking charts can be useful during long analysis (as indicated by experts), this locking should be very clearly visible to not confuse novices (e.g. fading the entire locked chart, or making it appear as depressed).

Data filtering considerations: Data filters, although very useful for data exploration, can be challenging for novice users. We found that novices preferred to create global filters from scratch, select categories, values, etc. While experts preferred to create them graphically (through visual selections) and then refine them. Both options should be available. When selecting categories to filter, we recommend presenting them in thematic groups (expert choice) that promotes dataset learning. But this organization should be supplemented by text search functionality and a possibility to order them alphabetically.

Novice participants want filter details to always be visible (both local and global), while experts found they take up too much space. Designers should consider an auto-hide, or on/off visual representation of filters. Approaches such as interactive legends [START_REF] Riche | Understanding interactive legends: a comparative study with standard widgets[END_REF] could also reduce clutter of local filters, since most charts include labels of some sort.

For both groups, being able to distinguish between global and local chart filters is important, and it should always be clear which categories and values contribute to existing filters. Finally, since novice users have trouble predicting conflicts in global and local filters, it is important to clearly mark such conflicts.

Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 54

Language & terminology: Novice users are often not aware of international terms for different charts, metrics or functions. Thus local language support is important.

Moreover, buttons, labels and icons (even universally understood ones) are not enough to provide novices with adequate information for their behavior in the context of dashboards.

Detailed explanations need to be provided on demand (e.g. tool-tips).

Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 55

Summary

We have presented Exploration Views, a prototype system that allows users to easily create, customize and interact with Business Intelligence visualization dashboards. EV was build following previous guidelines for novice visualization users. Through a user-study with novice and expert visualization users we verified that EV enables novices to create dashboards for answering complex questions. We found some differences in how experts and novice users interact with the interface, reported in our study. More importantly, based on our observations we provide new guidelines that augment previous work on designing for visualization novices, especially in the context of interactive visualization systems in the form of dashboards.

Chapter 4

Annotating BI Visualization Dashboards

Abstract

Annotations have been identified as an important aid in analysis recordkeeping and recently data discovery. In this chapter we discuss the use of annotations on visualization dashboards, with a special focus on business intelligence (BI) analysis. In-depth interviews with experts lead to new annotation needs for multi-chart visualization systems, on which we based the design of a dashboard prototype that supports data and context aware annotations.

We focus particularly on novel annotation aspects, such as multi-target annotations, annotation transparency across charts and data dimension levels, as well as annotation properties such as lifetime and validity. Moreover, our prototype is built on a data layer shared among different data-sources and BI applications, allowing cross application annotations. We discuss challenges in supporting context aware annotations in dashboards and other visualizations, such as dealing with changing annotated data, and provide design solutions.

Finally we report reactions and recommendations from a different set of expert users.

Introduction

The main problem that plagues dashboards today is that they don't say enough and what they do say they don't say very well. In other words, they are not very informative.

-Stephen Few [START_REF] Stephen | Information Dashboard Design: The Effective Visual Communication of Data[END_REF] Well-designed interactive visualizations help users gain insights into their data, identify patterns, and make decisions. During visual analysis, a single analyst or a group, access information from multiple sources to combine insights. These analysis sessions may last from a few hours to many weeks. Given the limits of human memory [START_REF] Cowan | Attention and Memory: An Integrated Framework[END_REF] remembering all reasoning details and milestones in the analysis process becomes challenging. Thus the importance of "externalizing" memory, knowledge, and insights [START_REF] Hutchins | Cognition in the Wild[END_REF] has been identified early on. Analysts achieve this by making sense of their visualizations, annotating them with their findings [START_REF] Heer | Design considerations for collaborative visual analytics[END_REF], organizing their findings, and presenting them to others [52][65].

Annotations are thus an integral part of complex visual analyses, aiding analysts in revisiting and reusing past analysis sessions [START_REF] Babu Shrinivasan | Connecting the dots in visual analysis[END_REF]. Recent work has started to explore the potential of utilizing the context surrounding annotations to better support users in data discovery and analysis (e.g. [80][70]).

Despite the growing use and research work on annotations for visual data analysis in general Section 2.4, related research on the domain of BI dashboards and other complex multichart visualization environments (coordinated views [START_REF] Roberts | State of the art: Coordinated & multiple views in exploratory visualization[END_REF]), has not equally progressed. Our work attempts to rectify that.

Based on interviews with expert BI analysts, we derive new requirements for dashboard annotations that current dashboard tools fail to support. Following these requirements and using a user-centered design approach, we develop and evaluate an annotation-enabled dashboard, addressing several challenges.

The contributions presented in this chapter are: (1) A new set of requirements for annotations, targeting dashboards and other coordinated view visualization systems.

(2) The design and evaluation of a prototype dashboard that supports context-aware annotations.

Using this system, we explain how annotations support advanced analysis functionality such as: "annotate once, see everywhere" annotations for multi-chart visualizations (as was done for text [START_REF] Cheng | Context-based page unit recommendation for web-based sensemaking tasks[END_REF]), annotation recommendations to promote knowledge discovery, and the use of the common data model layer present in BI systems (e.g. [116] [START_REF]SAP businessobjects semantic layer[END_REF]) to link annotations with external data sources and enable cross application annotations and information foraging. (3) We identify and provide solutions to challenges when using context aware

Approach

In our work we followed a user-centered design approach. Based on in-depth interviews with expert BI analysts, we derive requirements and needs for annotations on visualization dashboards: this lead to the choice of context aware annotations. Following these requirements and scenarios of use provided from our experts, we developed a dashboard prototype with annotation functionality that supports different analysis tasks.

While building our prototype, we identified challenges in using context aware annotations with dynamic data and evolving annotation context. Based on a second interview with experts, we refined our system to address these challenges, following user recommendations.

Finally, we evaluated our prototype with different experts and report results and reactions on the benefits and drawbacks of our designs.

Interview 1: Annotation Requirements & Needs

To better understand BI expert annotation needs, we conducted a set of in depth interviews with 8 BI experts. Our goal was to investigate their annotation activities when performing analysis using dashboards, as well as their hopes for future annotation enabled dashboards.

Experts' experience in the BI domain ranged from 3-11 years. Three used dashboards daily while 5 several times a week.

Interviews lasted 50-90 minutes, and they were recorded and transcribed, and the emerging themes identified using a card sorting analysis approach. Bellow we summarize our major findings:

BI Dashboards Annotation Practices

All experts confirmed the importance of annotations in their analysis and the need of having annotations directly embedded in their BI dashboard tools. This way analysts can share their analysis with tactical and operational decision-makers simply by using their dashboards and relevant annotation indicators [START_REF] Golfarelli | Beyond data warehousing: what's next in business intelligence[END_REF]. An expert explained: "The data has no meaning without our analysis and contribution we add to it, so charts and dashboards would be useless without explanations".

When asked about their actual analysis annotation practices, all experts mentioned taking static snapshots of dashboards (Figure 4.2) and including them in emails or power point presentations, with text comments accompanying them. Their annotated comments are either for dissemination purposes to a larger audience, or serve as personal reminders and analysis findings records. Experts explained that for each annotation they also copy a data query, as a reference to the exact data the annotation refers to.

BI Dashboards Annotation Needs

We further prompted experts to elaborate on their ideal vision of annotation-enabled (2)

(3)

Multiple Target Annotations

Experts need to attach their annotations to multiple targets (multiple charts on the dashboard). Seven out of eight experts annotated multiple charts on a single dashboard. An expert commented, "The aim of a dashboard is to tell a story, that's why we need a way to link all the dashboard's charts that bold a connection and give the story we need".

Another expert said "two different dimensions that cannot be gathered in one chart could answer a specific question, so we need to link them by one annotation to explain and show the result or solution".

Chart Transparent Annotations

Six experts mentioned they need to attach annotations to the actual data points, rather than static visualizations "because usually I change the chart view and then have to rewrite or re-comment for the new representation of data". Another explained that this is useful as a verification or exploration mechanism: "it is good not only for having the comment for both chart representations, but when seeing the comment on a different chart, it may show that the interpretation of this point was not right, or maybe a different chart would give a better interpretable view of your data". One expert mentioned that it "would be useful to find other visualizations or annotations for a specific event in that data", i.e. use the annotated data as a focus point from which to find related annotations and visualizations.

Here experts highlighted the importance of being able to go from an annotation, or a list of annotations, to the referenced real data on the dashboard and back.

Visible Annotation Representation on the Dashboard

Five experts preferred to see visual indicators (icons, flags or special visual highlights) of annotations attached to the actual data "like small indicators on the dashboard and clicking them to see the content". They also wanted to have the option to expand and collapse all annotations' text: expand them "to see all dashboard related annotations at a glance and their story", and collapse them "because having them all on the dashboard would clutter it". Moreover they commented that a list of all dashboard annotations should be available, with visual links to their data targets.

Granularity Transparent Annotations

Four experts mentioned they would like to have annotations "transparent" through different data dimension granularities (e.g. country/city, year/quarter/month), and mentioned there are business use cases where they want to mark a specific event of importance in all visualized levels (e.g. mark a market trend). They would like this feature as an option in special business cases, and deactivate it when not relevant.

Annotation Validity & Lifetime

The lifetime of dashboard annotations was an important issue raised, since there are cases where the data referred to by an annotation can change. All experts confirmed the need to archive annotations "we would like to have them archived to be used in future analysis and to access them for comparisons". They would also like to archive with the annotation a snapshot of the visualization on which it was made. Seven experts preferred to have the option to explicitly define the lifetime of annotations either in time periods (sessions, days, weeks, months, etc), or based on validity rules (e.g. valid while the value of sales ranges between 3000 and 5000).

Sharing BI Dashboard Annotations

Experts expressed the need to share annotations, and in particular being able to define user groups to share annotations with. They also requested to be alerted when a specific user or group of users edits or shares an annotation. Nevertheless, all experts expressed the need for private annotations to use as reminders or analysis record keeping.

Formal Annotation Representation on Dashboards

When prompted, all experts stated a preference for a more formal annotation design (e.g. use a toolbar with predefined annotation categories, shapes, arrows and text boxes), rather than hand-drawing shapes or notes [START_REF] Denisovich | Software support for annotation of visualized data using handdrawn marks[END_REF]. They explained that they very often share annotated dashboards with peers and embed them in presentations, thus they see a "first priority for structured, clean annotations" that look more professional.

Summary

To summarize, we identified the following annotation needs expressed by BI experts, grouped in two categories:

a. Functional

F1. Multiple Target Annotations

Annotations should have multiple data and chart targets. Navigation to the different targets should be possible from the annotation. With few exceptions (e.g. [START_REF] Babu Shrinivasan | Connecting the dots in visual analysis[END_REF]), most visualization systems annotate single visualization charts.

F2. Chart Transparent Annotations

Annotations need to be attached to data-points, rather than charts. Few systems (e.g. [START_REF] Tableau | [END_REF]) actually allow data, rather than visualization annotations, and generally they do not show a data point's annotation in different charts.

F3. Granularity Transparent Annotations

Annotation transparency should be optionally preserved across dimension granularities. We are aware of no system that supports this.

F4. Annotation Validity & Lifetime

Annotations should be archived even if the context of the annotation (e.g. annotated data) changes, together with a visual snapshot of the data at the time of the annotation.

Moreover, users should also be able to define the annotation lifetime, based on a time period or data related rules. We are aware of no system that supports both these aspects.

F5. Sharing Annotations

Users should have the option to share annotations with specific users groups, or kept them private for personal use. Systems supporting sharing (e.g. [START_REF] Viegas | Manyeyes: a site for visualization at internet scale[END_REF]), do so for single charts.

b. Design

D1. Visible Annotation Representation on the Dashboard

Annotations should be attached to the annotated data using small visual indicators that won't clutter the overall view, but with an option to expand them in place.

D2. Formal Annotation Representation on Dashboards

BI experts prefer structured and professional looking annotations to hand drawn ones, for presentation purposes.

Context Aware Annotations Prototype

Based on these findings we implemented a dashboard prototype which supports annotations that keep a record of their surrounding context (i.e. the multiple data points and charts they are attached to), as well as special annotation properties, such as their validity and lifetime. Using annotation context also allows us to provide annotation recommendations [START_REF] Babu Shrinivasan | Connecting the dots in visual analysis[END_REF].

Chapter 4. Annotating BI Visualization Dashboards

64

In this section, we provide an overview of our prototype. Apart from the described interview, experts were engaged throughout the design of our system.

Architecture

BI dashboards provide users with a multi-chart interface, connecting one or more data sources and data sets that are represented as charts. Based on our user interviews, we want seamless annotations that work across data sources and charts. To that end we allow annotations to refer to one or many "data targets" on a common data layer.

A common data layer is present in most BI systems (e.g. Oracle, SAP, Microsoft) and provides a layer of data abstraction that unifies different data sources. Thus data queries are performed directly on this layer and retrieve relevant results from multiple sources. Our annotations work on top of this layer: each annotation is stored in a database, together with a reference to its data targets on the common data layer. Although the common data layer is not present in all coordinated views system, there is usually a common querying layer (e.g. a database with SQL queries) that works across charts and controls chart coordination. In such systems our annotations can be connected to the querying layer.

Given this approach, our system permits context aware annotations that can be shared among different applications with a common data layer. Once an annotation is made on data points in one application, it is attached to this data irrespective of application, allowing cross-application annotation and annotation re-use.

The context of an annotation includes the following entities Figure 4.3, similar to the model proposed by [START_REF] Chen | Toward effective insight management in visual analytics systems[END_REF]:

• Content: the text provided by the user.

• Authoring metadata: automatically captured information, such as the author and creation date of the annotation.

• User defined properties: these include annotation ranking and categorization of annotations (derived from previous research [START_REF] Mahyar | A closer look at note taking in the co-located collaborative visual analytics process[END_REF][122] [START_REF] Marshall | Work practice study: Analysts and notetaking[END_REF]). As well as new concepts such as annotation "validity rule" or "lifetime" (F4 discussed in great detail in the Annotation Challenges section). Defining these properties is optional.

• Data targets: these are one or more entities that the annotation refers to (or is attached to). Each target includes: an Overview data context that defines the larger data context of the annotation, for example the chart or data table on which is was made; and more importantly, a Detailed data context, the specific data points that the user is annotating. Both Overview and Detailed data context are stored as data • Threads: other annotations connected to this annotation (e.g. a discussion thread in collaborative annotating).

Interface

How to Use Context

In this section we discuss in detail how our annotation prototype follows the experts' requirements and aids analysts in their tasks on dashboards. The tasks and scenarios described here come either from the interview or from intermediate feedback sessions with experts.

Transparency

Our annotations are transparent through applications, dashboards, charts, and datasets following a common data layer. Moreover, they can be transparent through different hierarchical data granularities, discussed later on.

Our annotations store detailed data context, so whenever this data context appears, so does the annotation (annotate once, see everywhere as it is done for text in [START_REF] Cheng | Context-based page unit recommendation for web-based sensemaking tasks[END_REF]). During the analysis phase this can help in 4 different ways:

(i) Analysts can switch visualizations (chart types) without loosing their annotations (F2), experimenting with alternative representations that may be better suited to their data.

(ii) Our annotations provide a list of all other charts containing the same detailed annotated context. This allows analysts to find new charts that may present these data in different analysis scenarios, e.g. in combination with different measures or data categories.

Thus analysts may gain new insights from their data by seeing them in different analysis situations.

(iii) Through annotations, analysts can reach different material (other applications or data-sources) that incorporate the same data context, aiding in knowledge discovery.

(iv) Finally, users have the option to see annotations from lower or higher data dimension granularity (F3). For example when viewing sales data for France they can make an annotation transparent for the hierarchical dimension 'location' and see it appearing in sales in Paris. Noticing such annotations that are explicitly visible across dimension granularities can lead to new exploration paths and insights. We elaborate on this last point and discuss challenges with annotations across granularities in the next interview section.

Chart linking

Our system provides an easy way to add/remove data contexts to/from the annotation, either at the detailed annotation window, or by selecting interactively a data context (e.g. a bar in a bar-chart) and choose the option to link/unlink it to an existing annotation.

Annotations can target multiple detailed data contexts from multiple charts, as well as multiple data sets (Figure 4.4.a). This was a specific requirement from our interviews (F1), made possible through the use of the common data layer across charts and datasources. Analysts can thus use annotations as a means to chunk or group diverse material together for insight externalization and organization (an important task in sense-making [START_REF] Babu Shrinivasan | Connecting the dots in visual analysis[END_REF], [START_REF] Klein | A dataframe theory of sensemaking[END_REF]).

Moreover, annotations can be seen as a way to navigate through the linked charts: by expanding the annotation of a data context in one chart, analysts can see all related contexts in other charts, select them and load them in the dashboard. This cross-linking and navigation has been identified as important in reading sense-making tasks (e.g. [START_REF] Tashman | Liquidtext: a flexible, multitouch environment to support active reading[END_REF])

and visualizations representing one chart at a time (e.g. [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF]).

Recommendations

We use annotation context similarly to [START_REF] Babu Shrinivasan | Connecting the dots in visual analysis[END_REF] on text, to recommend annotations or charts of similar data contexts, promoting knowledge discovery and peer learning from annotations of others.

To identify similar contexts we compute a simple similarity measure. Similarity between two data contexts is the arithmetic average of all common dimension IDs across contexts, all common measures IDs and their values.

Recommended annotations are presented in a list by order of similarity (Figure 4.1.d). A set of two visual similarity bars was included initially in our prototype, the 1st to show the overall similarity of all dimensions and measures between two annotations, and the 2nd to show the similarity of dimensions only. The goal of this was to help analysts understand the ranking and, if they choose so, to focus the recommendations to dimensions only.

Nevertheless experts indicated that the overall similarity was enough for their purposes and we adjusted our prototype (see "User Study").

External search

To aid knowledge foraging and discovery, our system allows users to use their annotation context as a search term outside applications with the same data model. For example we use it to perform system wide searches for documents, as well as to construct a set of well formatted search terms to use in web search engines, like Google.

Automatic annotation

We do not perform automated or semi-automated annotation [START_REF] Chen | Click2annotate: Automated insight externalization with rich semantics[END_REF]. Although we do have all the required components, this previous work targets single chart visualizations to define annotation templates, and requires further work to extend to multiple charts. Nevertheless, in our system there is an implicit automated annotation taking place through the transparency of annotations across applications and data sources.

Interview 2: Design Challenges & Solutions

Early on in our interviews, we came across two major design challenges for annotations not addressed in previous work: (1) how should annotations behave when their underlying data has changed? (2) When are annotations applicable across granularities? We conducted a follow-up interview with the same 8 experts, where we discussed alternatives and collected their reactions and suggestions.

Dynamic context vs. dynamic annotations

Often annotations in BI target dynamic data points (e.g. the averages of sales records that are being updated every month). There are two design alternatives for how annotations could behave during such data changes.

The first is to make annotations dynamic: when the context of the annotation changes in a small way, then the annotation can be attached to this new context. The challenge is to clearly define what is "a small change".

Based on input from all our participants this "depends on the situation". Some mentioned that changes in data value only could be considered as a small change in some scenarios (while measure and dimension IDs remain the same). For example a participant attaches an annotation regarding sales peak on the data point with dimension ID= month, value = December, and measure ID = sold-quantity, value= 3000. They want to keep this annotation valid even if the value of the measure changes, e.g. becomes 3500. Others mentioned that sometimes if the dimensions or measures go through small changes, the annotation can remain attached to the data points. Clearly the nature of "small change" is unique to each analysis scenario and requires users to define "change" thresholds. On the when the annotation is deleted a specific data threshold has been reached.

The second alternative is to acknowledge dynamic context, and assume that the annotation is no longer attached to the data. When the data context changes the system assumes the annotation is no longer valid, and keeps a snapshot of the context under which the annotation was made (dimensions, measures, values), as well as the visual chart it was made on. Nevertheless, all participants stressed that this should not mean the annotation gets lost, but archived.

This last approach is the default approach we chose in our system. The unattached annotation obviously has a context very similar to that of the changed data. Thus if the user revisits the changed data, the old annotation will appear as a highly ranked recommendation, easily accessible. The visual representation of such "archived" annotations should be clearly distinguishable from "active" ones: following the inactive metaphor of UI interfaces, archived annotations are grayed out (Figure 4.6). We chose this way of addressing dynamic data context, as it is simple to define, and provides a way to archive data changes (see Sec. User Study).

An orthogonal treatment comes as a direct suggestion from our experts. They stated that their analysis is often affected by events through time, or is valid for short time periods.

This prompted the notion of a user-defined "validity period" of an annotation. If she so chooses, the user can explicitly define an expiration date or a validity period for annotations, after which annotations get archived or deleted from the system (Figure 4.7). At this point experts requested the option to be sent a notification inquiring if the annotation should be archived or deleted. This lifetime functionality is also implemented.

Granularity

In hierarchical datasets, as is the case in many BI data sources (e.g. OLAP databases), users can drill down or up in dimensions. For example for the dimension time we can have 3 hierarchical levels: year, quarter and month, where a year has 4 quarters, and a quarter 3 months. The question that arises is: if an annotation is made at a lower/higher hierarchy level, when should it appear on the dashboard?

In our system, users can reach annotations from different dimension levels through annotation recommendations (as their contexts are similar). Nevertheless, experts asked to have some annotations of other hierarchical levels visible as icons on the dashboard all the time. When asked to identify under which conditions this makes sense, they commented that this also depends on the nature of the analysis. They underlined the importance of viewing such important cross-level annotations for 2 reasons: First, to use annotations as a "bookmark" to an important point in the analysis, while the analyst explores new paths (e.g. annotate 2008 to mark the economic recession, while exploring stock market values per month). Second, to highlight important findings that could influence future analysis.

The use of annotations to bookmark important analysis points is closely tied to the idea of "base views" discussed in Chapter 3.

Through our discussions with the experts, we could not conceive an automated way of detecting such important annotated points, and thus provided an option to explicitly make some annotations visible across hierarchical levels. We also provided users with an option to toggle on/off the visibility of all annotations from different levels.

The visual representation of such annotations was deemed very important by our experts:

it should be different from annotations on the same level, but not too complex. After a few design iterations, most agreed on using the same icon used for navigation across dimensions in the dashboard (up/down icon for annotations, drill-up/down in the hierarchy)

User Study

After several iterations of the prototype with the original group of experts, we evaluated it with a different group of experts, to explore if they can use it and benefit from the different aspects of context aware annotations for their analysis. We requested that participants conduct a set of tasks similar to the ones identified in our interviews and user-centered design process. We report our observations and experts' comments and suggestions.

Participants

Seven participants took part in our study. They were BI experts, and only one was previously involved in the design of the system. They all had experience using dashboards.

All were familiar with the concept of annotation, mostly in annotating text documents while conducting analysis tasks.

Procedure and Apparatus

Participants performed the study on a 15.6" HD screen laptop. One observer was present to give instructions, explain tasks and observe participants. Sessions lasted from 45 to 90 minutes, were recorded and later analyzed. They started with a survey of participants' annotation practices and ended with an exit interview. During the study we followed a think-aloud protocol, requesting users to vocalize their thoughts and actions while performing tasks.

Tasks and datasets

We used 4 different datasets, each presented on a different chart on the dashboard: (i) and (iv) US historical auto sales (25 records, 2 dimensions). Datasets were new to users, but their type was familiar (e.g. the measure sales and dimension year ).

As indicated in our interviews, real analysis contains different data sources. Thus using 4 datasets allows us to examine if users could easily use annotations to link different contexts from different datasets.

Participants were told they are analysts exploring different datasets to find trends, and record their findings. We instructed them to think loud, and if they failed to do so, the observer prompted them to vocalize their insights and expectations. Participants were given a brief explanation of the prototype functionalities (e.g. adding new and explore existing annotations). They performed tasks in 3 main groups, focusing on different functionalities of context aware annotations. (The detailed task scenarios are presented in (Table 4.1))

T1: Usability and understandability. Participants were requested to analyze part of a dataset and create a new annotation with their insights, attached to specific data. Then they explained their understanding of characteristics of that annotation, such as lifetime, privacy, category, and transparency across dimension granularity levels. Finally they were asked to experiment with other visualizations for the same data found through their annotations.

T2: Retrieval of annotations of the same context. Participants were asked to create an annotation on a specific data context and look for other annotations on the same exact context, or ones linking multiple contexts, including the current. When they detected these annotations, they were asked to determine if they were related to their analysis and explain why (based on the creator, context, datasets, characteristics, etc).

T3: Knowledge discovery. Participants were asked to search for possible similar annotations, and retrieve external search results related to the annotated data (through search in the databases sharing a data layer, or external searches on the hard disk and Google).

Participants were asked to collect relevant material and add them to their analysis annotations.

Tasks began by loading a dashboard containing 4 charts, each representing a data set.

Participants could change the chart visualizations and were given time to interact and familiarize themselves with the datasets and dashboard before starting the tasks. At the end of the tasks a short interview (Table 4.2) was conducted to understand how context aware annotations impacted participants' analysis, to clarify observations made during the study, to prompt users to identify difficulties/barriers they faced, and to suggest improvements.

Results

We present here feedback from participants on how the system supported their analysis, and on its visual design.

# Task Scenario Purpose of Task

Context

You are an analyst that explores datasets, and analyzes them to find trends and contribute information. We provide you with an exploration system (dashboard), that permits you to:

-annotate data points -annotate multiple targets (charts) -explore annotations done by other users -retrieve previous annotations context -recommend annotations -give a similarity measure for recommended annotations -retrieve information from Google search related to a context on your dashboard We would like to get your feedback of those functionalities on your information retrieval while analyzing different datasets.

Permit participant to understand the context of the tasks and introduce the system use.

T1

Read the first chart top left in the dashboard and describe your interpretation (Internet usage) -What are the top 3 countries in internet usage in the world? -Annotate France with its order over the world.

-Explore the creation interface that appears and explain your interpretation of the presented features.

Supporting Analysis

Participants completed all tasks successfully without any help from the experimenter. We present here observations made regarding the use of context aware annotations in our tasks.

T1: Usability and understandability.

• All participants understood the detailed data context(s) of the annotations, and explained that understanding the context was easy when "having the annotated data visually highlighted in a chart", and having a "detailed textual description of the data context" (dimensions' and measures' names and values) on the annotation body.

• The transparency of annotations between charts allows users to understand new chart types, using the annotated data as reference points across the two visualizations. A participant mentioned "the radar chart was unfamiliar for me, so I switched it to a scatter plot and I was able to still see the annotated data points. The annotation helped me to understand the radar by reviewing the scatter plot and see that I am looking at the same data point".

• Annotating data points vs. charts helps users see changes of important data points.

Our system takes snapshots of annotated data points whose context changes. A participant commented that "annotating important data points acts as a way to see their changes through time".

T2: Retrieval of annotations of the same context.

• All participants found attaching annotations to data points (vs. entire charts) very important for verifying annotation relevance to their task, both for searched and recommended annotations (relevant to T3). As one participant mentioned, this "maximizes the analyst's focus on targeted data and not all the chart, and prevents loosing time trying to verify the context for each searched or recommended annotation".

• All participants appreciated the use of annotations as reminders of past work, and as highlights of the important data in charts ("at a glance I can determine the important data points, no matter where I am").

• Finally they all appreciated the use of annotation transparency for all users across all visualizations, as a way to avoid work replication (for example prevent duplication of annotations among multiple analysts).

T3: Knowledge discovery.

• All participants were able to navigate the list of recommended annotations, and interpret their similarity. They mentioned they understood why each recommended annotation had the presented similarity, by comparing the context of the targeted data points and that of the ranked recommendations. Five found that the global similarity measure that combines dimensions and measures is enough, while two would have also liked a text similarity metric.

• All noted the usefulness of recommending annotations for re-use or peer learning ("from notes of others"). They commented that it is especially useful because the reason for the recommendation is clear, due to clear data contexts.

• All participants found retrieving external data related to the annotation context very useful. They all commented that searching for supporting material during analysis will be easier if they can retrieve data from their company's portal using the data context to search other BI applications with the same data layer, or search for files containing the context keywords on their machine or on Google.

Visual Design

Participants provided us with design suggestions to improve the visual interpretation of annotated targets. Their feedback was used to reach the final design presented in this chapter. Here we present some of their comments that influenced the look and feel of our The other recommendation was to visually link annotations targeting multiple charts with lines across the dashboard. One participant had at first some trouble understanding that the charts in the detailed annotation window showed the multiple targets of an annotation.

She explained that this could become more intuitive if annotations with many targets on the dashboard were connected with lines to the targeting charts. Thus at the end of the study we provided three design suggestions to our participants: the first (

Introduction

Stories are one of humanity's communication structures and storytelling a means of passing on wisdom and culture. Individually and collectively, stories help us make sense of our past and reason about the future. Johnson [START_REF] Johnson | The moral imagination[END_REF] and Maclntyre [START_REF] Maclntyre | After virtue[END_REF] argue that story narrative goes beyond communication, it is also a process of extracting meaning from events, that is central to human experience and conduct. It is thus not surprising that intelligence analysts, people who make sense of data and identify links between disparate pieces of intelligence, are interested in storytelling. Their analysis process is supported by the construction of stories and narratives. Bier et al. [START_REF] Bier | Entitybased collaboration tools for intelligence analysis[END_REF] point out that "a story is a powerful abstraction used by intelligence analysts to conceptualize threats and understand patterns as part of the analytical process".

As analysts continue to work with increasingly large data sets, data visualization has become an incredibly important asset during analysis, and when communicating findings to other analysts or to a broader public [START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF][START_REF] Shrinivasan | Supporting the analytical reasoning process in information visualization[END_REF]. Given the importance of storytelling in the analysis process it is clear there is a need to enhance visual analysis tools with storytelling support. Nevertheless this process is not simple [START_REF] Wojtkowski | Storytelling: its role in information visualization[END_REF][START_REF] Ma | Scientific storytelling using visualization[END_REF], as analysts need to highlight and explain items or events of importance and their connections, within very large data resources. Despite the growing use and research work on storytelling and narrative visualization in the visualization domain [START_REF] Segel | Narrative visualization: Telling stories with data[END_REF][START_REF] Heer | Voyagers and voyeurs: Supporting asynchronous collaborative visualization[END_REF][START_REF] Eccles | Stories in geotime[END_REF], related research on the domain of BI has not equally progressed (see section 2.5). Our work attempts to rectify this.

As previously discussed, in Business Intelligence (BI) analysis the most popular visualization tools are dashboards. Nevertheless, just collections of visual representations do not speak for themselves; to become meaningful they require interpretation and explanation, often presented in a story narrative.

Our findings indicate that although the work practices of BI analysts' may differ, in the end they all construct stories that include: collections of visual components (dashboards, charts or tables) to support their stories, verbal explanations during presentation(s), or detailed textual annotations and descriptions in the form of written reports and external resources such as wikis. This process remains cumbersome and requires frequent switching between different software (dashboard analysis tools, screen captures, text editors, presentation or drawing tools, etc).

Our work aims to aid BI analysts with their storytelling needs, and makes the following contributions: (1) Based on interviews with expert BI analysts (story "creators" and "readers"), we derive and provide a better understanding of the actual practices in creating BI stories.

(2) We emphasize the need of BI storytelling support and extract new requirements for enhancing BI visual analysis tools with storytelling capabilities. (3) Following these requirements and using a user-centered design approach, we design and implement a prototype storytelling tool that is fully incorporated in an existing visual analysis dashboard, to fluidly transition between analysis and story creation. (4) We report on feedback from BI experts creating a story on the usefulness of the prototype as a communication and teaching medium, and from BI novices reading a story on its effectiveness for story communication.

Interviews

To support visual storytelling in BI, we attempted to understand current practices in transmitting BI knowledge and analysis results, and identify the challenges experts face when creating and sharing their stories or reading and interpreting the stories of others.

We interviewed 5 BI experts occupying positions in SAP which is a leading business intelligence development company, with experience in the BI domain ranged from 6 months to 12 years. Participants job descriptions included development project manager, project manager, development director, data warehouse engineer and delivery manager. Three experts used dashboards and communicated their analysis or read analysis from others daily, while 2 several times a week. Interviews were held either in person or by phone and lasted on average 1 hour. We report next our findings.

BI Reports

All experts communicate their analysis or read analysis from others in the form of BI reports. These contain an entire dashboard, often accompanied by several single charts and tables. Their purpose is to help monitor aspects of business performance, by highlighting with charts key performance indicators (KPIs) that indicate success of project management and progression of development teams. Our experts were report creators, but occasionally acted as readers of others' reports.

A BI report can be a single page, with only a title and a dashboard as in Figure 5.1 and Figure 5.2. This one page report summarizes the most important data, and can serve as a starting point for longer reports, up to 50 pages in length, that give more details.

Details can be additional visualizations, tables, annotations, links to the data used in the visualizations, and finally block text, although all participants preferred limiting text to 5% of the space in a report. Thus BI reports have a very strong visual component, with little text added for explanations.

The one page report is preferred by clients such as company managers that want an "at-a-glance view" of a project status. They usually see many such reports during a

What is a BI Story

Experts explained that reports are difficult to understand without detailed explanations from their creator. Thus they don't represent a complete BI story on their own. Before The audience can ask questions during the training session or afterwards, through emails or arranged meetings.

Content very similar to the presentation, with more textual explanations is often put on a wiki page to help the audience and answer follow-up questions. The session recording, wiki page, and presentation slides, are made available to the audience as reminders and reference material. Experts explained that this material is not a complete story either, as it does not include the visualization and data details shown on the BI report. Thus a complete BI story is a BI report with visual representations of the most important data, accompanied by instructions on how to read the report, in the form of presentations, and verbal or textual instructions on a wiki.

Teaching BI storytelling

According to our experts, the material making up a BI story is also a tool for training new analysts. As one recently trained report creator explained, this material taught her the key aspects of a BI report and how to interpret them. When analysts first start out, they read in detail older reports and all their supporting material to understand how to read and analyze visualizations, see their relations, and how to identify the important points and their link to KPIs.

Experts mentioned that designing a report from scratch is hard and needs a lot of experience, thus they often use a template that they modify according to their needs. Three stressed the influence of a senior report creator (the 5th interviewed expert), and their reliance on his BI report templates to create their own. They still occasionally contact the senior creator when they face difficulties or are unsure of the clarity and completeness of their messages in a report.

The supporting material is also often reused. All experts explained that data (and thus reports) change, but often the structure of BI stories repeats itself. Moreover, they sometimes create a sequence of reports on dynamically evolving data (e.g. sales over several years). These stories change very little. Thus material on how to read reports can be reused by adapting it to new stories. Nevertheless, as experts pointed out, this adaptation can be a tedious and repetitive process.

Creating a BI Story

Experts explained that report creation follows several iterations of communication with a client (usually a decision maker, like a manager or executive), who is one of many possible report readers (a process identified in Chapter 3).Thus the focus and structure of the story changes and evolves through the communication between the story teller and the story reader over time, a process that does not appear in most other visual analysis domains and visualization storytelling systems, where the needs of the reader are not directly reflected into the creation of the story. This interactive report evolution process can be heavyweight (through phone calls, emails, etc), and can take several iterations to iron out, while the communication details are often lost between different versions of the report. All interviewed creators use an in-house reporting tool, that gives access to different data sources and can extract interactive dashboards, individual charts and tables from their dashboard analysis system. It also provides the capability of adding text. Three experts also used Excel for creating less complex reports. Our experts identified several issues with this process. Firstly, when extracting interactive visualizations from their analysis tools, metadata and annotations from their analysis are not extracted and have to be recreated.

Moreover, the annotation capabilities of report creating software is very limited. There is no ability to point to specific data points, and sequences and connections cannot be displayed graphically but have to be explained in one of the supporting material. Finally the report creation tool can be in-house software, thus to share their reports with others, creators often extract static snapshots of the tool output and save it in PDF format to share with their readers.

Some report creation tools give access to interactive visualizations and dashboards through hyperlinks. Our experts like this option and add these links to their non-interactive reports (PDF, Excel) and presentations. Readers within the organization may have access to the in-house reporting tool, and can then interact with individual visualizations or dashboards (e.g. query and filter data, or drill down/up). This interactivity is lost when reports are communicated outside the company.

To overcome the shortcomings of reporting tools, creators are forced to provide the supplementary material (presentations, wiki pages) and when possible links to interactive visualizations for in-company clients. They described this process as limiting and requiring a lot of work duplication.

We requested clarification on two points regarding the possible integration of analysis and storytelling tools: (i) Given that analysis dashboards are exploratory environments that allow users to interact fully with data, should such interactions be allowed in the storytelling? (ii) If creators had the opportunity to create their entire story with all support material in one place, how would they prefer the story to be visualized?

Interactivity Due to the constraints of current reporting tools, shared reports are often non-interactive when accessed outside the organization. When asked if it were possible for the visualizations in their reports to be completely interactive, would they encourage their readers to interact with them (e.g. using drill down/up, filter, link & brush), four of our experts prefer to have interactive visualizations that permit linking and brushing (i.e. data selection). But they would limit the more advanced interactions such as drill down/up or filtering. They felt that all needed data to tell the story should be displayed clearly in the report without the need to explore the data further.The fifth participant would not be opposed to fully interactive visualizations. Thus authors feel business stories and related visualizations should be mostly author-driven and constrained, a practice that is known to work best when the goal of a visualization is storytelling or efficient communication [START_REF] Segel | Narrative visualization: Telling stories with data[END_REF].

Story templates Going from the current practices of storytelling (BI report, presentations, wiki) to a dedicated storytelling tool is not straightforward. We thus showed our experts a group of story templates identified by Segel et al. [START_REF] Segel | Narrative visualization: Telling stories with data[END_REF] to see if they met their needs. All chose the "Annotated chart" as the preferred template (Figure 5.4), with the modification that it should have multiple charts on the same page (dashboard) that they can annotate. Four experts identified "Partitioned poster" as a potential template, where the side list of charts display details that support the main chart/charts in the central region. Three mentioned that the "Slide Show" template is useful both as a means to focus attention on each chart and zoom to details, and as a step-by-step presentation that clarifies the analysis path and the ideal reading sequence. One expert mentioned the "Flow

Chart" would be useful to show some business scenarios, like following bugs during development (discovered, tested, fixed etc). And another mentioned that the "Comic strip" could be a useful template, but with added annotations. So besides the templates identified by Segel et al. [START_REF] Segel | Narrative visualization: Telling stories with data[END_REF], for BI stories we need a new template that consists of an annotated dashboard, that combines multiple templates from that group. Our users attempt to do this with their current reporting tools, but they are limited (cannot annotate detailed points and relations), or they create it manually in their presentations.

Participatory design session

We brought back the senior BI expert, that was involved in training of other analysts, to identify the best design for a BI storytelling tool. She provided us beforehand with one of their old reports and supplementary material. During this session we gave the expert a copy of one of her material, and several cut-outs of the entities in the report (charts, titles or tables). She was also given "narrative aids", such as arrows, lines, numbered items, grouping containers of different colors, and annotation bubbles (Figure 5.5). She was asked to construct a stand-alone story of the report on an A3 paper, with accompanying audio if necessary, in a way that it can be interpreted by a general audience with no training.

The session lasted 2 hours and was recorded. Other structures used by the participant were chart grouping rectangles (enclosures for placing charts that need to be read together), sequence markers (what needs to be read first), and highlight markers (to draw attention to part of the story).

At the end of the session, we asked our expert how she would prefer that the readers see the story, as a static image or as an animated presentation. Our expert explained that both are needed, the static representation shows the "entire story and gives context", whereas the animation "focuses the audience where I want". She then played out for us how she wanted the animation to be presented by explaining when and how to zoom to specific areas of the story. This play-back was recorded using static shots.

Requirements for Creating BI stories

BI reports, the main communication medium of BI analysis, consist of snapshots of visualizations and textual descriptions, made in a different environment than the one used for the analysis. These reports are fact-based and can be interpreted only by an expert audience. To increase their audience, report creators use supplementary material, such as wiki pages, presentations, and when possible links to interactive visualizations. Based on our experts' comments and the participatory design session. To create a more effective BI story, we identified a set of requirements for such as storytelling tool:

R1. Fluid Transition. Report creation tools are different than the analysis tools used to explore data and create visualizations. This costs time and effort to export visualizations and limits the possibility of embedding meta data or annotations created during the analysis. To ensure that creators do not recreate this missing information, they require a fluid and integrated way to transition from their analysis and meta data associated with it, to report creation.

R2. Integration. To tell their stories, BI creators need tools that combine all materials used currently in their story creation: BI reports, interactive visualizations, ways to indicate story structure, highlighting capabilities, presentation of the story in sequence, and textual or audio explanations.

R3. Narrative visual aids. Report creators need to add focus expressions to draw attention to specific visualization data, such as highlighting, coloring, annotating and zooming. They also require ways to indicate reading sequence (e.g. vectorial references, such as arrows [127][128]). These are not available in reporting tools or BI analysis tools.

R4. Interactive visualizations. Visualizations on shared reports are often non-interactive when read outside the organization. A storytelling tool should have completely interactive visualizations, although the way that readers interact with the data should be limited (by default to brushing and linking) and be controlled by the creator. This balance has been identified as a challenging aspect of storytelling [START_REF] Ma | Scientific storytelling using visualization[END_REF].

R5. Appropriate BI Story templates. BI stories have specific structure not necessarily shared by other story narratives identified by Segel et al. [START_REF] Segel | Narrative visualization: Telling stories with data[END_REF]. Our experts identified templates of interest (Figure 5.4) and highlighted the need for a new template that consists of an annotated dashboard.

R6. Reuse. Although BI reports and data changes from analysis to analysis, often the underlying structure of BI stories remains the same. It is thus important to be able to easily reuse the structure of stories created within the tool, both for stories of evolving data and similar future stories.

R7. Playback.

Readers should be able to see a static representation of the whole story to get a general overview, but also be provided with a guided playback of the story to help them follow analysis paths that are not easy to discover, and ensure they understand the important points according to the creator.

Such storytelling support would alleviate some of the burden associated with creating and learning how to read BI stories. We are not aware of any BI dashboard or other multi-chart visualization system that currently provides all the above functionalities.

Chapter 5. Storytelling in Visual Analytics tools for Business Intelligence 96

BI Narrative Prototype

Following these requirements, we extended the dashboard exploration tool described in Chapter 4, to support the creation of BI stories.

5.6.1 Exploration/annotation dashboard (Figure 5.8)

The user starts from a traditional analysis dashboard, a collection of coordinated (synchronized) charts connecting one or more data sets. Our dashboard provides advanced exploration capabilities such as data selection and filtering, and OLAP functionality (e.g.

drill-up/down) [START_REF]OLAP and olap server definitions[END_REF]. We build our narrative tool on top of this visualization dashboard system [START_REF] Elias | Annotating bi visualization dashboards: Needs & challenges[END_REF] that supports annotations on "data targets".

After or while conducting her analysis on the dashboard, the analyst can choose to create a story. The menu option "Create story", opens up the narrative board window. All 5.6.2 Narrative board (Figure 5.9)

Here users can create a BI story. Visualizations and annotations from the dashboard appear on the main part of the window. By default they appear as the dashboard (to support the dashboard template identified in our interviews R5). Users can then resize and re-locate all story entities freely, or choose to organize them using other appropriate narrative templates (R5). They can edit visualization titles, add/remove and edit annotations of entities, and add text entities. Users can add visual entities that indicate relations, grouping and emphasis (R3) to help create stories, available at a tool palette on the right.

The story can be seen in: a static representation, where all entities remain one screen; and a playback representation, where entities are highlighted sequentially according to an author defined sequence (R7). We categorize the entities available on the narrative board as:

1. Information entities: visualizations, text, annotations. These are mainly the imported visualizations and annotations. For annotated visualizations, the annotation text, the visualization, as well as the annotated data target (the specific part of the visualization that is annotated), are all defined as story entities. In addition to Chapter 5. Storytelling in Visual Analytics tools for Business Intelligence 98 the entities created during analysis, users can place text anywhere on the narrative board to add further explanations to their story. Moreover, new data points on visualizations can be selected and highlighted with different colors to focus attention on and add them to the sequence of entities in the story.

2. Relational entities: arrows, lines, html links. A story can include relationships between entities, such as causality. We provide visual arrows, lines, html links and other vectorial references to define relations between story visualizations and text entities (Figure 5.9.c).

3. Organization entities: visual grouping and sequences. Previous work [START_REF] Pirolli | Introduction to this special issue on sensemaking[END_REF] and our interviews emphasized the need of visual grouping of entities that are to be seen together in a story. We support this with entity grouping borders. Moreover, our experts indicated the need to define a reading sequence for each entity to show readers how to move through the story. We achieve this by allowing users to define the order of appearance of all entities through a sequence list (Figure 5.9.d). Users can change the entity's sequence in the list, delete, or rename any entity. They can also define a playback time for each entity, a time for the entity to be in focus in the playback presentation mode. Audio can also be recorded for any entity, to be played when the reader watches the story in playback mode. Interactive visualizations: Our visualizations are imported with time-stamps referring to a "snapshot" of the dashboard data at the time of the story creation. Nevertheless, they are still connected to that version of the data and are still live. Thus users can still interact with them to perform brushing and linking actions (R4). By default we have deactivated more advanced interactions such as drill-down/up, but the authors can choose to reactivate them through a properties window for the visualizations.

Moreover, the creators may decide to take a new snapshot of the data for the visualizations (i.e. change the timestamp for the visualizations and thus the values of the data). This way they can reuse the story structure for a new version of the data (R6). This is of particular importance for evolving dynamic data. Finally, users can select any visualization in the story and replace it by another visualization from another dashboard that they can load in the system. This way they can reuse the story structure not only for the same dynamic data, but also for completely different datasets.

Thus our narrative board integrates all the material used currently in BI story creation (R2): BI reports (visualizations and annotations), interactive visualizations, ways to indicate story structure, highlighting capabilities, presentation of the story in sequence, and textual or audio explanations. The entire narrative board can then shared with others (e.g. readers) that have access to the prototype.

User Feedback

We conducted two user feedback sessions to asses the usability and benefits of the system, both from the creator's and the reader's perspective.

Story creation by experts

We first invited back two of the interviewed experts, and showed them the system in individual sessions (lasting 90 min). One of the experts has been an analyst for almost 12 years and has trained several other analysts. The second expert has been an analyst for a year and was trained by the first expert. They were able to explore the prototype, see story examples, and create stories themselves.

General feedback and recommendations

Both experts were very enthusiastic with the prospect of having access to such a system for their work. They found the prototype easy to use, and very helpful, specially as it Chapter 5. Storytelling in Visual Analytics tools for Business Intelligence 101 is integrated in the exploration/annotation dashboard tool. As one mentioned it "saves me from recreating any charts or annotations to present to others". The prototype saves a lot of time and effort compared to their actual practices, not just in terms of creation, but also in communicating the reports to others. As one expert explained, by creating a story with the prototype, there is no need for an explanation presentation to clients, for creating a wiki page or interactive charts.

Moreover the story can be archived and used for any new arriving employee that needs to learn how to create reports (comment from our trainer analyst). This expert stressed how important such an archiving is for knowledge passing between analysts. The recently trained expert mentioned how this tool could help him further improve his reports by looking at the story structure of others.

Both participants appreciated the ability to use the annotated dashboard template, saying that "this is how we want to present our analysis story, same as we present our dashboard".

Moreover they both commented that annotations attached to data points are very important in pointing out important data values to focus on "I add a lot of manual arrows to point to annotations that refer to specific data points on a chart, and their ideal value".

When it came to how to show the story to readers, both creators reaffirmed that story reading must be strictly guided by the story creator, else the goal of the story may get lost. In their current practices they enforce this by the order of explanations used in their BI reports. They commented that the prototype supports this well by adding a sequence for the entities. Participants found the playback mode (with fade out and zoom) useful to guide readers through the story and keep their focus on a single entity each time. Both participants mentioned that they liked the fact they could create a story on one page (even if scrolling is required), as opposed to the current multi-page reports, because as one mentioned "people tend to read only the first page, and explore less the following pages". Finally, they appreciated the static representation of stories, because "the reader can always understand the position and link between parts and the overall story".

Constraining and opening up the story

Our experts suggested that our system should support three types of BI narrative stories:

• Static stories. Stories that present snapshots of data sets at specific points in time, yet are interactive (as is the case in our prototype), to be shared as tutorials, explanations, or reports with any audience. This is the case in our system.

• Online or Live stories. Stories that present live data. This is needed for dynamic data that often have the same analytic scenario, regardless the values of data. For example they may have the same description for each chart (e.g. data measures and dimensions), have the same relation between KPIs, and the same reading sequence.

In this case the visualizations in our stories are no longer snapshots, but completely live and updated with data changes. We have implemented this extension.

• Collaborative stories. Based on our interview findings, in our current version the analyst designs and creates the story, and controls all aspects of its presentation.

One expert commented on how this storytelling prototype can also be used as a means to evolve stories. Multiple analysts can integrate their own comments and knowledge in the story, encouraging peer learning, but also creating more complete and detailed stories.

One expert also mentioned she could envision using the system to iterate the definition of the story directly with their clients (a process conducted now by email or phone-calls). Using our prototype they envisioned cases where their clients add further explanations on specific entities and their relations between entities (to explain patterns), possible summaries about the decisions they took based on the report or what was the information missing, etc. This goes beyond collaborative analysis and storytelling, as it empowers the reader and becomes a way for her to communicate what they want from the story.

Story reading by novices

We then ran a user feedback session to evaluate the prototype from a reader's perspective, and thus close the story communication cycle. We conducted private sessions that lasted between 40 and 50 minutes, with 5 BI novices. All were IT professionals, knew what a dashboard is but had never worked with one. Two had heard of BI reports but never used one.

Participants were asked to (1) Read a BI report created by one of our experts, (2) Read a BI story (created by an expert report creator in the previous session, and (3) Explore our prototype.

The story presented the progress of a system development project, from different perspectives: a General (how are the projects evolving in terms of finished code in a sprints time line, how many code components are added to a waiting list, and how many are done each week with a categorization of components as critical, major or minor), and a Detailed perspective (the progress of each development team in coding and testing each code component). In the story the development is not progressing according to plan because many bugs fixed are not critical, whereas new bug reports coming in add critical bugs to the

• Persistence: All users tended to remember even detailed aspects of the story. They only went back to the system when answering detailed quantitative questions (e.g.

to retrieve numbers from charts), while all general comprehension questions were answered without looking back to the story visualizations and comments. They also tended to answer using the same terms used by the expert analyst in their comments.

When asked similar questions after reading just the report, they went back to the report each time to search for answers.

• Confidence: Readers were not confident about their understanding of the report since they had to draw their own conclusions. They expressed worry that they may have misunderstand or not noticed some important points.While all readers felt confident when reading the story, specially since their interpretations are confirmed by another person (analyst comments).

• Guidance: all readers appreciated the guidance in reading the story both static and playback. Participants commented on the importance of both modes: "the static mode permits to understand the whole story", while playback "is easier to follow" the story sequence. Particularly for the playback, they commented on how it helped them focus and prevented them from spending time on unimportant data. Four participants preferred the fade mode over the default highlight mode, and 3 preferred to add the max mode when focusing on an entity.

• Understanding: all users understand the entire story using our tool, and found it easy to read and interpret. They also commented how it was easier to find answers both to qualitative and quantitative questions. The story structure showed clearly what is the problem, how to analyze it and how to find the cause. On the other hand they described the report as ambiguous, as they couldn't find the relation between charts or KPIs. When reading the simple report, each reader can choose his own way in structuring and analyzing the data and spend time in exploring in detail each chart to find relations. In the end most could just deduce that all charts represent the same data but from different perspectives.

• Transmitting knowledge: All readers found the annotations very helpful in explaining the relationships between charts and KPIs, and in teaching them the analysis logic they should follow. Four mentioned that the system can "aid in transmitting different knowledge in the company between different users", and 2 participants said they would like to be able to use the system to communicate with their team their own data (even if they are not analysts).

• Engagement: Three readers out of 5 got very engaged with the story and began searching in other detailed charts to find how this problem can be solved. As one mentioned "stories are more encouraging than static reports".

The comparison of BI reports without supplementary material over our story prototype is clearly unfair and our goal is not to prove its superiority in terms of communication value. It mainly serves to illustrate how our stand-alone prototype could be an effective means of communicating BI stories without additional material. The comparison of our tool against the collective BI story material used today in terms of comprehension is future work, but our experts already identified the gains it provides in terms of saving time on report creation, and allowing the sharing of interactive stories. Using these requirements we implemented a prototype tool that is integrated in BI analysis dashboard and allows fluid transition from analysis to storytelling. We then evaluated in two sessions. First we gain feedback from story creators on the use and benefits of the tool. Our findings indicate that the requirements and resulting prototype meet the authors needs and prompted further suggestions for collaborative storytelling and story archiving. We then presented a story created by an expert to novice readers, as a first step of testing the understandability of stories seen in our system, and its potential value as a stand-alone means of communicating BI analysis stories. Our readers were all able to understand the created stories without any training and to answer complex comprehension questions.

Chapter 6

Conclusions and Future Work

The incredibly quick rate of growth of data poses challenges in analyzing large data sets.

BI as a field aims to better analyze large data sets related to business and one of the more important tools available to BI analysts are visualization dashboards. Dashboards are still only created by BI experts due to a complex and long creation procedure, that often cannot be conducted by end-users, involving a large amount of communication between business analysts and end users. Moreover, they still fall short of supporting detailed analysis record keeping, and extracting results to communicate to others is a tedious process. To address these challenges, this thesis focuses on enhancing user interaction with BI dashboards, in three aspects of the life of a dashboard (1) creation: easy creation & customization of dashboards for a broader user audience, (2) analysis: adding novel annotation support and

(3) communication: the use of visual storytelling in communicating dashboard analysis results.

Review of Thesis Results and Contributions

The central problem addressed by this thesis is how to enhance user interaction with BI dashboards in order to access a broader user audience and enable them to analyze large data sets and communicate their findings with others.

Creation: To that end we have presented Exploration Views, a prototype system that allows users to easily create, customize and interact with Business Intelligence visualization dashboards. EV was build following previous guidelines for novice visualization users.

Through a user-study with novice and expert visualization users we verified that EV enables novices to create dashboards for answering complex questions. We found some differences in how experts and novice users interact with the interface, such as different functionality needs for each group of users, for example visualization novice users need more interface exploration and navigation capabilities (e.g. base views, undo) and more explanations and tutorials, while experts require advanced data exploration and customization functionality (e.g. OLAP functions and color/label customization). Moreover each group of users has different understanding of concepts, for example when novices select a part of a visualization in the dashboard, they expect a filter action on the dashboard visualizations, while the same action for experts translates to traditional crashing and linking.

More importantly, based on our observations, we provide new guidelines that augment previous work on designing for visualization novices, in the context of interactive visualization dashboards, such as the need for extensive exploration aids (e.g. basic views and self reviling functionality).

Analysis: After an easy creation of a BI dashboard, we need to support advanced VA in order to answer complex questions. Recent years have seen an increase of use of context aware annotations to support VA tasks. Based on in-depth interviews with expert Business

Intelligence analysts, we extend this previous work by defining a new set of requirements for annotations, targeting visualization dashboards and other coordinated view systems.

With these requirements and a user-centered design approach, we developed a dashboard prototype that supports context aware annotations.

We discuss how such annotations support new functionality, like "annotate once, see everywhere" for visualizations (not just text [START_REF] Cheng | Context-based page unit recommendation for web-based sensemaking tasks[END_REF]), multi-chart annotations, and annotations that are transparent across hierarchical data dimensions. They also support knowledge discovery through annotation recommendations, as well as the use of the common data model layer present in most BI systems (e.g. [116] [START_REF]SAP businessobjects semantic layer[END_REF]) to perform cross-application annotation, and information foraging outside the visualization system.

While building our prototype, we identified and provided solutions to challenges in using context aware annotations, notably issues arising when the annotation's "context" changes (e.g. annotated data are deleted). Based on feedback from experts, such annotations should be archived with a snapshot of their context, as long there is an easy way to retrieve them (e.g. through recommendations). Users should also be able to define the lifetime of annotations.

Our prototype was evaluated with a different set of BI experts that were able to easily perform several tasks that they deemed important in their work Communication: Finally, BI analysts often need to organize visually and communicate their analysis story and findings to others. Nevertheless their tools do not allow them to easily transition from visual analysis to storytelling. They are often required to use multiple tools, replicate work, and train their audience to understand their analysis. Based on interviews and a paper prototyping sessions with expert BI analysts, we derived requirements on how to extend BI visual analysis dashboards to support storytelling, including the need for reuse of story structure, and the constraints to impose on interactive stories.

Using these requirements we implemented a prototype tool that is integrated in a BI analysis dashboard and allows fluid transition from analysis to storytelling. We then evaluated it in two sessions. First we gained feedback from story creators on the use and benefits of the tool. Our findings indicate that the requirements and resulting prototype meet the authors needs and prompted further suggestions for collaborative storytelling and story archiving. Our system was also identified by experts as having great potential

for training other analysts. We then presented a story created by an expert to novice readers, as a first step of testing the understandability of stories seen in our system, and its potential value as a stand-alone means of communicating BI analysis stories. Our readers were all able to understand the created stories without any training and to answer complex comprehension questions. Our preliminary results indicate that our prototype can help authors reach a broader audience, and little or no training is required to read the results of an analysis.

Limitations, Discussions and Future Work

By conducting research into enhancing user interaction with BI dashboards, this thesis opens the door to new lines of inquiry; we hope it serves as a prelude to a continuing stream of research. There is a great deal of future work that can be done to further enhance dashboard usability. Here we elaborate on some of the limitations of this thesis and corresponding opportunities for future research.

BI Dashboard Creation and Customization

As described in Chapter 3, Exploration Views was evaluated in a short term study. Further work is needed to gain more insights in how novices use dashboards, and it would be interesting to observe their exploration behavior in a long-term study. Such a study Chapter 6. Conclusions and Future Work 110 could provide us with a clearer understanding of the evolving needs of a novice user that gains more and more experience. Indeed an important question to explore is if in fact systems designed for novices with limited functionality, actually help novices to evolve and transition to expert users.

Moreover, as study participants were not data experts, and they performed tasks on unfamiliar data sets, it could be interesting to see if our findings extend to data experts, and find out how would they use EV to explore and analyze their data sets. We also tested EV for business data, but it would be interesting to discover its usability in other different domains' data, such as science, mathematics and weather forecast, etc.

Finally, our work focused more on the user interface design and use of a visualization environment for novices, and less on the insights gained from the data to help improve decisionmaking. Clearly this is an avenue of future work. For example it would be interesting to observe the evolution of exploration strategies (e.g. use of base views and undo), how the different visualization combinations effect understanding and decision making, and whether mechanisms such as the clipboard and history are enough for archiving analysis findings and resuming older analysis sessions.

Context-Aware Annotations

Our work in Chapter 4 highlights the importance of data-centered annotations, and how analysts need to attach annotations to data points (rather than images), so that annotations remain attached to different data visual presentations used by analysts to see data from different perspective, in order to gain better insights. In our work, we have presented a new set of requirements for context-aware annotations for the BI domain, and it would be valuable to test if those requirements are applicable to other visualization domains.

For example level transparent annotations, could be useful in layered visualizations (e.g. systems supporting semantic zooming). Annotation validity and lifetime would be of interest in dynamic data visualization systems (e.g. stock market visual analysis tools).The set of requirements and design decisions in this work are highly influenced by the domain of focus (BI). Thus some may not apply to other domains (e.g. the need for formal annotation presentation). Nevertheless, some of the more general requirements on annotation transparency across charts and dimensions, as well as ways of addressing changing data context can be applied to other visualization domains.

For example: annotation transparency and re-use saves user effort in any visualization system with many possible data views (e.g. multidimensional visualization systems); granularity transparent annotations that highlight important data across data levels can benefit any system with a hierarchical or layered data representation (e.g. systems supporting Chapter 6. Conclusions and Future Work 111 semantic zooming); whereas issues related to annotation validity and lifetime affect any dynamic data visualization system (e.g. stock market visual analysis tools). Nevertheless, further work is needed in this regard to clarify the importance of the identified needs and to validate the proposed designs with domain expert users of these visualization systems.

More generally, our work highlights a deeper need for data-aware and transparent note taking: for example in todayâĂŹs spreadsheet applications users can comment specific cells, but when cells are turned into charts the comments are not visible on them. Our system takes advantage of a common data model to annotate specific data contexts and have them available in all data representations and views (even across applications). But the idea of annotating directly data or queries for annotation re-use and recommendation, can be modified to work with any data archiving system that has a querying mechanism for the annotation to be attached to, and a notion of "distance" of the archived objects for recommendations. Thus it can be applicable to interactive applications ranging from spreadsheets to medical imaging archives or large document collections.

Although initial reactions to our system are very positive, a long-term study is needed to determine how context-aware annotations can impact analysis behavior in the long run.

Our prototype design is now integrated in different analysis systems that would permit a deeper study of its usability, as well as how central annotations will be used in different analysis systems by using the common data layer.

Another topic of interest would be collaborative annotations. The capability to extend an annotation to an information exchange medium, where each user can add new comments to an existing annotation, could effect decision making and raises questions regarding the quality of new contributions and trust of its author. Further more, tracking new annotations on a specific data context could be frustrating, as a big amount of annotations could cause clutter, and furthermore if an opinion conflict exists between annotations on the same context it can become confusing for other analysts.

Storytelling

Although our work focuses on the BI domain, the broader need for storytelling is true for all visual analysis domains where analysts need to communicate their findings. Nevertheless such support exists in only few visual analysis systems. We hope our system motivates more work on visual storytelling, we feel that some of our storytelling requirements, for example the need to both playback the story as an animation but also have the option for a static overview, can be directly applied to other domains. Other requirements, such as the need to often reuse the same story structure and constraint user interaction, maybe domain specific and need to be re-examined for other domains.

Our system was identified by experts as having great potential for training other analysis.

Although it was designed as a stand alone tool, this does not mean that it aims to replace the expert trainer analysts. We think that in training scenarios, as mentioned by one expert, it will act to aid novice analysts understand how to read the analysis results so they can then focus their questions to their trainer on more complex aspects, such as analysis methodology, the analysis goals and content details. It can help authors reach a broader audience if little or no training is required to read the results of an analysis.

BI analysis is highly driven by the analysts' client and BI stories evolve based on client needs and feedback, a characteristic not present in most other analysis domains. Nevertheless the details of this process (done through emails, and phone calls) are often lost.

Experts suggested using our prototype as a collaborative storytelling system shared between analysts and clients, that can help reduce the turnaround time for the final story, and that captures the evolution of the story structure and focus. Thus the storytelling tool can be act as means for the audience to influence the story outcomes themselves. We currently exploring this possibility.

Moreover, it was suggested by our experts that our prototype can help to improve BI story quality and creators expertise, through communicating with readers and learning from their comments and feedback directly on the story (i.e. collaborative annotation).

Some experts even expressed the desire for a collaborative storytelling tool, where the story (not just the annotations) is open to being edited by both readers and authors.

In both cases of collaborative story editing, the prototype can move from simple communication to becoming a collaboration medium, where the authoring of a story opens-up and evolves with the contributions of many users, and acts as an archive of knowledge and different points of view. This opening up of the story is somewhat contradictory to the need to control the story, expressed by our experts in their requirements. In such a system our experts explained that readers need to be able to clearly differentiate between the contributions of individual users (e.g. filtering the author of added explanations and annotations). To support this process we need to also provide a powerful history and comparison mechanism, that displays different story versions and shows to the reader clearly what changed through time or from a storyteller to another, while providing advanced temporal navigation and filtering functionalities. We are currently exploring this extension.

Our prototype is currently being finalized to be given to BI analysts in order to study its long term use and how its impact on analysis collaboration and communication practices.

  Amélioration de l'interaction des utilisateurs avec des dashboards de Business Intelligence par Micheline Elias Docteur de Philisophy Ecole Centrale Paris Laboratoire MAS (Mathématiques Appliquées aux Systèmes) L'objectif de cette thèse est d'améliorer les tableaux de bord (dashboard) de visualisation afin d'aider à la prise de décision et d'analyse BI, et les rendre accessibles à un public plus large. Nous identifions les tendances actuelles et pratiques d'utilisation des dashboards de visualisation dans le domaine BI, et après une approche de conception centreé, nous proposons des améliorations et des solutions. Plus précisément, nous relevons les défis suivants: faire la construction du dashboard et d'utiliser accessible aux novices de visualisation, l'amélioration des dashboards avec des capacités d'annotation pour aider les analystes BI dans le processus d'analyse, et ajouter des fonctionnalités du storytelling pour aider à la communication entre les analystes et les décideurs. Notre approche centrée-utilisateur consiste à recueillir les besoins des utilisateurs (grâce à la recherche de fond, des entrevues en profondeur ou des séances de prototypage avec des experts BI), le développement itératif des prototypes, et l'évaluation des prototypes issus avec des utilisateurs représentatifs.

  Les analysis BI (experts ou novices) ont besoin d'enregistrer leurs idées et leurs connaissances les organiser et communiquer aux autres, et les mis à disposition pour les futurs analystes. Cette étape de communication est très important, comme souvent l'analyste de données et le décideur sont deux personnes différentes. Pour atteindre cet objectif nous avons besoin d'une mécanisme de partage facile afin de faciliter le transfert des connaissances, des expériences et des histoires avec les décideurs et d'autres analystes. Nous avons interviewé des experts en analyse BI et recueillies de nouvelles exigences liées à la narration visuelle BI. Sur la base de ces exigences, nous avons conc . u et mis en oeuvre un prototype de narration (storytelling) qui est intégré dans un dashboard, et permet une transition facile entre l'analyse et la création d'une histoire visuel et son partage. Notre système a été identifié par les experts comme ayant un grand potentiel pour la formation d'autres analystes. En outre, il peut aider les auteurs à l'analyse BI d'organiser et de communiquer leurs conclusions rapidement aux décideurs, et les analyse peuvent atteindre un public plus large, puisque les résultats d'une analyse présenté dans l'outil de narration peut être lu avec peu ou pas de formation.

4 . 3

 43 database schema for Sales database .................. 1.2 OLAP data cube example ........................... 1.3 ISO UCD .................................... 2.1 Commercial BI dashboards .......................... 2.2 Improvise .................................... 2.3 Snap-Together ................................. 2.4 Visual Analytics (VA) systems ........................ 2.5 Types of Narrative Visualization proposed by Segel et al. [1] ........ 2.6 Geotime Interface ................................ 3.1 Dashboard life-cycle .............................. 3.2 Exploration Views ............................... 3.3 Exploration Views creation dialog ....................... 3.4 Exploration Views Chart Suggestion ..................... 3.5 Exploration Views Data Category and measures selection. ......... 3.6 Exploration Views Tool Bar .......................... 3.7 Drag and Drop visualization .......................... 3.8 Filters and Faceted navigation creation. ................... 3.9 Filters and Faceted navigation creation, drag-and-drop. ........... 3.10 Active Filters .................................. 3.11 Graphical explanations ............................. 3.12 eFashion ..................................... 3.13 follow-up interview results ........................... 3.14 An example of EV Filtering functionality .................. 3.15 Understanding Linking & coordinated views by novice and experts .... 4.1 Our dashboard prototype with context aware annotations. (a) The main dashboard contains icons with the number of annotations for each datapoint (b). Bellow the list of all dashboard annotations in (c), is a set of recommended annotations (d) emphasizing their similarity. ......... 4.2 Three examples of the actual practice of annotating visualizations provided by our participants. (1) A dashboard represented in the middle, and associated annotations on right and left. The left annotations represent the bad remarks, and the right side contains the good remarks. (2) and (3) are two slides from a report training powerpoint presentation, that contains annotations made on the presentation tool. ................. xiv List of Figures xv This figure shows the context-aware annotation structure followed in our prototype. An annotation can be attached to multiple targets, and each target is identified by its data context (dimensions and measures, names and values). The annotation saves with each target the dataset name, visualization type and a snapshot at the time of annotating. Moreover, an annotation has different properties such as: author, creation date, annotation ranking, categorization and lifetime. Targets .............. 4.4 Detailed annotation view, including (a) all related data contexts & detailed data context in text form, (b) content, (c) metadata, (e) discussion thread, and (d) annotation properties (lifetime, privacy, etc). ............ 4.5 Chart Transparent Annotations ........................ 4.6 This figure represents how our prototype distinguish invalid archived annotations (grayed out and written in red "Invalid"), from the valid annotations, in the annotation list. ............................. 4.7 Two variations of the dialog for creating a new annotation: (a) permanent annotation that would be always valid, (b) periodic annotation in which the user has to identify the life time of the annotation in time periods. The user's choice of "permanent" or "periodic" periods defines the view of the dialog. ...................................... 4.8 Level Transparent Annotations ........................ 4.9 An annotation example that the participant explored during performing Task 3. (a) Radar chart presents the "50 Richest Americans in 2006". (b) Annotation icons that declares the existence of annotations attached to two data points on the chart. (c) In a new window we see the content of an annotation that says "Second Rich man!". (d) The original chart type (scatter plot) when the annotation was created. (e) Context of the data point, the annotations is attached to. (f) List of all existing annotations. .7 4.10 Three designs linking annotations in the annotation list with their data targets. Experts preferred the last design. ..................

  An example of explanations included in a power point presentation used as an introduction session to report readers. Example provided by one of our interviewed experts. .............................. 5.4 Genres of Narrative Visualization selected by BI experts from a group of templates defined by Segel et al. ....................... 5.5 Participatory design session .......................... 5.6 Story created by an expert during the design session. ............ List of Figures xvi 5.7 The final story mockup created by an expert (fully annotated and sequenced story seen in Figure 5.6). It represents how a static story on a system should look like. ..................................... 5.8 Dashboard with visualizations on the left and list of annotations on the right. ...................................... 5.9 Narrative board (a), Sequence of entities (b), Relational (c) and Organization (d) entities, and (e) Playback control. .................. 5.10 Storytelling playback modes. .........................

Chapter 1 .

 1 Introduction 2

Figure 1 . 1 :

 11 Figure 1.1: Example of an OLAP database schema for Sales database. The facttable usually represents measures. In this example the fact table represents Sales which are categorized by dimensional tables (Product, Customer, Time and Location). Each dimensional table needs to include a primary key that corresponds to a foreign key in the fact table that represents the sales measures.

Figure 1 . 2 :

 12 Figure 1.2: OLAP data cube example.

Figure 1 . 3 :

 13 Figure 1.3: (ISO 13407: Human-centred design process) defines a general process for including human-centered activities throughout a development life-cycle. The four activities are: (1) Specify the context of use: Identify the people who will use the product, what they will use it for, and under what conditions they will use it.(2) Specify requirements: Identify any business requirements or user goals that must be met for the product to be successful. (3) Create design solutions: This part of the process may be done in stages, building from a rough concept to a complete design. (4) Evaluate designs: The most important part of this process is that evaluation -ideally through usability testing with actual users -is as integral as quality testing is to good software development.

Figure 2 . 1 :

 21 Figure 2.1: Commercial BI dashboards: (a)Dundas [42], (b)Spotfire [35], (c)Tableau [36], (d)SAP BusinessObjects dashboard [44], (e)Choosel [45], (f)OracleBI 10g [43].

Figure 2 . 2 :

 22 Figure 2.2: Improvise [50]. The visualization represents the election results in Michigan from 1998 to 2002. (A) Selection of counties visualized in table view and a map. (B) When selecting a race from races visualization, the election results for that race will be loaded (from a file) and shown in the "Results for selection candidates and counties" visualization. (C) When selecting from the "Candidates" table, the pie chart will be filtered to compare results for selected candidates only. (D) Selected counties in the scatterplot visualization are highlighted with gray bars. (E) A four-layer scatterplot colors counties by winning candidate party. (F) A semantic zoomed in "state map" visualization labels counties with nested bar plots.

Figure 2 . 3 :

 23 Figure 2.3: Snap-Together[START_REF] North | Snap-together visualization: A user interface for coordinating visualizations via relational schemata[END_REF]. A coordinated visualization environment constructed using Snap-Together Visualization. This visualization represents the Census data of U.S. states and counties. When selecting a state from the left side visualizations, displays detailed county and industry information for that state in the table and Treemap on the right.

Figure 2 . 4 :

 24 Figure 2.4: Visual Analytics (VA) systems that supports annotating a single visualization: (a) sense.us [52] (b) ManyEyes [37]

  reports [99]. Segel et al.[1] explore different aspects of narratives from a variety of sources and identify distinct genres of narrative visualization, see Figure 2.5.

Figure 2 . 5 :

 25 Figure 2.5: Types of Narrative Visualization proposed by Segel et al. [1]

Figure 2 . 6 :

 26 Figure 2.6: Geotime shows events in time and space, in a X, Y and T coordinating space. [103]

Figure 3 . 1 :

 31 Figure 3.1: Current BI dashboard creation process involves multiple actors, end-users and business analysts, who require a large amount of communication in different stages of dashboard design.

32 (

 32 sual templates and representations to learn what meets their needs. Requirements and tasks can also change over time, requiring different data representations and data filtering mechanisms, or new data sources. EV should support iterative visualization specifications Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices suggested by Grammel et al. [38]) by being easily customizable and adaptable. It is important for novices to see the effect of their choices immediately, promoting exploration and experimentation with visualizations. Thus in EV customizations become active immediately, ensuring that customization is tightly coupled with the visual analytics process (recommended by Grammel et al. [38]). EV further supports the visual analytics process with functions such as saving and sharing, text search mechanisms, fully linked visualizations and visual queries, and other data exploration mechanisms. Finally, if the design of EV supports both low-cost experimentation with visual templates and visual analytics processes, it can help novice users to become more accustomed to visualization creation and analysis, promoting learning.

Figure 3 . 2 :

 32 Figure 3.2: The Exploration Views consists of: (a) the main Dashboard and its components. (b) The Clipboard drawer widget that includes templates for charts (top) and data widget samples (bottom) that the user has created as a pallet. The drawer can be closed when not needed. (c) A visual representation of the global data filters that are active at any point. (d) And a set of other functions, such as search, save, share, etc.

  3). The visual representation of possible dashboards helps users recall designs (as opposed to remembering dashboards Chapter 3. Understanding Dashboard Creation and Customization for Visualization Novices 33 solely based on names [106]), and their titles are descriptive so novices can start from a reasonable existing dashboard design, minimizing new content creation. These dashboard templates can be also shared with others. If none of the existing dashboard templates meets the user needs, she can choose to create a dashboard from scratch.

Figure 3 . 3 :

 33 Figure 3.3: Creation dialogs with visual examples of templates or saved dashboards.

Chapter 3 .

 3 Understanding Dashboard Creation and Customization for Visualization Novices 34 [105],[107]) by providing reasonable mapping and visual template recommendations (Figure 3.4).

Figure 3 . 4 :

 34 Figure 3.4: Chart creation dialog with recommendations of charts appropriate to data categories and analysis types, sorted from simpler to more complex.

Chapter 3 .

 3 Understanding Dashboard Creation and Customization for Visualization Novices 35

Figure 3 . 5 :

 35 Figure 3.5: Data Category and measures selection.

Figure 3 .

 3 Figure 3.6:The "explore" mode is a detailed view of a chart, where users can undo/redo actions, choose data categories, sort items, change data sources, switch chart templates, and create local data filters. To avoid switching to the "explore" mode, part of this functionality is also available as a drawer toolbar for each chart in the main dashboard, with added options to lock the chart.

Figure 3 . 7 :

 37 Figure 3.7: (a) Users select subset of their data, that get highlighted in all charts. (b) They drag their selections outside the chart to create a cut-out clone visualization, of the same type (here bar chart) with the selected data only. A blue highlighted bar indicates the new chart will be placed on the right of the original, with existing charts shifting to accommodate it. (c) The new bar chart of the data subset created and the dashboard visualizations resized to accommodate the new visualization.

Chapter 3 .

 3 Understanding Dashboard Creation and Customization for Visualization Novices 38

Figure 3 . 8 :

 38 Figure 3.8: Filters and Faceted navigation creation. User can select global dashboard filters to apply directly on the whole dashboard charts.

Figure 3 . 9 :

 39 Figure 3.9: The user can drag-and-drop a new global filter and choose the data categories that will be visible on the dashboard. The filter is either directly applied, or can become a filter control component for faceted navigation.

Chapter 3 .

 3 Understanding Dashboard Creation and Customization for Visualization Novices 39

Figure 3 . 10 :

 310 Figure 3.10: (a) All active global filters are visible at the top of the dashboard, (b) and local filters on the top of charts.

Chapter 3 .Figure 3 . 11 :

 3311 Figure 3.11: Graphical explanations used for training the novice participants in the user study: (a)Measures and dimensions on a chart axis (b)Filtering a chart (c)Drill down/up example.

Figure 3 .

 3 Figure 3.12: E-fashion dashboard that participants were asked to replicate.

Chapter 3 .Figure 3 . 13 :

 3313 Figure 3.13: Results of 7-point Likert scale questionnaire: (a)Novel participants answers scale (b)Expert participant answers scale.

Chapter 3 .Figure 3 . 14 :Figure 3 . 7 )Figure 3 . 15 :Chapter 3 .

 3314373153 Figure 3.14: An example of EV Filtering functionality. In this example visualization of Sales Revenue (a)the user selects data for Q1 and Q2 in year 2005, then clicks on the "filter" button. (b)The filtering result presented in EV.

Chapter 3 .

 3 Understanding Dashboard Creation and Customization for Visualization Novices 53 When providing coordinated views, designers should remember that novice users confuse linked selections of visual queries to applying global data filters based on the selections. To help disambiguation, they need clear visual differentiation between selection for linking and for data filtering. A linking selection should highlight selections without affecting the visual representation of unselected data; a filtering selection should also alter the visual representation of the unselected data (e.g. fade them out) to indicate that all data are being altered due to the selection and if the user requires it only show the filtered data.

Chapter 4 .

 4 Annotating BI Visualization Dashboards 58 annotations, such as issues arising when the "context" changes (e.g. annotated data are deleted or changed at the data source).

Figure 4 . 1 :

 41 Figure 4.1: Our dashboard prototype with context aware annotations. (a) The main dashboard contains icons with the number of annotations for each data-point (b). Bellow the list of all dashboard annotations in (c), is a set of recommended annotations (d) emphasizing their similarity.

  dashboards. Experts brought to the interviews annotated examples made by them or other BI experts in their groups. These examples indeed consisted of either screenshots of dashboards on paper with hand-written comments, snapshots attached to emails with annotated explanations, or presentation slides with annotated text (Figure 4.2). By examining this material with them, we extracted annotation functionality they would like to have embedded in their dashboards systems. Seven major needs emerged from the interview and expert's examples:

Figure 4 . 2 :

 42 Figure 4.2: Three examples of the actual practice of annotating visualizations provided by our participants. (1) A dashboard represented in the middle, and associated annotations on right and left. The left annotations represent the bad remarks, and the right side contains the good remarks. (2) and (3) are two slides from a report training powerpoint presentation, that contains annotations made on the presentation tool.

Figure 4 . 3 :

 43 Figure 4.3: This figure shows the context-aware annotation structure followed in our prototype. An annotation can be attached to multiple targets, and each target is identified by its data context (dimensions and measures, names and values). The annotation saves with each target the dataset name, visualization type and a snapshot at the time of annotating. Moreover, an annotation has different properties such as: author, creation date, annotation ranking, categorization and lifetime. Targets

Figure 4 .

 4 Figure 4.1 shows a screenshot of our prototype's main user interface. The left portion of the dashboard screen is the main canvas where the retrieved data is visualized (Figure 4.1.a). Users explore and analyze data by interacting with each visualization chart on the dashboard. We provide functionality such as data filtering, sorting, as well as advanced OLAP functionality [46] (e.g. drill-down/up). On the main canvas (Figure 4.1.b),

Figure 4 . 4 :

 44 Figure 4.4: Detailed annotation view, including (a) all related data contexts & detailed data context in text form, (b) content, (c) metadata, (e) discussion thread, and (d) annotation properties (lifetime, privacy, etc).

Chapter 4 .

 4 Annotating

Figure 4 . 5 :

 45 Figure 4.5: Chart transparent annotations are attached to the actual data points, rather than static visualizations, that permits to show a data point's annotation in different charts. (a) A radar chart shows the attached annotations to two data points. (b) The visualization is switched to a vertical stacked bar chart, and still shows the attached annotations to the two data points.

Chapter 4 .

 4 Annotating BI Visualization Dashboards 71 flip side, this requirement essentially turns annotations into data monitoring mechanisms:

Figure 4 . 6 :

 46 Figure 4.6: This figure represents how our prototype distinguish invalid archived annotations (grayed out and written in red "Invalid"), from the valid annotations, in the annotation list.

Figure 4 . 7 :

 47 Figure 4.7: Two variations of the dialog for creating a new annotation: (a) permanent annotation that would be always valid, (b) periodic annotation in which the user has to identify the life time of the annotation in time periods. The user's choice of "permanent"or "periodic" periods defines the view of the dialog.

Figure 4

 4 Figure 4.8.

Figure 4 . 8 :

 48 Figure 4.8: Annotation transparency through dimension levels. (a) A bar chart presents the total retail price from 2003 to 2006, a "drill down" icon indicates the existence of annotations in a lower dimension level for year 2004. (b) The user selects the 2004 bar, and right click, a list of existing actions will appear, drill down or drill up, the user selects drill down. (c) The drilled down chart to the Quarter dimension level for year 2004, shows the existence of 3 annotations attached to that level on Quarter 3.

Forbes 50 Richest

 50 Americans in 2006 (50 records, 4 dimensions); (ii) Population per age group (110 records, 5 dimensions); (iii) World internet users (220 records, 3 dimensions);

Figure 4 . 9 :

 49 Figure 4.9: An annotation example that the participant explored during performing Task 3. (a) Radar chart presents the "50 Richest Americans in 2006". (b) Annotation icons that declares the existence of annotations attached to two data points on the chart. (c) In a new window we see the content of an annotation that says "Second Rich man!". (d) The original chart type (scatter plot) when the annotation was created. (e) Context of the data point, the annotations is attached to. (f) List of all existing annotations.

  prototype. The first point raised was the use of icons on the main dashboard to indicate annotation on specific data points. Although they found this useful, they suggested that if a data point has many annotations we should use a different icon, that when clicked shows a drop down list of all annotations. This would introduce another annotation icon (apart from that of simple annotations and annotations at different granularities). So we chose an icon containing the number of annotations attached to the data point. The two annotation icons are seen in Figure 4.1 and Figure 4.8.

Figure 4 .

 4 10.a) highlights targets in red, the second (Figure 4.10.b) draws arrows to the targets, and the third (Figure 4.10.c) fades out all related targets. All participants preferred fading out. As one explained 'this shows only points I should focus on while still seeing the whole context; moreover I can print this representation and share it because it has less clutter and more interpretable".

Figure 4 . 10 :

 410 Figure 4.10: Three designs linking annotations in the annotation list with their data targets. Experts preferred the last design.

Chapter 5 .

 5 Storytelling in Visual Analytics tools for Business Intelligence 86 day, thus require answer to their question(s) in a timely and targeted manner. Longer detailed reports are used more to communicate findings to other analysts, project teams and product owners. Experts mentioned that BI reports are used to: spacing label Answer specific questions (e.g What is the progress of code development in project X the last 3 weeks?); label Investigate specific data (e.g Investigate the increase of bugs in section Y of project X); label Manage conflicts and highlight problems (e.g Team X completed less use cases than planned, because test team Y did not complete testing); label Interpret past data and predict future trends (e.g Given last year's sales last, what is the sales projection for this year).

Figure 5 . 1 :

 51 Figure 5.1: First page of a BI report given by an expert: a dashboard snapshot that presents the bug status in a specific development project. (Used with the experts' permission)

Figure 5 . 2 :

 52 Figure 5.2: An example of a BI report given by an expert that was used in the evaluation of our system. This dashboard represents the "technical backlog" & "functional users stories" items status in different perspectives. (1) Green group, represents the status of the items per team. (2) Yellow group, shows the created items versus the completed "done" items, and the category (critical, major, etc) of each. (3) Pink visualization presents the remained items to be completed and its progression through time.

Figure 5 . 3 :

 53 Figure 5.3: An example of explanations included in a power point presentation used as an introduction session to report readers. Example provided by one of our interviewed experts.

Chapter 5 .

 5 Storytelling in Visual Analytics tools for Business Intelligence 90 At each communication iteration, creators explore the desired data using interactive visualization dashboard tools, create initial visualizations, provide additional information and details about the analytical techniques used, and finalize reading paths (i.e. how to read a report).

Figure 5 . 4 :

 54 Figure 5.4: Genres of Narrative Visualization selected by BI experts from a group of templates defined by Segel et al.

Figure 5 .

 5 Figure 5.6 shows the final version of the manually created story by the expert, where we can see the intense usage of annotations and explanations, connected by arrows and lines.

Chapter 5 .Figure 5 . 5 :

 555 Figure 5.5: Participatory design session settings, that took place at SAP UX room.

Figure 5 . 6 :

 56 Figure 5.6: Story created by an expert during the design session.

Figure 5 . 7 :

 57 Figure 5.7: The final story mockup created by an expert (fully annotated and sequenced story seen in Figure 5.6). It represents how a static story on a system should look like.

  visualizations and their annotations (annotation text + data targets highlighted) are immediately placed in the narrative board. Thus the analyst can transition fluidly (R1) from analysis to story creation. Because dashboards can present evolving data, but a story can be an instance in time, by default each visualization is placed on the narrative board with a time-stamp to indicate when the visualization was taken from the data.

Figure 5 . 8 :

 58 Figure 5.8: Dashboard with visualizations on the left and list of annotations on the right.

Figure 5 . 9 :

 59 Figure 5.9: Narrative board (a), Sequence of entities (b), Relational (c) and Organization (d) entities, and (e) Playback control.

4 .Figure 5 . 10 :

 4510 Figure 5.10: Three storytelling playback modes. While playing the sequence of elements (the element sequence list on right) in a story, each element (highlighted in the element sequence list on right) will be: (a) Highlighted, (b) Maximized, (c) or Faded out, on the storyboard in the play back mode.

Chapter 5 . 106 5. 8 Summary

 51068 Storytelling in Visual Analytics tools for Business Intelligence BI analysts often need to organize visually and communicate their analysis story and findings to others. Nevertheless their tools do not allow them to easily transitioning from visual analysis to storytelling. They are often required to use multiple tools, replicate work, and train their audience to understand their analysis. Based on interviews and a paper prototyping sessions with expert BI analysts, we derived requirements on how to extend BI visual analysis dashboards to support storytelling, including the need for reuse of story structure, and the constraints to impose interactive visualizations.

Chapter 6 .

 6 such as: use annotations to discover other visualizations with the same data context and look at data in a new perspective; use annotations to link and organize multiple charts related to their analysis; Conclusions and Future Work 109 discover charts and annotations through annotation recommendation and understand the reasons behind it; and use annotation context to find information outside the system.

  Understanding Dashboard Creation and Customization for Visualization Novices 42All novice users were familiar with the concept of data filtering, but at least half did not understand concepts such as drill down-up. Novice users were recruited from our university community, with a wide professional expertise (IT, medicine, and pharmacology). Experts were product users of a BI dashboard provider (SAP) with different areas of expertise

	(finance, human resources, analysts, etc). Experts reported using charts and dashboards
	on a daily basis and were familiar with multiple types of charts, data analysis types (such
	as correlation, trends, comparisons, etc), and data exploration concepts (filtering, drill-
	down/up, pivot, roll-up, sorting, etc).
	Overall 15 participants (4 women), ages between 24 and 43 years old (median of 29) took
	part in our study: 8 were visualization novice and dashboard design users, 7 were BI
	dashboard experts. Visualization and BI dashboard novices had no previous knowledge

of visualization creation and were only familiar with basic charts (e.g. pie charts) seen in everyday life in newspapers or presentations. They reported being exposed to charts on a monthly basis. When asked, only 2 novice users reported that they knew what a dashboard was, and one of them had used the Google analytics dashboard. Chapter 3.

Table 3 . 1 :

 31 Tasks users performed during each session of the study, and their purpose.

Table 3 .

 3 2: Average ratings of EV for different attributes, by Novice and Expert users (* sig: indicates significant difference between the 2 groups)

	User	Utility	Functionality	Ease of Use	Consistency	Satisfaction
	Novice 5 (std. 1.12) 6* (std. 1.12) 5.5 (std. 1.58) 5.7* (std. 0.83) 6.5* (std. 0.74)
	Expert 5 (std. 1.63) 4* (std. 0.75)	5 (std. 0.53)	6.1* (std. 0.95)	5* (std. 0.69)
	3.6.2 General Observations on Dashboards for Novices

3.6.2.1 Chart Creation and Customization

Chart recommendations. As suggested in previous work on visualization novices, we provided a chart recommendation feature. All participants commented on the value of EV in helping them create appropriate charts fast. Although all users chose charts from the recommendation, only novices experimented with different alternatives, in order to "see what this new visual looks like", "look at a chart I've never seen before". We've had at least 2 users discovering and learning in this way a new visualization that they used often until the end of the study. Nevertheless, in most cases, novice and expert users ended up using familiar visualizations (bar and pie charts) for answering questions ("this new one looks complex, let's stick with what we have").

Table 4 .

 4

	Usability and under-
	standability

1: Tasks users performed during the study of context-aware annotations on dashboards.

Table 4 .

 4

2: Post-test (context-aware annotations tasks) questionnaire.

Acknowledgements

The acknowledgements and the people to thank go here, don't forget to include your project advisor. . . viii

List of Tables

Summary

We have collected a new set of requirements from BI experts for annotating dashboards. Context-aware annotations across multiple visualizations on a dashboard can meet these users needs. Based on the collected requirements and following a user-centered design approach we built our context-aware annotation prototype. We identified and provided solutions to challenges in using these types of annotations, notably issues arising when the annotation's "context" changes (e.g. annotated data are deleted or changed). Based on feedback from experts, such annotations should be achieved with a snapshot of their context, as long there is an easy way to retrieve them (e.g. through recommendations). Users should also be able to define the lifetime of annotations.

Our prototype was evaluated with a different set of BI experts that were able to easily perform several tasks that they deemed important in their work: use annotations to discover other visualizations with the same data context and look at data in a new perspective; use annotations to link and organize multiple charts related to their analysis; discover charts and annotations through annotation recommendation and understand the reasons behind it; and use annotation context to find related information outside the system.

Chapter 5

Storytelling in Visual Analytics tools for Business Intelligence

Abstract

Stories help us communicate knowledge, share and interpret experiences.

In this chapter we discuss the use of storytelling in Business Intelligence (BI)

analysis. We derive the actual practices in creating BI stories from in-depth interviews with expert BI analysts (both story "creators" and "readers"). These interviews revealed the need to extend current BI visual analysis applications to enable storytelling, as well as new requirements related to BI visual storytelling. Based on these requirements we designed and implemented a storytelling prototype tool that is integrated in an analysis tool used by our experts, and allows easy transition from analysis to story creation. We report experts' recommendations and reactions to the use of the prototype to create stories, as well as novices' reactions to reading these stories.

# Question

Q1

Why is the trend line of "user story" in the pink line chart stable in the first spring?

Q2

Why is there no progress in the project "user story" in the pink chart?

Q3

What is the difference between the two yellow line charts?

Q4

What is the number of critical items done last week?

Q5

Why are the critical non developed items increasing? Q6 Is the "User Story" project meeting the expectations? Why?

Table 5.1: Users were asked this group of comprehension question related to the content of the story and charts, after reading first the report then the story.

waiting list. On chart in the bottom right highlighted the problem and the rest provide details.

Reading the report

When given the report, all BI novice readers read the first page dashboard from left to right and up to down (which in fact is not the "author" suggested order). They characterized the report as a presentation of facts, that they understood the goal (progress of a project)

and what the charts displayed (e.g. bugs in waiting list) with the aid of titles and legends on each chart. But they all struggled to finding what was the problem illustrated in the report. Only one participant figured out that the project development is not progressing over time which is not normal, nevertheless she could not understand why.

This clearly demonstrated our experts' comment that reports cannot be read without the supporting material.

Reading the story

Participants found that reading a story was easier "the story showed the facts in an understandable manner". The 5 readers found the system easy to use, understood the story, and were able to answer correctly several comprehension question related to the content of the story and charts Table 5.1.

We report here our main observations and readers' comments: