Markku Turunen
email: markku.turunen@sis.uta.fi

Daniel Sonntag
email: daniel.sonntag@dfki.de

Klaus-Peter Engelbrecht
email: klaus-peter.engelbrecht@telekom.de

Thomas Olsson
email: thomas.olsson@tut.fi

Dirk Schnelle-Walka

Andrés Lucero
email: lucero@mci.sdu.dk

Interaction and Humans in Internet of Things

Keywords: IoT, novel interaction means, automation

Internet of Things is mainly about connected devices embedded in our everyday environment. Typically, 'interaction' in the context of IoT means interfaces which allow people to either monitor or configure IoT devices. Some examples include mobile applications and embedded touchscreens for control of various functions (e.g., heating, lights, and energy efficiency) in environments such as homes and offices. In some cases, humans are an explicit part of the scenario, such as in those cases where people are monitored (e.g., children and elderly) by IoT devices. Interaction in such applications is still quite straightforward, mainly consisting of traditional graphical interfaces, which often leads to clumsy co-existence of human and IoT devices. Thus, there is a need to investigate what kinds of interaction techniques could provide IoT to be more human oriented, what is the role of automation and interaction, and how human originated data can be used in IoT.

Introduction

Most successful Internet of Things applications nowadays provide quite straightforward user interfaces in comparison to the vast capabilities a smart device with smart materials, sensors and web access could actually provide. When thinking of the most successful practical IoT applications, human-technology interaction is mostly focused on graphical mobile or web interfacesand in some cases embedded interaction devices such as touch screen panels. Typical examples include "smart home" devices, such as thermostats, ovens, and light fixtures. Since it is assumed that the IoT devices operate mostly automatically, the control interfaces are mainly targeted for initial setups or other more individual configuration operations. Another frequent task is monitoring of the device state. Situated control of IoT devices is less frequent in current applications, but may gain importance once more of our everyday objects are connected to the IoT. For situated control, it is particularly desirable that we can interact with the device directly, rather than indirectly using graphical user interfaces. Moreover, the smart capabilities of IoT devices should be exploited for a more human-oriented, i.e. natural and tangible, interaction. To make IoT devices more human oriented we need to explore novel ways to interact with them in our daily life, research how user-friendly automation can be achieved, and analyze how human originated data can be utilized. In this panel, the key topics related to interaction in IoT are covered from these perspectives.

Interaction techniques for IoT

As stated, most human-technology interfaces in IoT are still rather traditional, and usually based on GUIs. However, more natural multimodal interaction means, such as spoken and gestural interaction, for example, could provide more efficient and pleasant interaction with IoT devices. Still, such interfaces are not studied widely in IoT context, even there are some cases, which are mostly related to "non-serious" applications such as toys and other gadgets. What we need are success storiesor at least some storiesfrom experiences of different interaction techniques which have been studied in everyday IoT scenarios. The participants of the panel will demonstrate concrete examples and findings from novel IoT interfaces.

Automation and Interaction

One of the most interesting aspects of many IoT applications is the relation of automation and human control. For example, when IoT devices are used to automate certain functions in home environments, this creates huge challenges for acceptance of the technology, including critical factors such as safety, feeling of control, and privacy, among others. The key question is in which cases automation is more suitable and when human control should be preferred. The major factors for successful design for co-existence of automation and explicit interaction of IoT devices are discussed.

Human-related information in IoT

In addition to the explicit interaction between humans and devices, human originated information is often highly valuable or even crucial in many IoT applications.

  Most obvious examples include IoT applications monitoring humans. Practical examples include self-monitoring applications, which are typically related to health and wellbeing. Other examples include applications for monitoring of other people, such as chil-dren and elderly. Topics for discussion include novel means to gather, process and distribute human-oriented data, as well as key questions related to the human-factors, such as safety and privacy. Finally, it will be discussed how all of this affects and changes human behavior in the long run.Markku Turunen is a professor of Interactive Technology at the School of Information Sciences in University of Tampere, Finland. He has been worked in the Tampere Unit for Computer-Human Interaction (TAUCHI) since 1998, leading a group on pervasive interaction. His fields of expertise include novel interaction techniques, software architectures for interactive systems, pervasive applications, interactive solutions for industrial settings, user experience of multimodal interaction, ecological valid evaluation methods (including showrooms, living labs, and long-term pilot studies) with representative user groups, and commercialization or research results. His teaching portfolio includes multiple courses on pervasive and multimodal interaction. Postdoc at the Quality and Usability Lab, Telekom Innovation Laboratories, Technische Universität Berlin. He studied Communication Research and Musicology and received his Magister degree in 2006 from Technische Universität Berlin. In July 2011 he successfully defended his Dissertation thesis "Estimating Spoken Dialog System Quality with User Models". His more recent work deals with modeling how users experience interactions with automated dialog partners, and how this impacts their quality judgments. From 2013 through 2014, he managed the Forschungscampus project "Interaktion & Sensoren" funded by the Federal Ministry of Education and Research of Germany. In this project, novel ways of interacting with technology in the smart home were designed and evaluated.Thomas Olsson is an adjunct professor of User Experience of Socio-Technical Systems at the Department of Pervasive Computing in Tampere University of Technology (TUT), Finland. He received the Dr. Tech. degree in 2012 with a thesis addressing user experience and user expectations of future mobile augmented reality systems. Currently he leads a research team focusing on social technologies that aim to enhance social interaction and collaboration between co-located people, utilizing technologies like proximity sensing, wearables and smart networked objects. His other research interests include user experience in various ubiquitous computing systems, covering, e.g., context awareness, internet of things and smart environments, as well as user-centered design and user expectations of new interactive technology.Dr. Dirk Schnelle-Walka lead the "Talk&Touch'' group at the Telecooperation Lab at TU Darmstadt until end of 2014. Since then, he works as a function owner speech&dialog for S1nn in the automotive industry to take his research portfolio to an industrial level. His research focus is on voice-centric multimodal interaction in smart spaces. He authored more than 50 book chapters, journal article and conference papers and is chairing the IUI workshop on Interacting Smart Objects and the EICS Workshop on Engineering interactive Systems with SCXML. He is also the head behind several open source projects around speech technology, e.g. the open source voice browser JVoiceXML Andrés Lucero is an Associate Professor of interaction design at the University of Southern Denmark in Kolding. His recent work at Nokia focused on the design and evaluation of novel interaction techniques for mobile phones and other interactive surfaces. He got his Masters degree in Visual Communication Design from Universidad Tecnológica Metropolitana (UTEM), Santiago, Chile. In 2004 he received a Professional Doctorate in Engineering (PDEng) degree in User-System Interaction from the Eindhoven University of Technology (TU/e) in the Netherlands, which included a one-year project in Philips Research. In 2009 he completed his PhD at the TU/e on co-designing mixed reality support tools for industrial designers. As part of his PhD work, he was a visiting researcher at the University of Art and Design Helsinki (TAIK) in Finland. His interests lie in the areas of mobile human-computer interaction, co-design, and design research.

	5	Panel members

He is also a CEO of Multisense Oy, a startup company specializing on advanced human-technology interaction.

Dr. Daniel Sonntag is a principal researcher and principal investigator/project leader at DFKI (Intelligent User Interfaces), reader at Saarland University and University of Kaiserslautern, and permanent member of the editorial board of the German Journal on Artificial Intelligence (KI). He has worked in natural language processing, multimodal interface design, dialogue systems, and knowledge representation for over 17 years and leads a team of more than 10 full-time researchers and engineers at DFKI, and 6 (PhD/Msc) students. His research interests include common-sense and machine learning methods for multimodal human computer interfaces and knowledge discovery, mobile interface design, cognitive modelling with ontologies, and usability for the Internet of the Future. He is particularly interested in practical methods for Augmented Cognition and Mixed Reality, Compensating Cognitive Impairments, (Medical) Cyber-Physical Systems, and Human-Machine Collaboration in Production.

Klaus-Peter Engelbrecht is working as a


Patrick Bader
email: baderp@hdm-stuttgart.de

Valentin Schwind
email: schwindv@hdm-stuttgart.de

Norman Pohl
email: pohl@hdm-stuttgart.de

Niels Henze

Katrin Wolf

Stefan Schneegass

Albrecht Schmidt

Self-Actuated Displays for Vertical Surfaces

Keywords: self-actuated, display, vertical surface, mobile

Most current devices are passive regarding their locations by being integrated in the environment or require to be carried when used in mobile scenarios. In this paper we present a novel type of self-actuated devices, which can be placed on vertical surfaces like whiteboards or walls. This enables vertical tangible interaction as well as the device interacting with the user through selfactuated movements. In this paper, we explore the application space for such devices by aggregating user-defined application ideas gathered in focus groups. Moreover, we implement and evaluate four interaction scenarios, discuss their usability and identify promising future use cases and improvements.

Introduction

The variety of input and output devices that can be used to interact with computing systems is steadily increasing. Traditionally we can discriminate interaction devices into two groups.

The first group covers stationary devices, including desktop computers, TVs, and public displays. These are not mobile while in use. In many cases they are installed and become part of the environment. Notebook computers, even though they are often carried and used in different settings fall in this group, too, as they are stationary while in use. The second group describes mobile devices including, smart phones, tablets, and interactive glasses that are carried or worn by the user. Interaction with these devices takes place while the user is mobile. These two groups of devices are well explored and the design space is well understood (e.g., for input devices [START_REF] Card | A morphological analysis of the design space of input devices[END_REF]). In recent years a third group of devices is emerging: devices which can move themselves and act autonomously, called self-actuated devices.

Interactive self-actuated devices combine the advantages of stationary devicesas the user does not have to carry themwith the advantages of mobile devicesas the device can always be with the user. Prominent examples of this device category are known from robotics. Domestic robots, such as Wakamaru [START_REF] Shiotani | World's first full-fledged communication robot "wakamaru" capable of living with family and supporting persons[END_REF] which provides companionship to elderly and disabled people, can autonomously serve the user. In recent conferences attendees participated through a robotic device, e.g. using the Beam re-mote presence system 1 . Besides systems that are designed for a specific application domain recent work in HCI proposed interactive self-actuated general purpose devices (e.g. [START_REF] Schneegass | Midair displays: Concept and first experiences with free-floating pervasive displays[END_REF][START_REF] Seifert | Hover pad: Interacting with autonomous and selfactuated displays in space[END_REF], [START_REF] Tominaga | Around me: A system with an escort robot providing a sports player's self-images[END_REF]). These works introduce devices that move freely, while providing rich input and output possibilities which is similar to state-of-the-art mobile devices, but without restricting the application purpose. Fig. 1. Some exemplary use cases for self-actuated displays. They can (1) guide a person in using a coffee machine (2) write formulas on whiteboards with an attached pen (3) guide a user through an exhibition and give additional information by being placed besides an exhibit (4) change the display size according to in-situ needs by joining multiple devices when multiple persons join a video conference.

Much like current stationary and mobile devices, interactive self-actuated devices can conceptually facilitate a large range of applications by combining different device behaviors and fitting several user roles. In this paper we explore an application space of how interactive self-actuated displays can be used from a human-centered perspective. Based on the idea that devices can move on any vertical surface, we implemented a prototype that can freely move on ferromagnetic planes. With the size and abilities of a standard tablet computer it can easily be carried and moved by the user while it has the additional abilities of a self-actuated device. Thus, the device combines the advantages of mobile, tangible, and self-actuated devices, which allows to support a broad set of use cases (see Fig. 1). Steerable projector systems [START_REF] Pinhanez | The everywhere displays projector: A device to create ubiquitous graphical interfaces[END_REF] or display walls can provide visual output across large spaces, however, a free-moving device enriches the Everywhere Display with a tangible dimension. In contrast to flying devices like Midair Displays [START_REF] Schneegass | Midair displays: Concept and first experiences with free-floating pervasive displays[END_REF] less power is consumed during operation. Furthermore unlike self-actuated devices for vertical surfaces flying and floor based devices share a movement space with users and thus may get in the user's way.

Using the interactive self-actuated prototype as a stimulus, we conducted a series of focus groups to explore the space of promising applications. Participants were asked to envision and discuss potential use cases. They proposed a truly broad range of corresponding ideas which we grouped in four categories: role, context, application, and device behavior. Using these categories, we identified four promising application scenarios that were implemented as cinematic showcases. Through a survey we further investigated the usability and emotional impact of the presented scenarios.

After revising the related work on interactive self-actuated devices, we investigate the concept and implementation of the self-actuated display device. This implies its use cases and application scenarios that we investigated by different focus groups.

Afterwards, we present four exemplary scenarios, their implementation, and the results of their evaluation that lead to a promising conclusion about the potential of selfactuated displays. The contribution of this paper is as follows:

─ The concept and implementation of a novel self-actuated display device. ─ An application space for the self-actuated displays for vertical surfaces. ─ An evaluation of promising application scenarios for these devices.

Related Work

Before self-actuated user interfaces were proposed, user-actuated interfaces were used to manipulate digital information through manually moving physical representations of virtual information. This concept of tangible user interfaces (TUIs) has been explored as passive physical user interface and advanced towards self-actuated physical user interfaces. Nowadays, a wide range of autonomous or semi-autonomous moving user interfaces has been proposed and built, including self-actuated TUIs, devices, and robots.

Physical User-Actuated Interfaces

Even before coining the notion Tangible User Interface, Fitzmaurice, Ishii, and Buxton introduced Graspable Interface [START_REF] Fitzmaurice | Bricks: Laying the foundations for graspable user interfaces[END_REF] that allow direct control and manipulation of digital objects through moving physical wooden bricks. Ishii and Ullmer later introduced Tangible Bits [START_REF] Ishii | Tangible bits: Towards seamless interfaces between people, bits and atoms[END_REF], a vision to use the whole real world as medium for manipulation of the virtual world. One of these prototypes was transBOARD, a digitallyenhanced whiteboard system that monitors the activity of physical objects on its vertical surface and has the capability of storing pen strokes. Another example was Urp [START_REF] Underkoffler | Urp: A luminous-tangible workbench for urban planning and design[END_REF], a TUI for collaborative urban planning using physical models of buildings on a tabletop system. Video projection and electromagnetic tagged wireless mice were used as pucks on the Sensetable [START_REF] Patten | Sensetable: A wireless object tracking platform for tangible user interfaces[END_REF], while the music interface reacTable [START_REF] Jordà | The reactable: Exploring the synergy between live music performance and tabletop tangible interfaces[END_REF] works with optical markers that are placed underneath the tangibles are moved on a tabletop system to play music. Geckos [START_REF] Leitner | Geckos: Combining magnets and pressure images to enable new tangible-object design and interaction[END_REF], Magnetic Appcessories [START_REF] Bianchi | Designing tangible magnetic appcessories[END_REF], and Gauss-Bits [START_REF] Liang | Gaussbits: Magnetic tangible bits for portable and occlusion-free near-surface interactions[END_REF] use magnets to attach passive tangible elements on vertical surfaces and thus demonstrate that interaction with TUIs is not limited to horizontal planes. In addition to magnetic solutions, vacuum adhesion forces for sticking tangible objects on vertical surfaces were used in Vertibles [START_REF] Hennecke | Vertibles: Using vacuum self-adhesion to create a tangible user interface for arbitrary interactive surfaces[END_REF].

Physical Self-Actuated Interfaces

Technologies proposed for actuating tangibles were for instance merged arrays of electromagnetic coils embedded in a tabletop system [START_REF] Pangaro | The actuated workbench: Computercontrolled actuation in tabletop tangible interfaces[END_REF], [START_REF] Weiss | Madgets: Actuating widgets on interactive tabletops[END_REF], the six-legged Hexbug TM [START_REF] Somanath | Integrating a robot in a tabletop reservoir engineering application[END_REF], and vibrating bristles [START_REF] Nowacka | Touchbugs: Actuated tangibles on multi-touch tables[END_REF]. Moreover, tabletop systems were using robots instead of TUIs (Touch and Toys [START_REF] Guo | Touch and toys: New techniques for interaction with a remote group of robots[END_REF], RoboTable [START_REF] Krzywinski | Robotable: A tabletop framework for tangible interaction with robots in a mixed reality[END_REF], RoboTable2 [START_REF] Sugimoto | Robotable2: A novel programming environment using physical robots on a tabletop platform[END_REF], Re-moteBunnies [START_REF] Guerra | Remotebunnies: Multi-agent phenomena mapping between physical environments[END_REF], and TabletopCars [START_REF] Dang | Tabletopcars: Interaction with active tangible remote controlled cars[END_REF]). PhyBots [START_REF] Kato | Phybots: A toolkit for making robotic things[END_REF] introduced a prototyping toolkit for adding locomotion on floors to everyday objects. Curlybot [START_REF] Frei | Curlybot: Designing a new class of computational toys[END_REF] was a driving educational toy robot that allows to be equipped with a pen extension. The PMD system uses tracked physical objects, whereby physical elements are moved by both, users and computers [START_REF] Rosenfeld | Physical objects as bidirectional user interface elements[END_REF]. Self-actuated devices have also been developed for vertical surfaces. WallBots [START_REF] Kuznetsov | Wallbots: Interactive wall-crawling robots in the hands of public artists and political activists[END_REF] are magnetic as well as self-actuated, autonomous wallcrawling robots equipped with a tri-colored LED and used in street art. Interactive self-actuated objects can also move in a three-dimensional space. ZeroN [START_REF] Lee | Zeron: Mid-air tangible interaction enabled by computer controlled magnetic levitation[END_REF], a magnetic-controlled volume with a levitating tangible element tracked by a Kinect demonstrates a physical computer-actuated 3D interface. The Midair Displays [START_REF] Schneegass | Midair displays: Concept and first experiences with free-floating pervasive displays[END_REF], a display mounted to a quad-copter, is conceptually a spatially unlimited levitating and moving interface. Similarly, Seifert et al., developed Hover Pad, a tablet that is attached to a static crane construction and can thereby freely move within a 3D space [START_REF] Seifert | Hover pad: Interacting with autonomous and selfactuated displays in space[END_REF].

Summary

Previous works on self-actuated interactive devices mainly focused on the technical aspects to realize novel types of devices, and mostly the device was build for one single application to demonstrate the technical concept. In contrast, our aim is to explore the application space of self-actuated interactive displays for vertical surfaces. Considering self-actuated devices as a new class of devices in this paper we explore potential applications of such devices from a human-centered perspective.

Self-Actuated Displays for Vertical Surfaces

In this section, we introduce the concept of self-actuated displays for vertical surfaces. Additionally we describe our prototypical implementation which realizes the main aspects of the concept using currently available technology.

Concept

In this work, we explore the possibilities of self-actuated displays that are able to move on horizontal as well as vertical surfaces. We present a device that can be grabbed and placed on surfaces (similar to tangible user interfaces [START_REF] Ishii | Tangible bits: Towards seamless interfaces between people, bits and atoms[END_REF]) and freely moved on these surfaces. This enables the usage of vertical surfaces such as walls, whiteboards, or ceilings. In contrast to most prior work, we thereby focus on devices that are actuated by the user as well as self-actuated depending on context and task.

To cover a broad range of interactions possibilities, we envision several input and output modalities. As input modality we mainly focus on touch screen, camera, and further sensors such as an accelerometer. The primary output modalities are visual and auditory. Since we envision a self-actuated device, we take the movement of the display itself into account as well. Moreover, the device can be equipped with traditional tools. For example, a pen can be attached to it, serving as an additional output means.

F M F G F F F M Magnets Ball caster
Motor F S Fig. 2. Section view of the prototype showing main components relevant to vertical movement. Gravity F G , stiction F S and magnetic forces F M are also shown.

Embedded sensors which are integrated in the device not only enable user tracking, they also facilitate coordination with other devices since we envision an active communication between multiple devices, so they can interact with each other and/or create a unified display space. Thus, the maximum display size is only limited by the number of devices used.

Whereas most current self-actuated interfaces are limited to horizontal surfaces, like interactive tables, our approach focuses on vertical surfaces. Therefore the device is also attachable to walls and whiteboards for example.

Prototype Implementation

We transferred the concept of our self-actuated display into two working prototypes (see Fig. 3).

Both are based on the commercially available 3pi robot platform by Pololu2 with the 3pi expansion kit without cutouts3 attached. We added support for external LiPobattery usage and charging to compensate for increased battery drainage caused by vertical movement. For wireless communication with external and attached devices a Bluetooth enabled microcontroller was used.

We enabled the prototype to move on ferromagnetic vertical surfaces by attaching a 3D printed frame 4 to its bottom that holds up to 22 neodymium magnets. To allow upwards movement on vertical surfaces, the motors have to generate enough torque to overcome both gravity F G and rolling resistance F F as depicted in Fig. 2. Thus, we replaced the default 30:1 gear motors with 298:1 gear motors. Furthermore, wheel slippage has to be prevented by generating enough stiction F S . In our case this is done by increasing magnetic force F M and contact pressure by adding magnets. However, increasing contact pressure also increases rolling resistance and reduces acceleration and maximum speed. We empirically determined the number and locations of the magnets needed to enable stable operation and ended up using 15 magnets with some bias towards the ball caster.

To expand the robot's input and output modalities, we attached a 3D printed frame that encloses a Google Nexus 7 tablet 5 . Besides its display, the tablet also provides the robot with additional peripherals like cameras, inertial sensors, Wi-Fi, and speakers. The tablet frame may also be extended with additional tools. As examples for such tools (see Fig. 3) we built servo motor actuated pen and eraser holders to draw on whiteboards. Each prototype is approximately 205 x 117 x 49mm in size and weighs 495g.

Creating Potential Use-Cases

To increase our understanding of the application space and to explore potential use cases for the device we conducted a series of focus groups [START_REF] Kitzinger | The methodology of focus groups: the importance of interaction between research participants[END_REF] and evaluated the results. We first presented the developed prototype to the focus groups to create a common understanding of the possibilities and interaction modalities. In the following, we first describe the design of the focus groups and afterwards, we present the results and a discussion.

Study Method

Three focus groups with 19 participants (15 male, 4 female) aged 22 to 41 (M=26.9 years, SD=4.3) were conductedsix to seven participants took part in each of them. We recruited participants through our mailing lists and from our peer group. We strove for a broad cultural background and, thus, we invited participants originating from five different countries, namely the U.S., Germany, Egypt, Belgium and Argentina. Each participant was compensated with 15 €.

After welcoming a group of participants and providing them with basic information about the procedure, we asked them to fill in a consent form and to answer a brief demographic questionnaire. After an introduction round, we introduced the main goal and procedure of a focus group to the participants. This was followed by a demonstration of the prototype (see Fig. 4) on a whiteboard and its capabilities as a stimulus for participants. We also highlighted the ability to control a pen and an eraser to show its potential for further extension. Directly after the presentation, we asked participants to write down their initial reactions (Result R1). We asked them to discuss them afterwards, and we took notes during the discussion (R2). Fig. 4. Photo of the prototype demonstration we showed participants as stimulus. Some lines were drawn with the pen that is attached to the device.

After the discussion of the participants' initial reactions, we asked them to write down potential use cases on large post-it notes (R3). This was followed by a discussion about the most promising and the most controversial cases, which were recorded in a written protocol for post-hoc analysis (R4). During this discussion participants could write down additional ideas on post-its (R3). After the discussion, we closed the respective session.

Results

Participants' first impression (R1) and their discussion (R2) were mainly positive. Answers can be categorized in three main categories. 1) Participants were impressed by the overall idea. They, for example, stated that "it looks impressive" (P5), "opens a new space" (P6) and is a "pretty interesting technology" (P16). 2) Participants also imagined applications for similar devices. They stated that it would be "useful if you have hands full" (P18) and could be used "in the kitchen" (P17). Finally, 3) two participants expressed concerns about the presented technology. One asked "For what?" (P13) and another participants wondered if it "must be able to move?" (P19).

In total, participants created 137 potential use cases (R3) for the presented technology. Using a bottom-up analysis and open coding, we identified 49 groups of ideas in total that again could be grouped into four main categories. An emerging group, for example, contains 31 ideas that propose to use the device in home environments and another group with 25 ideas proposed ideas where the device follows the user's position. The groups were categorized by their Role, Context, Application, and Device Behavior (see Table 1). Role can be further divided in ownership, audience, and controlling subject. The device can, for example, be autonomous, part of the infrastructure, and with a single person as audience. Context was mostly provided in the form of description of a location, such as an office or a classroom but also through specific situations such as emergencies. Participants' ideas provide diverse applications. Table 1. Aggregation of participants' ideas along the four dimensions. The numbers in the n columns denote the number of ideas that fit a particular group. Groups which are implemented by a specific scenario are highlighted using color coding. Blue groups are implemented by the kitchen scenario, green represents the classroom scenario. The museum scenario is highlighted with a yellow background, and the office scenario is red. Exemplary applications include navigation and route guidance, sending messages, as well as providing alarms and notifications or using the device as smart companion. The fourth category describes device behaviors. For example, arranging multiple devices in a grid generates a large display. Another example proposes that the device follows a predefined path and expects the user to follow for delivering locationand situation-based information to the user.

For a final improvement of the category consistency, we revised all ideas by going through the individual post-it notes and categorized them using the identified four idea categories. This procedure ensured that all ideas are covered by the four identified categories. During that process, we determined how often particular ideas appear in each group.

Discussion

The three focus groups identify a wide range of use cases for self-actuated displays on vertical surfaces (see Table 1). Using a bottom-up analysis we identified 76 groups to structure the ideas that can further be fused into the four categories Role, Context, Application, and Device Behavior. Moreover, these categories can be used to generate new application scenarios, which were not explicitly envisioned by one of our participants. Cells of Table 1, for example, can be fused to the following scenario: multiple devices that are part of an office's infrastructure (Role, Context), can build a large display (Device Behavior), to form a window by enabling the user to see through walls (Application). Whereas this particular scenario was not envisioned by one of our participants, it could be derived by combining idea groups across the four categories. However neither the results of the focus group can be generalized nor the amount of design ideas per group should be over interpreted. That means that ideas mentioned the most are not necessarily the most interesting ones. Yet we were able to cover a broad range of potential application scenarios.

Implementation of Example Scenarios

In the previous section, we developed application ideas for self-actuated displays. In this section, we select four scenarios that cover a broad range of application possibilities for self-actuated displays. For a later survey evaluation, we extended the previously described device prototype and prepared video prototypes. It will be described in the next chapter.

Selection of Scenarios

We designed four scenarios (kitchen, class room, museum, and office) through combining idea groups of each of the four idea categories we presented in Table 1. This allowed us to cover a broad variety of different application types. During that process, we aimed for covering diverse scenarios that are well represented through the ideas generated by the focus groups.

As shown by color coding in Table 1 the scenarios cover each sub-category of the role dimension: ownership, audience, and controlling subject. Moreover, we consider diverse context types as well as combinations of them: home (kitchen), classroom and teaching, public space (museum), as well as office and meeting. Afterwards, we se-lected applications that suit the selected contexts, while also representing a good coverage of the idea groups: guidance (task), teaching and presentation/data visualization, guidance (navigation) and podcast as well as communication/telepresence. Finally, we chose a device behavior for each scenario which fitted the combination of role, context, and application best.

Video Prototype

To evaluate the scenarios we created videos of the prototypes being used for each scenario. We present these videos to participants in an online survey. Thus, four storyboards, one for each scenario, were designed which describe how a user interacts with the self-actuated display during the four scenarios (see Fig. 5). The storyboards explain the context the scenario takes place in (e.g., museum or kitchen), the interaction sequence when a user is using the device in a specific application, and the content that is displayed on the device's screen during the interaction. Fig. 5. Stills of the four video prototypes presenting the scenarios. 1) Device gives directions for preparing coffee. 2) Graph is visualized using a whiteboard marker attached to the device.

3) Device follows a user through an exhibition. 4) Remote participant joins a conversation during a video conference. Scenario 1 -Kitchen. A person is in an unknown environment, for instance she just started to work in a new office. In the kitchen of this office is a commonly used coffee machine. She would like to have a coffee but she neither knows how the machine works nor where the cups and ingredients are. This video shows how a selfactuated display guides the person to find the cups and the ingredients as well as how to use the coffee machine.

Scenario 2 -Class room. The teacher is teaching trigonometry and explains a new formula. The self-actuated display assists through being an interactive display where the teacher can write any formula on. The self-actuated display draws the formula on the whiteboard to present it to the class.

Scenario 3 -Museum. Visitors may want some kind of additional information about the exhibits, but some of them want to follow their own path and not being guided in a tour. A self-actuatedand at the same time tangible displaycan follow the visitor in an exhibition and provides further information about exhibits of interest. Furthermore, placing the device manually (like TUIs) enables the user to gain further background information about an exhibit of interest.

Scenario 4 -Office. In this scenario the display can increase its size if that is necessary, for example, if additional people join an ongoing video conference meeting. For communicating with a single person, the display is sufficiently large enough to show that person. However, when another person joins the conversation, the display may be too small and, thus, one cannot see all conversation parties at once. In the video prototype we show how self-actuated displays allow to change their size through automatically gathering together and extend the display size similar to puzzle pieces.

The content displayed on the device during the video (text, images, and video) was pre-produced and presented in a remotely controlled slideshow while the video prototypes were recorded. The device has been remotely controlled in scenarios 1, 3 and 4 using a Microsoft Xbox360 game controller. For scenario 2 the predefined curve has been drawn autonomously using the control scheme described below. In summary, we developed four video prototypes with an average duration of 69 seconds.

Device Prototype

To realize the video prototypes we implemented two control schemesremote control and autonomous behavior. For remote control we used a commercially available game pad connected to a laptop. Motor speeds were calculated according to the direction of the analogue sticks and sent to the prototype via Bluetooth.

The whiteboard task also requires the display to move autonomously so the position and orientation needs to be detected. As the prototype has a tablet attached, we exploited its gravity sensor to determine the orientation. This orientation data is send to a laptop using Wi-Fi. To obtain the position of the device, we used an external Asus Xtion depth camera 6 which was oriented perpendicular towards the prototype. The device position is obtained by segmenting depth values in a short distance to the surface (2 to 7 cm). After filtering out small segments we determined the exact position on the surface by calculating the center of mass for each segment in screen space. We chose this method for its simplicity. In future versions the position could also be obtained by the prototype without an external sensor using the built in camera and feature tracking, for example.

Based on the position and orientation data we implemented a control scheme using a proportional-integral-derivative controller which allowed us to implement two simple movement commands: ─ look at: rotates the robot around its center until it is heading towards a target point. ─ move to: reaches a specific point by following a straight line to the target.

We used these simple commands to build scripts for drawing axes and plotting simple mathematical functions such as a sine curve by linear approximation. The autonomous behavior is, for instance, used in the classroom scenario in which the self-actuated display draws mathematical functions to assist the professor.

6

Scenario Evaluation

Method

For evaluating the four scenarios, we conducted an online survey, which we distributed via mailing lists and social networks. The survey started with an introduction about its purpose and a questionnaire that records the age and the gender of the participants. Then the scenarios were presented in randomized order. At the beginning, a brief introduction was given, then the video was shown, and afterwards we asked (using a 5 item Likert scale) if the presented scenario was liked. Moreover, we used the AttrakDiff mini questionnaire [START_REF] Hassenzahl | The inference of perceived usability from beauty[END_REF] to collect opinions about the scenarios. Finally, in two open questions the participants were asked to report positive (e.g., strengths or possibilities) and negative (e.g., weaknesses or risks) aspects of the scenario.

Results

In total, 57 participants (13 female, 44 male) aged 20 to 58 years (M=33.7, SD=9.6) completed our online survey. Thus, we collected 57 completed AttrakDiff questionnaires for each scenario as quantitative results and 269 (64.9 %) out of 456 possible qualitative answers to open questions regarding positive or negative aspects.

Quantitative Results. The user perception of the emotional impact was evaluated according to the AttrakDiff scheme (based on a 1-7 Likert scale). Table 2 compares the mean values of the scores of each presented scenario. The hedonic quality (HQ) consists of the HQ-Identity (HQ-I) and HQ-Stimulation (HQ-S). In terms of pragmatic quality Scenario 1 performs slightly better than the others. Relating to attractiveness Scenario 4 performs best. Also in terms of hedonic quality Scenario 4 achieves the highest scores. The small and overlapping confidence intervals indicate that the participants generally assess the presented scenarios similarly.

We further analyzed the data using a Friedman Analysis of Variance. However, the Friedman ANOVA yielded no significant differences for the AttrakDiff scales of hedonic qualities (HQ-I:  2 (3)=0.587, p=.899, HQ-S:  2 (3)=3.322, p=.345) as well as for attractiveness (ATT:  2 (3)=5.818, p=.121), we found a statistically significant difference in the pragmatic qualities (PQ:  2 (3)=8.051, p=.045). Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied, resulting in a significance level set at .008. However, we did not find any statistically significant differences for the perceived pragmatic qualities between the four scenarios (p>.008).

Table 2. Results from the AttrakDiff evaluation (N=57) of the four scenarios show the mean of the pragmatic quality (PQ), hedonic quality (HQ) and attractiveness (ATT). Confidence interval at 95 percent probability level. Qualitative Results. We analyzed the 269 qualitative answers of the questionnaire by manual assessment. Analysis was done in two iterations: derivation of a categorization and answer reassignment afterwards.

Scenario

In the first iteration two researchers independently derived categories for each scenario based on the answers and counted their occurrences. Then both categorizations were discussed and merged into unified categorization schemes. Analysis revealed seven scenario independent categories, which we then separated from the scenario specific ones.

In the second iteration we went through the answers once again, reassigned them to the previously derived categorization and counted the occurrence of each category. Furthermore, occurrences of independent categories were also summed up over the scenarios.

Concept. We assigned 24 answers to the concept category, which deals with the device being attached to walls. This property has been mostly commented on in the kitchen and office scenarios. One answer stated that the "contents of the [kitchen] drawer may be changed without having to tell someone" (P96) and another one wrote "it's nice to have additional screen space as needed and helpful that the screen aligns itself with the one already there." (P86) or simply that the system is "cool and universally applicable" (P97). Whereas others mentioned "the device seems to be limited to a 2d-region. If the procedure extends beyond this region (e.g., throwing the coffee filter into the wastebin at the door), the display is not usable adequately" (P40), and "is it able to move around corners or does it have to be detached?" (P97).

Usefulness. In total 82 answers directly addressed the usefulness of the device. It "fulfills its tasks" (P59), gives a "simple and lucid explanation, useful for complex tasks" (P90), and "might help some physically limited people" (P73). Likely due to the simplistic scenarios some users asked "what is its purpose?" (P76), and regarded the device a "gimmick" (P48).

Attention. Another set of 35 answers addressed the attention of users. The system was described to have a "good entertainment value" (P72) and it "attracts attention to itself" (P90). Although some of them noted that the "show-off & wow effect (works probably just once)" (P73). In the museum scenario there were concerns that "the device may attract negative attention from the viewer or disturbs him from viewing artworks since movement of the device automatically attracts attention to itself" (P100), as stated by one answer.

Alternatives. Sixty eight answers compared the prototype with various alternative products including mobile phone applications ("better [realized] with indoornavigation app" (P105)), camera projection systems ("[I] think projector+software are clearly better suited for this [kitchen scenario]." (P14)), or head mounted displays ("I'd rather consider this [museum] a scenario for Google Glass or CastAR." (P16)). These approaches, however, have additional drawbacks that need to be taken into account as well (e.g., stationary setup for projection, challenges in augmented reality). Others mentioned advantages like (e.g. ""pointing'' [at something] is easier than using static displays." (P79)). This was especially the case in the museum scenario where there is "no need for one's own tablet/smartphone" (P36), and it also "replaces guide, [and has] individuality" (P42).

Complexity. In total 39 answers dealt with the influence of task complexity in the scenarios. The devices are a "very nice possibility to maintain eye contact with many participants at video conferences" (P10), "It is quite easy for students, they can write the formula and the machine draws for them" (P98), and "managing new situations is definitively simplified" (P87).We chose simple tasks that were easy to implement, like drawing simple functions. On such tasks there is a risk that it "complexes a simple process" (P16). Also when preparing coffee one participant found that it "looks way too inconvenient for something such simple to me" (P17).

Multiple users. Multi-user support has been identified as another category which was mentioned in 30 answers. In the office scenario "the number of participants involved in the conversation may be easily varied" (P87) using multiple devices. In public spaces like a museum where many people act in a quite limited space questions regarding the use of multiple displays that need to be addressed arose. For example: "What happens when multiple people are looking at the same piece of art or passing each other?" (P86) and "how should probably dozens of displays be controlled and be distinguished from each other?" (P45). Technical characteristics. We categorized 102 comments into technical characteristics which describe particular aspects of the prototype. Our prototype is driven by two gear motors with relatively high gear transmission ratios, so "the device is slow and noisy" (P3). Especially one participant said "it is too noisy to hear that this machine follows you. I need quiet and silence to visit the art exhibition." (P98) In contrast, in the office the device was "quiet-running" (P59).

In the kitchen one answer remarked "it is good that I don't need to ask anyone or spend more time to find out" (P98) how to prepare coffee, whereas "the teacher probably would have manually drawn the curve faster" (P87) in the classroom.

Asthetics were also subject to potential improvement in future versions, especially in the museum scenario the "metal driving plates around the exhibits are little aesthetic" (P35).

Scenario specific categories. Besides the independent categories described above, we identified some categories that applied to specific scenarios only:

─ kitchen: Nine answers dealt with the social aspect of preparing coffee. As with other technology that was introduced before (e.g. smart phones), some participants saw the risk that "communication between colleagues is lost" (P64). ─ classroom: We identified two more categories with sixteen participants commenting on the precision and seven on the didactic meaning. The prototype used for the video clearly lacks precision and is "quite scrawly, [and] surely problematic for more complex functions" (P14), but future versions may as well produce a "potentially better/more exact drawing of functions than quickly sketched hand drawings" (P57) can. Thus some answers were skeptic about "what is the didactic meaning and learning success?" (P65). ─ museum: Five answers saw the device as a "personalized guide" (P52) that is "more personal than fixed video installations" (P14). Additionally three answers emphasized the self-actuated aspect, for example "further multimedia information [is available] at any time without having me to carry something with me" (P61). ─ office: Dynamically scaling the display with multiple devices was mentioned in ten answers. One participant found "it's nice to have additional screen space as needed and helpful that the screen aligns itself with the one already there" (P86), and another wrote "having multiple displays merge into a single large one is fantastic" (P3). Six participants were not sure "where did the second display come from?" (P14) since the scenario looked somewhat constructed with only a whiteboard and two devices.

Discussion

We evaluated four scenarios that have been derived from the application ideas that resulted from the focus groups. For each scenario, we collected qualitative comments as well as the scenarios' pragmatic and hedonic qualities. Participants' quantitative assessment was similar for the four scenarios. Despite the limitation of the concrete prototype used, the overall reaction is positive in terms of hedonic and pragmatic qualities with a tendency towards being desired.

Comments regarding the scenarios' usefulness were mainly positive, describing specific use cases. Some participants, however, also wondered about the additional value the device could provide. Accordingly, participants compared the device with existing devices that can support similar tasks. In this sense the self-actuated display is similar to devices that fill a position complementary to existing devices, such as tablets that fill a position between large static displays and small mobile smart phones. Participants also highlighted unique aspects of self-actuated displays and that in certain situation they could replace static but also mobile displays.

Participants widely addressed the technical characteristics of the concrete prototype. Comments suggest that self-actuated displays must be fast enough to draw or to follow a walking user. Furthermore, participants criticized the noise level. These technical limitations can be tackled by using more powerful gear motors with reduced noise emission. Concerns about the margin of the display's border could be addressed by further developments in display technologies that reduce the frame thickness. This would allow seamless display connections with several devices.

Challenge might emerge if a large number of self-actuated devices are used at the same time. On the one hand, devices might interfere with each other and on the other hand it might become difficult for a user to identify his or her devices.

Participants appreciated the general concept of a device that is attached to and can move on walls. They envisioned a general purpose device that can provide additional screen space. A limitation of the current prototype is the restriction to a single 2D surface. For the device to be more general purpose, the device must be able to change surfaces itself.

It was appreciated that the device attracts attention. Participants partially assigned this to the device's novelty but also to its ability to move into the users' field-of-view. While this can be seen as an advantage it can also distract users from other tasks or content.

Participants discussed the potential benefit of the developed scenarios. They agreed on using the device could reduce the complexity of new tasks through being a support, for instance by providing in-situ information (assistance when acting in an unknown environment, e.g. kitchen scenario or museum scenario).

Conclusion

In this paper, we explored the space of applications for self-actuated displays. Assuming that self-actuated devices are a third class of devices that fill a space between mobile and static devices we developed the concept for a novel type of self-actuated display device. We implemented this concept through a prototype that is able to autonomously move vertically on ferromagnetic surfaces. Based on the results of a series of focus groups we derived a categorization for applications of self-actuated displays. To further explore this space we derived four application scenarios and implemented them as video prototypes showing interactive self-actuated displays in four application domains. Evaluating the video prototypes revealed that participants see advantages but also limitations of self-actuated displays. In particular, it is important that self-actuated devices are quiet and sufficiently fast to follow or guide a moving user. If this is the case, the device's physical position and movement provides a way to attract users' attention and can also encode information.

In this work, we used a particular self-actuated display to explore use-cases and to further explore them through concrete scenarios. Therefore, we are interested in extending the work through the use of other self-actuated devices [START_REF] Schneegass | Midair displays: Concept and first experiences with free-floating pervasive displays[END_REF][START_REF] Seifert | Hover pad: Interacting with autonomous and selfactuated displays in space[END_REF], [START_REF] Tominaga | Around me: A system with an escort robot providing a sports player's self-images[END_REF]. On a technical level we are interested in approaches that extend the mobility of self-actuated displays for vertical surfaces. In particular, ferromagnetic wall paint could be used to make existing surfaces accessible for the current prototype. Further options are adding moveable suction cups and adhesive pads that could either be used to get over non-ferromagnetic spaces or to enable free movement on arbitrary surfaces.

Fig. 3 .

 3 Fig. 3. A self-actuated display prototype without any extensions attached (left) and a bottom view of another prototype with pen and eraser holder attached (right). Both extensions are driven by servo motors.

https://www.suitabletech.com/

http://www.pololu.com/product/975

http://www.pololu.com/product/978

3D-models to reproduce the robot are available at: https://github.com/patrigg/WallDisplay

http://www.google.com/nexus/7/

http://www.asus.com/Multimedia/Xtion_PRO_LIVE/

Acknowledgements. This work was supported by the graduate program Digital Media of the Universities of Stuttgart and Tübingen, and the Stuttgart Media University.


Diana Löffler
email: diana.loeffler@uni-wuerzburg.de

Wolfgang Paier
email: wolfgang.paier@hhi.fraunhofer.de

Takashi Toriizuka
email: toriiduka.takashi@nihon-u.ac.jp

Mio Ikeda
email: mio.ikeda1220@gmail.com

Jörn Hurtienne
email: joern.hurtienne@uni-wuerzburg.de

Augmented Happiness: Simple Color Changes Influence Users' Conceptual Choices

Keywords: Augmented Reality, Color, Embodied Cognition, Conceptual Metaphor

Color is a powerful visual property and is used to make interferences about the world. However, no theoretical framework is available that explains precisely where color associations come from and how they affect psychological functioning, making it difficult to predict how color affects human-computer interaction. This paper aims at closing this gap by suggesting an Embodied Cognition view on color, which assumes that the aggregate of our perceptual color experiences is part of the mental representation of tactile object attributes and thus systematically influences our abstract thinking via the process of metaphorical mapping. An empirical study is presented in which hues and saturation of objects were manipulated via Augmented Reality. Participants matched objects to abstract concepts, e.g., happiness. 83% of the participants' choices were correctly predicted, suggesting that color information is considered during the processing of abstract information. Eight color-to-abstractconcept mappings are recommended and possible areas of application are discussed.

Introduction

Color is a ubiquitous perceptual experience that influences perception, cognition and behavior. However, color research mainly focuses on establishing relations between color and psychological functioning and often lacks a rigorous theoretical understanding of why these relations occur [START_REF] Elliot | Color and psychological functioning[END_REF]. This might be a main reason why findings on the impact of color on psychological functioning are largely inconsistent, unreliable and inconclusive [e.g., 22; 13]. In the field of human-computer interaction (HCI), design recommendations on color are usually restricted to issues of readability, aesthetics or color preferences [START_REF] Bonnardel | The impact of colour on Website appeal and users' cognitive processes[END_REF]. This paper aims to advance our theoretical understanding of how and why color affects psychological functioning to leverage a systematic applica-tion of color in the design of human-computer interaction. To achieve this, the framework of embodied cognition, which holds the promise of providing a unifying perspective for psychological research in general [START_REF] Glenberg | Embodiment as a unifying perspective for psychology[END_REF], is briefly reviewed and then applied to the field of color.

In recent years the predominant views of the human mind, such as the computational theories of information processing, have been increasingly challenged by findings that cognition is embodied (cf. [START_REF] Clark | Being There: Putting Brain, Body and World Together Again[END_REF]). Embodied cognition grants the body a central role in cognitive processing. Physical experiences we have, moving through the material world, scaffold the development of our conceptual knowledge. Many of such embodied experiences are manifested in language [START_REF] Lakoff | Metaphors We Live By[END_REF]. Expressions such as "the heavy burden of guilt" or "that warmed my spirits" reflect how the mental representations of guilt and happiness are metaphorically conceptualized. These conceptual metaphors involve a mapping from a physical source domain (weight, temperature) to another, more abstract, target domain (guilt, happiness) [START_REF] Lakoff | Metaphors We Live By[END_REF]. Being equipped with conceptual metaphors in our embodied minds, even seemingly unrelated visual or haptic properties of objects or technical devices influence our perception, feeling, and behavior (cf. [START_REF] Hurtienne | Sad is Heavy and Happy is Light -Population Stereotypes of Tangible Object Attributes[END_REF]), in a way predicted by the framework of embodied cognition.

An Embodied Cognition View on Color

Color is an important perceptual feature and in natural environments often diagnostic for other perceptual properties [START_REF] Cant | Independent processing of form, color and texture in object perception[END_REF], like edibility and taste [START_REF] Shankar | Grape expectations: the role of cognitive influences in color-flavor interactions[END_REF], weight [21; 1] or temperature [START_REF] Ho | Colourtemperature correspondences: When reactions to thermal stimuli are influenced by colour[END_REF]. Thus, it is likely that color affects psychological functioning in a similar way than related perceptual properties via the process of metaphorical mapping [16; 20].

Many studies already investigated associations between color and other perceptual attributes like weight, size and temperature. For example, according to [START_REF] Cheng | Quantitative evaluation of colour emotions[END_REF], temperature attributes are solely related to hue (red is warm, blue is cold). Increasing weight is related to decreasing brightness and an increase in saturation [START_REF] Alexander | Influence of hue, value, and chroma on the perceived heaviness of colors[END_REF], while when the size of a stimulus is increased, it is perceived as brighter and more saturated [START_REF] Xiao | Colour Appearance for Dissimilar Sizes[END_REF].

In recent years, a few studies have been conducted to empirically test these associations between colors and perceptual properties. For example, in an HCI relevant application, Ban and colleagues [START_REF] Ban | Augmented Endurance: Controlling Fatigue while Handling Objects by Affecting Weight Perception using Augmented Reality[END_REF] proposed an Augmented Reality (AR) system to manipulate weight perception through an increase in object brightness. They demonstrated that this facilitated the lifting of medium-weight objects. However, as with most of the research on relations between color and psychological functioning [START_REF] Elliot | Color and psychological functioning[END_REF], no theoretical background is provided on how and why this association is shaped, making it difficult to transfer the results into other situations and contexts. An embodied cognition view on color may close this theoretical gap, as it explains that the association between weight and brightness arises through the repeated experience that heavier objects appear to be darker than light objects (e.g., when a shirt gets wet it will be heavier and darker, [START_REF] Walker | The brightness-weight illusion: darker objects look heavier but feel lighter[END_REF]). Thus, the perception of brightness and weight mutually influence each other. In addition, this association between brightness and weight might be extended to more abstract concepts via metaphorical mapping [17; 20]. For example, weight is metaphorically linked to psychological significance, or, in other words, weight is conceptualized as an embodiment of importance [START_REF] Jostmann | Weight as an embodiment of importance[END_REF]. Therefore, it can be assumed that the perception of the color property brightness will bias information processing in contexts where psychological significance plays a role. This prediction can now be put to direct empirical test.

In this paper, we aim to experimentally test the embodied cognition view on color and investigate whether perceiving certain color characteristics will influence the cognitive processing of more abstract information (like happiness, emotionality, or power), which is metaphorically linked to the tactile properties weight, size and temperature.

Method

Participants

Fourteen volunteers were recruited at the campus of Nihon University, Japan, (seven male, seven female) to participate in this study. The participants' ages ranged from 21 to 24 years (M = 22.7, SD = 0.8). All participants had Japanese as their mother tongue and had no reported defective color vision. They were all naïve as to the subject under investigation.

Procedure

The experimental procedure was adopted from a study by Hurtienne et al. [START_REF] Hurtienne | Sad is Heavy and Happy is Light -Population Stereotypes of Tangible Object Attributes[END_REF]. Each participant was asked to complete a short demographic questionnaire. The participants were then told that they would wear AR glasses and, in each trial, would be presented with two colored objects (spheres) and an adjective describing an abstract property (e.g. difficulty). AR technology was utilized because it allows for a convenient manipulation of object colors in the viewers' sight.

The participants were instructed to touch the objects with their dominant hand that they felt corresponded most closely to the adjective (Figure 1). Participants were told to decide quickly and instinctively in favor of one object. Each session consisted of 30 trials. To avoid sequence effects, the adjectives and colored objects were presented in random order. The whole experiment lasted about 15 minutes.

Apparatus and material

Each of the two wooden spheres was 9 cm in diameter and weighed 250 g. They were colored in blue and green (controlled for saturation and brightness) in order to facilitate technical object recognition against the background and against the participants' hands and arms. These spheres were the target objects for the color changes, which did perfectly obscure the spheres. The spheres' position was counterbalanced across participants. The participants were seated in an experimental room; the table and wall in front of them were covered in gray cloth. The spheres were placed on the table within the reach of the participants, with 15 cm distance between them. Fig. 1. A participant wearing AR-glasses that change the object colors in the participants' sight.

The AR system used in the study was composed of a video see-through realizing Head Mounted Display VUZIX Wrap1200AR (resolution 852 x 480, 35 degrees diagonal field of view) a notebook (Intel Core i7 4x2.4 GHz 8.00 GB RAM, Windows 8.1 64 bit). The color manipulation software was based on OpenCV, which provides convenient interfaces for processing and capturing images from the stereowebcam integrated in the AR glasses. Each pixel was classified in background, green sphere or blue sphere, using a histogram-based color-classifier. The classifier was trained beforehand using six images with proper fore/background masks to fill the color-histograms for each pixel-class. The masks were generated using a semiautomatic approach for foreground extraction [START_REF] Rother | GrabCut": interactive foreground extraction using iterated graph cuts[END_REF]. To guarantee the realtimecapability of the system, a static look-up table, containing the color-transform (RGBin to RGBout) for each camera was calculated after the color classifiers were trained.

Three HCI-relevant tactile properties with well-documented metaphorical extensions were selected for the experiment: weight (heavy-light), size (big-small) and temperature (warm-cold) [START_REF] Hurtienne | Sad is Heavy and Happy is Light -Population Stereotypes of Tangible Object Attributes[END_REF]. A pre-study was conducted to determine the relationship between these tactile properties and color characteristics (hue, saturation and brightness). In an online questionnaire, 80 Japanese subjects rated how well 35 different colors (varying in hue, saturation and brightness) matched the properties weight, size and temperature. The results showed that weight and size were mainly associated with the color properties of saturation and brightness, and the temperature is almost solely associated with hue (red vs. blue). These results confirmed those of previous research [5; 23]. The tactile properties of weight, size and temperature were therefore operationalized through their associated color properties in the following way (numbers depict HSB values): heavy, big vs. light, small: violet (270/100/100) vs. orange (20/100/100). Note that, although both, saturation and brightness, influence the perception of weight, size and strength, the brightness of the objects in the experiment was not altered. This would have led to a loss in texture and surface shading, thereby negatively affecting the three-dimensional impression of the objects. Hues in the weight and size condition were randomly chosen to create some variation and should, according to theory, have no effect, because hue is not diagnostic for these tactile properties.

For each condition, well-documented metaphorically linked abstract concepts were taken from literature [12; 17] (Table 1). Two lists of adjectives were prepared. One word from each adjective pair was placed on list A (e.g., important), and its opposite was placed on list B (e.g., unimportant). Both groups were balanced with regard to the number of positive and negative words they contained. Half of the participants were presented with adjectives from list A, and the other half with adjectives from list B. The hypothesis for this experiment was that since specific color characteristics are diagnostic for certain tactile properties, viewing these color characteristics will trigger choices consistent with metaphorically related abstract concepts. If the color characteristics are not associated with the abstract concepts, participants will choose one of the two objects by chance.

Results and Discussion

The results are depicted in Table 1. The columns show the percentage of answers that were consistent with the metaphorically related concept (averaged over all color samples) and Cohen's kappa values are used as an index for associative strength (str) that takes into account the agreement occurring by chance (50%). According to [START_REF] Landis | The Measurement of Observer Agreement for Categorical Data[END_REF]'s categories of agreement, negative str values indicate "poor" agreement, 0 < str ≤ 0.19 indicate "slight" agreement, 0.20 ≤ str ≤ 0.39 "fair" agreement, 0.40 ≤ str ≤ 0.59 "moderate" agreement, 0.60 ≤ str ≤ 0.79 "substantial" agreement, and str ≥ 0.80 "almost perfect" agreement. Because of the limited and non-representative sample size we only interpreted strength values of above 0.60 as supportive of the color-toabstract-concept mappings (8 of 12). However, across all 12 color-to-abstract-concept mappings, the mean value of str was 0.67 (SD = 0.26), suggesting that at least 83% of the participants' choices were consistent with the conceptual metaphor. This value differs significantly from a chance str of zero, t(11) = 8.94, p < .001, Cohen's d = 3.64, and indicates non-randomness in the choices of the participants.

Overall, the results are interpreted in favor of an embodied cognition view on color: 8 out of 12 evaluated abstract concepts showed on average "almost perfect" agreement, while the remaining four concepts showed on average "fair" agreement [START_REF] Landis | The Measurement of Observer Agreement for Categorical Data[END_REF]. These results were predicted by associations between color attributes and tactile properties: because heavier objects tend to be darker and less saturated [21; 1], darker and less saturated hues are assumed to bias our thinking about metaphorically related abstract concepts, like psychological significance, because weight is an embodiment of psychological significance. The link between brightness, saturation and size can be explained by the bodily constraint of a nonuniform distribution of photoreceptors across the retina, resulting in different color vision in the periphery compared to vision in the fovea [START_REF] Xiao | Investigation of colour size effect for colour appearance assessment[END_REF]. The association between physical temperature and hue can be explained by the internalization of correlations between stimuli that are present in the environment (like fire, sun, water) and our bodies (blood circulation) [START_REF] Ho | Combining colour and temperature: A blue object is more likely to be judged as warm than a red object[END_REF]. As predicted, multiple McNemar tests (Bonferroni corrected) revealed no difference between the different hues tested across the weight and size conditions.

Conclusion and Future Work

With the help of the AR system it could be demonstrated that simple color changes in the users' sight resulted in a bias in decision-making in favor of metaphorically related abstract concepts. This is explained by an embodied cognition view of cognition on colors: because abstract concepts are grounded in observables in the world, we automatically consider perceived physical properties, such as color and tactile properties, during reasoning about these abstract concepts. The value of this embodied cognition view on colors lies in its ability to explain and predict why and which effects of color on psychological functioning occur, taking the guesswork out of color theory and design.

Moreover, this research has shown that not only hue but also saturation plays a great role in conveying abstract content [START_REF] Elliot | Color and psychological functioning[END_REF]. Three abstract concepts were identified that can be influenced by a change in saturation: difficulty, importance, significance and quantity. Three concepts were influenced by hue: emotionality, activity and happiness. The concept of difficulty was influenced by both saturation and hue. We expect that the effects would have been even stronger had the brightness parameter been manipulated as well. Although selected well above chance level, four color-toabstract mappings were weaker than expected and require further investigation.

By extending this approach to applied contexts, the validity and effectiveness of predictions derived from an embodied cognition view on colors is an area for future research. Changes in saturation, for example, could be used to indicate the most significant, or most-often used objects or functions (significance is saturation); they could be used to express priority among different items, such as the most valued menu option in a restaurant, or the best selling products in an online shop (importance is saturation). Reddish vs. bluish hues could be used to communicate or visualize subtle mood changes in social networks, online games or in real life situations (happiness is redness). Changing font or background color could influence the perceived difficulty of a computerized cognitive training (difficulty is saturation) -and this should also work for people with cognitive impairment, since embodied cognition effects are assumed to be rather independent of higher cognitive processing [START_REF] Hurtienne | Facets of prior experience and the effectiveness of inclusive design[END_REF].

Although the results of this pilot study are promising, the sample size of the study reported here is very small (n=14). Larger scale empirical studies have yet to be conducted in order to validate the results obtained here, also with regard to different cultures. Since color and also, to some extent, conceptual metaphors are subject to cultural influences, studies on the applicability of the results within and across countries and cultures are required.

  (270/60/100), green (140/100/100) vs. (140/60/100), red (360/100/100) vs. (360/60/100), yellow (60/100/100) vs. (60/60/100), orange (40/100/100) vs. (40/60/100); cold: blue (200/100/100), cyan (180/100/100); warm: red (360/100/100),

Table 1 .

 1 Tactile properties, conceptual metaphors, stimulus adjectives and results of color-toabstract-concept mappings; all mappings with str < 0.60 have been grayed.

	Tactile Property	Conceptual metaphor with linguistic examples	Stimulus adjec-tives	Related erty color prop-	%	str
		Difficulty: heavy reading; light duties	difficult, easy	Saturation	93	0.86
		Importance: heavy matters of state; light, idle chatter	important, unim-portant	Saturation	91	0.82
	Weight is..	Guilt: the heavy burden of guilt; easing conscience Sadness: heavy news; light-hearted girls	guilty, innocent sad, happy	Saturation Saturation	61 61	0.21 0.21
		Significance: he is a big man in industry; only a small crime	significant, insig-nificant	Saturation	96	0.93
		Quantity: I only have a small amount of money	much, less	Saturation	95	0.90
	Size is..	Education: he expanded his horizons	educated, unedu-cated	Saturation	75	0.50
		Activity: a hot debate; frozen accounts	active, inactive	Hue	93	0.86
		Emotionality: a warm smile; cold logic	emotional, unemo-tional	Hue	93	0.86
	Temperature is..	Difficulty: a smoldering con-flict; the debate cooled down Happiness: that warmed my spirits; a deep cold sadness Intimacy: giving the cold shoulder; warm feelings	problematic, un-problematic happy, sad intimate, distant	Hue Hue Hue	86 82 75	0.71 0.64 0.50

Acknowledgements. We thank the Japan Society for the Promotion of Science for funding this research (grant ID PE14746).


Miriam Greis

Thorsten Ohler

Niels Henze

Albrecht Schmidt

Investigating Representation Alternatives for Communicating Uncertainty to Non-Experts

Keywords: . Uncertainty, Representations, Visualizations, Non-Experts 1

Non-experts are confronted with uncertainty of predictions everyday when, e.g., using a navigation device or looking at the weather forecast. However, there are no standards for representing uncertain information and representations could be easily misleading. Thus, we selected twelve representations that provide different levels of uncertainty information. We compared the representations in an online survey with 90 participants where we asked participants to judge their support in decision-making, familiarity, easiness to understand, and visual appeal. We further evaluated the four most promising representations in a turn-based online game. Players had to make decisions in a farming scenario based on a displayed weather forecast. The results of the survey and the game indicate that a function graph of a probability distribution function is the best way to communicate uncertain information. Nevertheless, our results also show that presenting more uncertainty information does not necessarily lead to better decisions.

Introduction

Simulation is a very powerful technique used in different fields to explore the behavior of complex systems as, e.g., the flow of ground water, the human walking behavior, and the world's climate. Due to its applicability to many problems, simulation is one of the most used techniques in research and management sciences [START_REF] Law | Simulation modeling and analysis[END_REF].

The results of simulations are uncertain due to, e.g., assumptions made in the modeling process or the parameter choice. Non-experts, who have no specialized knowledge about simulations, are confronted with simulation results and uncertain data everyday, e.g., when looking at the weather forecast that shows the possible temperature for the next day. They rely on uncertain data to implicitly predict the future, plan activities, and make decisions. These decisions could be even unintentionally manipulated by choosing a specific representation.

Although several aspects of the communication and visualization of uncertain data were examined before, still work has to be done to get more insights in how different representations influence decision-making. From existing work, we selected twelve representations (three textual and nine graphical representations) used for uncertainty communication for experts and non-experts. This leads to a wide range of used representations with different degrees of included uncertainty information.

First, we present the results of an online survey where participants had to rate the representations according to the perceived help for decision support in a farming scenario, familiarity, easiness to understand, and visual appeal. We found significant differences in the rating of the representations and also correlations between ratings. Surprisingly, we did not find any strong correlations between subjective ratings and the degree of uncertainty information included in a representation.

Second, we present the results of a small experiment with a turn-based online game. The four representations that performed best in the online survey were used to display a weather forecast. Decisions of players were analyzed on the basis of optimal decisions. The function graph of a probability distribution function performed best, but the representation with no information about the included uncertainty did not perform dramatically worse.

The contribution of our work is twofold. First, we show that people do not judge representations for the degree of uncertainty information they show. Other factors are more important for their judgment. Second, we show that a higher degree of uncertainty information does not necessarily lead to better decisions and that people are able to make good decisions with the help of a probability density function.

Related Work

The topic of uncertainty visualization is well explored for experts. Multiple areas such as vector field, surface and glyph visualizations are for example explored by Pang et al. [START_REF] Pang | Approaches to Uncertainty Visualization[END_REF] and Zuk and Carpendale [START_REF] Zuk | Theoretical analysis of uncertainty visualizations[END_REF]. Additionally, new versions of basic representations (e.g., box plots) are for example developed by Potter et al. [START_REF] Potter | Visualizing Summary Statistics and Uncertainty[END_REF]. In our paper, we do not take into account such visualizations and focus on basic representations developed for experts and non-experts.

For communicating uncertainty to non-experts, usually quantitative information, especially probabilities are used. One problem when using quantitative information is the inability of even well-educated adults to solve easy numeracy probability questions [START_REF] Lipkus | General performance on a numeracy scale among highly educated samples[END_REF]. But also qualitative information, e.g., labels such as low uncertainty or low risk, could be misleading, as already examined by Wallsten et al. [START_REF] Wallsten | Measuring the vague meanings of probability terms[END_REF]. Additionally, the formulation of a risk or uncertainty, whether negative or positive, has a huge influence on decision-making [START_REF] Mcneil | On the elicitation of preferences for alternative therapies[END_REF].

One strand of work investigated uncertainty information in weather forecasts. Morss et al. [START_REF] Morss | Communicating uncertainty in weather forecasts: A survey of the US public[END_REF] found that most people are aware of the uncertainty in deterministic weather forecasts, although the range of this uncertainty was perceived very differently. Additionally, 70 % of the people preferred forecasts that contained information about the uncertainty of the forecast. Studies by Roulston et al. [START_REF] Roulston | A laboratory study of the benefits of including uncertainty information in weather forecasts[END_REF] and Joslyn et al. [START_REF] Joslyn | Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error[END_REF] showed that people make better decisions when having information about the uncertainty of a forecast and that information about the uncertainty also increases the trust in a forecast. They used a small number of alternative representations in a decision task.

Another strand of work investigated and compared visualizations including uncertainty information. Ibrekk et al. [START_REF] Ibrekk | Graphical communication of uncertain quantities to nontechnical people[END_REF] compared nine visualizations for uncertainty by giving non-experts specific tasks (e.g., finding the mean). They suggest displaying a normal probability distribution function together with a cumulative probability distribution function. Pappenberger et al. [START_REF] Pappenberger | Visualizing probabilistic flood forecast information: expert preferences and perceptions of best practice in uncertainty communication[END_REF] made a study with experts in meteorology and asked them about their preferred representation for a probabilistic forecast. The most used representation were quantiles.

Additional studies focus on one specific visualization or aspect. Olston et al. [START_REF] Olston | Visualizing data with bounded uncertainty[END_REF] examined visualizations for presenting bounded uncertainty by adjusting the visual elements and including transparency in the visualizations and Tak et al. [START_REF] Tak | The perception of visual uncertainty representation by non-experts[END_REF] used seven variations of a line graph to investigate the perceived certainty. Correll and Gleicher [START_REF] Correll | Error bars considered harmful: Exploring alternate encodings for mean and error[END_REF] proposed a redesign of bar charts and found that less well-known visualizations improve performance for interferential tasks.

Previous work mainly focused either on a small number or representations, variations of one representation, or a very specific task (e.g., finding the mean). They found that uncertainty information leads to better decisions, but it is not clear how this conclusion relates to different degrees of uncertainty information and how aggregated uncertainty information influence decision-making. In contrast to previous work, we decided to take a wide range of basic representations and concentrate our research on the degree of uncertainty information that is included in these representations.

Online Survey

To understand if the presented degree of uncertainty changes the perceived value of a representation for decision support and the easiness to understand the representation, we conducted an online survey. Building upon prior research, we selected twelve representations (see Figure 1) with different properties for communicating uncertainty information. All representations show the expected rainfall for the next three days. Three representations use a textual representation of the information, whilst the other nine representations are graphical. We use a line chart, a box-and-whisker plot, bar charts, stacked bar charts, stacked area diagrams, shaded bars, and function graphs. The representations also communicate different degrees of uncertainty information, from no information about uncertainty at all up to detailed information. The displayed degree of uncertainty information for each representation is depicted in Table 1.

Questionnaire

On the first page of the questionnaire, we asked participants for demographic information: age, gender, their highest degree, and their field of work.

We then displayed a scenario description that told participants to imagine that they are a farmer and want to grow plants. The plants need a certain amount of water to grow and survive. A weather forecast will be available to support participants in making a decision, but it will be uncertain.

Participants then had to navigate through twelve pages with one representation on each page. The order of the representations was randomized across participants. For each representation, participants had to indicate their level of agreement on a fivelevel Likert scale from totally disagree to totally agree with four statements:

1. The representation supports me in making a decision.

Participants

We invited participants through social networks to participate in our survey. In total, 90 participants (36 female, 54 male) fully answered our online survey. Participants' age ranged from 18 to 82 years with a mean of 31 years (sd: 12.6). 45 % of the participants had a university degree, 28 % had a high school diploma, further 20 % finished a vocational training, and all other participants had a minor degree or no degree at all. Participants worked in different field such as computer science, economics, commerce, teaching, mechanics, services, and others. They had no specialized knowledge about simulations.

Results

For each representation, we calculated the mean for each of the four statements and the overall mean (see Table 2). For each statement, we also conducted a Friedman test and for the post hoc analysis Wilcoxon signed-rank tests with an applied Bonferroni correction. The Friedman test showed that there is a statistically significant difference between the twelve representations for each statement.

The Wilcoxon singed-rank tests showed that representation 1, 4, 7, and 11 performed significantly better than the majority of other representations in at least one judgment each. Representation 3 was rated significantly worse than the majority of representations on three scales.

We ran a Spearman's rank-order correlation to determine relationships between our 1080 Likert items and the degree of uncertainty information of the different representations. As expected, we found strong, positive correlations between all pairs of Likert items, which were all statistically significant (p < 0.0005), see Table 3 for detailed values. Surprisingly, we did not find any significant positive or negative correlation between the Likert items and the degree of uncertainty information of the representations, except one moderate positive correlation with the Likert items for visual appeal (see Table 3). We assume that this correlation occurred because we used textual representations with low degrees of uncertainty information.

Experiment

Based on the results of the online survey, we implemented a turn-based online game that displayed a weather forecast using the four representations that performed best in our online survey. With the help of the weather forecast for the next three days, players had to decide which crops they want to plant. Crops needed specific values for rainfall, wind, and sun that were always displayed and gave different amounts of money when fully grown and harvested. Players had the goal to get as much money as possible. If one weather condition for a planted crop was violated, it withered and it gave no money at all.

We conducted a controlled experiment with 12 participants (4 female, 8 male), recruited with the help of social networks, who had to play the game with each of the four representations. The order of the representations was changed for each participant and randomly assigned to them. Each game lasted for 10 rounds. Participants were not trained and did not have any specialized knowledge about simulations. For the analysis, we logged all relevant information, which included the forecasted weather, the real weather, and all crops that were planted.

Participants played in total 480 rounds of the game, 120 rounds with each representation. For our analysis, we only considered rounds in which the players at least clicked on the button to open the weather forecasts once. This resulted in 442 rounds.

For each round, we calculated the optimal decision based on the weather forecast displayed and compared it with players' decisions. With representation 11, participants made the most optimal decisions, in 69 % of the rounds, with representation 7 in 64 % of the rounds, with representation 1 in 60 % of the round and with representation 4 in 57 % of the rounds.

At the end, we asked participants which representation they liked most. 8 participants selected representation 11, three participants selected representation 7, 1 participant selected representation 4, and no participant selected representation 1.

Discussion & Conclusion

The results of our online survey show that the representations were judged very differently. Surprisingly, the four best-rated representations taking the overall mean show a different degree of uncertainty information each. Correlating the judgments and the degree of uncertainty information provided by the representations, we found that there are no significant correlations between the perceived support in decisionmaking and the degree of uncertainty information. Thus, we showed that participants do not judge the perceived support of decision making for a representation in regard to the degree of uncertainty information presented. In contrast, factors such as familiarity, easiness to understand, and visual appeal have a huge influence on the judgment. Our work indicates that other factors besides the presented degree of uncertainty information have to be considered when displaying uncertain data.

Comparing results from the online survey and the experiment, we found that the line chart with the highest rating in the survey performed worse than the other representations in the experiment. Although other studies showed that participants make better decision when having uncertainty information, our experiment suggests that aggregated uncertainty information do not provide enough details to make better decisions than only information about the expected values. We assume that the aggregated uncertainty information, the confidence interval, was not enough information to make a better decision but instead alienated participants. Nevertheless, most participants in our experiment preferred using the representation of the probability distribution function. Additionally, participants made very good decisions using this probability densi-ty function although earlier studies suggest that people are not very good at intuitively interpreting statistical information. This indicates that a probability function should be used to communicate uncertain data for non-experts and that too much aggregation of uncertainty information should be avoided.

Our experiment is clearly limited by the small number of participants and the scenario, but nevertheless shows interesting factors that are relevant for uncertainty communication. Future work on different scenarios with more participants will help to generalize the findings.

Fig. 1 .

 1 Fig. 1. All 12 representations compared in the online survey. Explanations: 1 -3 Textual representations, 4 -Line chart with area diagram, 5 -Box-and-whisker plot, 6 -7 Bar charts, 8 -Stacked bar charts, 9 -Area chart, 10 -Shaded horizontal bars, 11 -12 Function graph

Table 1 .

 1 Degree of uncertainty information included in the representations.

		Textual Representation	Graphical Representation
	No Uncertainty In-	REPR 1: Expected values	/
	formation
	Aggregated Uncer-	REPR 2: Expected values	REPR 4: Expected values
	tainty Information	and standard deviation	and confidence interval
		REPR 3: Quantiles	REPR 5: Quantiles
			REPR 6: Expected values
			and standard deviation
	Detailed Aggregated	/	REPR 7 -9: Aggregated
	Uncertainty Infor-		probability density function
	mation		REPR 10: Color-coded
			probability density function
	Detailed Uncertainty	/	REPR 11: Probability Densi-
	Information		ty Function
			REPR 12: Cumulative Prob-
			ability Density Function

Table 2 .

 2 Calculated mean values for the level of agreement on a five-level Likert scale from totally disagree (1) to totally agree[START_REF] Lipkus | General performance on a numeracy scale among highly educated samples[END_REF] with the statements: S1 -The representation supports me in making a decision., S2 -I am familiar with the representation., S3 -The representation is easy to understand., S4 -The representation is visually appealing., and Othe overall mean values for all 12 representations.

	REPR	S1	S2	S3	S4	O
	1: Expected values	3.63	4.29	4.28	2.30	3.63
	2: Expected values and standard deviation	3.68	3.90	3.34	2.13	3.26
	3: Quantiles	2.86	2.79	2.37	1.70	2.43
	4: Line chart with confidence interval	4.12	3.74	4.12	4.01	4.00
	5: Boxplot	3.22	2.73	2.52	2.48	2.74
	6: Bar chart with error bars	3.48	3.11	3.08	2.98	3.16
	7: Histograms as bar charts	3.93	4.21	3.67	3.67	3.87
	8: Histograms as stacked bar charts	3.47	3.49	3.16	3.52	3.41
	9: Histograms as area chart	3.16	2.84	2.74	3.42	3.04
	10: Shaded horizontal bars	3.73	2.31	3.59	3.59	3.31
	11: Probability distribution function	3.89	3.88	3.46	3.60	3.71
	12: Cum. probability distribution function	3.44	3.50	2.88	3.52	3.34

Table 3 .

 3 Spearmen's rho for a Spearman's rank-order correlation between Likert scale items of the online survey and the degree of uncertainty information of the presented representations.

	Statistically significant values are marked with asterisk(s). **p < 0.01, *p > 0.0005
		Decision	Familiarity Easiness to	Visual
		Support		Understand	Appeal
	Degree of Uncertainty	0.048	-0.040	-0.084**	0.365*
	Decision Support	-	0.505*	0.670*	0.527*
	Familiarity	-	-	0.609*	0.530*
	Easiness to Understand	-	-	-	0.530*

Acknowledgements. The authors would like to thank the German Research Foundation (DFG) for financial support of the project within the Cluster of Excellence in Simulation Technology (EXC 310/2) at the University of Stuttgart.