Tess Speelpenning

Alissa N Antle
email: aantle@sfu.ca

Tanja Doering
email: tanja.doering@uni-due.de

Elise Van Den Hoven

Exploring How Tangible Tools Enable Collaboration in a Multi-Touch Tabletop Game

Keywords: Tangible interaction, collaboration, CSCL, tabletop gaming, multitouch, Futura, interaction design

Digital tabletop surfaces afford multiple user interaction and collaboration. Hybrid tabletops that include both tangible and multi-touch elements are increasingly being deployed in public settings (e.g. Microsoft Surface, reacTable). Designers need to understand how the different characteristics of tangible and multi-touch interface elements affect collaborative activity on tabletops. In this paper, we report on a mixed methods exploratory study of a collaborative tabletop game about sustainable development. We explore the effects of tangible and multi-touch tools on collaborative activity. Forty-five participants, in trios, played the game using both versions of the tools. Our analysis includes quantitative performance measures, qualitative themes and behavioral measures. Findings suggest that both tangible and multi-touch tools enabled effective tool use and that collaborative activity was more influenced by group dynamics than tool modality. However, we observed that the physicality of the tangible tools facilitated individual ownership and announcement of tool use, which in turn supported group and tool awareness.

Introduction

Since the concept of graspable user interfaces was first defined by Fitzmaurice and colleagues [START_REF] Fitzmaurice | Bricks: Laying the Foundations for Graspable User Interfaces[END_REF] interactive user interfaces have developed in many areas and are appreciated because of their advantages over traditional mouse and keyboard based interaction. Interactive user interfaces can provide multi-touch and multiple user interaction and opportunities to weave the physical and digital world together in a seamless matter [START_REF] Ullmer | Emerging Frameworks for Tangible User Interfaces[END_REF]. Although there are many similarities, a multi-touch interface is distinct from a tangible user interface (TUI). Multi-touch interfaces enable "hands-on computing" [START_REF] Kirk | Putting the Physical into the Digital: Issues in Designing Hybrid Interactive Surfaces[END_REF], direct manipulation, joint actions and spatial visibility of movements [START_REF] Kirk | Putting the Physical into the Digital: Issues in Designing Hybrid Interactive Surfaces[END_REF]. In addition to the characteristics of multi-touch interfaces, TUIs may provide affordances that are more natural [START_REF] Jordà | The reacTable: Exploring the Synergy between Live Music Performance and Tabletop Tangible Interfaces[END_REF], enable manipulation in 3D space and may improve spatial cognition [START_REF] Maher | Do Tangible User Interfaces Impact Spatial Cognition in Collaborative Design?[END_REF]. These differences are also described as embodied facilitation [START_REF] Hornecker | A Design Theme for Tangible Interaction: Embodied Facilitation[END_REF].

Technologies such as SMART boards, plasma screens, and projected wall displays all have been studied in order to better understand how to design them to support collaborative interaction [START_REF] Rogers | Configuring Spaces and Surfaces to Support Collaborative Interactions[END_REF]. Several different tabletop systems have also been developed to support multi-user interaction that facilitates collaboration (e.g. Microsoft Surface, ReacTable). For designers it is important to understand how the characteristics of these different interface styles affect user experience. Some comparative studies have explored how multi-touch tabletops enable social interaction (e.g. [START_REF] Waldner | Tangible Tiles: Design and Evaluation of a Tangible User Interface in a Collaborative Tabletop Setup[END_REF][START_REF] Scott | Guest Editors' Introduction: Interacting with Digital Tabletops[END_REF][START_REF] Rekimoto | DataTiles: A Modular Platform for Mixed Physical and Graphical Interactions[END_REF]).

Several studies suggest that multi-touch tabletops stimulate more collaboration compared to traditional user interfaces (e.g. [START_REF] Rogers | Configuring Spaces and Surfaces to Support Collaborative Interactions[END_REF][START_REF] Scott | Guest Editors' Introduction: Interacting with Digital Tabletops[END_REF][START_REF] Price | The Effect of Representation Location on Interaction in a Tangible Learning Environment[END_REF][START_REF] Piper | SIDES: A Cooperative Tabletop Computer Game for Social Skills Development[END_REF]). In addition, tangible artifacts that are part of a user interface may invite people to interact more, because tangible objects lower the threshold for participation [START_REF] Hornecker | Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interactions[END_REF]. Thus, multi-touch interactive tabletops with additional tangible objects may provide new opportunities for socializing, collaborative learning and working. Nevertheless, as Hornecker and Buur [START_REF] Hornecker | Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interactions[END_REF] stated, the research field lacks an analysis and understanding of the aspects that support the social aspects of tangible interaction. A design framework that provides concepts or guidelines that help designers understand how to support collaboration related to multi-touch-tangible tabletops does not yet exist.

In this paper, we report on findings from an exploratory comparison of multitouch and tangible controllers for a visualization tool that were developed to augment a collaborative, sustainability learning game, called "Futura: the Sustainable Futures Game" (original game described in [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF]). We abbreviate the game title as "Futura" in the remainder of this paper. Our study addressed two research questions: "How does tool use differ between multi-touch and tangible interaction?" and "How does the user interface (UI) style of the tools affect collaboration?" Our goal was to understand how the characteristics of the tangible and the digital controllers for the visualization tools affected activity and collaboration in the context of a tabletop game.

The learning goal of Futura is to increase players" awareness of the complexity of sustainability issues. Previous work with Futura indicated that players had trouble linking individual actions to cumulative, global effects [START_REF] Hornecker | Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interactions[END_REF]. In addition, many of the groups played the game with a parallel play or individual style at the beginning of game sessions, and shifted to a more collaborative play style only after they played the game several times [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF]. Therefore, the goal for the visualization tools was to support players to understand cumulative effects and to enable an earlier shift from individual to collaborative play. We implemented visualization tools that addressed these needs and that could be controlled with either tangible or multi-touch objects. This enabled us to compare the tangible and multi-touch controllers of the visualization tools to determine how UI characteristics changed or enabled interaction and collaboration. For brevity we call the tangible and multi-touch controllers that are part of the visualization tools, the "tools". Compared to prior work, this exploratory study focused on a comparison of two versions of visualization tools, which were novel elements in the Futura game.

Prior research based on the Futura game has included two observational studies focusing on understanding how people learn [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF] and a survey-based case study that explores how Futura can be used to engage the public with issues of societal importance [START_REF] Antle | Balancing Act: Enabling Public Engagement with Sustainability Issues through a Multi-Touch Tabletop Collaborative Game[END_REF]. This paper presents a comparative study focusing on how the UI style of these tools affects collaboration. The study was conducted with a redesigned Futura application.

Related Work

Tabletop-based games have been in the focus of several case studies on interaction with tangible and multi-touch UI elements. Examples of studies of these kinds of hybrid tabletop gaming can be found in [START_REF] Magerkurth | Towards the Next Generation of Tabletop Gaming Experiences[END_REF], [START_REF] Dang | Hand Distinction for Multi-Touch Tabletop Interaction[END_REF] and [START_REF] Leitner | Physical Interfaces for Tabletop Games[END_REF]. Few studies explore how the different affordances of tangible or multi-touch interface elements affect interaction and collaboration. There are no existing studies that compare the effects of tangible and multi-touch interaction on collaborative game play activity. In this section, we will summarize some research that compares users" performance and preference using mouse, multi-touch and tangible user interfaces for a variety of application areas. In a user study in which participants had to implement warehouse layouts with either multi-touch or tangible shelves, Lucchi et al. [START_REF] Lucchi | An Emperical Evaluation of Touch and Tangible Interfaces for Tabletop Displays[END_REF] found that the completion time for the tangible group was faster or more efficient. The authors assumed that this was caused by users" familiarity with physical artifacts in the everyday world. In another user study where users had to do several matching tasks [START_REF] Tuddenham | Graspables Revisited: Multi-Touch vs. Tangible Input for Tabletop Displays in Acquisition and Manipulation Tasks[END_REF], the authors also found that tangible artifacts were easier and more accurate to manipulate. However, they emphasized that effects likely depend on the task, tangible objects and quality of technological implementation.

A user study with children solving jigsaw puzzles using three different UI styles (physical, TUI and GUI) showed no significant differences in children"s enjoyment. Observation data indicated that children"s engagement was higher for both physical and tangible conditions, compared to the graphical condition [START_REF] Xie | Are Tangibles More Fun? Comparing Children's Enjoyment and Engagement Using Physical, Graphical and Tangible User Interfaces[END_REF]. A follow-up study using video analysis of hand-actions indicated that there were significant differences in the frequency, duration and type of actions used to manipulate puzzle pieces between UI styles [START_REF] Antle | Exploring How Children Use their Hands to Think: An Embodied Interactional Analysis[END_REF] and that tangible and physical UIs facilitated many fast actions, and a more sophisticated puzzle solving strategy that evolved over time.

Wang [START_REF] Wang | Comparing Tangible and Multi-touch Interfaces for a Spatial Problem Solving Task[END_REF] performed a comparative study of a tangible and a multi-touch interface in which he investigated the effect of interface style on users" strategies and performance solving a jigsaw puzzle. Both multi-touch and tangible interaction enabled complementary actions (i.e. using physical actions on objects to reduce cognitive load) but interaction was more precise and efficient with the TUI version. The author suggests that these performance benefits may be a result of the tangible puzzle"s 3D manipulation space and tactile feedback.

In a study by Rogers and colleagues [START_REF] Rogers | Equal Opportunities: Do Shareable Interfaces Promote More Group Participation Than Single User Displays?[END_REF], the authors compared a mouse, multitouch and physical-digital condition for a collaborative design task. The mouse condition led to frustration because it resulted in unequal participation. However, it resulted in the most utterances. The physical-digital condition led to more equitable participation in terms of verbal contribution. The multi-touch condition had the most equitable participation in terms of physical actions. Based on their findings, we focus on physical as well as verbal interactions between the users.

Background Theory

In order to investigate how the characteristics of tangible and multi-touch tools affect collaboration, we first provide a working definition of collaboration. Roschelle and Teasley [START_REF] Roschelle | The Construction of Shared Knowledge in Collaborative Problem Solving[END_REF] (page 70) defined collaboration as "a coordinated, synchronous activity that is the result of a continued attempt to construct and maintain a shared conception of a problem". This activity results from individuals that share and negotiate meanings to solve a problem. In this model, collaboration is measured in terms of communication. Brehmer [START_REF] Brehmer | Distributed Decision Making: Some Notes on the Literature[END_REF] describes the relationship between collaboration and communication as follows: if the necessity for coordination and cooperation increases, the need for communication also increases. The theory of Computer Supported Collaborative Learning (CSCL) provides an important perspective on collaborative learning which applies to groups playing a learning game, like Futura. A review of the CSCL literature suggests three elements that are required to support collaborative learning: shared objects of negotiation [START_REF] Suthers | Representational Guidance for Collaborative Learning[END_REF], referential anchors [START_REF] Clark | Grounding in communication[END_REF] and metacognitive processes [START_REF] Duffy | Critical Thinking in a Distributed Environment: A Pedagogical Base for the Design of Conferencing Systems[END_REF]. In our study, we focus on shared objects of negotiation, because we are interested in characteristics of objects -tools --that may affect or improve collaboration. We also draw on empirical findings from the tangible interaction literature because designers need to understand how the different characteristics of tangible and multi-touch elements affect collaborative activities on tabletops. We next summarize the following main theoretical concepts from the CSCL and tangible interaction literature that we use to inform our analysis: objects of negotiation, access points, physical constraints and awareness. In our thematic analysis of observational data (below), we also search for emergent concepts.

Objects of Negotiation

Objects of negotiation are objects that externally represent information that is important in some way in the collaborative task [START_REF] Suthers | Representational Guidance for Collaborative Learning[END_REF]. An individual or group can modify this representation during the learning process. If an individual changes a shared representation, he may feel an obligation to announce, discuss or negotiate his action with the rest of the group in advance [START_REF] Suthers | Representational Guidance for Collaborative Learning[END_REF]. In Futura, players must collaborate and therefore communicate about various different external representations in order to win the game and through winning attain the game"s learning goal. The tools we developed enabled players to change the main external representation in the game, as described below in section 4.3.

3.2

Access Points Access points are physical or spatial interface characteristics that enable a user to join a group"s ongoing interactions with a system. They provide perceptual and manipulative access and enable fluidity of sharing [START_REF] Hornecker | Entry and Access -How Shareability Comes About[END_REF]. These characteristics may enable a user to interact, participate and join a group"s activity. The amount and type of input devices that are used in shareable interfaces are important factors that shape interaction and affect the type of social interaction. Manipulative access and fluidity of sharing are two elements that are amplified by tangibility. The fluidity of sharing describes how easily one user hands over control to someone else [START_REF] Hornecker | Entry and Access -How Shareability Comes About[END_REF]. One of the benefits of tangible tools that result from their physical and spatial qualities is that they may better enable access compared to similar multi-touch tools [START_REF] Hornecker | Entry and Access -How Shareability Comes About[END_REF].

3.3

Physical Constraints Hornecker [START_REF] Hornecker | Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interactions[END_REF] describes embodied facilitation as physical and perceptual affordances and spatial properties that shape and determine people"s interactions with a physical artifact. Some of the affordances of physical artifacts enable specific actions that are not possible or less obvious when using digital artifacts (e.g. a multi-touch tool). For example, physical or tangible tools can be manipulated in 3D space, whereas the use of multi-touch tools is constrained to 2D space. Because of this, a wider range of actions may be performed with tangible artifacts, and such actions are more visible to other users [START_REF] Wang | Comparing Tangible and Multi-touch Interfaces for a Spatial Problem Solving Task[END_REF].

3.4

Awareness Awareness is involved in creating a context for a person"s activities by understanding the activities of others [START_REF] Dourish | Awareness and Coordination in Shared Workspaces[END_REF]. Ideally, individual contributions are relevant to group activities in that they support the group to collaborate. Hornecker found that multitouch input resulted in more positive indicators of workspace awareness compared to mouse input in a collaborative task including an increase of helping behaviors and more handovers of objects [START_REF] Hornecker | Collaboration and Interference: Awareness with Mice or Touch Input[END_REF]. She suggested "being able to see another"s physical actions can enhance awareness, which in turn can support fluid interaction and coordination" [START_REF] Hornecker | Collaboration and Interference: Awareness with Mice or Touch Input[END_REF] (page 167). Fleck et al. suggest that "maintaining joint attention and awareness" is critical to coordinated, collaborative activity [START_REF] Fleck | Actions Speak Loudly with Words: Unpacking Collaboration Around the Table[END_REF]. They suggest that a physical artifact may better enable collaboration because of what we can do with it. For example, a physical artifact draws attention when a user picks it up from a tabletop and moves it in 3D space in front of their body.

4

The Futura Game and Visualization Tools

Overview

Futura (Figure 1) is a multiplayer, tabletop game that challenges players to make decisions that help preserve the environment, and at the same time meet the needs of a growing population. There are three roles to be played in the game: providing food, energy or shelter. Each role can access specific resources. Players can place resources on the world map to support the growing population, but they have to make sure that the resources do not damage the environment. The goal of the game is to balance supporting the population and damaging the environment in a simulated world. The game spans many years and the first version of the game lasts about three minutes.

For more details about the original game, see [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF][START_REF] Antle | Balancing Act: Enabling Public Engagement with Sustainability Issues through a Multi-Touch Tabletop Collaborative Game[END_REF]. For a video overview of the Futura project, see http://www.antle.iat.sfu.ca/Futura.

Design Goals: The New Version of Futura

Players can only win the game when they collaborate since all three roles must work together. Coordination and communication are necessary. Based on results of a previous study we wanted to better enable player understanding of the cumulative impact of actions on the world state, and to support collaboration based on this information [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF]. We augmented the original version of Futura (described in [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF]) with two visualization tools.

Visualization Tools

We developed two visualization layers that provide information about the cumulative impact of player actions on the environment, and on the population. Each of the two layers is activated using magnifying glass tools. There are two tangible magnifying glasses (TUI), and two magnifying glasses implemented as digital objects activated through multi-touch (MT). We refer to these as the TUI and MT conditions. In both conditions, the two tools provide players with more information about the cumulative impact of each resource that has been placed on the world map up to that point in the game (on the environment and population respectively). One of the magnifying tools is labeled with a tree and represents the impact on the environment (Figure 2). The other is labeled with people and represents the ability of the current resources to meet the needs of the population. The impact visualizations are generated in real time based on the current state of the game world. The impact visualizations are in the form of "heat maps" in which colour indicates the strength of effect. We used two slightly different colour mappings for the two maps so they could be differentiated (red/dark pink is hot or negative; green/blue is cool or positive). For example, for the environmental layer, the colour red indicates areas of high damage caused by resources located in those areas. In the population layer, dark pink indicates areas where resources are not efficiently supporting the population.

Tool Usage

When a player uses one of the magnifying glass tools to activate a visualization layer, the game pauses for everyone, and the appropriate impact visualization appears as an overlay on the game world (Figure 3). In the TUI condition, the tangible magnifying glasses are placed on the edge of the table. Any player can place a tool on the table"s active surface to activate the corresponding impact visualization. Removing the tool removes the visualization. Only one of the visualizations can be displayed at a time. In the MT condition, the two magnifying glasses are static buttons placed in each lower corner of the tabletop. Touching a multi-touch tool button will activate the corresponding visualization. Touching the resume button will remove the visualization. All other game features are identical between the two conditions. The tools may provide opportunities for collaborative communication and coordination. For example, players may use the displayed impact visualization to trigger discussion of the contribution of each resource type to the current game state and then modify their game strategy.

System Implementation

The Futura game runs on a multi-touch version of EventTable, a custom-made digital tabletop system that facilitates both multi-touch and tangible interaction, the latter using the reacTIVision-tracking library [START_REF] Kaltenbrunner | reacTIVision and TUIO: A Tangible Tabletop Toolkit[END_REF]. Fiducial markers are used to track the tangible tools. The platform is initially described in [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF]. Some modifications were made to improve finger tracking. For example, three cameras were added to the existing camera configuration. This is described in detail in [START_REF] Wang | Comparing Tangible and Multi-touch Interfaces for a Spatial Problem Solving Task[END_REF]. Fiducial markers are used to track the tangible tools.

Study Methodology

In order to understand the differences between tangible and multi-touch tool use, and the effects of tool interface style on collaboration, we designed an exploratory comparative study between a tangible interface (TUI) condition and a multi-touch (MT) condition of the visualization tools for the game Futura. We used a within subjects design, because the group dynamics and personality of users may have a major influence on the results [START_REF] Meerbeek | Pipet: A Design Concept Supporting Photo Sharing[END_REF]. Another advantage of a within subjects design is that users are able to compare the two UIs. The order of the conditions was counterbalanced (AAABBB or BBBAAA).

Participants

We collected data from 45 participants who played the game in groups of three players resulting 15 different groups. Participants (26 male, 19 female) in our study were predominantly graduate and undergraduate students from different programs at Simon Fraser University (Canada). Ages ranged from 18 to 36 years old (M =23, SD = 3.69). Most participants reported that they were in the third year of their study (14 out of 45). Most participants reported that they played computer games sometimes (16 out of 45) or often (13 out of 45), but had never (22 out of 45) or rarely (17 out of 45) interacted with a digital tabletop.

Participants were recruited through a user study pool at the university. The groups were randomly assigned and thus participants knew each other to different degrees: some were friends (13 out of 45), classmates (16 out of 45), others did not know each other at all (16 out of 45). Participants were rewarded with a $10 gift card for participating in our study.

Procedure

To minimize potential learning effects, we explained the mechanics of the game carefully before participants started playing using an in-game tutorial. To stimulate the use of the magnifying glass tools, we demonstrated and encouraged the use of them at the end of the tutorial. Participants played each of the two conditions (TUI and MT) three times. In previous studies, we noticed that many groups played the game about three times before they were able to win (without a tutorial) [START_REF] Antle | Futura: Design for Collaborative Learning and Game Play on a Multi-touch Digital Tabletop[END_REF]. They were directed to try to win the game. Each game took about five minutes (longer than the original three minute version of the game because players used the tools to pause the game). The total duration of the study was about 50 minutes for each group. The participants were asked to fill in the game questionnaire after playing in both conditions. The study setup is shown in Figure 4.

Data Collection and Analysis

The mixed methods study design involved collecting data about demographics, game performance, tool use behaviors, player opinions, as well as observations of interaction and play patterns. Quantitative measures included game performance, game scores, and tool use. Tool use data was analyzed for both frequency of occurrence and temporal patterns of usage. In the questionnaire, we asked about learning outcomes, collaboration and tools use. For example we asked the participants if they used the magnifying glasses (TUI and MT) to play the game and if they collaborated with others. We also asked participants if they thought the physicality of the tools made a difference. We used both open and closed questions, the latter ranked with a 7-point Likert scale. We analyzed numeric data with Wilcoxon test to compare the difference of two means, since the data involved repeated measures and included non-Normal count data and ordinal Likert ratings. Since we were interested in how the tangible tools influenced collaboration, we also collected qualitative observational notes and video. Real-time observation of the table surface on an additional screen helped the observers to see what was going on in the game without being intrusive (Figure 4, right). We used a structured note taking approach to record how they interacted and collaborated with each other, when players started interacting, how they used the tools, and how this changed over time.

Afterwards three researchers looked for patterns in collaboration. We did this by analyzing our observational notes and video through the lenses of theoretical concepts (objects of negotiation, access points, physical constraints and awareness) as well as searching for other repeated patterns or themes. Based on previous work comparing physical-digital and multi-touch tabletops, in which verbal and physical acts were both deemed important [START_REF] Rogers | Equal Opportunities: Do Shareable Interfaces Promote More Group Participation Than Single User Displays?[END_REF], we considered verbal interactions including: utterances, discussion, information exchange, negotiation, and suggestions, and physical interactions including: gesturing and performing actions on game objects.

Results

Our results address two research questions: "How does the tool use differ, comparing multi-touch and tangible interaction?" and "How does the UI style of the tools affect collaboration?" Results provide insights about how players use the tools, how they collaborate with and through the tools, and with each other. We report our general tool use related findings first, then follow with our four theoretical themes: object of negotiation, access points, physical constraints and awareness. We introduce new themes under each of these, as appropriate.

General Tool Use

We observed that all 15 groups used the environment and population magnifying glasses in both the TUI and MT touch conditions. Groups used the tools during most of their games (72 out of 90 games). In total, we counted 204 magnifier usages in 90 games (90 = 2x3 games x 15 groups). Of these, 108 usages occurred with the TUI tools and 96 with the MT tools. Neither condition significantly differed regarding frequency of usage (average number of usages per game TUI: 2.4, MT: 2.1). However, the frequency of usage varied from group to group. The maximum number of magnifying glass usages (summed over all six games in both conditions for each group) was 20 and the minimum was three. Basic usage patterns were similar for the TUI and MT groups. For example, we found that users would normally not use the magnifier glasses early in the first game (in either condition), since they were still focused on understanding their own game roles and possibilities in the game. Sometimes players seemed to forget about the tools, but started to use them later, "Maybe we should try these" (man, age 26, group 2, game 2). Furthermore, if people figured out a strategy to win the game, which normally took at least a few games, they used the tools less, since they did not need the additional information.

We also looked at how long people used the tools and if there were temporal patterns of usage (e.g. using a tool at a specific moment in the game or using both magnifier glasses directly one after each other). We found that early magnifier glass usages (within the first 45 seconds) occurred more in the second condition, regardless of which condition came second (first session = MT 2, TUI 3; second session = MT 8, TUI 10). The pattern of using both magnifier glasses shortly after each other occurred more often when players played their first three games with the TUI tools. We suggest that this could indicate that the TUI tools were more suitable for shared explorations. This suggestion needs further research.

6.2

Object of Negotiation

The tools affected the game play of everyone in the game when they were used. Our analysis of qualitative observational data indicated that people used the tools in different ways independent of condition. For example, some players only quickly glanced at the impact visualizations without having a conversation and others explored the impact visualizations in depth but did not talk. Others used it to discuss either game functionality or game strategy, sometimes including gestures like pointing to specific points on the visualization and thus, focusing all players" attention to this spot. We observed these different behaviors equally between players in both conditions, and we did not find any significant differences in how they used the tools to negotiate with each other or play the game. The characteristics and quality of collaboration we observed depended primarily on the group dynamics and personality of individuals. As can be seen in Figure 5, some groups communicated and interacted frequently while other groups played the game largely individually. The differences in group dynamics can be explained by the different relationships in groups: some players knew each other well, others were strangers; some groups consisted of quiet and shy people and other groups had one or two main "talkers". The finding that group dynamics is an influencing factor is in line with findings from other studies [START_REF] Piper | SIDES: A Cooperative Tabletop Computer Game for Social Skills Development[END_REF][START_REF] Rogers | Equal Opportunities: Do Shareable Interfaces Promote More Group Participation Than Single User Displays?[END_REF][START_REF] Meerbeek | Pipet: A Design Concept Supporting Photo Sharing[END_REF].

Analysis of the questionnaire data indicated that although users tended to like the second tool they used more, many preferred the TUI tools regardless of if they used them in the first or section sessions (26 vs. 17). This contradiction between use and preference may be explained by the learning effect of the game: when players understood the game better, they were more likely to win and thus liked everything about the game more (including the tools). We did not see differences in collaboration responses in the questionnaire data. For example, the Likert rating for the statement "The magnifying glasses helped me to collaborate with other players" was similar in both conditions (Z = -.938, n.s. = .348). One person said he would like to have tools for everyone individually.

In both conditions and in both session orders, players used the environment tool more than the population tool (60% vs. 40%). We suggest that players may have understood the environment impact visualization better than the population one. In the environment impact visualization, the colour red represented damage, which was easily understandable for all users, "See, here where the marine resource is, it's less polluted" (woman, 22, group 12). However, the population impact visualization showed how well the population was supported. Dark pink meant that the population was not well supported. This was not always clear, "we need more population" (man, 22, group 15). Other players related the population impact visualization to density.

Access Points

We would expect a difference between the conditions related to access points, because the physicality of tangible tools enables mobility and handing over or passing around the tools, which could have influenced collaboration. However, our results showed no differences in how players accessed the game in either condition. This may be because all three players started the game together. The effect of access points may be more important when an additional player is joining an existing game.

Physical Constraints

In the TUI condition, we observed that initiating and resuming of the impact visualizations were done significantly more often by the same person than in the MT condition; 77,4% vs. 56% (Wilcoxon test; Z = -5.83, p<.001). In the MT condition, a different person more often resumed the game by touching the resume button in the middle of the screen. However, this does not suggest more collaboration because it often happened without the resuming player checking with the other players. In the MT group, players often reached over and canceled the impact visualization that someone else activated, without clear announcement or communication (Figure 6). This may have occurred less in the TUI group because of the physical and spatial constraints on the actions that can be taken with a physical object [START_REF] Hornecker | Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interactions[END_REF]. This finding can be explained by looking to meaning rather than behaviors alone. Each type of action that is enabled by an artifact"s affordances can change the meaning of the artifact. Physicality opens up meaning making. We suggest that the finding that many more players resumed the game themselves in the TUI condition may be explained the concept of ownership. The physicality of the tool enables the action of picking it up and holding it, which may in turn cause a feeling of ownership. Psychological ownership is defined as the state in which individuals feel an object is theirs [START_REF] Beggan | On the Social Nature of Nonsocial Perception: The Mere Ownership Effect[END_REF]. Thaler [START_REF] Thaler | Toward a Positive Theory of Consumer Choice[END_REF] uses the term "endowment effect" to describe the idea that goods that belong to one"s endowment are valued higher than identical goods that do not. Endowment contributes to a feeling of ownership. When a player picks up and holds one of the TUI magnifying tools, they, and other players, may all sense that the object temporarily belongs to that player. Other players may then hesitate to take action on the object which was placed by, and temporarily owned by, another player. Further investigation is needed to find out whether it is this sense of ownership that inhibits the initiating and resuming of other players, when using physical tools. For both conditions, from left to right: 1) Player activates impact visualization; 2) Impact visualization is activated; 3) Player resumes the game.

Awareness

Our analysis of observational data indicated that players more often announced that they were going to use one of the tools in the TUI condition compared to the MT condition. Announcements can be verbal, gestural or take the form of actions on objects. For example, announcements include visibly holding up a physical tool or waiting for others to finish their tasks before using a tool. Using the tools affected the game play of everyone and so an individual may have felt an obligation to obtain agreement from the other players before using a tool. In our analysis, we found more instances of gestures, eye contact, and verbal announcements in the TUI condition. This may be because tangible interaction in 3D space makes individual actions more legible, which leads to better peripheral awareness. People need to make body movements in order to use both the multi-touch and the TUI magnifying tools. However, the 3D interaction space and the physicality of the TUI tools make the use more visible when they are lifted off the table in preparation for placement or to resume play [START_REF] Wang | Comparing Tangible and Multi-touch Interfaces for a Spatial Problem Solving Task[END_REF]. This visibility may have made people more aware of each other and the tools (Figure 7). When asked why they liked the TUI tools more, some users (6 out of 26) said that they liked being aware of using a physical artifact. Awareness contributes to group coordination and collaborative context [START_REF] Dourish | Awareness and Coordination in Shared Workspaces[END_REF]. Coordination of actions is an important part of collaborating [START_REF] Hornecker | Collaboration and Interference: Awareness with Mice or Touch Input[END_REF]. Announcing the use of the tools may contribute to the coordination of collaboration [START_REF] Fleck | Actions Speak Loudly with Words: Unpacking Collaboration Around the Table[END_REF]. In a collaborative learning game, individual contributions are relevant to group activities. Awareness is critical when designing a collaborative system [START_REF] Dourish | Awareness and Coordination in Shared Workspaces[END_REF] and announcing the use of the TUI tools can be seen as an increasing awareness about the game. Gestures and utterances make it clear that people are going to use the tools. The findings that players in the TUI condition more often activated the visualization, that the same person more often resumed the game, and players more frequently announced tool use support the suggestion that the TUI design better enables tool awareness. We suggest that these findings support the effectiveness of the TUI tools for enabling awareness. Enhancing awareness supports fluid interaction and coordination during collaboration.

Limitations

The tools we added to the game are not mandatory to win the game. Although all groups used the tools at least once, it may be possible that players did not think they were useful and decided not to use them frequently. In some cases, the players linked information to the impact visualization that was not related at all. For example, some players interpreted the game world and impact visualization as location specific "maybe we can put highly polluted things far away" (woman, 27, group 7). While the impact map shows the impact of each resource at the location where the resource has been placed, the spatial locations of resources on the map do not mediate their impact in any way. For example, placing a highly polluting resource anywhere on the map would affect the game state in the same way. This possibly confused players. Furthermore, since we did not test the effectiveness of the impact visualizations, it is possible they were confusing instead of helpful. However, any negative effects would have occurred in both conditions. The tangible object recognition system (using fiducial markers) did not work as precisely as the touch recognition of the system. This means that activation and deactivation of the visualizations using the multi-touch (buttons) was easier and faster than using the TUI objects. This may explain why 17/45 participants liked the multitouch tools better. Of these participants who liked the multi-touch tools more, all of them gave these reasons: the MT tools worked better, faster or were easier to use. There was also some latency between activating a tool and the appearance of the impact visualization (about 0.5s). Although the latency was present in both conditions, it was slightly longer in the TUI condition. Neither of these limitations appears to directly impact our results about ownership or group and tool awareness.

Generalization of our results should take into account that we performed this study in a lab setting. For example, this limits our ability to make inferences about tool use, collaboration and public engagement in a public setting. In addition, the statistical conclusion validity is limited because of the small sample size of 15 groups.

Conclusions

In this paper, we present the results of an exploratory study that compared the differences between tangible and multi-touch tool use, and the impact of tool use on collaboration in a digital tabletop game. Analysis of tools use and its impact on collaboration was viewed through a concept from the theory of CSCL: objects of negotiation, and concepts from previous empirical research in tangible interaction: access points, physicality and awareness. Research that includes analysis using theories about collaborative learning and previous findings from tangible interaction studies for tabletop game applications is one of the unique contributions of our work. This paper presents new and valuable insights about tool use, objects of negotiation, access points, physical constraints and awareness in collaborative tabletop settings. We provide insights about how people participate and interact with both tangible and multi-touch tools in the context of a tabletop game. Our gamespecific findings include that the tools were used in different ways regardless of UI style. Some players used it individually, through quiet pondering, and others used the tools intensively and collaboratively to helped them to discuss the game strategy. We also found that players in the MT group often "took over" tool usage that a different player had initiated without consultation. We suggest that the 3D interaction space and physicality the TUI tools may have contributed to the feeling of psychological ownership which prohibited this from happening as much in the TUI condition. This is supported by the finding that tangible tool use was correlated with more player announcements (both verbal and behavioral). We suggest that better awareness of group activity, and therefore better collaboration, can occur when the same player activates the visualization and resumes the game when this is accompanied by some form of announcement. In terms of player preferences, we found that, despite implementation limitations, players still preferred the tangible tools.

Overall, our results provide limited support for the benefits of tangibility. Physical affordances can change the meaning of the artifact as well as actions taken with it. We introduced the concept of ownership which we used to explain player behaviors in the TUI condition. The physicality of the tools and the 3D interaction space may enhance ownership, enable announcements of use, and facilitate awareness. Together these processes and forms of interaction support coordinated collaborative activity. We presented the details of our analysis to support designers to make decisions about tangible and multi-touch elements of digital tabletops deployed to facilitate collaborative activity.

Fig. 1 .

 1 Fig. 1. The Futura game (in the multi-touch condition).

Fig. 2 .

 2 Fig. 2. Tangible (left) and multi-touch (right) magnifying glass tools.

Fig. 3 .

 3 Fig. 3. Impact visualization -environment (left), impact visualization -population (right).

Fig. 4 .

 4 Fig. 4. Playing Futura in groups of three (left). The setup of the user study, showing the Futura game tabletop, three players, microphones, video camera and a additional screen (right).

Fig. 5 .

 5 Fig. 5. Group dynamics; collaborating group (on top) and individual play (bottom), in both conditions (MT left, TUI right).

Fig. 6 .

 6 Fig. 6. Initiating and resuming tools is more often done by different players in the MT condition (top); and more often done by the same player in the TUI condition (bottom).For both conditions, from left to right: 1) Player activates impact visualization; 2) Impact visualization is activated; 3) Player resumes the game.

Fig. 7 .

 7 Fig. 7. No tool usage announcing in MT condition (left), clear gesture of tool usage announcement in TUI condition (right).

Acknowledgements. The authors would like to thank Sijie Wang and Nahid Karimaghalou for their technical support, Anna Macaranas, Allen Bevans, Katie Seaborn and Josh Tanenbaum for their ideas, thoughts and practical support. Thanks to Wijnand IJsselsteijn for his coaching. In addition, we would like to thank all the participants of our user study for their participation. This research was funded by the NSERC RTI and the GRAND NCE grants from Canada.


Beant Dhillon

Rafal Kocielnik

Ioannis Politis

Marc Swerts
email: g.j.swerts@uvt.nl

Dalila Szostak
email: szostak@tue.nl

R D Dhillon

I Kocielnik

D Politis

Culture and Facial Expressions: A Case Study with a Speech Interface

Keywords: Multicultural study, culture, cultural differences, uncertainty avoidance, expressiveness, speech interface

Recent research has established cultural background of the users to be an important factor affecting the perception of an interface's usability. However, the area of cultural customization of speech-based interfaces remains largely unexplored. The present study brings together research from emotion recognition, inter-cultural communication and speech-based interaction and aims at determining differences between expressiveness of participants from Greek and Dutch cultures, dealing with a speech interface customized for their culture. These two cultures differ in their tendency for Uncertainty Avoidance (UA), one of the five cultural dimensions defined by Hofstede. The results show that when encountering errors, members of the culture that ranks higher in the UA scale, i.e. Greeks, are more expressive than those that rank low, i.e. Dutch, especially when encountering errors in a low UA interface. Furthermore, members of the high UA culture prefer the high UA interface over the low UA one.

Introduction 1.Culture

There are more than 150 ways in which culture has been defined over the years [START_REF] Kroeber | Culture: A Critical Review of Concepts and Definitions[END_REF]. It is a broad concept that incorporates a wide range of features. Merriam-Webster online defines Culture as "customary beliefs, social forms, and material traits of a racial, religious, or social group." Various new terms have been created by different authors [START_REF] Hall | The silent language[END_REF][START_REF] Hofstede | Culture and organizations. International Studies of Management & Organization[END_REF][START_REF] Trompenaars | Riding the Waves of Culture: Understanding Cultural Diversity in Business[END_REF] to break up culture into smaller components and study the differences. Fons Trompenaars, a Dutch management consultant [START_REF] Trompenaars | Riding the Waves of Culture: Understanding Cultural Diversity in Business[END_REF], defines the various levels of culture, namely, National, Corporate and Professional. Hall [START_REF] Hall | The silent language[END_REF] presented a four-factor model which distinguishes between cultures in terms of low vs. high context, multi-tasking vs. single tasking etc. Further, Hofstede [START_REF] Hofstede | Culture and organizations. International Studies of Management & Organization[END_REF] defines Culture in terms of five dimensions which include Masculinity vs. Femininity, Individualism vs. Collectivism among others. Though these cultural models may have been defined at different times by different researchers, at times, they tend to be overlapping or refer to similar components of culture. In fact, Khaslavsky [START_REF] Khaslavsky | Integrating culture into interface design[END_REF] demonstrated an instance of merging the existing cultural models by Hall [START_REF] Hall | The silent language[END_REF], Trompenaars [START_REF] Trompenaars | Riding the Waves of Culture: Understanding Cultural Diversity in Business[END_REF] and Hofstede [START_REF] Hofstede | Culture and organizations. International Studies of Management & Organization[END_REF] for applications in interface design. The present study is based on Hofstede's model thus, it relates to exploring differences between national cultures. The following section discusses some of the empirical studies carried out in context of culture and interface design.

Culture and Interface Design

Culture has been shown to affect perception [START_REF] Nisbett | The influence of culture: holistic versus analytic perception[END_REF], judgement [START_REF] Gardner | I" Value Freedom, But "We" Value Relationships: Self-Construal Priming Mirrors Cultural Differences In Judgment[END_REF] and decision making [START_REF] Briely | Reasons as carriers of culture: dynamic vs. dispositional models of Cultural influence on decision making[END_REF]. Moreover, several studies have shown how the cultural background of users affects their perceptions as well as preferences for interfaces. Researchers Cyr and Trevor-Smith [START_REF] Cyr | Localization of Web design: an empirical comparison of German, Japanese, and US Web site characteristics[END_REF] analyzed design elements such as the use of symbols and graphics, color preferences, and site features for 30 municipal websites in Germany, Japan, and the U.S to find out the effect of design elements in culture. They concluded that layout, symbols, navigation, and the use of color are amongst important design elements that vary across cultures and need to be taken into account for localization. Singh et al [START_REF] Singh | A cross-cultural analysis of German, Chinese and Indian consumers' perception of website adaptation[END_REF][START_REF] Singh | The culturally customized website -Customising websites for the global workplace[END_REF] provided empirical evidence that in comparison to standardized websites, consumers from Germany, Spain, Netherlands, Switzerland, Italy, China, and India preferred culturally adapted websites. As described, there are several studies that emphasize the importance of cultural customization of graphical user interfaces to the cultural background of users [START_REF] Cyr | Web site design, trust, satisfaction and e-loyalty: the Indian experience[END_REF][START_REF] Reinecke | Culturally Adaptive Software: Moving Beyond Internationalization[END_REF][START_REF] Sheppard | The effects of cultural markers on Web site use[END_REF] but research dealing with cultural customization of speech interfaces has not led to very conclusive results so far [START_REF]Text, Speech and Dialogue[END_REF].

Facial Expressions, Emotion and Speech Interfaces

Speech based interaction started with telephony but people quickly found telephonic conversations with machines (voicemail, booking systems) rather uncomfortable. Because of that reason, systems with human facades which try to mimic human interaction are becoming a preferred way of interaction [START_REF] Pitt | Design of speech-based devices: a practical guide[END_REF]. One way of making the interaction more human-like is to develop systems that can interpret the facial expressions of humans to determine their emotional state and adapt their behavior and respond accordingly (Affective Computing). There are technological solutions available that can already provide a robust estimation of a person's emotional state by interpreting facial expressions [START_REF] Balomenos | Emotion Analysis in Man-Machine Interaction Systems[END_REF], [START_REF] Caridakis | Modeling naturalistic affective states via facial and vocal expressions recognition[END_REF], [START_REF] Jarkiewicz | Anthropometric Facial Emotion Recognition[END_REF]. Further, existing research suggests that interpretation of certain facial expressions (e.g. frustration) could help a system deal with errors more effectively [START_REF] Barkhuysen | Audiovisual Perception of Communication Problems[END_REF]. Along the same line, the present study aimed to explore if members with different cultural backgrounds differ in their expressiveness towards speech-based interfaces adapted for culture. Based on the results, a few pointers are provided for cultural customization of a speech-based interface.

The following section describes the cultural dimension on which the interface design was based as well as the rationale for choosing Dutch and Greek culture. Since Hofstede's cultural model is one of the most well-known and widely applied models in both the field of management as well as interface design [START_REF] Dhillon | Effects of cultural background of users on interface usability[END_REF][START_REF] Kondratova | Proceedings of EdMedia 2005 -World Conference on Educational Multimedia, Hypermedia & Telecommunications[END_REF], it was used for the purposes of the current study.

Culture and Expressiveness

One of the cultural dimensions defined by Hofstede [START_REF] Hofstede | Culture and organizations. International Studies of Management & Organization[END_REF] that is related to frustration is Uncertainty Avoidance (UA). Uncertainty Avoidance Index (UAI) is used as a measure of UA, i.e., the higher the UAI of a country, the higher the UA tendencies of members of that culture. Cultures with high UA are associated with higher anxiety, stress, avoidance of ambiguity and tendency to be normative, as emphasized in rules. Hofstede [START_REF] Hofstede | Culture and organizations. International Studies of Management & Organization[END_REF] also suggests that cultures rating high on UAI are emotionally more expressive. As he reports, this might be because people from cultures with high UA experience higher stress and have less internalized emotional control. On the contrary, people living in cultures with low UA tend to experience lower stress and less conflict between norms and experience. Due to this "weak superego" and acceptance of deviation, people from these cultures tend to display less emotion [START_REF] Gudykunst | Bridging Japanese/North American differences[END_REF]. According to Hofstede's research, Greek culture has the highest UAI, i.e. 112, while Dutch has a considerably lower score in the same index, i.e. 53, (mean UAI=66.4, min=8, max=112). Since the UAI of Greeks and Dutch differ substantially, representatives of these two cultures were chosen as participants in the study.

Hypotheses

As described above, research suggests that emotional expressiveness differs across cultures. Additionally, guidelines have been developed for adapting interfaces to different cultures [START_REF] Marcus | Cultural dimensions and global web user-interface design: What? So what? Now what[END_REF], based on the assumption that such an adaptation will lead to a more positive user experience. This study focused on determining whether there are differences in facial expressiveness of members of different cultures when they interact with an error prone interface. Further, the study tested whether the level of expressiveness differs when the interaction involves an interface which is adapted to one's culture vs. one that is not. Thus, intensity of expression was the dependent variable. These expectations are justified by the researchers' intuition that any deviation from the expected system behavior would elicit a reaction from the user and the reaction would, very likely, include a facial expression. This deviation was achieved by introducing errors in the system which resulted in an error-prone interaction. Pilot studies by independent evaluators showed that intensity of expression was the variable that most clearly reflected the perceived difference between conditions where there was an error encountered and conditions where there was no error. Two different versions of an interface that varied in the dimension of UA were designed. This enabled the researchers to assess differences in expressiveness during the interaction with each version. A more expressive response was expected for an interface that did not correspond to the cultural expectations of the user. Moreover, a general tendency towards higher expressiveness was expected in high UA cultures. The following hypotheses were formulated to be tested during the study:

• H1: When encountering errors, high UA cultures will overall be more expressive than low UA cultures. • H2: When encountering errors in a low UA interface, high UA cultures will be more expressive than low UA cultures. • H3: When encountering errors in a high UA interface, low UA cultures will be more expressive than high UA cultures.

Methods

Experiment Design

The study was set up as a 2X2 mixed factorial design with two independent variables: culture and type of interface. Culture was divided into Greek and Dutch and type of interface was divided into high UA and low UA. The dependent variables were frustration experienced by participants and facial expressiveness. UAI for each culture group was measured using the Values Survey Module 94 (VSM94) questionnaire, frustration was measured with the NASA Task Load Index (NASA-TLX) questionnaire and expressiveness was measured through a perception test.

There are some disadvantages of having a 2X2 mixed design, e.g. learning effects and fatigue. However, the advantage of the design is that a lower number of participants is required. Additionally, statistical power increases since participants' individual differences are kept constant between conditions.

Application Design

The study was conducted using a Wizard of Oz approach which involved interacting with a speech based Train Booking System. One of the experimenters controlled the system as a response to the users' utterances where the participants were asked to make a booking from London to Newcastle for a specified date and time. Two versions of the system were developed, one customized for a high UA culture and the other for a low UA one. The interfaces were developed using the guidelines available for cultural customization of Graphical User Interfaces [START_REF] Marcus | Cultural dimensions and global web user-interface design: What? So what? Now what[END_REF]. Guidelines were adapted for use in a speech only interaction. Furthermore, during pilot testing, two independent reviewers were asked to interact with the interfaces and the interaction was modified according to the comments provided by reviewers. For the high UA interface, the following recommendations were applied:

• Guided interaction with confirmations at every step.

• Explicit cues to prevent users from feeling lost. In the second case, the system would ask the participant to repeat what he/she had last said, e.g.: S: Do you wish to take a direct or an indirect train? U: A direct one. S: Sorry, can you repeat your train preference? ... An equal number of misrecognition and repetition errors were induced in both high and low UA versions of dialogue, two of each type for each interface. As a result, each user would encounter, in total, 4 errors per interface.

Participants

Participants were recruited from the Eindhoven University of Technology. There were 16 participants, 8 Greek and 8 Dutch. All the participants selected for the study were university students as the VSM94 questionnaire is validated only for participants from similar professional backgrounds. Their age ranged from 18-30. The participants were asked to fill in the VSM94 designed by Hofstede [START_REF] Hofstede | Values Survey Module manual[END_REF] in order to measure the UAI and make sure their responses were in line with Hofstede's findings. Participants were offered coffee/tea with a snack as well as a small reward (5 Euros Coupon) for participation. Details on participants' demographics can be found in Table 1.

Questionnaires

Two questionnaires were utilized during the experiment: the Values Survey Module (VSM94) and the NASA Task Load Index (NASA-TLX).

The VSM94 [START_REF] Hofstede | Values Survey Module manual[END_REF] is a 26-item questionnaire developed by Geert Hofstede for comparing cultures, based on region or country, across 5 dimensions: Power Distance, Individualism, Masculinity, Uncertainty Avoidance, and Long-term Orientation. 20 questions (4 questions for each) are used for the dimensions, and the remaining six questions are demographic. Scores for each dimension are computed on the basis of four questions. Each question is scored on a five point scale and the index scores are derived using the mean scores of respondents from these questions.

The results are computed using a formula provided with the questionnaire. Score for each dimension is calculated using mean scores for related questions for a group of participants. For calculating UAI, Formula 1, as provided below, is used: UAI = +25*mean Q13 +20*mean Q16 -50*mean Q18 -15 mean Q19 +120.

(1)

In the present study, the UAI score for Greek participants was found to be 105 and 40.7 for Dutch participants. A large difference between these scores supports the selection of Dutch and Greek participants based on differences in UAI.

The NASA-TLX is a tool that allows users to perform subjective workload assessments of tasks performed with human-machine systems [START_REF] Hart | Nasa-task load index (nasa-tlx) : 20 years later[END_REF]. The ratings are collected on six sub-scales: Mental Demand, Physical Demand, Temporal Demand, Own Performance, Effort and Frustration. The overall workload is computed from weighted averages of these ratings.

Procedure

Participants filled in the VSM94 questionnaire after which they were briefed about the experiment by one of the investigators. While interacting with the interface, the participants were video recorded using the built-in webcam of the computer. The average duration of the interaction was 2:03 minutes for the low UA interface (Std. Dev. 0:17) and 2:22 minutes for the high UA interface (Std Dev. 0:14). After having completed the task with the first interface, participants were asked to fill in the NASA-TLX questionnaire which was followed by the interaction with the second interface. The order of interfaces was balanced across subjects to counterbalance for learning effects. After the second interaction, participants were once more asked to fill in the NASA-TLX questionnaire followed by a debriefing session as well as a short interview session. The location and set up of the study was chosen in order to recreate a comfortable and friendly environment for participants. All the questionnaires were used in their entirety to preserve validity.

Perception Study

Perception study is a widely used method for evaluating facial expressiveness, in terms of type and/or intensity of expression [START_REF] Barkhuysen | Problem detection in human-machine interactions based on facial expressions of users[END_REF], [START_REF] Ekman | A new pan-cultural facial expression of emotion[END_REF], [START_REF] Ekman | Universals and cultural differences in the judgments of facial expressions of emotion[END_REF]. It involves showing a series of videos or pictures to the participants and asking them what kind of expression is perceived by the viewer and/or rating how intense an expression is.

The videos recorded during the previous phase of the experiment were edited and instances where a participant encountered an error with the system were extracted. An equal number of videos from Dutch and Greek participants with misrecognition and repetition error were used. The situations considered as erroneous were only the ones where the users explicitly repaired the errors by repeating their command or correcting the system. At times, the participants did not correct the mistake made by the system, giving no evidence of realizing a misrecognition error. In such cases, their videos were not included in the perception test. Also, for each participant, one video with a neutral expression, i.e. a facial expression in a neutral, non-erroneous context, was included in the perception test. The neutral videos were selected from dialogue fragments without errors, and without clearly perceivable muscle movements within a timeframe of 5 seconds, following best practice procedures outlines in related studies [START_REF] Barkhuysen | Problem detection in human-machine interactions based on facial expressions of users[END_REF]. Neutral videos were used later as a baseline in terms of expressiveness. The results from the perception study validated this set-up.

The total number of clips presented during the perception test was 56. Each clip was about 2 seconds long. The participants of the perception study rated the intensity of the expression shown in each video on a scale of 1 to 5 (from neutral to very strong). All 17 participants for the perception study were employees of Eindhoven University of Technology (TU/e). In order to balance possible cultural differences in expressiveness perception, participants from at least 9 different cultures were selected. These participants did not have any information regarding the nationality of the participants in the videos or the context of the interaction. Furthermore, none of them had participated in the initial experiment.

Results

Quantitative

The ratings obtained from the perception study were averaged for each rater by video type and culture. This resulted in 6 average scores for each rater (2 cultures X 3 Video types): 3 videos of the Greek participants (high UA, low UA and Neutral), and the same for the videos of the Dutch participants. This procedure was followed for all 17 scorers. These scores were analyzed using a 2X3 repeated measurements ANOVA with 2 independent factors (nationality -2 levels; video-type -3 levels). The results display significant main effects for nationality F(1,16)=27.109, p < 001. Thus, hypothesis H1 is supported.

Further, video type F(2,15)=125.466, p<.001 as well as an interaction between nationality and video type F(2,15)=31.804, p<.001 were found to be significant, both for low UA and high UA levels. To tease apart this interaction, a post hoc pair wise comparison (Bonferroni test) was conducted. This test revealed that the perception ratings for neutral video type differ significantly from the two other versions (low and high), but they don't differ among themselves. Furthermore, unlike high UA or low UA conditions, the neutral videos scored close to 1 (No expression) for both cultures. The mean score, on a scale of 1-neutral to 5-very strong, was 1.309 (Std. err: 0.070) for Greek participants and 1.322 (Std. err: 0.092) for Dutch participants. The neutral videos were indeed rated as having 'no expression', thus validating the set-up for the study. The bar graph (Fig. 1) presents the mean scores from perception test for each video with respect to each culture.

The results of pair wise comparison for the main effect of nationality and of the interaction between nationality and video-type can be found in Tables 2 and3.A spliced analysis confirmed that the differences between UAI levels are significant (using separate t-test), both for Dutch and Greek participants separately (P < .001). Thus, the null hypothesis can be rejected and hypothesis H2 and H3 are supported. Finally, the values from the NASA TLX Questionnaire were analyzed using multivariate ANOVA. Culture and system were the independent variables, and workload, frustration and frustration weights were the dependent variables. No significant main effects were found for this interaction.

Qualitative

According to the statements from users in the semi-structured interviews conducted after each experiment, five out of eight Greeks preferred the high UA interface, commenting that they "liked the step -by -step and uncomplicated structure and the clear confirmations provided". Five out of eight Greeks disliked the low UA interface, describing it as "stressful and frustrating" and claiming that it presented them with "too few confirmations". It is worth noting that none of the Greek participants preferred the low UA interface. Concerning Dutch participants, the preferences were equally distributed between the interfaces. Thus there are no strong conclusions to be made for Dutch people out of the interviews conducted. Only some of the participants expressed strong opinions about their preferences or dislikes when asked. Others, however, had no strong attitude towards either type of interface. Tables 4 and5 summarize the main findings of this qualitative analysis.

Discussion

Based on the aforementioned results, it can be concluded that the three alternative hypotheses, i.e. H1, H2 and H3, are all supported. When encountering errors, cultures that score high on the UA index are more expressive than cultures with low UA index. Also, when encountering errors in a low UA interface, high UA cultures are more expressive than low UA ones. This holds true for low UA cultures encountering errors in a high UA interface too, but the results are not as strong.

The results suggest that Greek participants, who belong to a high UA culture, prefer interfaces with more confirmations and a step-by-step guided navigation. They have stronger reactions when encountering errors during an interaction with an interface with fewer confirmations. This was confirmed by the exit interviews where five out of eight Greek participants commented that they disliked the low UA interface and five out of eight expressed a liking for the high UA interface.

The Dutch participants seemed to have slightly stronger reactions when they encountered errors in the high UA interface. However, as discovered during the interviews, three Dutch participants disliked the low UA interface and three of them liked it, thus no strong conclusions can be derived from this observation. Overall eight out of sixteen participants disliked the low UA interface, conveying that if no clear information regarding the culture is available, it is probably better to provide an interface designed for high UA.

These findings can be helpful in building systems which detect the expressiveness of a user and adapt their features, e.g. a navigation style that suits the needs of the user. If an interface could detect the UA (based on expressiveness) of its user, especially in context of errors, the interface could adapt its communication style to enable a less frustrating and more efficient interaction. An example could be the case of a virtual agent -based system for flight booking.

If the system detects a high level of frustration in the user, then, it could adapt itself to another context more suited to the user, e.g. adapt its navigation style to become more flexible. Since such a system takes into account the individual differences of the user, this adaptation could even be independent from the cultural background of the user and rather focus on the individual reactions of the user without binding the users into cultural stereotypes. Since the present study was conducted using a speech interface, care should be taken while extending them to other kinds of interfaces.

Another interesting finding was the fact that the results from the NASA TLX questionnaire provide minimal insights into participants' preferences for an interface. There was no match between the participant ratings for frustration in the NASA TLX questionnaire and their comments or the way their expressions were perceived in the perception test. Thus, this finding demonstrates possible limitations of quantitative methods when it comes to providing insights into human behavior, especially in relation to human computer interaction.

Possible bias

The results of this study could have been affected by various factors such as cultural adaptation and the lab set up. Since culture was an important aspect of the study, it is essential to note that the study was conducted in the Netherlands and Greek participants living in this country could potentially have shown different behavior from Greeks living in Greece, although measuring the UAI might have taken care of this bias to some extent. While Hofstede's cultural model advises against making statements from responses from less than 20 participants from a given country, the researchers found a substantial difference between the two cultures even for 16 participants. Finally, testing in a lab environment with a Wizard of Oz approach could have introduced bias in comparison to usual context of interaction. Although the researchers are aware of the limitations of Wizard of Oz, it was still chosen as the best option for this type of research because it facilitated a controlled testing of specific hypothesis. Since it was required to control the number and type of errors that the participants experienced during the interaction, a script-based interaction was a good fit. We note that 14 out of the 16 participants stated they were convinced they interacted with a real system, even though they were briefed that the interaction dialogue was being tested and not the system. The remaining two participants did not know that their expressiveness would be analyzed and their expressions didn't appear fake or staged either.

The sample size could be another point of concern in this study, but it should be noted that our initial work with a limited number of participants already displays observable differences. It can be expected that these promising effects will only become clearer when larger groups would be analyzed.

Conclusion and Future Work

Facial expressions are very important in the field of human-machine interaction, not only as an output modality (e.g. avatars), but also as an input modality. Automatic speech recognition is known to improve when systems interpret information from multiple modalities, but facial expressions as a modality have not been explored in depth in HCI. Facial expressiveness constitutes a big part of human communication and as language and gestures differ across cultures, so might facial expressiveness in the context of machine interactions. This study contributes to the area since it explores and finds out that these presumed differences are indeed true, for the dimension related to expressiveness and do not just refer to prejudices. Furthermore, this study demonstrates that such differences could be exploited to improve humanmachine interactions by tuning interaction styles to specific groups of people. In this respect, the current study is both novel and relevant.

An important future step for this research could be to include a larger number of participants for both the test as well as the perception study. A larger number of participants would make the cultural difference statement stronger, regardless of the strong statistical values that the current research shows. Additionally, testing other cultural dimensions with similar interfaces that produce controlled errors, or testing the same dimensions but in a real interface that is naturally prone to errors could also be of interest for further research. Finally, testing Greek participants in their native country and comparing the results to those of the described study could provide richer insights in this field.

For

  the low UA interface, the following recommendations were applied: • Free interaction with fewer confirmations • More flexible navigation • Variety of choices The following example demonstrates the low UA interface used in the study: S: Welcome to the Train Booker! You can start with places or times, which do you prefer? U: Places. S: Between which places do you wish to travel? U: From London to Newcastle. S: What day and time do you want to travel? You can always go back or cancel. U: Sunday at 12 o'clock. ... Two types of errors, i.e. misrecognition and repetition, were introduced. In the first case, the system would respond as if it had misheard the participant, e.g.: S: What class would you like to travel with? U: Second class. S: First class then? ...

Fig. 1 .

 1 Fig. 1. Perception test results -expressiveness mean values for video excerpts from each interaction context (Low UA interface, Neutral, and High UA interface) and nationality.

•

  Restricted amounts of data.• Limited choicesThe following is an example of the high UA interface used in the study:

	System: Welcome to the Train Booker! Where do you want to
	depart from?
	User: London.
	S: Ok, you are departing from London.	What is the
	destination? You can always go back or cancel.
	U: Newcastle.
	S: Ok then, what day do you want to travel between London
	and Newcastle?
	U: Sunday.
	S: And what time do you want to travel on Sunday?
	U: 12 o'clock.
	S: So you are traveling from London to Newcastle on
	Sunday around 12 o'clock, right? You can always go back
	or cancel.
	U: Right.
	...

Table 1 .

 1 Demographics for participant study.

	Nationality UAI*	Average age Average time abroad	Gender
					Male Female
	Greek	105.0	24-29	1-3 years	5	3
	Dutch	40.7	20-25	0-1 years	6	2
	* UAI stands for Uncertainty Avoidance Index measured for this group

Table 2 .

 2 Pair wise comparison for expressiveness by nationality.

	Nationality Mean	Std. Error	95% Confidence Interval
				Lower Bound	Upper Bound
	Greek	2.268	0.074	2.111	2.425
	Dutch	2.077	0.079	1.909	2.245

Table 3 .

 3 Interaction between variables nationality and context of interaction (video clips fromLow UA interface, Neutral, High UA interface respectively).

	Nationality Video Type	Mean Std. Error	95% Confidence Interval
				Lower Bound	Upper Bound
	Greek	Low UA interface*	2.900 1.109	2.669	3.131
		Neutral	1.309 0.070	1.161	1.456
		High UA interface* 2.597 0.092	2.402	2.791
	Dutch	Low UA interface*	2.230 0.094	2.031	2.429
		Neutral	1.322 0.092	1.126	1.518
		High UA interface* 2.678 0.088	2.491	2.865

* Low/High UA interface: video clip from interaction with low/high UA interface

Table 4 .

 4 Preference towards a type of interface based on comments from interviews.

	Nationality	Total		Preference
			Low UA	High UA	None
			interface	interface
	Greek	8	0	5	3
	Dutch	8	3	3	2

Table 5 .

 5 Dislike of a type of interface based on comments from interviews.

	Nationality	Total		Dislike
			Low UA	High UA	None
			interface	interface
	Greek	8	5	0	3
	Dutch	8	3	0	5