Aneta Takhtamysheva

Tim Decker

Heuristics for Social Games with a Purpose

Keywords: social gaming, playful applications, games with purpose, human computation, image tagging

Current project introduces a prototype of a game with a purpose constructed using heuristics of social game genre to attract long-term player involvement, and user content generation, to meet the constrains of time and budget.

Introduction

Social games have experiencing unprecedented success rate. Millions of players, encompassing a large range of age categories play social games to relax, socialize and compete with Facebook friends on daily basis [START_REF]PopGames: Social Gaming Research[END_REF]. Social games are known for comparatively small costs of development, minimalistic graphics, simple game mechanics, and viral promotion, that seem to have fit the goals and development facilities of developers of games with a purpose (GWAP) [START_REF] Ahn | L: Human computation[END_REF].

In the current work we will discuss the heuristics of social games that can be utilized by games with purpose (human computation games) and propose a prototype of social game with purpose that also incorporates mechanisms that allow to deal with the financial and time constraints of researchers.

Social Game Heuristics Principles for Games with Purpose

Most games with purpose (GWAP) published on social networks so far have not been developed using heuristics specific for social network games; rather they were developed as simple casual games (i.e. Digitalkoot [START_REF] Chrons | Digitalkoot: Making Old Archives Accessible Using Crowdsourcing[END_REF]), or gamified playful environments like Sentiment Quiz [START_REF] Weichselbraun | Using games with a purpose and bootstrapping to create domain-specific sentiment lexicons[END_REF] that utilize social platforms only for the purpose of game deployment. This issue resulted in the lost opportunities of maximizing the reach and engagement of players that social networks like Facebook can potentially offer.

While existing game heuristics frameworks offered by Desurvire [START_REF] Desurvire | Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration[END_REF] and Pinelle [START_REF] Pinelle | Heuristic evaluation for games: usability principles for video game design[END_REF], cover many elements that are also related to social games, some elements that are related specifically to the social game genre must be added. Paavilainen [START_REF] Paavilainen | Critical review on video game evaluation heuristics: social games perspective[END_REF] suggests a set of Heuristics for Social Games, based on the guidelines developed by Järvinen [START_REF] Järvinen | Game design for social networks: interaction design for playful dispositions[END_REF], which besides the points common to all classic games, includes the following additional elements: rewards for coming back; interruptability; using themes from popular culture; virality and sharing (rewarding players for inviting and gifting friends); narrativity (broadcasting of ingame events as a narration on the player's social network news timeline). Also, other authors mention sophisticated return mechanisms [START_REF] Thornton | 7 -FarmVille, Making Great Games[END_REF] rhythm design [START_REF] Tyni | Howdy pardner!: on free-to-play, sociability and rhythm design in FrontierVille[END_REF] ampleness of positive feedback and very clear game visuals and next step guidelines [START_REF] Isbister | Chapter 10 -The Strange Case of the Casual Gamer, Game Usability[END_REF] as part of a successful social game model.

We believe that in order to benefit from social game genre, social GWAP should strive to incorporate above mentioned elements whenever relevant.

In addition, Kirman [START_REF] Kirman | User Powered Games for Research[END_REF] suggest utilizing user empowerment mechanism for asset creation and management of the game, to save efforts, time and money for researchers, and yet creating an engaging experience for players. This tactic has been successfully used in some commercial multi-player online games like Second Life, or Spore and might prove to be a valuable approach in research environment where developers might be limited in expertise, time and budget.

World Architect Game Prototype

While so far games with purpose were constructed so that the major challenge of the game is the serious tasks itself, we propose a concept of a serious game constructed using the heuristics of social games, where serious tasks are alternated with purely fun tasks.

The goal of this approach is test whether prioritizing fun tasks can leverage the overall sustainability of players' interest to the GWAP and actually increase the number of "serious" contributions of players in the long-run.

We present a concept/prototype of a game called World Architect, which represents a social network game with elements of serious tasks (image tagging). The game consists of variety of quests, some of which are related to serious tasks and others are not but serve the purpose of maintaining the attractiveness of the game.

Players have to set up villages, cities, states and countries by constructing buildings. To avoid costly process of game asset development (buildings) we incorporate user asset generation (Construction Quest). Users should create their own buildings (using graphic tools provided by the game) and can submit those for competitions. Each player is rewarded for participating in the evaluation of submitted pieces (Judgment Quest), the results of which serve as a ground for personalized game reminders. The next Shopping Quest requires users to create shipping tags for construction materials that were purchased to be "transported" to the construction location. The process of creation of shipping tags involves generating image tags. The quality of image tags is verified in the Gardening Quest where players have to match generated tags with images that they were created for. Each successful matching will bring the "gardener" a virtual tree, certain number of which is required to accomplish the construction process, and the "shippers" that created the successful image-tag combination will "safely" receive their construction materials at the construction spot. Thus, both sides will reach perfect equilibrium only when providing honest image tagging and validation.

The game environment includes elements that are typical to many existing social network games: a popular culture theme, never-loose multi-session game play; return mechanics; simple strategy genre with simple puzzle elements; user's own unique space with a autonomous management rights; freedom to progress through the game at the desired pace; narrativity; social network login with open access to the user's profile data; and sociability (friends invitations and propagation mechanisms).

In addition to the above, the following guidelines were developed:

-Serious tasks positioned as secondary material to help deliver fun experience to the players rather than being a primary reason for playing the game.

-Serious tasks constructed as light "puzzles" that adds to the engagement of the game.

-Serious tasks deconstructed into smaller pieces and spread among several quests, thus decreasing the risk of breaking the flow of the game [START_REF] Jones | Creating electronic learning environments: Games, flow, and the user interface[END_REF].

-Game consist of a variety of fairly independent blocks (mini quests) that are connected through storytelling, but can be replaced, improved or removed without destroying the whole game. This makes the game space more flexible for further improvement and extension.

-Mini quests represent a variety of playful activities to chose from -rating, selecting, drawing, puzzle solving, etc, thus giving user freedom of choice in activities instead of limiting to one type of activity.

-The voting scores that each user receives for works submitted for voting competition will in its turn serve two important points: boost efforts and motivation for the players, and:

-Serve as a ground for anticipated updates on how well the submitted works are performing in terms of ratings (narrativity element), and inconspicuously remind players about the game itself.

Further Steps

The next stage of the work is to conduct user studies and test the usefulness of the proposed approach where serious tasks are presented only as secondary elements of social games, and see if proposed guidelines, in particular those related to serious tasks, prove effective in terms of soliciting long term players' contributions. Also, the studies should help to find a proper balance between the number of serious tasks in proportion to the number of playful tasks so that to keep sustainable interest of players while receiving useful input at a pace and quantity that makes the game worth the investments that it required.


David Milam
email: david_milam@sfu.ca

Seif Magy
email: magy@neu.edu

El-Nasr

Lyn Bartram

Bardia Aghabeigi

Perry Tan

Similarity in Visual Designs: Effects on Workload and Performance in a Railed-Shooter game

Keywords: Game Design, Visual Design, Cognitive Load, User Research 1

Games are a popular form of digital entertainment and one elusive question is how complex visual designs affect the player experience. We address one aspect of this topic in terms of similarity of visual features, explored both as an organizing principle in Gestalt psychology and as a theory in visual attention. To address this issue, we developed a 3D railed shooter game with adjustable visual features of size, speed, and density of targets and nontargets. Based on these features we evaluate 105 player's performance in 4 visual conditions. In addition, we employ a cognitive workload assessment as a means to understand the perceived demands on players. Results show effects of expertise on performance and cognitive workload, per visual condition. Our methods and implications on game design are discussed.

Introduction

3D video games engulf users in sensory rich environments made possible through the visual design. The visual design helps distinguish the importance of elements among less important elements. Often without our awareness, activities, such as searching for artifacts or shooting enemies are comprised of visual search tasks. Visual search tasks require both perception of the environment and attention to elements with expressive features. This work considers visual search in situations, where the environment is too busy or contains many features at once causing a player to misperceive what is important. This disruption can lead to mistakes or a breakdown in performance [START_REF] Ryan | Evaluating Interactive Entertainment using Breakdown: Understanding Embodied Learning in Video Games[END_REF]. Game designers remedy this problem by making game elements tied to goals clearly visible [START_REF] Smith | Helping Your Players Feel Smart: Puzzles as User Interface[END_REF] and allude to the player's focus of attention [START_REF] Lemarchand | Attention NOT Immersion: Making your games better with psychology and playtesting, the Uncharted Way[END_REF][START_REF] Björk | Games and Design Patterns[END_REF]. However, these works have not been empirically validated and are disconnected from a formal theory of attention. The goal of this paper is to apply a theory of visual attention to visual game elements and then analyze differences in players' performance and perception. We argue that this approach can improve game difficulty settings and accessibility.

Researchers within the fields of attention and perception proposed different theories through controlled experiments. Among the most foundational are feature inte-gration [START_REF] Treisman | A feature-integration theory of attention[END_REF] and guided search [START_REF] Wolf | Guided Search 2.0: A revised model of visual search[END_REF], whereby bottom-up (stimuli driven) and top-down (goal driven) approaches work in parallel in a given search task. Tests such as these are comprised of target and non-target elements combined with expressive features, such as color, shape, and motion. Attention researchers tested expert gamers and nongamers in these tests [START_REF] Green | Effect of Action Video Games on the Spatial Distribution of Visuospatial Attention[END_REF], and found the experts have greater attentional resources than novices in spatial attention acuity, such as peripheral vision. Researchers in game approachability [START_REF] Desurvire | User Experience Design for Inexperienced Gamers: GAP -Game Approachability Principles. Evaluating User Experiences in Games[END_REF] reached similar conclusions, that inexperienced players perceive and perform differently in the face of challenge. While these insights are not surprising, the take away design lessons are not clear since the experiments either occur outside the context of a game or do not consider the visual relationships between elements in sufficient detail.

Visual search in sensory rich 3D games is complex due to the continuous change of visual stimuli, such as player's viewpoint and presentation of elements, over time. Gestalt organizational principles [START_REF] Wertheimer | Untersuchen zur lehre von der Gestalt [Laws of Organization in Perceptual Forms[END_REF] support one well known approach to perceptually characterize this kind of complexity. The principles include proximity, similarity, common fate, continuity, and closure. Regardless of the underlining goal, the principles only consider visual relationships between element groups and the background.

We focus on similarity, defined as the degree of similarity between target and nontarget elements sharing a common feature. Similarity is also chosen due to its application as a theory in attention [START_REF] Duncan | Visual search and stimulus similarity[END_REF], whereby a target search task decreases in efficiency and increases in reaction time as the non-targets become similar or dissimilar in appearance to targets. In this effort, we evaluate 4 experimental visual conditions according to the theory in a simple game called EMOS (Expressive MOtion Shooter). Only the presentation of target or non-target elements change, specifically speed, size, and density. Our research question is based on an analysis of players' play performance and self report between novice and expert players.

 RQ1: What design variables (speed, size, and density) applied to game target and non-target elements, tested in 4 different conditions (produced based on designers' choices), produce performance differences perceived and actual? Based on previous work, we know that expertise will have an effect on both performance measurements, thus we also look at expertise as a factor.

The previous work section introduces relevant work in visual attention and game user research methods. This is followed by our methodology including the experimental railed-shooter game developed, definition of visual conditions, and analysis procedure. Results are organized respective to each experimental condition.

Relevant Work

Many empirical approaches investigated visual design in games, such as dynamic lighting [START_REF] El-Nasr | Dynamic Intelligent Lighting for Directing Visual Attention in Interactive 3D Scenes[END_REF], color [START_REF] Niedenthal | Complicated Shadows: The Aesthetic Significance of Simulated Illumination in Digital Games[END_REF], and visual cues [START_REF] Hoeg | The Invisible Hand: Using Level Design Elements to Manipulate Player Choice[END_REF][START_REF] Milam | Analysis of Level Design "Push & Pull" within 21 games[END_REF][START_REF] Samarinas | Illuminating Minotaur's Lair: Light Design and Navigation in Gameworlds[END_REF], to influence emotion or performance. To our knowledge, with the exception of our work [START_REF] Bartram | Filtering and Brushing with Motion[END_REF], no one addressed multiple elements in motion within the context of a game. Within the field of visual perception, there is abundant research on the expressive properties of motion includ-ing speed, shape (the path a motion follows), phase (periodic motion), flicker (flashes), smoothness, and direction [START_REF] Bartram | Filtering and Brushing with Motion[END_REF][START_REF] Lockyer | Simple Motion Textures for Ambient Affect[END_REF]. For instance, many of these properties are found to influence affective impressions, such as valence, comfort, urgency, and intensity [START_REF] Lockyer | Simple Motion Textures for Ambient Affect[END_REF]. Another work evaluated the similarity of properties [START_REF] Kingstone | Perceptual grouping and motion coherence in visual search[END_REF], specifically direction of motion and density of targets, on visual search.. Results found that reaction time increases not only with a higher density of targets, but also due to specific patterns of motion between target and non-target elements. The limitation with this work is that it is not situated within the context of a game. For instance, the random ordering removes any coherent escalation of difficulty, lack of task reward, or visual feedback, which is common in a game. We addressed these limitations in EMOS.

Game user research is a growing network of industry and academic groups [START_REF] Isbister | Game Usability: Advice from the Experts for Advancing the Player Experience[END_REF][START_REF] Tullis | Measuring the User Experience: Collecting Analyzing, and Presenting Usability Metrics[END_REF] with the goal to improve the play experience. Methods, such as triangulation and heuristic evaluation, have been applied to identify performance breakdowns. Triangulation methods include multiple data sources, such as the player's performance [START_REF] Tullis | Measuring the User Experience: Collecting Analyzing, and Presenting Usability Metrics[END_REF][START_REF] Pagulayan | User Centered Design in Games[END_REF] and attitudinal questionnaires [START_REF] Erfani | The Effect of Previous Gaming Experience on Game Play Performance[END_REF][START_REF] Ijsselsteijn | The Game Experience Questionnaire: Development of a self-report measure to assess player experiences of digital games[END_REF]. The most common metrics include time on task, task success, errors, and learnability (performance change over time) [START_REF] Tullis | Measuring the User Experience: Collecting Analyzing, and Presenting Usability Metrics[END_REF]. Others applied heuristic evaluation [START_REF] Desurvire | User Experience Design for Inexperienced Gamers: GAP -Game Approachability Principles. Evaluating User Experiences in Games[END_REF] to improve games for inexperienced players. For instance, game approachability principles [START_REF] Desurvire | User Experience Design for Inexperienced Gamers: GAP -Game Approachability Principles. Evaluating User Experiences in Games[END_REF] incorporate the player's self-efficacy [START_REF] Klimmt | Effectance, self-efficacy, and the motivation to play video games[END_REF] beliefs since the same level of challenge can be interpreted positively or negatively depending on expertise. These methods allow designers to gain in depth understanding why players feel the way they do, so corrective action can be taken.

Methodology

EMOS Railed Shooter Game

We investigate the impact of visual features on users' performance in the EMOS railed shooter game, developed in the Unity™ engine. Only one button is used to point and shoot. Additionally, there is no punishment as levels auto advance after 11 or 13 seconds. Like the perceptual tests, EMOS is comprised of 6 game elements; 2 types of targets, 2 non-targets, and 2 types of visual feedback. Speed, size, and density variables attach to the targets and ambient non-targets. The boss targets (TB) restrict advancement through levels and the minions (TM) increase the score. The non-target elements include ambient (NTA) and background ring (NTR) distractor elements. The ambient non-targets are the same color as boss targets, but do not contribute to the score or advance levels. The non-target rings rotate and are only variable by speed. Their purpose is to convey an illusion of camera movement. Finally, visual feedback for each shot fired (NTS) and on target explosions (NTE) is adjustable by size.

The game elements are abstract since we did not want the art-style to inform a strategy. To this extent, we also used simple geometric shapes, monochromatic color, basic sound effects, and periodic motion trajectories to the elements. Figure 1 shows an example of the game and how elements can visually change. The left image has small targets and many small ambient non-targets, while the targets on the right are larger with fewer ambient non-targets. In these levels, the boss and minion targets and ambient non-targets move along a circular trajectory (yellow arrow).

Visual Conditions

In our previous work [START_REF] Milam | Toolset to explore visual motion designs in a video game[END_REF], 8 expert game designers manipulated the variables: speed, size, and density, associated to target and non-target elements, in a toolset to manipulate the design of EMOS. Designers identified suitable designs for novice, intermediate, and expert levels. We then developed patterns from these designs. The study described in [START_REF] Hart | Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research[END_REF] discusses in depth the methodology, the study and the results. For the study presented here we will assume that the patterns used to construct the 4 conditions for the experiment are valid designs confirmed by eight expert game designers. We thus describe the 4 conditions here. Condition 1 is designed for intermediate difficulty level, and conditions 2-4 are designed for expert difficulty level. To preserve ecological validity as a game (as this is how designers indicated the design should proceed), condition 1 is played first by all players, followed by a random assignment to one of the expert difficulty conditions.

Due to the amount of elements that could change using the toolset, all four conditions modify multiple elements once. This approach is different from traditional perception experiments, which typically change one variable at a time. However, these conditions are more in sync with how designers approach level design. Designers typically change many elements between play tests. Therefore, to produce valid design lessons, the conditions need to adhere to designers' philosophy of how they typically manipulate the designs. The conditions produced are:

 Condition 1: Increase T Density, Increase NT Speed: Increases the density of targets, speed of targets, ambient, and ring non-targets. Boss and minion target density increases from 1 to 3, and 3 to 5, respectively. Target size is 3.5 times larger than the ambient non-target size. Target and ambient non-target speed increase 14% and 19% respectively, and ring speed increase 52%.  Condition 2: Increase T Density and Increase in NTR Speed: Similar to condition 1 except target speed is held constant since fast speeds will be physically 1. Ambient (NTA) 2. Boss (TB) 3. Minion (TM) 4.Visual feedback (NTE) 5. Rings (NTR)

1.
1.

2.

3.

3.

4.

4.

5.

5.

harder to shoot for novice players. Boss and minion target density starts from 2 and 4 respectively and end with 3 and 7, respectively. Non-target ring speed is twice the speed in comparison to condition 1.

 Condition 3: Increase similarity between T NT Size by Decreasing T Size:

Target and ambient non-target size become more similar with all remaining values are held constant. Target size decreases 24% respective to condition 1 and is now only 12% larger in comparison to the ambient non-target size.  Condition 4: Increase NT Density, NTE Size, and NTS Size: Increases the ambient non-target density, visual feedback explosion and sparks size. In comparison to condition 1, the ambient density is 2.5 times higher, explosion size is 2 times larger, and sparks size is 28% larger.

Data Collection, Analysis, and Study Design

We collected 3 game metrics per level: level time (seconds), number of mouse clicks (#), and enemies shot (#). From these metrics, we generate the mean and rate of change, based on the slope of the linear regression line through data points in the known y axis (performance metric) and the known x axis (across 5 levels in each condition). Also, we administered two post-play surveys: 1) the task load index as a measure cognitive workload [START_REF] Hart | Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research[END_REF], and 2) five questions gauging gaming habits [START_REF] Erfani | The Effect of Previous Gaming Experience on Game Play Performance[END_REF] as a basis for expertise. The workload survey contains questions regarding: mental demand (defined as following the target and ignoring distractions), physical demand (moving and clicking the mouse), temporal demand (feeling hurried or rushed), effort, success, and frustration. Each question is rated on a 7-point Likert scale (1=lowest and 7=highest). A response to an expertise questionnaire was also collected through surveys at the end so as not to bias participants [START_REF] Boot | Do action video games improve perception and cognition[END_REF]. The questionnaire gauged expertise based on the types and time spent playing games on a weekly basis. Players are novice if they play less than the average, prefer casual games, and non 3D genres.

Results

All analysis are within conditions, where expertise is evaluated in relationship to the independent variables (metrics and self report) using T-tests. Due to space limitations, we present values for statistically significant results only. The 105 participants include 48 females, 57 males, 55 experts, and 55 novices with an average age of 22.

 Condition 1: Increase T Density, Increase NT Speed: Mental, temporal, and effort are rated higher for the novice player in comparison to experts (p < .001, .004, and 002 respectively). Conversely success is perceived at a higher rate (p < .019) for expert players (mean rating is 5.4 vs. 4.9). Regarding performance, experts completed levels on average 28% faster with a lower rate of change (p < .001 and .007). Experts also fired 19% fewer shots with a lower rate of change (p < .004 and .005). As expected, experts perceived higher success in comparison to novices, and novices perceived higher cognitive load than experts. These perceptions suggest that even the intermediate difficulty was not as easy as we thought and may be problematic for novices. This is an important baseline.  Condition 2: Increase T Density and Increase in NTR Speed: 20 experts and 15 novices. The success rating was perceived higher for experts (p < 0.029) in comparison to novices (mean 4.1 and 3.5 respectively), even though no significant change in performance was found within the condition. Thus, even though novices demonstrated improved performance from condition 1, and their performance was comparable to experts, they did not perceive high success rates in comparison to experts. In comparison to condition 1, all subjects improved performance, increased the enemies shot rate, while the level time and shots fired rate decreased.  Condition 3: Increase similarity between T NT Size by Decreasing T Size: of 19 experts and 16 novices. No difference in self report ratings were found, even though on average experts finished levels 16% faster and shot 29% more enemies (p < 0.004 for both) in comparison to novices. In comparison to condition 1, all mean performance values increased, except for the enemies shot, which decreased for novices and remained constant for the expert players.  Condition 4: Increase NT Density, NTE Size, and NTS Size: 16 experts and 19 novices. Success is perceived again higher for the expert (p < 0.005) in comparison to the novice player (mean 5.5 and 4.2 respectively) even though no significant change in performance was found within the condition. One possible explanation is that the large visual feedback size was perceived to be a higher reward even though the rules never change.

Discussion

Supported by a theory of stimuli similarity [START_REF] Duncan | Visual search and stimulus similarity[END_REF] in an experimental game, these findings are based on a definition of visual features attached to target and non-target, elements. In regard to the research question, we found that most differences in self reports and performance occur in the intermediate difficulty condition 1, rather than the more difficult conditions 2-4. This finding not only underscores the importance of training and preparation, but also sets a baseline in the perception of performance for the difficult levels that follow. Within conditions 2 and 4, novices demonstrated improved performance on par with expert players, yet report less success in comparison to experts. For novice players, feelings of success are dampened by the higher mental, temporal, and effort ratings in condition 1. In contrast, expert players report high success as a response to increased density of targets in condition 2, or the increased nontarget visual feedback explosion size in condition 4. Within condition 3 by contrast, performance differences are found, yet no difference in self report ratings are found.

Conclusion and Future Work

This work investigated intermediate and complex visual designs informed by the similarity theory of visual attention, in an experimental railed shooter game. Our analysis found effects of expertise on performance and self report of performance in the intermediate difficulty condition. The same analysis in the complex visual settings found consistent disagreement in performance and perception of performance. Expert players report higher success in response to an increased density of targets or size of visual feedback, even though no performance differences are found. By contrast, no self report differences are found in the condition with an actual performance difference. Our contribution lies in the discussion of these results as well as the methods used to uncover them. Our future work incorporates pupillometry (pupil size) metrics as a continuous physiological metric and identifies additional implications on design.

7.

Figure 1 :

 1 Figure 1: Adjustable visual balance (top and bottom)

Acknowledgements

This research is supported by the Natural Sciences and Engineering Research Council of Canada and Graphics, Animation, and New Media Network (GRAND).

8.


Damian Lilla
email: lilla@tzi.de

Marc Herrlich

Rainer Malaka
email: malaka@tzi.de

Dennis Krannich
email: krannich@tzi.de

The Influence of Music on Player Performance in Exergames for Parkinson's Patients

Keywords: exergames, auditory cues, kinect, Parkinson's disease

Music therapy and music and rhythm in general can support standard physiotherapy for people suffering from Parkinson's disease to improve the motion performance and quality, sometimes even helping to overcome motion blocks. With the availability of cheap motion-tracking devices, exergames have become an interesting option to complement traditional physiotherapy. However, the role of music and rhythm in the context of games for this special audience is still largely unexplored. Based on a prototype exergame we developed, a user study was conducted to compare the effects of different auditory clues and their absence in exergames for this target group. The results show significant performance differences with music versus without music, but surprisingly no differences were found between music synchronized with the interaction and unsynchronized background music.

Introduction

The most common occurring symptoms of Parkinson's disease (PD) are reduced movement in amplitude and speed, problems in initiating movement, freezing, postural control difficulties, resting tremor or rigidity [START_REF] Morris | Movement disorders in people with parkinson disease: A model for physical therapy[END_REF]. Physical therapy (PT) can slow down the progression of PD [START_REF] Suteerawattananon | Effects of visual and auditory cues on gait in individuals with parkinson's disease[END_REF], but due to its repetitive nature, it can quickly bore and tire patients [START_REF] Yu | An interactive multimedia system for parkinson's patient rehabilitation[END_REF].

Music therapy (MT) can greatly enrich traditional PT and help to improve the precision, range, speed and quality of upper body movement [START_REF] Bernatzky | Stimulating music increases motor coordination in patients afflicted with morbus parkinson[END_REF]. By instructing PD patients to step to the beat of music, significant improvements for stride symmetry, length, gait velocity and cadence were achieved [START_REF] Thaut | The connection between rhythmicity and brain function[END_REF].

Exertion games (exergames) can motivate and immerse patients to facilitate repetitions [START_REF] Flynn | Feasibility of using the sony playstation 2 gaming platform for an individual poststroke: a case report[END_REF] which are crucial in order to rebuild functional abilities [START_REF] Rand | Virtual reality rehabilitation for all: Vivid GX versus sony PlayStation II EyeToy[END_REF].

To utilize the advantages of auditory cues and exergames, we developed a prototype which blends both approaches into one system. We hypothesize that the auditory cues will have a performance enhancing effect in this game-like environment. To validate this hypothesis we contrast the performance of three groups of patients with PD. In the first group, the music's rhythm is directly connected to the timing of the movement. In the second, the music does not correlate with the movement. In the third, no music is used during gameplay. Our results indicate that music can, in fact, have a performance enhancing effect when used in a game-like environment. In our particular setup it did not matter whether the music had a clear, easily followed beat or a more fluid, calming tempo.

Related Work

Assad et al. developed a collection of mini-games specifically for PD patients in order to train memory as well as upper and lower extremities. Within this collection the game "Cinderella" makes use of auditory cues in order to motivate and guide the patient's pace of movement [START_REF] Assad | Motion-based games for parkinson's disease patients[END_REF]. However, these auditory cues are not directly connected to the timing of the motion. A system for PD patients by Yu et al. lets the user interact with the system by wearing a motion suit and also makes use of auditory cues in order to help the patients to pace their movements [START_REF] Yu | An interactive multimedia system for parkinson's patient rehabilitation[END_REF]. However, the timing of these movements is not directly synchronized with the beat of music. Contrary to both systems, ours strongly connects the beat of the music with the timing of the movement in the attempt to help train its temporal precision as well as to reduce motion blocks.

The Mole Game Prototype

While standing in front of the camera, the player's hands are tracked by the camera and represented as cartoon hands on the gameplay screen (cf. left side of figure 1). The game's goal is to grab a worm and feed the moles appearing from the holes repetitively. The earlier the player hits the appearing moles the higher the score, ranging from 100% to 0%. Visual cues, small helmets peaking out of the holes, act as a hint for the player where the next mole will appear.

Evaluation

We conducted nine meetings over the duration of three months at three places where Parkinson's disease patients meet for physiotherapy or socializing. Every meeting lasted for approximately 1 to 2,5 hours.

Procedure

We tested for differences between the groups' performances by logging their scores. Additionally, we videotaped the motion of the participants. They were randomly assigned by us into one of three groups. The participants played a training round which had no music, followed by three gameplay rounds with the same movement patterns and with the specific background song settings for their group. Each round lasted one minute and 14 seconds. The choreography for all three rounds was the same and consisted of simple one and two-handed reaching tasks (cf. right side of fig. 1). Group one's beat of the music exactly matched the timing when a mole appeared and should therefore have helped with the temporal precision of the movement. We did not, however, explicitly instruct the participants in both music groups to use the auditory cues to pace their motion.

Results

A total of 24 participants with a mean age of 70.4±6.9 took part in the evaluation. 62,5% were males and 37,5% were females. All three groups had the same proportion of males [START_REF] Thaut | The connection between rhythmicity and brain function[END_REF] to females [START_REF] Yu | An interactive multimedia system for parkinson's patient rehabilitation[END_REF].

We conducted Kruskal-Wallis tests in order to reveal significant differences in the performances among the groups. In round one (H(2) = 2,885, p=0,236) as well as round three (H(2) = 4,56 p=0,102) no significant differences among the groups could be found, whereas the test revealed significant differences among the groups in round two (H(2) = 6,5 p=0,039). In this round, the participants of group one achieved an average score of 77,59%±22,84. Slightly lower was the score of the participants of group two with a mean score of 76,72%±10,92. Lastly, by nearly 20% less compared to the previous groups, the control group achieved a mean score of 57,18%±20,50.

Afterwards, we conducted pairwise Mann-Whitney U post-hoc tests in order to evaluate which pairs of groups had significant differences in round two. The difference between group one and the control group was significant (Mann-Whitney U=10,0, n1=n2=8, p=0,021 two-tailed). By analyzing the differences between group two and the control group, we found a tendency (Mann-Whitney U=14,0, n1=n2=8, p=0,059 two-tailed) for group two to score higher than the control group. Lastly, the differences in scores between the two groups with music were not significant.

Exclusively based on the observations, we could not identify any participants from group one rocking themselves or noticeably nodding their heads to the beat of the music.

Discussion

Our results show that music indeed effects player performance positively, even if the players are not explicitly instructed to regard the auditory clues. In contrast to our assumptions, the participants did not follow the specific beat in group one. We conclude that explicitly instructing players about the synchronization of the beat and the interaction might further increase performance.

Conclusion and Future Work

We have presented first experimental findings that music can improve the performance of PD patients in a game-like environment. For a follow up study we particularly recommend to explore if PD patients would time their movements with the help of auditory cues when explicitly instructed to do so. Further research could also discover the other attributes of music that could have positively affected performance in gameplay and why.

Fig. 1 .

 1 Fig. 1. Gameplay screen and an extract of movements for the mole game.

Acknowledgements

We thank the "Deutsche Parkinson Vereinigung", particularly the physiotherapists Katharina von Sauken, Rieke Tischkewitz, Brigitte Kloker, Marlis Böger and Renate Stöver. We further want to thank all the participants who patiently and with great effort took part in this study.


Ali Mazalek
email: mazalek@gatech.edu

Michael Nitsche
email: michael.nitsche@gatech.edu

Claudia Rébola

Paul Clifton

Andy Wu
email: andywu@gatech.edu

Nick Poirier
email: nick.poirier@gatech.edu

Firaz Peer
email: firazpeer@gatech.edu

Pictures at an Exhibition: Design of a Hybrid Puppetry Performance Piece

Keywords: physical/digital puppetry, performance, experimental theatre, tangible interaction, virtual space

Pictures at an Exhibition is a physical/digital puppetry piece that uses tangible interface puppets to modify a virtual scene projected at the back of the stage in real-time. The piece merges traditional puppeteering practices with tangible interaction technologies and virtual environments to create a novel performance for the live stage. This paper describes the design and development of piece, as well as our lessons learned from this process and from on-stage performances of Pictures at an Exhibition in a puppetry theatre.

Introduction

Digital technologies are increasingly shaping novel performance practices [START_REF] Dixon | Digital Performance[END_REF] [START_REF] Giannachi | Virtual Theatres: An Introduction[END_REF]. Hybrid performance formats are found in traditional theatrical settings, as well as in film and video games. In theatre, digital media are used to augment performances, e.g. in the form multimedia projections, in a manner similar to traditional visual effects. Artists and researchers have also explored digital media in combination with real-time computation in order to create interactive experiences for audiences or performers. In film and television, the use of digital puppetry and motion capture technologies is a widespread production process. More recently, the emerging machinima art form uses real-time game engines as platforms for digital puppetry in the creation of short animated films.

Building on these ideas, our work explores the merger of traditional puppetry with tangible interfaces and virtual environments. In Pictures at an Exhibition, five physical puppets equipped with sensors communicate data about their movements to a virtual scene projected at the back of the stage, which changes in response (see Fig. 1). The piece was inspired by Zbig Rybczynski's Tango, which uses 36 overlaying loops to build a video performance. In order to create Tango, Rybczynski had to draw and paint roughly 16,000 cell-mattes and make several hundred thousand exposures on an optical printer, a lengthy process that took seven months [START_REF] Rybczynski | Looking to the Future: Imagining the Truth. Cinema & Architecture[END_REF]. Instead of looping the recorded image, our piece captures animation data from puppets during the live performance. This data directly affects the virtual scene but is also looped and layered much like the actions in Tango. However, recording and looping happens in real-time. The virtual space becomes a lasting reflection of the story unfolding in the physical space. The abstract mapping between puppets and virtual scene addresses key questions of digital performance that deal with the relationship of the body to the digital realm in hybrid performance pieces. Pictures at an Exhibition was created through a process of physical/digital co-design and was realized as a joint project across classes on experimental media, industrial design, and expressive virtual space. In this context, the conceptual, physical, and digital aspects of the piece co-evolved. The design and development of each aspect-story and performance, physical puppet interfaces, virtual environment-thus informed and altered the design of the other two. The process concluded with the performance of the piece on stage as part of the Xperimental Puppetry Theater show at the Center for Puppetry Arts in Atlanta, GA. The project enabled us to better understand the nature of hybrid performances that bridge the physical and digital worlds, and how to foster them in an interdisciplinary collaboration between artists and researchers.

First, this paper looks at related work from traditional and experimental puppetry. Next, we provide an overview of Pictures at an Exhibition and describe the design process across the physical, digital, and performance aspects of the piece. We discuss experiences and observations from the performances and conclude with lessons learned about the design of hybrid performance pieces for the live stage.

Related Work

Our work is inspired and informed by traditional puppetry and by digital technologies in performance arts, especially virtual environments and games. We offer an overview of related work in these areas.

Traditional Puppetry

As we have discussed elsewhere, there is a roughly inversely proportional relationship between ease of use and expressive potential with respect to traditional puppetry techniques [START_REF] Mazalek | Recognizing Self in Puppet Controlled Virtual Avatars[END_REF]. Marionettes, which use strings to control individual parts of the puppet, can lead to a very fine degree of control over the puppet. However, each body part requires an additional string, and each string adds a degree of complexity to the puppet's control. At times, the control is shared. In the Bunraku tradition, three puppeteers control a single puppet. The main puppeteer controls the right arm and the head, while the other two control the left arm of the puppet and its legs. Each puppeteer has a fine degree of control over one or two body parts, which leads to an extremely expressive but also complex performance. Mastering these formats often takes years of practice.

Hand and rod puppets are more limited in their manipulation. However, as Jim Henson's Muppets have shown, in the hands of a skilled puppeteer they can be very expressive. Hand and rod puppets are controlled using one hand inside the puppet and the other hand to move a rod attached to one of the puppet's hands. While hand and rod puppets do not provide a fine degree of control of each individual part of the puppet's body, they do provide a balance between ease of use and expressive range. They can gesture using their arms and can direct their gaze at the important parts of a scene. Certain features, like the eyes, are simplified and exaggerated. Pictures at and Exhibition uses hand and rod puppets designed as described below.

Digital Technologies in Theatre Performance

Pinhanez describes theatre practices that actively use computers during live performance while maintaining the dynamics of live theatrical performance [START_REF] Pinhanez | Computer Theater[END_REF]. This includes human-controlled digital lighting and scenery, as well as artificially intelligent actors sharing the stage with a human actor. For example, in Mark Reaney's production The Adding Machine, off-stage computers control 3D projected props and scenery that users at computer terminals move to simulate movement of the live actors on stage. This literally shifts the perspective of the audience. Off-stage actors are also projected in 3D. The size and angle of the projection changes as they interact with the actors performing on the stage [START_REF] Reaney | Virtual Scenography: The Actor/Audience/Computer Interface[END_REF]. Computation in The Adding Machine is used to frame the viewers' experience in a way that would not be possible otherwise. In comparison, It/I creates a digital character that reacts to the performance of the actor and to audience participation. The digital character communicates with words, pictures, videos, sounds, and stage lighting [START_REF] Pinhanez | It / I: A Theater Play Featuring an Autonomous Computer Character[END_REF].

In puppetry, digital technology has been used primarily for stage effects. Meridiano Theatre's Genesis uses projections to move the scene from underwater to outer space, and projection mapping to highlight certain props with changing surface textures [START_REF] Ravicchio | Genesis. Meridiano Theatre[END_REF]. In Terrapin Puppet Theatre's The Falling Room and the Flying Room, a projected television screen acts as a character in the performance. Animated versions of the puppets and actors move back and forth from the animated world to the real world. While the animated events are prerendered, careful staging and skilled puppeteering make the interactions look seamless [16]. Digital technologies also allow new forms of audience participation. Naked Puppets' Dark Earth uses technology to enable the audience to affect the scenery, soundscape, and imagery of the performance [START_REF]Naked Puppets: Dark Earth[END_REF]. In both theatre and puppetry performances, computer vision and projection mapping have been the tools of choice for facilitating interaction between performers, puppets, and digital content. Embedded sensor technology, though, remains undeveloped in live theatre, with few examples of puppets that sense their own movement, like we implemented in Pictures at an Exhibition.

Digital Puppetry and Virtual Environments

Ever since users gained control over interactive digital characters, this interaction has been characterized as a form of dramatic performance [START_REF] Laurel | Computers as Theatre[END_REF] and digital puppetry [START_REF] Hayes-Roth | Improvisational Puppets, Actors, and Avatars[END_REF]. A player character's actions usually depend completely on a user's input -much like that of a physical puppet. Calvillo-Gamez and Cairns draw a parallel between users learning game controls and puppeteers learning the functionalities of a puppet [START_REF] Calvillo-Gamez | Pulling the strings: A theory of puppetry for the gaming experience[END_REF]. But due to the standardized nature of video game systems, the puppet analogy also has deficiencies.

One problem is mapping the limited control schemes of existing interfaces onto the near limitless range of expression. The hardware limits the input to certain set parameters. Keyboard, mouse, joystick, gamepad, motion-control device, etc. each have their own restrictions. In traditional puppetry, each puppet features a control scheme customized for its particular construction. Since games depend on standard input devices, such fine-tuning is difficult. Mappings can be adjusted, as players can set their own control preferences, but remain limited by the hardware. Another challenge is the principle difference in design between games and performances. Video games are defined as goal-driven systems that encourage players to fulfill a certain objective [START_REF] Juul | Half-Real: Video Games between Real Rules and Fictional Worlds[END_REF]. They emphasize the notion of interaction as making a "choice" [START_REF] Salen | Game Design and Meaningful Play[END_REF] that has certain consequences in the game world. Due to the limitations of interaction possible in these worlds, these choices are often simplified (shoot the enemy or not, help the thief or the victim). In contrast, a theatrical performance consists not only of the action a character does, but also how any activity is performed. Due to the dominance of either pre-produced motion captured animations or procedural animation systems such fine-tuned control remains largely inaccessible to users.

Cases of successful digital puppetry grow in emergent play formats, when players either modify or repurpose game elements to achieve a level of expression that was not envisioned by the game designer. For example, fighting moves might be re-interpreted as dance for a machinima music video (see Bainst's Dance Voldo Dance). More open systems, like Second Life, allow creation of new animation cycles that can be used by virtual performers like the Ballet Pixelle [1] to stage choreographies with multiple dancers steered in real time by their users. However, their character control is achieved through elaborately timed animation activation. The players have no direct control over individual body parts of their dancing avatars.

Few titles allow for such a direct control over limbs, head, and expression through the interface. Media Molecule's Little Big Planet (2008) features typical jump'n run gameplay but allows players direct control over some facial animations and body parts of the virtual puppet. These expressions have no effect on achieving the main game objective but illustrate the focus on player involvement that is a key component of Little Big Planet. The game includes elaborate creation tools for players to generate game levels and customize their characters. It highlights a shift away from the pre-fabricated goal driven game and exemplifies the expressive range of digital puppetry. As it stands, this shift in commercial video games most often still depends on the inventive use of the underlying platform by the player and is rarely actively supported by the original producer.

Pictures at an Exhibition

Pictures at an Exhibition is a six-minute physical/digital puppetry piece that tells the story of a heist in an art gallery in seven scenes. The cast features five characters: security guard, cat, old lady, small boy, and gangster, each represented by a tangible interface puppet controlled by a single puppeteer. The piece is set in one room of the art gallery, where the old lady, small boy, and gangster conspire to steal a painting (represented on stage by a physical prop), while the security guard and cat try to foil their efforts. Another artwork is projected at the back of the gallery room and acts as a virtual mirror of the characters' actions, reflecting them in abstract animations on its surface, which are looped and layered over time. The piece is a comic reflection on the way the stories of visitors to an exhibition become part of the exhibited paintings themselves. Inspired by the structure of Rybczynski's Tango, the performance unfolds as follows:

-Scene 1 : a performer (security guard puppeteer) enters stage and tells a story with their puppet. This puppetry leaves traces on the artwork in the background. Once this section of the story is completed, the performer and puppet leave the stage, but the animations repeat in an endless loop on the artwork at the back of the stage. -Scene 2 : another performer (cat puppeteer) enters and their actions with the puppet animate another part of the virtual canvas. They are soon followed by the first performer (security guard puppeteer), whose new actions are reflected onto the painting and layer with those of cat. As the scene ends, the performers leave the stage but their past actions continue to loop on the virtual canvas.

-Scenes 3-5 : These scenes repeat the pattern, each bringing a different group of characters onto the stage as the story builds up to its climax. At the end of scene 5, the painting prop is stolen and all characters leave the scene while the artwork reflects the frenzy of actions. -Scenes 6-7 : As the story resolves itself with a comic twist involving the whole cast of characters, the layered animations on the painting build up to reveal the fully assembled image as the curtains close.

The story is conveyed through the puppet performance, which is set to a specially composed sound track that uses a different instrument for each character and helps to pace the performance. All puppets are custom-designed variations on hand and rod puppets with sensors at key points of articulation to capture expressive actions. The sensors in each puppet are connected to an Arduino and XBee, which transmit data to a computer that controls the virtual space.

Design Process and Performance

The piece was designed as part of a joint project across three research and studio classes on experimental media, industrial design, and expressive virtual space at Georgia Tech in spring 2011. One of our goals in producing the piece was to explore, engage in, and ultimately better understand how an interdisciplinary physical/digital co-design process can be applied to the creation of hybrid performance pieces that use real-time computational media on the live stage. Crucial to this is the seamless integration of the physical performance and the digital media on stage. Tangible interaction and puppetry are especially suited for this, since the puppets themselves can serve as an interface between the physical and digital worlds. The performers tell the story through the movements they make with their puppets, while the puppets capture real-time data of these actions and communicate them to the virtual space. The puppets can also serve as reflections of the captured data, for example through LED lights embedded in their bodies.

The project began with an initial concept of the structure and story for the piece. The realization of this concept unfolded across the three classes in a tightly coordinated manner. The progress in each class informed the work in the other two, and also evolved and altered the structure and story to its final state. Students in the three classes worked in teams to develop the different aspects of the piece. In the industrial design course, students did studies of form, materials, and movements, in order to inform the conceptual and physical design of the puppets. In the experimental media course, students explored physical interactions and sensor technologies, in order to realize the design, prototyping, and development of the puppet interfaces. In the expressive virtual space course, students explored the structure and animation potential of visual artworks, in order to create real-time animated versions that could reflect the story of the puppets into the virtual space.

The co-design process required sharing work-in-progress material across all three classes, including design sketches, renderings, technical specifications, and prototypes. E.g. an OSC-based communication protocol between the physical interfaces and virtual artwork was developed and refined as the piece evolved. Over the course of the semester, students from each class presented deliverables in the other two classes at key stages in the design process. Subsequent deliverables in those classes built on the ideas and artifacts presented: industrial design students built on the conceptual puppet designs from the experimental media students and presented additional design explorations back. Experimental media students used their output to revise their concepts and develop prototypes of the puppet interfaces. These prototypes were presented to the expressive virtual space students, in turn used them to inform the design of the virtual artwork.

The Xperimental Puppetry Theatre (XPT) show at the Center for Puppetry Arts in Atlanta accepted our piece. So, in addition to our class collaboration, we also worked with professional puppeteers involved in XPT. These puppeteers provided feedback on the puppet designs, story, and staging of the piece throughout the design process, and helped to coach the performers. One of the puppeteers was a composer and created the music for the piece.

Puppets and Story

The experimental media class was divided into five teams of 3-4 students. Each team was assigned one of the five story characters and tasked with building its corresponding puppet interface. The creation of the puppets and the script evolved in iterations, from concept designs, through two rounds of prototypes, to final products. The two industrial design students worked as a team, producing design concepts and prototypes, which were informed by and inspired the work of the experimental media students.

Phase 1: Concept Designs. Experimental media students first generated conceptual designs for their puppets. The group visited the workshops, stage, and museum at the Center for Puppetry Arts to draw inspiration from traditional puppetry. As they developed their designs, each team was encouraged to consider:

-How their characters personality could be reflected in the puppets physical design and expressive movements; -How their puppets actions could be captured with sensing technologies and in turn reflected within its body; -How their characters relationship with other characters could be conveyed through interactions with them; and -What kinds of story scenarios might unfold between the characters in the art gallery setting.

This conceptual design exercise yielded puppets with a lot of complexity in their mechanical design and operation, and a lot of detail in their physical appearance. The designs supported many different interactions between characters, and thus many possible relationships and story scenarios. The next task was to constrain and significantly simplify these designs. The industrial design students worked from the initial design concepts, distilling each character's personality to primary emotion. They explored how these emotions could be conveyed through abstractions of form and movement, and built a cardboard prototype that embodied the five characters as geometric shapes that were integrated into the space of the stage and interacted with one another through simple but evocative movements (see Fig. 2). This prototype was presented to the experimental media students, and encouraged them to radically re-think the nature of character and story in stage performance. Inspired by the work of the industrial designers, the experimental media students then revisited their initial concepts in order to represent each character as a simple geometric shape that communicated its essential qualities: the short stocky gangster was represented by a square, the broad-shouldered security guard was represented by an upside down triangle, the active chubby small boy was represented by a circle, the little old lady in a long dress was represented by a triangle, and the sleek cat was represented by a sideways oval.

Phase 2: Prototyping. Following the initial concept design phase, the experimental media and industrial design students went through two rounds of prototyping. The first prototype focused on the physical design of the puppets, including form, appearance, movement, control, and sensing. Each puppet was designed with sensors (accelerometers, bend sensors, and rotary potentiometers) at key points of articulation, which were connected to an Arduino microcontroller board. Puppets were also equipped with LEDs that highlighted key physical features. E.g., the cat could arch its back and its LED eyes would go from blue to red. The arching movement was detected by two accelerometers placed in the forward and rear segments of the cat's body. The second prototype refined the physical designs, and served as a step in wirelessly communicating puppet data to a computer via XBee, where it was visualized using 2D avatars created in Processing. A standardized XML format was developed to describe the moving parts and data value ranges for each puppet in order to enable calibration and mapping of the sensor data to the virtual artwork running in Unity.

Fig. 3. Final gangster puppet (left) and electronics housed inside the gangster's coat (right). The gangster used a bend sensor to detect movement in his rod-controlled elbow, a force sensing resistor (FSR) to detect contact of his cigar with his mouth, and two 3-axis accelerometers (one in the head, on in the body) to detect head and body movements.

Phase 3: Final Puppets. Building on the two rounds of prototyping, the industrial design students did studies and design exercises with different materials in order to improve the production quality, consistency, robustness, and performance controls of the final puppets (see Fig. 3). For example, they refined the production of the cat's eyes (using halved ping pong balls) in order make the LEDs more visible on stage, and they added volume to the old lady's dress so that she would better retain her shape while performing.

Story Script. As the puppets and their personalities took shape, so too did the story and dramaturgy. After the first prototypes were built, a story workshop was held with the experimental media students and professional puppeteers. Different story scenarios were storyboarded and eventually turned into a precisely timed script, which was shared with the expressive virtual space students, who used it to refine their designs for the virtual artwork. The script continued to evolve throughout the remainder of the design process, until the puppets, the virtual artwork, and the music were all finalized and the piece was in the final stages of rehearsal.

Virtual Artwork

The expressive virtual space class was divided into 4 teams of 3-4 students. Their task was to design and animate a virtual environment based on an existing artwork. They remained updated on the development of storyline and characters in the parallel course throughout the process.

Phase 1: Selection and Design. Students had to select an existing artwork to adapt for the virtual canvas. They explored materials and art historical references to envision interaction and animation designs that would reflect its specific qualities. The goal was interactive re-interpretation rather than reproduction. Four reference pieces were selected: French's Summer (1978), Kandisky's On White II (1923), Warhol's Silver Clouds (1966), and van Gogh's Starry Night over the River Rhone (1888); all created in different periods, using different techniques, and engaging audiences in different ways.

For each artwork, the mappings from expressive movements of the puppets to animations on the artwork were done in different ways. Each piece featured some specific quality that was highlighted in the interaction design. E.g., the design for Summer, a virtual stained glass window, referenced glass material in the animations (light reflections, melting, shattering, and bending glass based on external input). On White II, ultimately chosen for the performance, referenced the artist's abstract geometrical art. It does not claim to truly incorporate Kandinsky's complex approach, but uses its distinct components, breaks them into individual 3D objects (see Fig. 4), and maps them onto different characters. The designers looked for features in the painting that would reflect the story characters and their puppets, and the animations follow the lines of movement in the artwork in a way that relates to the key actions made by the puppets. For example, the black cat puppet who leaps and bounds across stage in the performance is reflected in the leaps of the black diagonal segments that cross in the center of the painting, while the gangster smoking his cigar creates bubbles that emerge from the smoke-like curves in the upper left.

Phase 2: Implementation. All four teams created 3D models of their pieces in Maya and imported them into Unity as real time rendered environments. Then they coded the interaction design. Changes in the digital artworks were planned in correlation to the storyline as it evolved from the parallel class.

The most challenging part was connecting the puppet data to the virtual artwork. Seamless communication and clear mappings between the two had to be established. Each puppet had different sensors, which changed throughout the design process. The mapping to animations in the virtual Kandinsky had to cater for these changes. Even during the rehearsal stage, this mapping was adjusted in collaboration with the puppeteers and puppet designers. Unity also had to record the animations in real-time and provide instant playback. This was achieved via a database that recorded all input from the puppets and fed it back into the render engine when activated by an operator. This system allowed the operator to orchestrate a smaller version of layered animation as seen in Rybczynski's Tango in real-time. We also included pre-set animations that reflected the intended effects. This allowed us to test the artwork while the puppets were still being built. It also served as back-up in case the connection to the puppets was lost.

Live Performance

The XPT show (see Fig. 5) consisted of five performances preceded by five nights of dress rehearsals. Overall, the system worked solidly during all rehearsals and dress rehearsals. However, some initially small inconsistencies observed in the the connection between puppets and virtual artwork more prominent after the first two show performances. This forced us to sidestep the live-recorded animation playback during two of the performances and employ our back-up plan of playing pre-recorded animations on the virtual Kandinsky painting.

Despite the technical difficulties we experienced, the piece received laughter from the audience at the right moments (e.g. when the old lady hit the boy with her cane) and much applause. The feedback received from audience members after each performance was positive, and many people approached us to say how much they enjoyed the piece. Nobody we spoke with had experience of real-time computation used in performances, and many were intrigued by the idea and interested to see more similar works. While everyone perceived the connection between the puppets and the virtual artwork in our piece, relatively few understood how the different characters' stories played out on the virtual canvas. This indicated that the mappings between the puppets' actions on stage and the virtual artwork were too abstract, and would have benefited from simplification.

Discussion and Conclusions

We provide here a brief summary and discussion of lessons learned from the design and performance of the piece.

Communication. One of the most important aspects of design of a hybrid performance piece is effective and timely communication between members of the interdisciplinary team that is realizing the physical and virtual aspects of the piece. The communication between the industrial design and experimental media students worked well at the first, but became more challenging as the prototypes progressed. Since much of the ideation took place separately, as the designs progressed, students from the two classes were not always in line with their refinements and design iterations. In the same way, communication between experimental media and expressive virtual space students met with challenges throughout the process. Most significantly, the expressive virtual space class would have benefited from having a complete set of captured data earlier in the design process, however this was impossible for the experimental media class to deliver as the piece and technologies were continually evolving. Overall, a more tightly integrated design process in which all three classes participated in all design sessions together would have been beneficial, but this was not feasible due to institutional constraints, such as class scheduling and coordination across different units.

Mapping. Hybrid performance pieces need to consider how the physical world on stage and the projected virtual world interact. We aimed for a balance between the two. However, the mappings were not obvious enough to the audience, and so the action on stage took precedence over the changes happening to the virtual canvas. In retrospect, this could to some extent have been avoided if we had been able to incorporate the virtual artwork into the rehearsals earlier on. But due to timing constraints and technical challenges, the real-time communication between the puppet interfaces and the artwork was not in place early enough for the rehearsals to happen with the virtual canvas running. A lot of rehearsing/coaching with the professional puppeteers thus focused on the physical performance, and the changes in the artwork did not seem to be designed into the story action from the outset. In the future, it would be worthwhile to explore how to mock up the virtual scene and rehearse with it early in the design process even as the puppet designs, technologies, and storyline continue to evolve.

Balance. Hybrid performances need to consider how to balance between action on the physical stage and action in the virtual space. There is no one correct way to realize this balance since it depends on the story and director's vision. However, it is difficult for the audience to focus in many places at once, so a hybrid piece should give the audience time to take in and switch between action on the physical stage vs. on a projection screen. In our case, replaying animations on the virtual canvas after each scene gave the audience time to process and relate the virtual performance to the physical one. This was especially important given the fast pace of the actions unfolding on stage.

Control. The physical actions made by puppeteers when their focus is on controlling a virtual entity are different than when their focus is generating a physical performance. Our puppeteers performed actions regardless of how they were unfolding on the virtual canvas. Even when communication lagged between the puppets and virtual artwork, the puppeteers maintained their focus on the physical performance. Shifting their focus to control of the virtual, e.g. to make an adjustment or accommodate latency, would have been at the expense of expressivity and narrative communication of the physical puppets. In this sense, a physical performance with a physical controller can be at odds with a virtual performance with a physical controller. This poses a challenge for the creation of hybrid pieces that blend both physical and virtual performances.

Error-Handling. Hybrid performance practices need to develop methods for real-time error handling that do not disrupt the performance in progress. Technological devices and networks rarely work flawlessly. However, common errorhandling methods usually involve pausing the normal workflow in order to address the error (e.g. respond to an error dialog, restart the application, restart the machine, etc.). This is not an option in a live performance, where any disruption will break the fourth wall for the audience. Having a backup plan is important, however the decision to use it needs to be made in a timely manner, before the performers are on stage. Solutions enabling the seamless transition to a backup plan in mid performance could be worth exploring in future hybrid performance works.

Fig. 1 .

 1 Fig. 1. Performance of Pictures at an Exhibition during the XPT show at the Center for Puppetry Arts.

Fig. 2 .

 2 Fig. 2. First design concept for the gangster puppet with complicated control mechanisms (left) compared with the heavily abstracted puppet designs (right).

Fig. 4 .

 4 Fig. 4. The virtual artwork -a 3D digital recreation of Kandinsky's On White II. Complete front view (left) and animated portion (right).

Fig. 5 .

 5 Fig. 5. Performers manipulate the puppets on stage during the XPT performance of Pictures at an Exhibition. The puppets' actions are mapped to the virtual painting in real time.

Acknowledgments

We thank students in our spring 2011 classes at Georgia Tech, puppeteer collaborators Lee Bryan, Dolph Amick, Mauree Culberson and Lynn Talley, XPT director Michael Haverty, the Center for Puppetry Arts, and BlinkM.


Konstantinos Mikalef
email: kostas.mikalef@gmail.com

Michail N Giannakos
email: mgiannak@idi.ntnu.no

Konstantinos Chorianopoulos

Letizia Jaccheri
email: letizia@idi.ntnu.no

Interactivity on Learning and Future Visits in a Museum

Keywords: Mobile learning, informal learning, performance, perceptions, mobile devices, museums. 1

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Mobile applications are increasingly used nowadays to assist many scenarios (e.g., museums, cultural heritage locations, outdoor activities). Educational medium effectiveness and students' perceptions for the medium highly affect the learning success. As the introduction of learning tools is often complex, students do not always use them as expected. For instance, students' perceptions regarding the importance and the interest for the medium are some of the most widespread barriers for the effective adoption (e.g. [START_REF] Giannakos | Using Webcasts in Education: Evaluation of its Effectiveness[END_REF][START_REF] Hsu | Acceptance of Blog Usage: The Roles of Technology Acceptance, Social Influence and Knowledge Sharing Motivation[END_REF]). In addition students' perceptions have an impact on what they have already learned and what they choose to do next [START_REF] Metcalfe | Evidence that judgments of learning are causally related to study choice[END_REF]. Especially, for the mobile devices, due to their portability and functionality, they can be used outdoors, in exhibitions and in other non classroom settings, extending the depth and breadth of informal learning and the importance of students' perceptions.

Research in educational games argues that well-designed games can meet some of the needs of students and pursue them to enroll in learning procedures [START_REF] Tarumi | KEI-time traveler: visiting a past world with mobile phones to enhance learning motivation[END_REF]. Evidence of learning performance of educational games has been shown in several areas, such as history, arts and cultural heritage [START_REF] Cabrera | Mystery in the museum: collaborative learning activities using handheld devices[END_REF][START_REF] Costabile | Explore! Possibilities and Challenges of Mobile Learning[END_REF].

Portable devices have been successfully employed as learning tools in both formal and informal learning contexts. In the formal learning, handhelds have been used in teaching both within the classroom setting and out-classroom (i.e. [START_REF] Cabrera | Mystery in the museum: collaborative learning activities using handheld devices[END_REF]). In addition, the potential of handhelds to assist informal learning has been explored through different frameworks. For instance, handhelds have been used as museum guidebooks [START_REF] Hsi | A study of user experiences mediated by nomadic web content in a museum[END_REF][START_REF] Sung | Designing an electronic guidebook for learning engagement in a museum of history[END_REF] or as learning systems [START_REF] Huang | Effectiveness of a mobile plant learning system in a science curriculum in Taiwanese elementary education[END_REF].

Researchers have often stressed the benefits of enhancing the functions of museums, parks and exhibitions by designing educational projects connected to learners' informal learning, [START_REF] Huang | Effectiveness of a mobile plant learning system in a science curriculum in Taiwanese elementary education[END_REF] and making non classroom environments more enjoyable and motivating for learning [START_REF] Sung | Designing an electronic guidebook for learning engagement in a museum of history[END_REF]. In the context of museums, many researchers have employed new technologies and customized mobile devices for attracting and engaging visitors. On that direction visitors' beliefs is highly important [START_REF] Cabrera | Mystery in the museum: collaborative learning activities using handheld devices[END_REF]. For instance, one of the most successful projects was conducted in the Exploratorium of San Francisco [START_REF] Hsi | A study of user experiences mediated by nomadic web content in a museum[END_REF]. This study was one of the pioneers in the direction of combining mobile technology and capturing experience in the real world.

Many studies have shown the potential of portable devices to increase learning opportunities. Informal learning with the real world experiences is some of the most remarkable advantages. Mobile devices' portability and functionality make them suitable for out-classroom learning, for instance bird-watching, plant hunting and museum guiding are some of the most successful case studies [START_REF] Hsi | A study of user experiences mediated by nomadic web content in a museum[END_REF][START_REF] Huang | Effectiveness of a mobile plant learning system in a science curriculum in Taiwanese elementary education[END_REF]. As such, it is subsequent that the use of mobile devices would bring many benefits and opportunities for informal learning in future. In particular, in this study we aim to provide insights of the following question: (How) Can interactive mobile technology be exploited to implement informal education?

As such, we designed an interactive mobile-based and a paper-based game and we conducted a controlled experiment in order to explore the educational impact and students' performance and perceptions affected by interactivity and mobile technology.

METHODOLOGY

The Game Design

A quiz was designed to include several game elements, such as purpose, rules, score, and limitations (space-time). The main purpose of the quiz activity is to identify a series of paintings, at an Art Gallery (fig. 1), based on descriptions of visual elements. During our experiment, the quiz was played by teams of two players, who were given Two free (one of them open-source) software tools were used for implementing the activity, a QR application and a quiz application. The only hardware requirement for running the software is a java enabled mobile phone. Camera feature was a requirement for the QR application11 . The design of the questions was implemented with MyMLE2 application, which allows creating mobile learning content. Some of the basic advantages of this game is the low cost of the infrastructure and that there is no coding required in developing the quiz of questions.

There are two versions of the activity, a Mobile Based (MB) and a paper-based (PB) version. In MB version QR codes are placed next to paintings. Each team provided with two mobile phones, one where questions displayed and answers typed in, and a second one that was used to scan QR codes. When the players identify a painting as an answer to a question, they scan the QR code next to it and type a four digit number as the answer (Fig. 2). If the answer is correct they get a message indicating so. If the answer is wrong, they get a message indicating that they have one more chance to answer correctly. Hence in total each team was given up to two chances to identify a correct answer.

The PB version of the game was identical, except from the fact QR codes were replaced with numbered post-its and the questions were printed on paper. However the major difference in the gameplay between the two versions was that players do not receive any feedback regarding their answers, during the PB game.

Sampling

The research methodology included a survey composed by the four main measures and by score tests. The initial sample of the study was sixty 1st grade lyceum (upper general secondary school) students, aged between 15 and 16 years. All the students were attending an elective course in ICT and multimedia at a general lyceum at Corfu, Greece. Three groups were formed up with high similarity in their prior class performance. The three groups of twenty students each, corresponded to the three different classes of the aforementioned course. Group A consisted of 16 males and 4 females, Group B consisted of 17 males and 3 females and Group C consisted of 16 males, 4 females. It should be noted that according to the school's policy students were assigned to the classes of the elective course in a rather "random" manner. At the initial sample of 60 students, we had 5 drop-outs, one female student from Group A, two female students from Group B and two male students from Group C, leaving a total of 55 (19 for the first group, and 18 for the remaining two).

Procedures

The study was conducted over a three-week period from 10th to 28th of May 2010.

During the first week formal classroom teaching was held. At the second week students visited the Art Gallery (Fig. 1) of the Municipality of Corfu and at the third week the posttest (questionnaire and assessment) took place. Upon completion of the tour students from Group A and Group B were briefed on the rules and played the game (mobile-and paper-based respectively). The students of Group A, were also provided with mobile phones with a sample quiz as well as sample QR codes in order to familiarize themselves with the software. The same devices were used during the game. Students of Group C where given instead 30 minutes extra time to spend at the gallery (extended tour), in order to identify visual elements on their own as they wished. For the final week of the study, the students from all three groups took the formal test (assessment) and answered the questionnaire.

2.4

Measures

In order to measure students' effectiveness, all groups completed the cognitive assessment at the end of the study. In addition, Groups A and B completed the survey with the four measures regarding students perceptions.

For the cognitive test a twelve-item assessment, consisting of multiple choice questions, was employed. Students were presented with a series of pictures and had to choose one out of the four descriptions that was more accurate. The description was referring to the same visual elements that were also employed during the game, which were combinations of all of the following: primary and secondary colors, warm and cold colors, geometric and organic shapes, depth and perspective. For instance, two questions were: "which painting has Religious Theme, Action Scene and Perspective in two spots?" (correct: The Beheading of the Precursor) and "which painting has Action Scene, Urban Landscape and Warm Colors?" (correct: Kapodistrias Assassination).

The second part included measures of the various questions identified in the literature from previous researches. These measures included Perceived Importance [START_REF] Joshi | Choosing IT as a Career: Exploring the role of Self-Eficacy and Perceived Importance of IT Skills[END_REF]; Perceived Comprehension [START_REF] Dunlosky | Metacomprehension: A brief history and how to improve its accuracy[END_REF]; and interest with the activity and the visit [START_REF] Berger | Measuring computer literacy of teacher trainers[END_REF]. In particular the following constructs were employed: (1) How important do you believe that the activity is to your future visits (perceived importance of activity for future visit, PIAFV), (2) How well do you feel you understand the concepts taught (perceived comprehension, PC), (3) How interest did you find the visit (perceived Interest for the visit, PIV) and ( 4) How interest did you find the activity (Perceived Interest for the activity/game, PIG). For each statement, students have to mark their response on a five-point Likert scale in relation to four measures.

Data Analysis

Firstly, in order to examine the differences among mobile-based, paper-based game and extended tour guide on students' effectiveness, a post hoc test was conducted. A Games Howell criterion was used in order to examine separately for each group the influence of 1) mobile-based game, 2) paper-based game and 3) extended tour guide on students' effectiveness. In addition for identifying any differences in students' perceptions for perceived importance, perceived comprehension and interest with the activity and the visit a t-test was conducted among mobile-based and paper based game. Finally, for identifying any correlations among students' perceptions Spearman correlation coefficient was conducted.

RESEARCH FINDINGS

To examine the hypotheses regarding students' performance, the Post-hoc analyses using Games -Howell criterion were conducted to verify whether the different learning procedures (Group) are related to the different effectiveness of the students. The results of three groups, Group A (M 1 =9.00, SD 1 =2.08), Group B (M 2 =7.28, SD 2 =1.67) and Group C (M 3 =8.50, SD 3 =1. 97), highlighted a significant difference only between Group A and Group B (p<0.05).

The results highlight that the enrolment with the mobile affects their performance. However, both (paper-mobile) games persuade the children in several ways. A team benefits from a strategy and game characteristics seem to motivate and excite them for the activity. In addition, based on the observations of the researcher students seemed to be more focused and immersed on the mobile application (fig. 3). Interestingly, mobile application did not push students to lose their interest for the museum as many paintings and its features are discussed from students by the end of the experiment and on the respective course on school. Another notable observation is that students from both groups spent approximately the same total time at the gallery (19 min on average). This may be derived from students' familiarity with mobile phones and the simple design of the game. Regarding students' perceptions among mobile-based and paper-based game, a t-test was used for identifying the differences, with four independent variables (PIAFV, PC, PIG, PIV) and the one dependent variable (Group) were included. All variables demonstrate very high levels on both groups (fig. 4), but no significant difference is identified on students' perceptions among the two groups.

Fig. 4. Students perceptions among mobile-based and paper-based game

Spearman's correlation coefficient between the measures was used, which is about quantifying the strength of the relationship between the variables. Spearman's test suggests that some of the measures are related, in some cases relatively strong. More precisely, perceived interest for the game is strongly related with all measures. Perceived interest for the visit has correlation only with perceived interest for the game. Moreover, perceived importance of activity for future visit and perceived comprehension have significant relation with all measures except perceived interest for the visit. Table 1 indicates the exact results of the test.

DISCUSSION AND CONCLUSIONS

The current study is one of the few so far, where a mobile application assists an informal educational process. Conducting a field study, of this kind with a sample of a 55 students, was not an easy task, especially when social interaction threats are concerned. The overall results prove that the mobile application had a positive effect, on students' performance at the final test and also does not affect their interest for visiting the gallery.

The results of the study allow us to argue that the enhancement of mobile device in a learning game context benefits students' performance in a highly significant way. This result may be explained from previous findings in the literature [START_REF] Huang | Effectiveness of a mobile plant learning system in a science curriculum in Taiwanese elementary education[END_REF] stating that the usage of mobile devices motivates students and offers sufficient learning opportunities to improve learning procedures. From research we do not have sufficient evidence to suggest that the usage of a mobile application affects students' perceptions regarding the non-classroom learning environment. However, students' perceptions for the game interest positively affect their beliefs for the visit, for a future visit and their comprehension. Hence, we can argue that the enrolment of mobile devise can improve the effectiveness of game based learning and can compete the extended guiding effectiveness in a non classroom environment. In addition, students' interest with the game helps them to understand the benefits of the visit and provide a vehicle for future visits.

These findings have important implications for understanding how students regulate their learning and achievement. Previous studies have shown that students' perceptions of what they have already learned affect what they choose to do next [START_REF] Metcalfe | Evidence that judgments of learning are causally related to study choice[END_REF]; thus, the differences that entity and incremental theorists exhibit in their perceived comprehension could potentially have long-term effects on their achievement, even if these differences are not currently reflected in their actual comprehension (as was the case in our experiment).

The findings of this study must be interpreted in light of some potential limitations. First, the generalizability of this study must be carefully made, as it was conducted in a single context with specific instructions. Secondly, the sampling of the study apart from students at one grade level and was not longitudinal; therefore the data could not reveal the continuation of the MB game behavior. Despite these limitations, the find-ings generate valuable insights, which can be used as part of hypotheses for representative follow-up studies in technological tools' educational effectiveness and experience.

Further studies may consider aiming to reveal several insights. First investigating the effect of the game in students' performance under different context and for a longterm period might help on the overall understanding. Also, another study involving different educational levels and students with different levels of computer experience might draw several important conclusions. Further research also needs to examine students' intention to adopt Augmented Reality based learning.

Fig. 1 .

 1 Fig. 1. The layout of the public art gallery and the 12 correct answers (paintings)

Fig. 2 .

 2 Fig. 2. The QR codes were placed next to the paintings (left), the mobile application (right)

Fig. 3 .

 3 Fig. 3. Students of the mobile-based game were immersed in questions displayed on their mobile phones

Table 1 .

 1 Spearman correlations between measures

		PIAFV	PC	PIG	PIV
	PIAFV	1
	PC	.411 **	1
	PIG	.342 **	.355 *	1
	PIV	.272	.213	.560 **	1
	* Correlation is significant at the 0.05 level. ** Correlation is significant at the 0.01 level.

http://qrcode.sourceforge.jp

http://mle.sourceforge.net

Acknowledgements

The authors would like to thank all of the students, the schools' staff and the museum curator for their participation in the experiment. The research introduces in this work is supported by the ERCIM "Alain Bensoussan" Fellowship Programme. This programme is supported by the Marie Curie Co-funding of Regional, National and International Programmes (COFUND) of the European Commission. This work is also supported by project CULT (http://cult.di.ionio.gr). CULT (MC-ERG-2008-230894) is a Marie Curie project of the European Commission (EC) under the 7th Framework Program (FP7).