Suwon Lee
email: swlee@paradise.kaist.ac.kr

Jinki Jung

Jihye Hong
email: jihyerish@paradise.kaist.ac.kr

J B Ryu

Hyun S Yang
email: hsyang@paradise.kaist.ac.kr

AR Paint: A fusion system of a paint tool and AR

Keywords: Augmented Reality, AR, AR Paint Recognition Input Frame Keypoints Extraction (FAST) Object Prediction (PRPDF) Keypoints Matching (KMPDF) Outlier Removal (PROSAC

In this paper, we present AR Paint, a fusion system of a paint tool and Augmented Reality (AR) that can augment in-place hand-drawn paintings onto an object. The system allows users to draw anything using the paint tool and then augment it onto an object whenever they want while providing a new experience and enjoyment during the process. It also has potential value if used in industry or for medical purposes. The hand drawing and visualization for AR is done through the screen of a mobile device, and all processes of the system are performed in real time.

Introduction

Have you ever desired to draw something directly onto a famous painting? When you visit an art gallery, you may wonder how a painting would look if something you drew could be added to the painting. For example, you can imagine drawing Mona Lisa's eyebrows onto the painting Mona Lisa by Leonardo da Vinci. This impossibility has motivated us, and we made it possible using Augmented Reality.

Augmented Reality (AR) is a type of human-computer interaction that enhances our perception and helps us to see, hear, and feel our environments in new and enriching ways while providing local virtuality [START_REF] Azuma | Recent Advances in Augmented Reality[END_REF][START_REF] Krevelen | A survey of augmented reality technologies, applications and limitations[END_REF]. With AR technology, we can gain a better understanding of education, find guidance through processes, be instructed in the assembly of objects, and have new experiences that we otherwise could not.

In this paper, we present a fusion system of a paint tool and AR termed AR Paint. The system allows users to draw anything using the paint tool and augment it onto an object in real time while providing a new experience and enjoyment during the process. The drawing and visualization for AR is done through the screen of a mobile device, and all processes of the system are performed in real time. The system also has the potential to be used in industry or for medical purposes. For example, surgeons may need to mark specific blood vessel for tracking while performing surgery. In such a situation, AR Paint can be the best choice when it supports full 3D object tracking with drawing in 3D space. In the AR Paint system, however, we currently consider only 2D surface tracking while drawing on a surface.

The rest of this paper is organized as follows. In Section 2, we first present the AR Paint system with its scenario and framework. We then show a demonstration with a performance evaluation in Section 3. Finally, we conclude with a brief summary and discussion in Section 4.

AR Paint

In this section we present the AR Paint system from a scenario based on end users' viewpoints to detailed accounts of its processes. The system can take many different forms depending on its application, but in this paper, we describe our system via an AR Paint application for an art gallery for a clear understanding of its capabilities.

Scenario

In the Louvre Museum in Paris, a man is standing in front of Mona Lisa, Leonardo da Vinci's most famous painting. While looking at the painting, he wonders how different the painting would look if Mona Lisa was given eyebrows. He runs the AR Paint application on his smart phone and then he points its built-in camera at the painting. The boundary of the painting is marked and tracked in every frame. Once he touches a screen of the smart phone, the current frame is fixed and the GUI for the paint tool pops up. He starts drawing the eyebrows on the touch screen using his fingertip. Once he finishes this task, the eyebrows he drew are visualized and tracked in every frame.

Recognition

Tracking with Augmentation

Framework

For the AR Paint system, only a mobile device equipped with a camera and a supporting touch-screen input are necessary, both of which are available on the latest smart phones. AR Paint system is composed of two main modules: the AR module and the Paint module, as shown in Fig. 1.

In the AR module, recognition and tracking are performed using training data that is learned from recognizable objects. The recognizable objects and the distribution of the training data depend on the application. In our application, the recognizable objects are limited to 2D paintings and the distribution of the training data is done in advance when the application is installed on the mobile device.

The AR module switches to the Paint module when the user touches the screen while tracking an object. When switching, the three-dimensional position of the object is remembered for use later. Users can add something to the painting in the current fixed frame using the paint tool. The three-dimensional position is used to remove any distortion from the user's drawing and update the model which will be augmented in the AR module.

Fig. 2. Recognition and Tracking process in the AR module

AR Module (Recognition and Tracking)

Fig. 2 shows the entire process of the AR module, which is described in this section. The recognition process is initially performed for every frame captured by the camera. For real-time object recognition with pose estimation, we use a Generic Random Forest (GRF) [START_REF] Cho | Markerless Visual Tracking for Augmented Books[END_REF]. The GRF is designed to be capable of simultaneous object recognition and keypoint matching through one pass over the original Random Forest [START_REF] Cho | Markerless Visual Tracking for Augmented Books[END_REF][START_REF] Lepetit | Keypoint recognition using randomized trees[END_REF]. As shown in Fig. 3, a GRF consists of N T random trees, and each tree conducts D binary tests comparing the intensities of two pixel locations of the input image patch surrounding a keypoint. Every leaf node of each tree stores n + 1 probability distributions (1 distribution is for object recognition, and each of the n distributions is for the corresponding object's keypoint matching). Therefore, for an image patch surrounding a keypoint, the GRF returns two results ('where' is the keypoint from among n objects, and 'what' is the keypoint from among the k keypoints of the corresponding object). For every frame, several keypoints are extracted via a FAST corner detector [START_REF] Rosten | D: Machine Learning for High-Speed Corner Detection[END_REF]. Every keypoint is passed through the GRF and the candidate object is predicted with matching pairs. Outliers are eliminated by the computation of a homography using PROSAC [START_REF] Chum | J: Matching with PROSAC-progressive sample consensus[END_REF], and only matching pairs with reprojection error amounts of less than 10 pixels become inliers. If the number of inliers is smaller than a threshold, the candidate object is rejected for recognition; otherwise, the initial sixdegree-of-freedom (6 DOF) pose of the candidate object is calculated [START_REF] Schweighofer | Robust Pose Estimation from a Planar Target[END_REF].

Once an object is recognized, its ID and 3D pose are estimated and the tracking process is started. During the tracking process, a new pose of the object in a new frame is predicted using the previous 3D pose and the current camera motions. For the predicted new pose, coarse-to-fine matching is conducted to calculate the final pose and the opportunity to update the keyframe is given only in the case of good tracking. All of the aforementioned tracking processes are performed in the same manner as [START_REF] Cho | Real-time recognition and tracking for augmented reality books[END_REF]. While an object is tracked, the boundary of the object is marked with a red line whose inner area becomes the canvas in the Paint module. Moreover, if there are some drawings of the object, they have to be augmented onto the object. In order to complete these tasks simply, we put one virtual layer per object and refer to the virtual layer as a model. The model is initialized by a transparent canvas with a red boundary considering corresponding to the object's shape and size. As a result, we can easily facilitate both augmentation and model update (Fig. 5). When the system is tracking, the current frame has an augmented model on the object as its image. By touching the screen during the tracking process, the current frame is fixed and the GUI for paint tool is displayed on the screen, as shown in Fig. 4. This state is ready for drawing and the inner-area of the model's boundary becomes a canvas for drawing. Users can draw something on the touch screen using their fingertip or a touch pen. In our application, currently, drawing is limited to 2D space, as our recognizable objects are 2D paintings and because the drawing is added onto their surfaces.

Once the drawing task is finished, the system starts the tracking process again and the users can see their drawing become augmented. This means the model for the object is updated immediately. When users draw something onto an object, the 3D position of the object differs depending on the viewpoint of the users. Therefore, we need to remove distortion in the drawing before applying it to a model. As shown in Fig. 5, the drawing is inversely warped to remove its distortion using 6 DOF pose of the object and the model is then updated according to the result.

Demonstration

To test our system, we implemented the AR Paint application on a tablet PC with a 1.4GHz dual-core processor and 1GB of RAM. For target objects, we printed out 100 paintings and trained them using a GRF which consisted of 40 trees of 12 depths, as mentioned in Section 2.3. At runtime, the application works as described in Section 2.1. Fig. 6 shows the demonstration of the application. All processes of the application were performed in real time.

For an evaluation of the performance, we first measured the recognition rate and recognition time as the number of training objects increased from 20 to 100. We randomly synthesized 100 new images per object, and they were used to measure the recognition performances. The average recognition rate and recognition time are shown in Table 1. We then measured the tracking time using a test movie with 500 frames. The test movie includes various events with tilts, scale variations, rotation, and fast movement. The averaged tracking time over all of the frames was 5.2ms. With the recognition time, it achieves more than 30 fps considering the total time requirements. This demonstrates that our system is fast enough in terms of its framerate performance.

# of training

Conclusion

In this paper, we presented AR Paint, a system which is able to augment in-place hand-drawn paintings onto an object. With the AR Paint system, users can immediately add their own drawing to a well-known painting, and the system provides a new experience and enjoyment for users as they use it. In the proposed AR Paint system, currently only 2D drawings on 2D surfaces are allowed. However, by allowing 3D drawings on 3D objects, it will become feasible to use this system for various purposes. As a future work, we first intend to find needs the AR Paint system can fulfill by conducting user studies. We also have plans to expand the system to support 3D drawings on 3D objects.

Fig. 1 .

 1 Fig. 1. The framework of AR Paint

Fig. 3 .Fig. 4 .

 34 Fig. 3. Generic Random Forest

Fig. 5 .Fig. 6 .

 56 Fig. 5. Augmentation and Model Update

Table 1 .

 1 Recognition Rate and Recognition Time

	objects	20	40	60	80	100
	Recognition Rate (%)	99.0	98.8	98.6	98.3	98.1
	Recognition Time (ms)	18.9	19.2	19.6	20.2	21.0

Acknowledgement This work was supported by the IT R&D program of MKE/MCST/IITA, [Development of learner-participational and interactive 3D Virtual learning contents technology]. This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education, Science and Technology (2011-0018291 / 2012-0005793)


André Pinto

António Coelho
email: acoelho@fe.up.pt

Hugo Da Silva
email: hugo.da.silva@fe.up.pt

A Ubiquitous Solution for Location-Aware Games

Keywords: location-based games, mobile games, location-based services, ubiquitous, indoor, pedestrian dead reckoning

Even though we now witness a popular use of location-based mobile games, the player experience in these applications is always limited by the errors of common location technologies, especially in indoor scenarios. This paper describes the way we minimize this problem in our game development platform, by levering the potential behind smartphone sensors to estimate players trajectories. Our approach is based on a Pedestrian Dead Reckoning (PDR) algorithm that combines methods to determine orientation, detect steps and estimate their length. Other typical multiplayer mobile games problems, like network latency, are also briefly addressed.

Introduction

During the last years, the proliferation of smartphones with location capabilities enabled the creation of several categories of location-based services (LBS). Among them, the mobile social gaming phenomena ended up attracting more attention to the opportunities created by these technological advances in the entertainment field. At the same time, ubiquitous computing, a model in which information is omnipresent and contextualized, is now more prevalent than ever before [START_REF] Greenfield | Everyware: the dawning age of ubiquitous computing[END_REF].

Although enabling interesting use cases for mobile games, everyone who used a GPS-enabled device, have certainly experienced its accuracy and availability bottlenecks, whether caused by indoor usage, city canyons or other physical obstacles. Even the alternative solutions, like WLAN positioning, can get its performance critically deteriorated in dynamic non-line-of-sight situations. These limitations are restraining mobile games (and many other LBS) from exploring truly ubiquitous experiences.

The solution presented in this paper was motivated by the development of a serious game that helps the player to explore a faculty building by engaging the user in a "treasure hunt". During the gameplay the player gets to know the most important places around him/her, whilst receiving basic useful information of the surroundings (Figure 1). By visiting specific points of interest within the faculty campus, the player collects points for his/her team. This use case implies some technical requirements such as indoor player positioning and intensive location reporting to the server. To enable the development of this type of games, we propose a multiplayer mobile gaming platform that was developed to diminish the above inaccuracies by implementing a pedestrian dead reckoning method that uses the sensors usually available on recent smartphones (accelerometers, magnetometers and gyroscopes) to estimate the players positions. Besides the location problem, our platform also offers a basic communication system created to minimize latency impact during the necessary data exchange in a multiplayer environment.

Platform Architecture

Our platform is composed by several client applications (one per player), which access one server application by using the available network technology in the place (e.g. WLAN, 3G, 4G). Both of these applications are written in Java. The client applications are responsible for reporting the player position to the server and engaging the user in the game mechanics. The server application manages the game communication and controls game-sensitive data, like the scoreboard. Finally there is a third entity, a PostGIS database, from which the client applications obtain the building plan data. In our test case, this data is used to render the plan and show information about the points of interest near the player.

Although we had used a Sparkfun Razor IMU with 9 Degrees of Freedom to gather the sensors data, the approach is equality valid for any smartphone which fulfil the necessary hardware requirements.

Related Work

The pedestrian dead reckoning approaches to location is not a new topic in the scientific community. From the various work already done, we highlight the 1996's patent [START_REF] Levi | Dead reckoning navigational system using accelerometer to measure foot impacts[END_REF], which provides an initial overview of the solution as a whole, and, more recently [START_REF] Syrjärinne | MEMS-IMU based pedestrian navigator for handheld devices[END_REF], that offers an implementation for handheld devices. Other solutions are also addressed in surveys like [START_REF] Liu | Survey of Wireless Indoor Positioning Techniques and Systems[END_REF] and [START_REF] Koyuncu | A Survey of Indoor Positioning and Object Locating Systems[END_REF], which enable the reader to better understand the complete panorama of indoor location methods. On the other hand, communication problems like network latency, which are common in these situations, have also been targeted in [START_REF] Wang | Experiences from implementing a mobile multiplayer real-time game for wireless networks with high latency[END_REF].

Pedestrian Dead Reckoning (PDR)

PDR is a method used by Inertial Navigation Systems. In this kind of systems, the positioning process is based on a relative estimation. The previous known position of the subject/object's is used together with an estimate for its displacement to obtain an approximation of the current position. For PDR solutions, the displacement estimate intends to measure the distance of pedestrian locomotion. The method is composed by 3 main tasks:

orientation determination, step detection, step length estimation.

In a "disturbance-free environment", orientation could be given directly by the magnetometer's readings. In the "real world", magnetic fields are very irregular which makes the former method a naive use of this sensor. On the other hand, electronic gyroscopes have offset and drift errors, so they can't be a reliable source for orientation without realizing further steps to treat their data. In order to fix this we combined the gyroscope's data with the values received from the magnetometers and accelerometers into a direction cosine matrix. This process includes several different methods that are generically described in [START_REF] Premerlani | Direction Cosine Matrix IMU: Theory. gentlenav[END_REF]. Still, this process is not free from the negative effects of magnetic disturbances, so we implemented an additional technique to detect interferences on the magnetometer's signal [START_REF] Ladetto | Digital Magnetic Compass and Gyroscope Integration for Pedestrian Navigation[END_REF]. We did this by monitoring sudden changes in the norm of the magnetic field, and temporarily ignoring the magnetometer's data while they are present.

The step detection process is based on a state machine that processes the accelerometer signal over the vertical axis and detects "step-like" waves. The state machine contains 4 states: IDLE, ACC UP, ACC DOWN and STOPPING. Each one of these states has its own transition conditions that reflect the mechanic of the different step phases.

Finally, as a step length depends on multiple factors that can be associated with the subject (e.g. height, sex), the environment (e.g. inclinations, stairs) or the walk dynamics (e.g. step frequency and walk gait), we used various different metrics obtained from the accelerometer to estimate its value, namely: the double integral of acceleration's norm (without gravity), the maximum amplitude of vertical acceleration, the minimum anteroposterior acceleration and a step frequency estimate. An additional constant factor is added to represent a reference value. During an initial calibration phase, the weights (w i ) of each one of these factors (v i ) are estimated by using multiple linear regression. Afterwards they are combined to calculate the step length (sL) estimate for every detected step:

sL = 5 i=1 w i • v i (1)
Fig. 2. Player's straight line trajectory (red) obtained by using our approach without (left) and with (right) magnetic disturbances detection.

Communication

There is a strong relation between the communication requirements and the game genre. When creating a platform for mobile serious games, it is not possible to predict the kind of games it will be used for, so the optimization work was focused on the system's reaction to the worst-case scenario. Many of the problems present in the worst-case scenario are greatly increased by mobile network characteristics, which are typically slower and more error-prone than the wired networks. This easily leads to package loss and latency. Considering the circumstances, different protocols (UDP and TCP) were assessed and ways to minimize the negative effects of latency (e.g. dead reckoning) were studied.

For location-based games that heavily rely on a continuous feed of messages reporting the player's position, the importance of a single message is very low as it becomes immediately outdated with the next one in the stream. Therefore, a protocol like TCP, whose methods for granting reliability introduce considerable delay on the network, is unsuited to our case. For instance, if one position update message fails to reach its target, skipping it and sending the new position instead, would be a better option than trying to send the obsolete position again. Additionally, the bigger segment header (UDP: 8 bytes, TCP: 20 bytes) and the considerable weight of acknowledgement messages in the network transfers (38% [START_REF] Chen | An empirical evaluation of TCP performance in online games[END_REF]), are all strong drawbacks for TCP in its comparison with UDP. Thus we ended up using UDP to handle the communication between the different actors. Our conclusions are also supported by [START_REF] Wang | Experiences from implementing a mobile multiplayer real-time game for wireless networks with high latency[END_REF] where UDP invariably outperforms TCP in every tested network (e.g. UDP UMTS: 458 ms avg, TCP UMTS: 933 ms avg).

Conclusions and Future Work

Overall, the final solution produced encouraging results when compared with the naive approach, but even with all the previous corrections, the PDR displacement estimation's accuracy still depends on the sensors data reliability. As can be seen in Figure 2, our approach creates some deviation from the real trajectory. This is especially worrying, as these errors accumulate in each iteration (as is typical in INS), which can lead to useless estimates relatively quickly. In order to minimize this accumulated error we plan to implement restrictions on the possible trajectories performed by the player. By removing impossible moves, like a person crossing a wall, we can keep the total uncertainty within an acceptable level in regions with physical obstacles. Additionally, it might be useful to introduce support for TCP communication to accommodate other use-cases that need to assure reliable message delivery.

As an alternative, the developer could eventually adopt a "seamful" design approach by adapting the game mechanics and bringing these limitations into the game itself. In some cases, informing the player about the existence of problems and exploiting them as game features, can constitute a valid and alternative [START_REF] Broll | Seamful Design for Location-Based Mobile Games[END_REF].

The present method can be used as a stand-alone solution, or as an auxiliary component of another location technology, which makes it also useful for outdoor-only games that require higher precision than a GPS-only solution can provide. Nevertheless, other location-aware products and services can also gain from the use of these techniques. A migration to a smartphone platform and a more intensive end-user experience would definitely foster further refinements and help consolidating the solution.

Fig. 1 .

 1 Fig. 1. Game screen showing the player (green) near a point of interest (yellow)

Acknowledgments. This work is partially supported by the Portuguese government, through the National Foundation for Science and Technology -FCT (Fundação para a Ciência e a Tecnologia) and the European Union (COMPETE, QREN and FEDER) through the project PTDC/EIA-EIA/114868/2009 entitled "Eras -Expeditious Reconstruction of Virtual Cultural Heritage Sites".


Sven Seele
email: sven.seele@brsu.de

Rainer Herpers
email: rainer.herpers@brsu.de

Christian Bauckhage
email: bauckhage@bit.uni-bonn.de

Cognitive Agents for Microscopic Traffic Simulations in Virtual Environments

Keywords: virtual environments, multi-agent systems, cognitive agents, traffic simulation

Traffic simulations in current open world video games and driving simulators are still limited with respect to the complexity of the behavior of simulated agents. These limitations are typically due to scarce computational resources, but also to the applied methodologies. We suggest adding cognitive components to traffic agents in order to achieve more realistic behavior, such as opting for risky actions or occasionally breaking traffic rules. To achieve this goal, we start by adding a personality profile to each agent, which is based on the "Five Factor Model" from psychology. We test our enhancement on a specific traffic scenario where simplistic behaviors would lead to a complete standstill of traffic. Our results show that the approach resolves critical situations and keeps traffic flowing.

Introduction

Recently, open world games have become very popular, since they allow players to roam freely through large areas and enable a non-linear playing experience. Obviously, this experience also requires virtual worlds to be populated by believable agents. If these simulated agents use means of transportation such as automobiles, simulations of vehicular traffic need to be realistic. However, stateof-the-art traffic simulations for digital games usually only display relatively simple, rule-based behaviors, so they do not mimic the reality of road traffic. This hampers the player's feeling of immersion. In serious games applications, such as driving simulators for road safety education (cf. the FIVIS bicycle simulator system [START_REF] Herpers | Multimedia Sensory Cue Processing in the FIVIS Simulation Environment[END_REF]), a lack of realism is in fact harmful.

Our goal is therefore to equip virtual agents with skills and reasoning mechanisms that produce real-world behaviors. With respect to traffic simulations, the critical observation is that, in everyday traffic, rules are frequently broken, either on purpose or because of human error. The aim of simulators, such as in the FIVIS project, is to teach children to identify and resolve potentially hazardous situations resulting from misconduct. If simulated traffic participants are to be useful for road safety training, they need to have a tendency to make errors and to occasionally defy traffic rules. However, in order for the misbehavior to make sense to an observer, it must be grounded in the momentary circumstances. Hence, the agents must be able to perceive and reason about their environment before making decisions. Unfortunately, current traffic simulations typically rely on scripted or randomized behaviors and therefore rarely produce realistic hazardous situations. In this paper, we therefore investigate the effects of adding psychological profiles to agents.

Related Work

Traffic simulations have been researched since the 1930s (see [START_REF] Treiber | Verkehrsdynamik-und Simulation[END_REF] for an exhaustive overview of different approaches). The field is mainly concerned with investigating emergent traffic behaviors in order to plan roads or predict jams. While applications like these require general models of driving behavior, they are not concerned with individual habits or actions of agents. In virtual environments, however, where users directly interact with a simulated world, individual behaviors are a major component of the user's experience.

Traffic simulations for virtual environments are often of limited complexity, typically because of scarce computational resources but also because of the current state of artificial intelligence. In fact, some researchers have criticized the limited character of game AI as one of the major shortcomings of current digital experiences and have focused on providing adaptive game AI using methods such as case-based reasoning or reinforcement learning (see e.g. [START_REF] Bakkes | Rapid and reliable adaption of video game ai[END_REF][START_REF] Sharma | Transfer learning in real-time strategy games using hybrid CBR/RL[END_REF][START_REF] Thurau | Learning Human-Like Movement Behavior for Computer Games[END_REF]).

Research on cognitive architectures naturally influences work on modeling cognitive traffic agents. While there are numerous different paradigms and concepts [START_REF] Anderson | An integrated theory of the mind[END_REF][START_REF] Byrne | Cognitive Architecture[END_REF][START_REF] Laird | Extending the Soar cognitive architecture[END_REF], ICARUS [START_REF] Langley | A Unified Cognitive Architecture for Physical Agents[END_REF] is an example of a corresponding framework applied to the domain of in-city driving. ICARUS enables an agent to learn how to perform basic driving tasks, for example, aligning itself in a lane or accelerating or decelerating for turns. Similarly, the work in [START_REF] Lüdtke | Modeling Pilot and Driver Behavior for Human Error Simulation[END_REF] used a cognitive architecture to model erroneous behavior of pilots and drivers. However, it focused on attention mechanisms for assistance systems and did not address decision-making.

Earlier work in [START_REF] Kutz | Urban Traffic Simulation for Games[END_REF] presented the concept of a more realistic traffic agent based on psychological personality profiles. Its core is the "Five Factor Model" (FFM) which states that every personality can be defined using only five distinct personality traits (openness, conscientiousness, extraversion, agreeableness, neuroticism) [START_REF] Borkenau | Neo-Fünf-Faktoren Inventar (NEO-FFI) nach Costa und McCrae[END_REF][START_REF] Herzberg | Beyond "accident-proneness": Using Five-Factor Model prototypes to predict driving behavior[END_REF]. However, the correlation between an agent's personality and its behavior in traffic was not worked out. Nevertheless, the FFM provides an auspicious starting point for the work presented here.

Modeling Cognitive Traffic Agents

While the approaches reviewed above apply many interesting concepts, they do not address the problem of modeling misbehavior. However, to achieve a realistic traffic simulation, agents need to behave and misbehave just as humans do. When they misbehave, they should only do so in situations where it makes sense to an observer, thus ruling out solutions solely based on randomization. We therefore aim to devise agents that are willing to intentionally take certain risks to reach their goals. Choosing risky actions could cause the agent to break traffic rules, which in turn would create potentially dangerous situations for other traffic participants. Our hypothesis is that modeling cognitive processes performed by human drivers and equipping agents with the ability to imitate these processes will result in the desired realistic behavior.

The first important aspect is to model what an artificial traffic participant should be able to perceive of its environment. As human drivers have limited perception, realistic agents need to be equipped with a "perception filter" that controls which information actually reaches the agent. We therefore model the visual field of an agent geometrically within a 3D game engine. Other agents residing within this geometric construct are considered to be perceivable. The concepts of attention selection and attention division presented in [START_REF] Lüdtke | Modeling Pilot and Driver Behavior for Human Error Simulation[END_REF] can be added in a modular fashion to refine this simple perception model. Figure 1 shows a corresponding example of a cognitive traffic agent. It draws heavily on existing cognitive architectures and features typical elements of this domain, for instance, a short and long-term memory, decision and learning processes, as well as links to the agent's environment in the form of perception and action modules. Additionally, emotions, mood and feelings can influence the current perception/attention of the agent as well as decision-making.

To ground the agent's possible cognitive processes in realistic traffic behavior, we add a psychological component to each agent. The "Five Factor Model", a common method in psychology, is used to classify personalities based on five distinct character traits. Researchers have tried to correlate the model's factors with driving behavior, but according to Herzberg the results show varying success, with conscientiousness being the only trait consistently connected to driving behavior [START_REF] Herzberg | Beyond "accident-proneness": Using Five-Factor Model prototypes to predict driving behavior[END_REF]. However, in his studies, Herzberg was able to show a correspondence between driving behavior and personality prototypes. The prototypes are based on trait configurations and three classes are especially prominent: resilient, overcontrolled, and undercontrolled drivers.

To identify a subject's personality, tools like the NEO-Five-Factor Inventory (NEO-FFI) can be used. The NEO-FFI is a questionnaire that assigns scores to the five traits of the FFM [START_REF] Borkenau | Neo-Fünf-Faktoren Inventar (NEO-FFI) nach Costa und McCrae[END_REF]. Individuals belonging to the resilient class score low in neuroticism but high for other traits. Overcontrollers score high in neuroticism and conscientiousness, low in extraversion and openness, and average in agreeableness. Undercontrollers show high scores in neuroticism and openness, low scores in conscientiousness and agreeableness, and average scores in extraversion. Drivers belonging to the class of undercontrollers tend to take more risks and drive more aggressively. On the other hand, overcontrollers tend to closely follow traffic rules. Resilients are individuals whose behavior lies somewhere in between the other two classes [START_REF] Herzberg | Beyond "accident-proneness": Using Five-Factor Model prototypes to predict driving behavior[END_REF].

To test our agent model, we defined a scenario which challenges regular rulebased traffic agents. If agents simultaneously approach an intersection from all sides, the rules of right-hand traffic demand that each agent yields to the one coming from the right. If every traffic participant abided strictly by the rule, a deadlock situation would result, leading to an irresolvable congestive state. To prevent this, drivers in real traffic would communicate -typically through hand signs -who is willing to waive the right of way. Interestingly, one of the participants would thus consciously break a traffic rule but resolve the deadlock.

To determine which traffic agent gives up its right of way in a deadlock situation, we considered the three psychological prototypes. Since undercontrollers tend to drive aggressively, they are not likely to give in. This explains the low agreeableness scores of this class. People with high scores in agreeableness usually oblige to other people's needs, tend to avoid confrontation, and adapt their behavior. High agreeableness scores among resilients suggest that drivers from this group would probably allow other drivers to pass. One would suspect overcontrolled drivers to do the same, since they are overly careful. Yet, they also strictly adhere to traffic rules and might thus be more reluctant to waive their rights. This is well reflected in the fact that overcontrollers score only average in agreeableness. Since overcontrollers and resilients are more likely to yield than undercontrollers, and since resilients are more likely to yield than overcontrollers, we use a combination of conscientiousness and agreeableness traits to determine which traffic agent waives the right of way in a deadlock situation.

Experimental Setup

To investigate the proposed ideas, a model of an uncontrolled intersection with a priority to the right system in effect, was implemented within the Unity 3D engine as shown in Figure 2. The basic system consists of a closed loop of about 1 kilometer of road around a single intersection in the center, i.e. each lane leads to and from the intersection. The system is populated with a specified number Fig. 2. For evaluation, a closed road system with an uncontrolled four-way intersection was modeled. Colored vehicles represent psychological prototypes: undercontroller (red), overcontroller (blue), and reslient (yellow). In the depicted situation a resilient driver waives its right of way (indicated by a yield sign) allowing the driver on its left to cross the intersection which resolves the deadlock situation.

of traffic agents which follow the road until they reach the intersection. There, they randomly decide whether to go right, left or straight on.

For the basic driving behavior, the intelligent driver model (IDM) was implemented. The IDM is a simple yet complete model for microscopic traffic simulations [START_REF] Kesting | Agents for traffic simulation[END_REF], which considers maximum acceleration, desired velocity or minimum headway. The parameters and their values were chosen according to suggestions in the literature [START_REF] Treiber | Verkehrsdynamik-und Simulation[END_REF]. At the core of the IDM is the calculation of the agent's current acceleration. In a free-flow driving state the agent simply accelerates, based on the given preferences and vehicle characteristics, until it reaches its desired velocity. In the experiment, the desired velocity was set to 30 km/h which typically is the speed limit in areas with priority-to-the-right intersections. If the agent is following another vehicle, a deceleration term based on the difference between the current headway and the desired headway results in adaptations of velocity.

The first part of the experiment was to fill the road network with a specified number of simple, rule-based agents. The agents were randomly distributed across the network and once the simulation started, the time taken to reach a congestive state was measured. For each specified number of agents (15, 20, 25, 30, and 35) fifteen trials were performed.

For the second part of the experiment the rule-based agents were enhanced with a component representing a psychological personality profile. Agents were divided into overcontrollers, undercontrollers, and resilients. Since Herzberg found an approximately even distribution of the three classes amongst drivers [START_REF] Herzberg | Beyond "accident-proneness": Using Five-Factor Model prototypes to predict driving behavior[END_REF], we also distributed our simulated traffic agents evenly across personality prototypes. Initial values for agents were set according to the NEO-FFI scores of the respective psychological traits, scaled to a range between -1 and 1.

In a deadlock situation, the personality traits of the agents determine who gives up its right of way. To implement this mechanism, agents were given the ability to perceive the information necessary to detect deadlock situations and to signal their willingness to waive their priority. To test the robustness of the system, several simulations with 35 vehicles were run for 15, 45 and 120 minutes.

Results

Assuming an average vehicle length of 4 m, theoretically the maximum capacity of the entire road network corresponds to about 530 vehicles. As expected, deadlock situations occurred for numbers far below this value. The exact moment when a deadlock occurs, depends on the initial vehicle distribution and routing decisions at the intersection. Thus, the fewer number of agents in the system, the higher the chance that a deadlock situation in the system will be delayed.

Nevertheless, even at only 3% of maximum capacity (15 vehicles), deadlocks were observed after only 333 seconds on average. At the same time, we also observed two occasions where the simulation continued for 1172 s and 957 s. However, since the median value (216 s) was considerably lower than the average, these cases can be considered as outliers. As illustrated by Figure 3, the time taken to reach a congestive state rapidly decreased as the number of agents in the system increased. An increasing number of vehicles, too, diminished the effect of advantageous routing decisions, since more agents had to wait at the intersection.

Introducing only five additional agents into the simulation decreased the average deadlock-free time by about 3 minutes, while still leaving room for times above the 10-minute mark. However, at 4.7% capacity (25 vehicles), the average and median times fell below one minute, with mean values of 43, 36, and 28 seconds and median values of 46, 36, and 26 seconds for trials with 25, 30, and 35 vehicles respectively. In accordance with the aforementioned observations, the values depicted in Figure 3 also show a decrease in deviations of simulation times as the number of agents increases.

Using our extended model, the simulation was run for three different periods of time. In each of the three trials the traffic never came to a complete standstill. 42 deadlock situations occurred during the 15-minute trial, 116 during the 45minute trial, and 327 during the 120-minute trial. All situations were resolved allowing the traffic to keep flowing. Figure 4 illustrates a direct comparison of the rule-based agents and our extended model. To achieve comparability between simulations, the random number generator was initialized with identical values, resulting in equal decisions for vehicle distribution and routing. Then the traffic flow at the intersection was measured by counting how many vehicles crossed the intersection in a time interval of one minute. The first deadlock occurred after 45 seconds, but as evident from Figure 4 (a), the flow never reached zero for the enhanced agents. At the same time, the average speed of each vehicle in the simulation was recorded for the duration of the simulation. Figure 4 (b) shows how the average speed continuously decreases after the first occurrence of a deadlock situation for both agent types. However, for the enhanced model the average speed never decreases below 10 km/h.

Conclusions and Future Work

In this paper, we discussed the need for more realistic traffic agents in games and simulations. We suggested extending regular, rule-based traffic agents with cognitive components. In particular, in an attempt to equip agents with more realistic behavior, we suggested the addition of psychological profiles based on the "Five Factor Model", dividing agents into three prototypical groups. We evaluated the impact of this extension in a specific traffic situation that cannot be handled by rule-based agents, and we found that our approach is able to seamlessly resolve deadlock situations at uncontrolled intersections. At the current state of development, the static personality of an agent determines which agent gives up its right of way. In the worst case, this would always be the same agent, forceing it to wait for an extended time to cross the intersection. Currently, we are working towards modifying static personalities by dynamic elements such as emotions, mood and feelings. In this way, frustration levels could be modeled to increase with prolonged waiting times, which would in turn decrease the likelihood of agents giving up their right of way too often. Other improvements concern the agents' perception and the number and complexity of their available actions. However, the most challenging task remains to implement models of internal perception and planning to increase the realism of the agents' overall behavior for traffic simulations in virtual environments.

Fig. 1 .

 1 Fig. 1. Cognitive traffic agents for virtual environments could contain the depicted modules from cognitive architecture research. The psychological profile is depicted in a separate layer to indicate that it may influence all cognitive processes.

Fig. 3 .

 3 Fig.3. The box plots depict the times until the simulation of rule-based agents resulted in a deadlock situation at the four-way intersection. For each number of vehicles 15 simulations were run.

Fig. 4 .

 4 Fig. 4. A comparison of rule-based agents with the extended model using psychological personality profiles. The simulations were run for 10 min. with the first deadlock occurring after 45 sec. (vertical line). The flow (num. of vehicles per min.) was measured at the intersection (a) and the average speed of all vehicles in the simulation (b).

Acknowledgements. The FIVIS project received additional funding from the Deutsche Gesetzliche Unfallversicherung (grant FP307). The AVeSi project is being funded by the FHprofUnt program of the BMBF (grant 17028X11).


David Kadish
email: dkadish@interchange.ubc.ca

Nikolai Kummer
email: nikolai.kummer@ubc.ca

Aleksandra Dulic
email: aleksandra.dulic@ubc.ca

Homayoun Najjaran
email: homayoun.najjaran@ubc.ca

The Empathy Machine Generated music to augment empathic interactions

Keywords: emotional music synthesis, facial expression recognition, empathic interaction

The Empathy Machine is an interactive installation that augments a visitor's empathic sense during a social conversation. Empathy is a key component of interpersonal interactions that is often neglected by modern communication technologies. This system uses facial expression recognition to identify the emotional state of a user's conversation partner. It algorithmically generates emotional music to match the expressive state of the partner and plays the music to the user in a non-disruptive manner. The result is an augmentation of the user's emotional response to the emotional expression of their partner.

Introduction

Empathy, the ability to share feelings with others, is an important part of interpersonal communication and plays an essential role in human relations. Many communication technologies such as telephones and e-mail inhibit one or more of the senses, becoming barriers between the communicators that reduce the opportunity for empathic interaction.

This paper presents a proposed demonstration of a system that augments empathy in live interpersonal interactions. The installation will allow a user to experience a conversation, while receiving direct feedback regarding the emotional state of the conversation partner in the form of music.

Empathy, Emotion and Music

The conceptual basis for the installation is rooted in the hypothesis that music can trigger an empathic response, similar to the response that occurs when one views an emotional expression. Empathy is the ability to identify with the feelings and emotions of another person or being. The neurological basis for empathy is the existence of so-called "mirror neurons" [START_REF] Preston | Empathy: Its ultimate and proximate bases[END_REF]. Mirror neurons are activated both when a subject sees an action or emotional display and when they perform that same activity. Emotional empathy, then, can manifest as a person feeling the same emotion as someone they are observing simply by seeing an emotional gesture on the part of the observed.

In addition to visual stimulation, music is often cited as a method of evoking an emotional response in a person [START_REF] Rickard | Intense emotional responses to music: a test of the physiological arousal hypothesis[END_REF]. In that way, music provides a secondary pathway to the emotional centres of the brain, which can be used to augment the emotional response derived from visual imput. The various properties of music that elicit or represent certain emotions have been studied by scientists since the 1930s [START_REF] Hevner | Experimental Studies of the Elements of Expression in Music[END_REF].

3 Proposed Installation

Overview

The Empathy Machine augments a social interaction between a user and their conversation partner. The emotional state of the partner is assessed using automatic facial expression recognition (FER) software and the feedback to the user is played back to the user in the form of emotional music. The music is generated using an emotional music synthesis (EMS) system.

The user and their conversation partner sit at a table in the demonstration area. A webcam peers over the shoulder of the user at their partner. Video of the partner's face is input into the FER system, the expression is recognised, appropriate music is generated, and the result is played at background levels to the user through small speakers.

Facial Expression Recognition

The FER system is a modified version of the system described by Valstar and Pantic [START_REF] Valstar | Combined support vector machines and hidden markov models for modeling facial action temporal dynamics[END_REF]. FaceTracker, an OpenCV-based algorithm for face registration and tracking based on [START_REF] Saragih | Deformable model fitting by regularized landmark mean-shift[END_REF], is used to track 66 points on a face across a series of frames.

The python machine learning package scikit-learn is used to perform classification on the features. For each facial action unit (AU) -the basic unit of facial motion -principle component analysis (PCA) is performed on the extracted features and a support vector machine (SVM) classifier is trained on the principle components to determine whether that AU is active. A second classifier is trained to map AU activation to the displayed emotion, using a multiclass one-versus-one SVM. This is used to distinguish between the display of one of four basic emotions and a neutral state.

Music Generation

The music used to represent the emotions is computer generated. Computer generated music lends itself to continuous generation and does not require an in-depth knowledge of musical composition. The music is generated using the AthenaCL algorithmic music generator. Algorithmic composition allows for the creation of non-repetitive music, thereby reducing strain on the user's ears. Musical parameters for the emotional music were selected from literature [START_REF] Gabrielsson | The influence of musical structure on emotional expression[END_REF][START_REF] Juslin | Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening[END_REF][START_REF] Wallis | A Rule-Based Generative Music System Controlled by Desired Valence and Arousal[END_REF]. A summary of the parameters used for the music generation in the user study can be found in Table 1.

Future Development

The initial prototype of the Empathy Machine focuses on four emotions in a discrete sense. Future implementations will consider a more nuanced notion of degrees of emotion and will seek to add more basic emotions to the set of four. More effort will also be put into making the FER system more robust and to adding further complexity to the EMS system.

Conclusion

The use of music to create an emotional response is common in film and performance. The Empathy Machine represents the first attempt to use music generated in real-time to augment the emotional response to a live interaction. The authors hope to provide an engaging demonstration of the possibilities that arise from the combination of real-time emotion detection and music generation.

Fig. 1 :

 1 Fig. 1: Overviews of the Empathy Machine system

Table 1 :

 1 Association between the emotions and musical features

	Tempo Pitch Rhythm Roughness MIDI Instrument	Additional Parameters
	Anger Fast	High	Few repeated notes	#2 Piano	Stoccato, Phrygian mode
	Fear Fast	High		#100 FX 4 Atmosphere Atonality
	Happiness Fast		Few repeated notes	#47 Orchestral Harp	Stoccato, Lydian mode
	Sadness Slow	Low	Repeated notes	#2 Piano	Legato

Acknowledgments. The authors would like to acknowledge the assistance provided by Dr. Sidney Fels as well as the support of the Advanced Control and Intelligent Systems Lab and the Centre for Culture and Technology at UBC.


Kathrin M Gerling
email: kathrin.gerling@acm.org

Frank P Schulte
email: frank.schulte@uni-due.de

Jan Smeddinck
email: smeddinck@tzi.de

Maic Masuch
email: maic.masuch@uni-due.de

Game Design for Older Adults: Effects of Age-Related Changes on Structural Elements of Digital Games

Keywords: Game design, older adults, accessibility, design recommendations

Recent studies report various positive effects on elderly persons playing digital games. Yet, games are rarely designed with an elderly user group in mind. In this paper, this issue is addressed by providing an overview of common age-related changes followed by a summary of game design considerations for senior audiences. The impact of age on game design is discussed based on an analysis of the most important structural elements of games. The analysis shows that age-related changes in users' cognitive and physical abilities affect the use of games on multiple levels, making the complexity of games and interrelations between different game mechanics a crucial factor when designing for older adults.

Introduction

Engaging with digital games has a positive impact on the emotional and physical wellbeing of elderly in nursing homes as recent research suggests [START_REF] Jung | Games for a Better Life: Effects of Playing Wii Games on the Well-Being of Seniors in a Long-Term Care Facility[END_REF]. However, many commercially available gaming products are not suitable for elderly persons [START_REF] Gerling | Exergame Design for Elderly Users: The Case Study of SilverBalance[END_REF]. Apart from the development of therapeutic applications and serious games, a structured approach to digital game design for an elderly audience based on age-related changes and impairments has rarely been the subject of academic work [START_REF] Nap | Senior gamers: preferences, motivations and needs[END_REF]. Understanding both chances and challenges of an elderly target audience is vital, if senior citizens are to be addressed by and engaged in video games [START_REF] Ijsselsteijn | Digital Game Design for Elderly Users[END_REF]. Core aspects that might foster or hinder the engagement of seniors in the use of video games as leisure activity must be identified. This paper provides an overview of age-related changes and diseases and discusses existing approaches towards game design for senior citizens, which are extended based on a structural analysis of digital games. This section summarizes age-related needs and changes that are caused by regular aging processes and provides an overview of game design recommendations to foster the creation of accessible games for older adults.

Common Age-Related Changes among Senior Citizens

From a biopsychological perspective, aging affects the quality of life on various levels. Besides sensory decrements in vision and hearing [START_REF] Gerling | Exergame Design for Elderly Users: The Case Study of SilverBalance[END_REF], older adults often experience:  Cognitive impairments, which affect problem solving skills, information processing [START_REF] Birren | Handbook of the Psychology of Aging[END_REF][START_REF] Gamberini | Cognition, technology and games for the elderly: An introduction to ELDERGAMES Project[END_REF] and result in a reduced attention span when working on complex tasks [START_REF] Birren | Handbook of the Psychology of Aging[END_REF][START_REF] Czaja | Information Technology and Older Adults[END_REF].

 A decline of motor skills, which includes a decrease in fine motor skills, changes in posture and balance [START_REF] Birren | Handbook of the Psychology of Aging[END_REF], and negatively affect motor learning [START_REF] Birren | Handbook of the Psychology of Aging[END_REF][START_REF] Czaja | Information Technology and Older Adults[END_REF].

 Chronic illnesses, which range from arthritis to severe heart conditions, and which have an impact on the physical abilities and mobility of senior citizens [START_REF] Czaja | Information Technology and Older Adults[END_REF].

Older adults living in full-care nursing homes are most likely to be severely influenced by age-related impairments [START_REF] Czaja | Information Technology and Older Adults[END_REF]. Additional diseases may lead to more physical impairments in late life. Generally speaking, age-related changes limit the patient's possibilities of participating in regular leisure activities and lead to special requirements for the design of interactive entertainment systems.

Game Design Considerations

In the early 80s, Weisman introduced video games to institutionalized older adults [START_REF] Weisman | Computer Games for the Frail Elderly[END_REF]: Based on observations, he claims that games with an adaptable level of difficulty are advantageous because they support individual preferences and sensomotor abilities best. The use of large and well-defined visual symbols and clear auditory feedback is recommended to address visual and auditory impairments. Almost 25 years later, Ijsselsteijn et al. [START_REF] Ijsselsteijn | Digital Game Design for Elderly Users[END_REF] identify the decline of visual and auditory abilities as well as a loss of sensomotor skills as most important age-related impairments, which influence the use and reception of digital games by senior citizens. They suggest visually adjustable games (e.g. regarding the font-and window-size, colors and contrast) that deliver multimodal feedback relying on more than one communication channel and feature user interfaces that are of low complexity. Flores et al. [START_REF] Flores | Improving patient motivation in game development for motor deficit rehabilitation[END_REF] include the design of appropriate cognitive challenges, which have the potential of keeping the interest of older adults. Based on findings from the ElderGames project, usability criteria for advanced technologies for the elderly were compiled [START_REF] Gamberini | Cognition, technology and games for the elderly: An introduction to ELDERGAMES Project[END_REF]. They include a reduction of steps necessary to complete tasks, a reduction of the cognitive load, the availability of immediate feedback and the adaptation of digital systems to the users' goals. Additionally, the importance of system consistency and fault tolerance is highlighted.

Furthermore, when designing entertainment systems for senior citizens, it seems important to consider the impact of game content, genres and benefits from playing digital games. Ijsselsteijn et al. [START_REF] Ijsselsteijn | Digital Game Design for Elderly Users[END_REF] identified games for relaxation and entertainment, games which support social activities, games to sharpen one's mind, and exertion games as suitable game types. Flores et al. [START_REF] Flores | Improving patient motivation in game development for motor deficit rehabilitation[END_REF] highlight the creation of meaningful play through learning objectives and social play. Results from the ElderGames project suggest that elderly players have a preference for simplistic puzzle and quiz games, which may have a positive impact on daily life [START_REF] Ijsselsteijn | Digital Game Design for Elderly Users[END_REF][START_REF] De Schutter | Meaningful Play in Elderly Life[END_REF]. De Schutter and Vanden Abeele [START_REF] De Schutter | Meaningful Play in Elderly Life[END_REF] identified connecting players, cultivating skills and contributing to society as important aspects of digital games for older adults. In terms of game features, the social aspect of games is frequently addressed by the recommendation of a multiplayer mode. A large amount of senior citizens engages in games as active non-players who watch others play, cheer and comment on in-game situations [START_REF] Shim | TableTalk Poker: An Online Social Gaming Environment for Seniors[END_REF].

The guidelines and design recommendations presented within this section cover the issue of aging and game design on a formal level and primarily address usability and accessibility issues. The majority of guidelines focuses on the impact of age-related changes and impairments on interface design and includes detailed recommendations for designers. However, those design guidelines that address game features and genres only discuss game design issues on a meta-level (e.g. in terms of genre recommendations) and do not give detailed information on the impact of age on structural elements of games.

Structural Models of Games

This section provides an overview of two theoretical approaches to a formal description of digital games by Adams [START_REF] Adams | Fundamentals of Game Design[END_REF] and Fullerton [START_REF] Fullerton | Game Design Workshop, Second Edition: A Playcentric Approach to Creating Innovative Games[END_REF]. For an overview of the development of the definition of games cf. Salen and Zimmerman [START_REF] Salen | Rules of Play: Game Design Fundamentals[END_REF].

Fullerton's [START_REF] Fullerton | Game Design Workshop, Second Edition: A Playcentric Approach to Creating Innovative Games[END_REF] formal elements of games include players, objectives, procedures, rules, resources, conflict, boundaries and an outcome. Adams' [START_REF] Adams | Fundamentals of Game Design[END_REF] structured approach towards the key components of video games primarily focuses on the interaction between core game mechanics and the user interface. Core mechanics include in-game resources, further attributes as well as general mechanics [START_REF] Adams | Fundamentals of Game Design[END_REF]. The user interface acts as a broker between core mechanics and the player. Because of the strong focus on interface design and its relation with game mechanics, Adams' model offers the opportunity of combining interface-related design guidelines with complex criteria derived from age-related needs and requirements which are expected to map on the complexity of digital games. In contrast, Fullerton provides a broad approach towards the structure of games that covers relevant areas of game design.

Creation of an Extended Model

Both Adams and Fullerton refer to games as complex systems that feature different sets of mechanics to create a unique user experience. These mechanics are affected by user input. According to Adams, the interaction model turns user input into actions, which may be interpreted by the core mechanics of the game [START_REF] Adams | Fundamentals of Game Design[END_REF]. Fullerton refers to actions as a part of procedures that are deeply rooted within the rules of the game, but does not define the precise relationship between user interface and game mechanics. Additionally, her concept of procedures includes both player actions and system procedures, which incorporate complex methods of play [START_REF] Fullerton | Game Design Workshop, Second Edition: A Playcentric Approach to Creating Innovative Games[END_REF]. While Fullerton suggests that the outcome of the game affects the player and determines whether the conflict was successfully resolved, Adams states that new challenges may be presented to the player via the user interface. Because Fullerton does not give detailed information regarding the relationship between game mechanics and the user interface, Adams' model offers the possibility of extending her definition. Other than that, Adams' definition of core mechanics is not as precise as Fullerton's description of the structure of games. In the extended model, Adams' camera model is referred to as feedback model, including visual, haptic and auditory output. Figure 1 shows a combination of the approaches.

Effects of Age-Related Changes on Structural Models of Games

Based on the joint model, the relevance of age-related changes and age-associated diseases in the context of game design can be analyzed with a focus on the impact of age on the structural elements of games.

Players and resources. When designing for today's senior audience, game designers have to create games for a target group with -at least nowadays -very limited gaming experience. Since research has shown that elderly persons enjoy both engaging in active play as well as following games as spectators and commenting on other players, designers should address the needs of those audience participants, e.g. by offering interesting visual sceneries, or fostering competition similar to sports events. Depending on the context of play, different competition types seem appropriate for elderly persons. "Local team versus game" competitions offer the opportunity of engaging groups of senior citizens in play, online multiplayer functionality offers the possibility of experiencing competition or cooperation to senior citizens who live an independent life. Resources are another important aspect that should be considered. The attention span of the player may be a relevant resource: When games require elaborate decision making or quick reactions, changes in short-term memory might affect the player's ability to recall in-game information at a later point. Sensorimotor skills are an important resource if certain game mechanics include complex interaction paradigms or repetitive user input and time-critical player feedback. Generally, the availability of in-game resources should not be as restrictive as for a younger audience and ideally be individually adjustable.

User interface. The use of input devices may also be limited due to age-related changes in motor skills. While active senior citizens may still be able to use regular game pads, institutionalized older adults may experience difficulties in holding the controller in their hands while pressing small buttons. Furthermore, the complexity of input sequences should be significantly reduced for an elderly audience to account for decrements of cognitive abilities, especially attention processes and short-term memory. In this context, possible difficulties with parallel player input (e.g. moving a haptic device and pressing a button at a certain point of time or performing complex point-and-click operations) should be considered. Additionally, the graphical user interface has to be adjusted to meet the needs of elderly persons (e.g. with regard to visual game element size, font size, high contrast, etc.) when designing the feedback model which presents the game to the player, for instance via visual, auditory or haptic output.

Core mechanics. Core mechanics describe internal processes and requirements of digital games. They consist of procedures, rules, objectives and conflict and define both challenges of and players' interactions with the game. Certain aspects of core mechanics (e.g. limitations caused by rules) have to be learned in order to master a game. Decrements in memory and elderly persons' attention span affect information processing from short-term to procedural memory. Thus, the ability of learning how to play a game is limited, and learning processes take more time if the player is affected. In-game actions available to the player -procedures -should be easy to understand and ideally incorporate metaphors of daily life to facilitate learning processes. Additionally, designers should avoid parallel input and quick sequential reactions to lower the player's cognitive load and account for changes in sensorimotor abilities. Rules should be easy to understand to facilitate the player's entry into game play. In this context, elderly players may need additional and more detailed information to understand restrictions and requirements of digital games due to their lack of gaming experience. Objectives have to be more clearly defined and communicated to the elderly player. Since objectives have an impact on the overall direction on the game, it is not only important to account for agerelated changes, but also to consider suitable game genres and features.

Outcome. Because age-related impairments are likely to reduce the player's ability to master games in comparison to a younger audience, it is important to consider the effects of impairments when defining the outcome of the game. Adjustable limits for the internal assessment of player success offer the possibility of providing the user with feedback which is based on his or her individual skills.

Application of the Model

Based on the evaluation results [START_REF] Gerling | Designing and Evaluating Digital Games for Older adults[END_REF] and observations during the design process of SilverPromenade, a game specifically addressing older adults, we discuss the applicability of the extended model of digital games in game design for senior citizens and potential extensions and future use-cases.

When creating the game design concept for SilverPromenade, we applied the extended model of digital games at an early stage of the development process. First, we created a general vision outline of the game concept. Second, we redefined single aspects using the list of structural elements and the associated recommendations regarding the impact of age. The model helped shape our game design document and provided a foundation for the more detailed analysis of certain game design decisions, the discussion of design alternatives and the integration of suitable game mechanics, e.g. the inclusion of the three different player roles to accommodate different abilities. Finally, when preparing the evaluation of the SilverPromenade, the model could be used to draw attention to important aspects of the game and provided a foundation for observations during the study. On a general level, the extended model of digital games with a focus on age-related changes provides a structure for designers to consider important issues associated with senior audiences during the design process. Its broad approach can help them to encompass basic needs of older adults and foster the creation of accessible games; the model embraces a wide range of game design elements supporting the systematic examination of the impact of ageing on the interaction with video games. In contrast to previous work on game design recommendations for elderly persons, the model accounts for the connection of interface and game design by linking the effects of core mechanics as well as the interaction and feedback model. Recognizing interrelations between user interfaces and game mechanics is also important when considering the growing popularity of motion-based games, in which interaction paradigms are frequently matched with game mechanics and which hold the promise of fostering physical activity.

Limitations of the model. Even when accounting for common age-related changes, older adults represent a heterogeneous target audience, thus general player abilities are difficult to capture. The evaluation of SilverPromenade showed that although our model covers the most important issues of game design for elderly, it lacks individual details that are difficult to integrate. To solve this issue, we suggest a user-centered design process which allows for user-testing at an early stage for further adjustments. Additionally, the current version of the model accounts for technical and structural aspects of game design only. In order to fully grasp the interaction between players and the game, the aspect of user experience needs to be examined and integrated into the existing approach. This is especially interesting when designing for a senior audience as only little research focused on the game experience of elderly players. Another opportunity which has not been explored and that could be examined based on structural considerations is the design of narrative and dramatic elements in video games. Generally speaking, the model presented in this paper provides a broad overview of common age-related issues as a starting point for further considerations. Individual adjustments have to be made based on an analysis of the particular target audience and application context to ensure the accessibility.

Conclusion and Future Work

In this paper, the impact of prevalent age-related impairments among senior citizens on structural elements of video games was discussed based on a joint model of digital games featuring the work of Adams [START_REF] Adams | Fundamentals of Game Design[END_REF] and Fullerton [START_REF] Fullerton | Game Design Workshop, Second Edition: A Playcentric Approach to Creating Innovative Games[END_REF], which was then evaluated through the design of SilverPromenade [START_REF] Gerling | Designing and Evaluating Digital Games for Older adults[END_REF]. The analysis showed that cognitive and sensory decrements as well as age-associated diseases affect various aspects regarding the use of digital games and have to be considered when designing for older adult players. While sensory decrements mostly affect the design of the user interface and appropriate design considerations have been presented by many authors, cognitive impairments and the lack of gaming experience among today's elderly population are an important factor when designing game mechanics. In this context, the analysis has shown that the complexity of digital games and interrelations between different mechanics are a crucial factor when designing for older adults: On the one hand, elderly players have to be provided with sufficient information to be able to interact with the game adequately. On the other hand, core mechanics of the game have to be simplistic and easy to learn in order to reduce the cognitive load of an elderly audience. A structured analysis of digital games and their interrelation with age-related changes helps fostering the integration of concerns regarding the target audience early in the development process. It may also help to guide designers even if they have no prior experience in working with elderly. Future work includes the further verification of the feasibility of design recommendations based on the structural approach towards digital games presented in this paper. We aim to examine the usage of our model beyond academic game development to gain further insights into the applicability of our work in an industry context.

Figure 1 .

 1 Figure 1. The extended model of digital games (based on Adams and Fullerton).


Lucio Gutierrez

Eleni Stroulia
email: stroulia@ualberta.ca

Ioanis Nikolaidis
email: nikolaidis@ualberta.ca

fAARS: A Platform for Location-Aware Trans-reality Games

Keywords: Game platform, trans-reality games, virtual worlds, mobile games

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Location-based games are a type of pervasive games that use the physical space of our entire world as a game board [START_REF] Magerkurth | Pervasive games: Bringing computer entertainment back to the real world[END_REF]. Their implementation relies on special-purpose middleware that integrates the players' smart-phones so that they can share information about their location, surroundings and actions. The players' game experience can be further enhanced with parallel virtual worlds, so that they can experience the game in the real world and in alternate realities, possibly at the same time. This technological convergence, on one hand, and the pervasiveness of the "gamification" concept across many aspects of our activities today, on the other, has brought to the forefront a new breed of pervasive games called "trans-reality games". This term refers to a relatively new, yet increasingly interesting, class of games, played using mobile devices indoor, outdoor and in virtual worlds. Trans-reality games were first conceptualized by Lindley in 2004. This type of games can be deployed and played in "natural" neighborhoods, such as a campus, a city, or a broader geographical area. Game players interact with the games (a) in the real world through location-specific clues communicated to them through smart-phones that "augment reality" and are aware of each other's locations using GPS or QR codes, and (b) in (possibly multiple) parallel virtual world(s), which represent alternate reality(ies) that reflect the real world in some dimensions and extend it in others.

The fAARS (for Augmented Alternate Reality Services) platform supports the implementation and deployment of trans-reality games in the virtual treasure-hunt and role-playing style. It consists of a set of components, flexibly integrated with its event-driven game engine, through RESTful APIs. So far, we have developed two different trans-reality games using the fAARS platform. The first game was the "Human Geometric Orientation" game. Through this game, players in a virtual world participate in an experiment, designed to study human orientation mechanisms. The second game, "Outbreak: Safety First", is a "serious game" designed to educate health-science students in precautionary procedures for avoiding nosocomial infections, and can be played in three different game-play modalities.

In the rest of this paper we discuss our work and experience to date. In section 2, we review the background for our work, by reviewing recent trans-reality games and existing platforms for location-aware games. In section 3, the design and implementation of fAARS and its constituent components are fully described. In section 4, we report on our evaluation of the platform through the development of two separate games. Finally, in section 5, we conclude with the contribution of our work.

Trans-reality Games and Pervasive-Game Platforms

The "pervasive games" heading includes a broad range of games that integrate ubiquitous technology with physical spaces to provide a gaming experience. Montola [START_REF] Montola | Exploring the Edge of the Magic Circle. Defining Pervasive Games[END_REF] introduced a conceptual framework for analyzing the spatial, social, and temporal features of pervasive games, leading to a classification of these games into six types. Mixed-reality games are played in physical spaces where virtual and physical components co-exist. Trans-reality games are played in physical and virtual spaces at the same time, with the actions of the player avatars in the virtual world affecting the game play in the real world and vice versa. Adaptronic games are played in a virtual world but react to real-world events. Crossmedia games involve different types of pervasive technologies. Alternate-reality games create the illusion that game-play events should be considered "real", thus creating an alternate reality where players act according to hidden clues. Finally, in Reality games, everything that happens in the game is real and can have consequences on the real life of players.

Trans-reality games are a particularly interesting and challenging class of pervasive games. "ARQuake" [START_REF] Piekarski | Arquake: the outdoor augmented reality gaming system[END_REF] was one of the first outdoor/indoor augmented-reality location-aware games based on the desktop game Quake. Two years later, "Human Pac-man" [START_REF] Cheok | Human pacman: a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing[END_REF] was the first outdoor augmented-reality game to blend the real and virtual dimensions. Both games required players to wear their particular versions of cumbersome equipment. In 2005, "Magic Land" [START_REF] Qui | Magic land: live 3d human capture mixed reality interactive system[END_REF] blended 3D avatars of real players with 3D computer-generated animations, fusing the real and the virtual worlds in a novel way. Between 2004 and 2006, trans-reality games, such as "Can you see me now?" [START_REF] Benford | Can you see me now?[END_REF], "Uncle Roy All Around You" [START_REF] Benford | Uncle Roy All Around You: Implicating the City in a Location-Based Performance[END_REF], and "I Like Frank" [START_REF]Blast Theory[END_REF], demonstrated the use of mobile devices in real-world playgrounds blurred with virtual worlds, with players in both worlds advancing the game, although the actions allowed to players in the virtual world were very limited. These games were developed on special-purpose hardware and software technologies, with no reuse potential. Furthermore, in the above trans-reality games, virtual worlds have been used primarily as an information resource, and there is still substantial work to be done towards the comprehensive integration of virtual-and real-world game-play. As fAARS is on the overlap between trans-reality and location-aware mobile games, we review above the features of some important platforms for location-aware games. The comparison of the features of these platforms is summarized in Table 1. As it can be seen in the table, a few of the platforms support indoors and outdoors game-play but none support virtual worlds; thus, they are not able to support transreality games. Most are tailored to individual game-play, except from FRAP that also supports teams. Most augment the players' game-play experience with game-related content; some include an augmented-reality browser that superimposes this content on the image that players see through their cameras, or on a map where they can see their (and other players') location(s). All support treasure-hunt games; Mupe also supports role-playing games and FRAP supports capture-the-flag games. All support communication between players through the game engine; Mupe and PCFPA also support peer-to-peer interactions of players in close proximity to each other. Finally, most of the platforms offer some game-authoring tools.

3

The fAARS Software Architecture fAARS was conceived as a new-generation pervasive-game platform, extending the space of the game play from the real world (indoors and outdoors) to parallel virtual world(s) in order to support location-aware trans-reality games. In these games (a) the real-world space and locations (and any information associated with them) are mapped to the virtual world; (b) the movement and actions of the real-world players is reflected through their avatars in the virtual world; (c) the movement and actions of the virtual-world players is communicated to real-world players through their smart phones; and (d) in addition to real-world and virtual-world players, non-playing characters, controlled by a simulation engine, can also change the game state. fAARS supports the deployment of trans-reality games in the virtual treasure-hunt and roleplaying game styles. fAARS integrates OpenSim (http://opensimulator.org/) as the virtual-world game-play environment and considers two types of mappings between the real-and virtual-world spaces. In the "parallel worlds" mapping, the virtual-world setting mimics the real-world setting and the coordinates of every location in the realworld playground are transformed to a pair of virtual-world coordinates. In the "points-of-interest" mapping, the virtual world may represent a conceptually different metaphor from the real world, and only specific points of interest may be mapped across the two. In this case, the players' locations can only be reflected across the two worlds when the players are close to any of the pre-defined points of interest. fAARS supports three different game-play modalities [START_REF] Lindley | Game space design foundations for trans-reality games[END_REF]. In the Dormant modality, players play the game in the real world using smart-phones. In the Dreaming modality, players play the game in the virtual world. Finally, in the Astral-projection modality, players choose to play in the virtual world or in the real world using smartphones; in the latter case, the real-world players' avatars interact with other players and objects in the virtual world at the same time. The Actors: In fAARS, every player in the real and virtual worlds is represented as an Actor. In the real world, Actors interact with the game using a mobile application running on a smart-phone. The mobile application senses the Actors' location and actions, using the smart-phone GPS and a built-in QR-code reader. In the virtual world, players have the same cross-platform application running on a HUD (Heads-Up Display). The HUD senses the virtual-world through collision, proximity, and location detectors. The virtual world also provides a Scheduler and a Broadcaster. The former supports the communication between the virtual world and the game engine, through a XML-RPC port, and maintains a set of global parameters related to the virtual-world game. The Broadcaster maintains a list of the active virtual-world objects and forwards them relevant communications from the Game Engine.

The Game Engine is responsible for maintaining and advancing the state of the game; it receives Events from the Actors, processes the game rules and accordingly updates the state of the game, and notifies the players and registered Subscribers. The Game Engine consists of (a) the Event-driven Machine, (b) the Message Manager, and (c) the Game Manager. The Message Manager receives Actor-generated events and forwards them to the Event-driven Machine, which communicates with the Game Manager to query the fAARS database for the relevant game rules 1 . Based on the rules, the Event-driven Machine updates the state of the virtual world and the state of the Actors and notifies the Subscribers about Events of interest. The Message Manager notifies the Actors as necessary and updates the cross-platform mobile applications and the virtual-world HUDs, without requiring players to explicitly request updates from the Game Engine.

The Subscribers RESTful Interface: fAARS provides a REST service that allows external systems to observe a game and to provide further downstream functionality to fAARS games, as if they were part of the platform. As shown in Figure 1, Subscribers can be either Active or Passive. Active subscribers can push events to the Game Engine (through the pushEvents API) where passive systems are only notified about Events (through the registerObserver API).

The fAARS RESTful API is composed of thirty APIs, which provide access to functionality of different internal pieces of fAARS, specifically to the internal pieces of the Game Engine as depicted in Figure 1. The APIs are divided in four categories: a) event-driven machine (push events: 2 APIs), b) game manager (query the game state: 16 APIs), c) message manager (send and receive messages: 2 APIs), and d) virtual world (update the virtual world: 10 APIs).

Two Games on the fAARS Platform

We have validated the usefulness and generality of fAARS by developing two games with it.

The "Human Geometric Orientation" Game (HGO): Systematically examining whether and how each geometrical property impacts animals and humans' orientation implies the need to "build" environments in a controlled manner with their properties systematically varied. This is practically impossible and this is why our psychology colleagues decided to conduct their study by developing a fAARS game. In this game, players were trained to locate two geometrically equivalent corners in a number of parallelogram-shaped rooms in a first-person navigable virtual world. With the HGO game, the researchers could systematically specify the types of room shapes they wanted, and we were able to systematically construct them in the virtual world using the OpenSim building tools. Of course, it was impossible to construct parallel realworld settings; therefore, the HGO game was played only in the virtual world, in the so-called Dreaming modality, with players logging in the virtual world and interacting with the room corners to generate virtual-world Events.

Referring to Fig. 1. , the fAARS components used for the HGO game are the virtual world, the game engine, and an external system as an active subscriber.

• A total of 99 Actor instances (one for each subject) with unique IDs were created to allow participants to interact with the corners in each of the rooms in the virtual world. The room corners were also represented as Actor instances, in order to rely on the virtual-world collision detection for generating corner-choice events • The HGO game engine has a single rule that upon recognizing a corner-choice event, it increases the score of the Actor, if the corner choice was correct. • The Virtual World was the setting of the game with rooms constructed manually in a systematic manner and the corners associated with invisible Actors. • There is a single active event subscriber that keeps track of the number of tries of every Actor for each of the rooms in the virtual world, and updates the location of the Actor instances based on the number of tries. The "Outbreak: Safety First" Game (Outbreak): The second game developed on fAARS, "Outbreak: Safety First", was designed as a learning activity for medical students in an epidemiology course. The game narrative places the players in a hospital where they have to visit a number of patients. As they move from one patient to another they have to avoid infectious viruses and to answer questions. At the patients' rooms, they have to follow appropriate precautionary procedures to protect themselves from infections, depending on the patient symptoms.

Taking full advantage of the fAARS functionalities, this game was deployed in three different variants. The first variant was played in the real world only, in a teaching-hospital ward. Players have to scan QR codes to see the symptoms of the simulated patients they visit, to select the appropriate gear to don, to answer questions, and to get treated if they were infected. As the game evolves, players "get infected" through actions dictated by the game (scanning QR codes attached on infected landmarks) or through the exposure of their avatars to virtual viruses in the simulation running in the virtual world. When this happens, they are informed through a message to their mobile device, at which point, they have to choose appropriate measures to protect themselves and the patients. The viruses, their infection cycle, their contagion to patients and players are controlled through a simple simulation in the virtual world, setup in a parallel mapping to the real world. This type of game play represents the so-called Astral-projection modality, where virtual-and real-world players can interact with each other in any world, and activities are reflected in both worlds.

The other two game Dreaming modality variants were deployed in a virtual world, as desktop video games. The first one uses the virtual world as a clickable 2D map (virtual 2D version) of a Pacman board, on which players and viruses interact with each other. The third game variant takes place in a 3D virtual world, where the virtual space configuration is similar to the ward of the mobile version, and players have a first-person view in a navigable virtual world.

The first game variant uses all the fAARS components, including two external systems as active subscribers. The other two game variants use the virtual-world actors, the game engine, and the same two external systems as active subscribers (similar to the HGO game). Let us now review the development process for the Outbreak game.

• We created Actor instances for each of our student participants; some were realworld Actors (for the first game variant) and some were virtual-world Actors (for the other two variants). In addition, a number of Actors acting as NPCs (Nonplayer Characters) were deployed to represent the virtual-world viruses. • The Outbreak game engine has many more rules than the HGO game that control the interactions of Actors with the QR codes representing questions, the QR codes outside the patients' rooms, and the collision with viruses. The Event types used to define the ECA rules of the game ware the decodeQR and onCollision Events. • The virtual world was an essential component for all three versions of this game.

In the Astral Projection modality, the virtual world is used as an extension of the real world, where all Actor instances can be infected by NPCs representing viruses flying around in the game. In the Dreaming modality, it is in the virtual world where the interactions among all Actors and NPCs happen during gameplay. • There were two event subscribers in this game that provided further downstream functionalities for all three versions of the game. The first was a repository of questions, which allowed players to increase their score if they correctly answered a question. The second was a mobility controller that moves the virus NPCs in the virtual world in a pseudo-random fashion.

Conclusions

The development of fAARS was motivated by our realization that, with the increased maturity and adoption of mobile and virtual-world platforms and momentum of the "gamification" paradigm in education there is a great opportunity for trans-reality games. The fAARS platform supports a specific style of trans-reality games, grounded on a complex spatial metaphor and following the virtual treasure-hunt and RPG styles. The fAARS platform extends the state-of-the-art in pervasive locationaware gaming platforms in the following ways. fAARS is the only platform that enables the deployment of location-aware games in a virtual world, as an extension of the physical indoors/outdoors playground. The virtual-world playground may "parallel" the real world (with all real-world and virtual-world coordinates mapped through a bi-directional function) or it may communicate a completely different metaphor to the game story (mapping only interesting locations across the worlds). fAARS supports peer-to-peer interactions among players in the virtual world. Mimicking the functionality of NFC, a special-purpose software device in the virtual world recognizes player proximity. This event can be used by the game story to notify the players and give them opportunities to advance the game play. Finally, fAARS enables a variety of game play combinations in the real and the virtual world. The two example games discussed in Section 4 demonstrate how

Fig. 1 .

 1 Fig. 1. The fAARS Platform Software ArchitectureAs shown in Fig.1. , the fAARS platform consists of two main components: (a) the Actors (and their devices) corresponding to the real-world and virtual-world players; and (b) the Game Engine, which is responsible for recognizing the Actors' actions and inferring the next game state based on the game rules. The Actors access the game engine through the fAARS API. Furthermore, external components (i.e., Subscribers) can be integrated with the Game Engine through the Subscribers API. The fAARS platform is implemented according to the Event-Driven SOA (Service Oriented Architecture) style.The Actors: In fAARS, every player in the real and virtual worlds is represented as an Actor. In the real world, Actors interact with the game using a mobile application running on a smart-phone. The mobile application senses the Actors' location and actions, using the smart-phone GPS and a built-in QR-code reader. In the virtual world, players have the same cross-platform application running on a HUD (Heads-Up Display). The HUD senses the virtual-world through collision, proximity, and

Table 1 .

 1 Pervasive-Games Platforms

	PLATFORM	Indoors	Outdoors	Individual	Teams	Map	Augmented	Browser	Raw Game	Data	Treasure Hunt	RPG	Capture-the-	flag

Wherigo -www.wherigo.com √ √ √ √ √ √ √ PCAFPEA [10] √ √ √ √ √ √ √ √

Note that, as of today, fAARS does not offer an authoring environment; therefore, the game rules have to be stored directly in the database by someone familiar with the fAARS database design.

fAARS can be used to develop real-world only games (like the platforms discussed in Section Error! Reference source not found.), or virtual-world only games, or true trans-reality games.