Tomoko Yonezawa

Shota Yanagi

Naoto Yoshida

Yuki Ishikawa

Accelerating the Physical Experience of Immersive and Penetrating Music Using Vibration-motor Array in a Wearable Belt Set

Keywords: Sound and vibration, Vibration-motor array, Realistic sensation's expressions, Physical penetration and localization

In this research, we aim to create a heightened and physical musical experience by combining electronic sound and time-lagged multiple vibrations that surround the user's neck, chest, and back. The purpose of the research is to elevate an immersive and extended experiences of music as though the sound source had a physical presence. We developed a wearable interface of a vibration-motor array with separately controlled multiple vibration motors to simulate both strong bass sounds and movement of the physical presence of the sound source. The control of intensity and the time differences among the motors produce not only the illusion of spatial presence but also the physical penetration of the ongoing sound. The results of the evaluations showed the effects of 1) the combination of vibration and sound in the musical experience and 2) time differences of the starting timings between the front and back vibrations when creating the illusion of physical penetration as though the sound had a physical presence.

Introduction

Since the 1980s when portable music players became popular, we have been to enjoy pre-recorded music with sophisticated and lightweight players in an easy and familiar way. These players have small speakers or need external headphones that are not advantageous for creating a high-powered deep bass sound, although musical experiences in live performances or movie theaters provide a strong presence of sound as a physical phenomenon with sensible vibrations. Such experiences are realized by a powerful output of volume using large-scale devices, such as large speakers or woofers, that are not suitable for personal use.

Humans perceive various stimuli as being multimodally combined. Sub-channel stimuli can change the perception of the main stimulus, as can be seen in McGurk effect [START_REF] Mcgurk | Hearing lips and seeing voices[END_REF]. Consequently, we considered that the sound and music experience of live performance or theater with real mega volume is not perceived by the sense of hearing alone. We especially focused on the sense of touch on sound vibration; as Merchel [START_REF] Merchel | Auditory-tactile music perception[END_REF] discussed in relation to the influence of the listener's experience of whole-body vibrations, the natural feeling of sound and music is expected to provide vibrations to a greater or lesser extent.

To achieve the experience of realistic sound and music of massive volume but without expensive audio equipment, we have developed a wearable interface with a vibration-motor array that produces vibrotactile expressions in synchronization with the sound of music. Multiple vibrators provide a localized illusion of both sound and vibration. Furthermore, we focused on an illusion of the physical penetration of a sound source through the user's body based on fictive physical presence by the differences in starting times among the vibration motors. Not just massive volume at live concerts, localized vibration would aid in creating a realistic experience of immersive opera and parades with actors' movements. The physical penetration is also expected to emphasize the presence of virtual objects passing through the user's body in 3D theaters. As described above, the surrounding vibrations reproduce these physical experiences.

Thus, the localized vibration and illusion of physical penetration would also provide a novel music experience. Moreover, the sound localization would become easier not only based on normal differences of level and time (ILD: interaural level difference and ITD: interaural time difference) among multiple vibrations but also based on the illusion of the physical penetration from the ongoing sound.

As an application of this configuration, the use of the vibration-motor array as a monitor combined with sound feedback would help a player in a live performance, especially for directional and expressive assignments while the wearable device does not restrict the user's actions. As another application, the proposed method is also expected to give an impact on people with hearing difficulties. The vibration can individually express the rhythm of music even without any sound.

RELATED WORKS

First, we refer to studies that have suggested the importance of vibration to the experience of music. Merchel and Altinsoy [START_REF] Mcgurk | Hearing lips and seeing voices[END_REF] discussed the effect of wholebody vibrations on the listener's experience of listening to concert recordings with a box-type chair with a large vibrating system in it. Their results showed that music with vibration, -though only from the buttocks,-was judged better than simple sound alone. Chang [START_REF] Chang | Audio-haptic feedback in mobile phones[END_REF] et al. suggested that audio-based vibrotactile feedback on a mobile phone is preferred to simple audio and is recognized as better sound. Thus, it is presumed that vibration improves the musical experience, especially with a small device and low volume. Yoshida et al. [START_REF] Yoshida | An examination of a music appreciation method incorporating tactile sensations from artificial vibrations[END_REF] provided vibration stimuli in the user's palm using an electromagnetic coil device driven by low-frequency components of the sound. Their evaluation showed that the different types of vibrations elevated the listener's sensitivity.

There are researches on educational systems in which vibration is used as a tool for learning music. Miura and Sugimoto [START_REF] Miura | Supporting children's rhythm learning using vibration devices[END_REF] aimed to give each person a vibration of a particular rhythm of the assigned part to learn on an instrument in ensemble. They showed that a vibration device could become an effective tool for rhythm learning. Similar to their application, our proposed system is also expected to be adopted not only for rhythm learning but also for understanding sound localization to develop musical experiences, though our main purpose is not education itself.

Second, there are discussions on various fixed/wearable systems of haptic expressions for realistic sensation of sound and musical experience. Lindeman et al. designed VR immersive devices with multiple haptic feedbacks covering the user's upper body [START_REF] Lindeman | Towards full-body haptic feedback: the design and deployment of a spatialized vibrotactile feedback system[END_REF][START_REF] Lindeman | Effectiveness of Directional Vibrotactile Cuing on a Building-Clearing Task[END_REF]. You et al. [START_REF] You | Sound-specific vibration interface: Its performance of tactile effects and applications[END_REF] developed a sound-specific vibrationarray interface according to a sound stream using 16 oscillators. They found the shoulder was the most effective place to fix the attachment among the wrist, forearm, upper arm, shoulder or neck to increase immersiveness. There is also a palm-sized mobile haptic device with dual-band stimuli to produce low-and high-frequency bands for musical experiences [START_REF] Hwang | Real-time dual-band haptic music player for mobile devices[END_REF]. The dual-band stimuli showed predominance in making a good impression; however, the user's action is limited while holding the device in the hand. In contrast, our wearable system does not restrict the action of the user. According to their dual-channel feedbacks, we applied different frequency bands to express various types of music. Sakuragi et al. [START_REF] Sakuragi | CollarBeat: Whole Body Vibrotactile Presentation via the Collarbone to Enrich Music Listening Experience[END_REF] reduced vibration motors to a minimum setup with actuators only at the clavicle to efficiently provide wide vibrations with as far-reaching stimuli as possible. For more extended experiences, we focused on the vibration-motor array on not only the clavicle but also other places on the user's body.

The above three music devices [START_REF] You | Sound-specific vibration interface: Its performance of tactile effects and applications[END_REF][START_REF] Hwang | Real-time dual-band haptic music player for mobile devices[END_REF][START_REF] Sakuragi | CollarBeat: Whole Body Vibrotactile Presentation via the Collarbone to Enrich Music Listening Experience[END_REF] cover a narrow surface of the body from only one direction except Lindeman's VR garment [START_REF] Lindeman | Towards full-body haptic feedback: the design and deployment of a spatialized vibrotactile feedback system[END_REF], which was not for a musical experience. Even the 4x4 table vibration array has difficulty in generating distinct localizations because it covers only a narrow surface. The palm-sized device and the simple placement on the clavicle also cannot provide vibrations on a wide or surrounding surface. For an immersive experience and localization of the sound from various directions, we considered that a wider surface and arrayed multiple vibrations should be prepared. Accordingly, we adopted multiple vibration motors placed in various directions from the center of the body. Also, the three above studies used small vibrators that made it difficult to create strong and slow vibrations like real loudspeakers. In contrast, we adopted two types of vibration motors: low-frequency and strong vibrators for the low frequency band and a small brushless motor for the midrange frequency band.

Chair-type systems provide vibration stimuli with sound for a similar purpose. Karam et al. [START_REF] Karam | The emoti-chair: an interactive tactile music exhibit[END_REF] suggested a chair-type haptic system fusing eight channels. The system expresses various vibrations of multiple musical instruments in music using haptic devices fixed on various parts of the body. There is also a chair-type vibration machine, Bodysonic [START_REF] Komatsu | Body-felt sound unit and vibration transmitting method therefor[END_REF], to extend the sense of hearing music. The chair-type device is considered to cover the user's body; however, the strong sound experience in a live performance provides the music experience with sound from the front to the back.

Differently to Karam et al., we focus on haptic vibrations from the front, back, left, and right to elevate sound localization and realistic sensation as though the user were surrounded with musical sound. The novel point of our proposed sys- !"#$%&'((")$)"*")+$,-$$ ),.$'(/$01//)"$-2"34"(%5* 6172'8,(90,:,2$%,(:2,)$4(1:* 6172'8,(90,:,2$'22'5$;<%&=* >1?1:')$+:"2",$04+1%* @1?&:$%&'((")$)"*")+$,-$$ ),.$'(/$01//)"$-2"34"(%5* Fig. 1. System structure tem is the differences in starting times of the vibrations among multiple vibration motors to create the illusion of physical penetration as though the sound had a physical presence.

Physical experience system of sound and music

Summary of system structure

In order to expand the sound and experience of using portable headphones/earphones, we implemented a synchronized vibration-motor array system as follows. The system consists of 1) a sound analysis unit and 2) a control unit of a vibrationmotor array (see Fig. 1). First, the sound-analysis unit analyzes the signal of ongoing music. Second, the control unit of the vibration-motor array 1) distributes the level of each vibration motor and 2) calculates the time delay for each motor based on the analyzed data and parameter settings. Finally, the original hardware device of the vibration-motor array produces vibrations in synchronization with the sounds of the music.

Hardware structure and vibration-motor-array device

The hardware of the system consists of a laptop PC, a set of headphones, an original wearable set of the vibration-motor array, an AVR microcomputer (Arduino UNO), and an Nch-independent MOSFET.

The proposed vibration-motor array device is configured as a wearable belt set for the chest with four vibration motors and a choker with four vibration motors (Figs. 2 and3). Two dual-axis vibration motors of 6000 RPM (revolutions per minute), which are assigned to the low-frequency power of the sound, are also placed on the upper rubber band of the chest belt in the vicinity of the third and fourth ribs. Two of the same 6000 RPM motors are placed on the upper rubber band of the belt on the left and right sides of the back 15 cm from the center of the spine. On the choker, two micro-brushless vibration motors of 13000 RPM, which are assigned to the middle-frequency power of sound, are placed on the left and right of the front of neck near the clavicles. Two of the same motors are also placed on the left and right at the back of the neck. The left and right pairs of micro-brushless vibration motors are placed at 8cm intervals. The AVR microcomputer performs pulse width modulation (PWM) controls for each of eight channels through the Nch-independent MOSFET in synchronization with the volume levels of each frequency band based on the data sent by the sound-analysis unit. The left motors are operated by reflecting the level of sound source from the left channel. The right motors reflect the right channel. Accordingly, the difference in the intensity of vibration between the left and right channels is automatically reproduced, including several-millisecond differences, such as the interaural time difference between the left and right ears. Therefore, localized sound can be felt emphatically by the differences in the intensity of vibrations between the left and right sides.

Sound-analysis unit

The sampling rate of sound processing is 44100 Hz. The analysis window size is 1024 samples. The FFT analysis of the sound at every 43 Hz derived the levels for following frequency bands: the low-frequency band is from 43 to 86 Hz, and the midrange frequency band is from 258 to 301 Hz. The FFT analysis is processed for each channel (left and right). The four values of the levels, the left/right and low/midrange frequency bands, are converted into 0 to 255 levels to control each vibration motor. The music sound is output by headphones connected to the PC at the same time as the vibration stimuli (see Fig. 4). The system is basically using a sound file, but it can also operate real-time processes of external input with the audio interface system. Accordingly, the system is expected to be applied to improvements of realistic sensation in both interactive sounds during game play and real-time audio in watching TV.

Vibration-motor control unit

There are two methods of vibration-motor controls. The first is the direct conversion of the frequency power to each motor without special delay except the interaural time difference. The second is the unique method of time-lag among the motors that was adopted. Vibration-motor control without additional delay: First, the sound-analysis unit sends the power levels of each frequency band of both the left and the right channels and converts them into levels from 0 to 255. Second, the vibrationmotor control unit operates the PWM controls for each vibration motor. Thus, the eight motors are controlled to correspond to the sound-analysis result.

Vibration-motor controls with designed delay for generating illusion of physical penetration by sound:

The vibration-motor control unit produces an illusion of physical penetration of sound by time lags in starting the vibrations among the motors. The processing frame rate is set to 120 fps. The delay time is set from 0 to 1000 ms every 100 ms. That is, the second vibration motor starts its operation after 0-1000 ms when the first vibration motor begins its vibrations (Fig. 5). When there is physical penetration by a real object through the human body, it is assumed that the first shock will be perceived on the front surface and the second shock will be perceived on the back surface after the passage through the flesh. Although sound and music do not have any physical presence, the assumed stimuli with the differences in starting times of vibrations are expected to create a physical penetration of sound.

To make the user perceive vibrations by an eccentric rotor, it takes x -t ms (x is the time of the length of the sound, and t is the starting time among the vibration motors), which is a longer period than short-time pulse signals. Accordingly, there is an overlap between the timing of the front and back vibrations. Moreover, the overlapped timing is expected to create the feeling of passage through the body, as we aimed physical penetration.

EXPERIMENT

Evaluation of the effect of creating realistic sensations with physical vibrations in synchronization with the ongoing music

Purpose of experiment: We tried to verify the effectiveness of vibration stimuli for creating realistic sensations of the ongoing sound of the music. The "low" is the lowest sound that the audio interface can output. The "middle" is 15 dB higher than low. The "high" is 5 dB higher than middle. The strengths of vibration for each condition were as follows. The "v.none" means without any vibration. The "medium" is the linear-mapped strength of the vibration synchronized with the volume levels of the sound converted from 0 to 255 levels. The "weak" is 0.5 times the power of medium. The "strong" is 1.5 times the power of medium. A branching table of the conditions is shown in Table 1. The 16 conditions with two factors were randomly arranged for counterbalance.

Procedures and Instructions:

The participants were instructed to listen to the music for 40 seconds (rock music played for 20 seconds and then changed into ballad music for 20 seconds) through the headphones and our proposed vibration-motor array device for each condition. The time difference between the motors was not generated in this experiment. The combined conditions of sound volume and vibration using each level in each factor were performed in the repeated measurement. When the 40 seconds of music finished, the participants were instructed to evaluate the statements that followed. The other 15 conditions were repeatedly conducted with both listening and evaluation. Evaluation Items: The participants made an evaluation using a five-point rating scale of the relevance (5: very relevant, 4: somewhat relevant, 3: even, 2: somewhat irrelevant, 1: irrelevant) of the following statements. -./,!" -./,#" -./,$" -./,%" -./,&" -./,(" -./,)" -./,*" -./,+" -./!," -./!!" -./!#" -./!$" -./!%" -./!&" -./!("

!" #" $" %" &" '!#" '!$" '!%" '!&" '!(" '!)" '!*" '!+" '#," '#!" ! ! " ! # "

Results:

The results of the mean opinion scores (MOS) are shown in Fig. 6.

The results of the analyses of variance (ANOVA) with repeated measurement among the conditions (p<.05) are shown in Table 2.

The subjective evaluations showed significance for both factors except questions 11, 12, and 16. No significant result by vibration factor was found without sound. There were no significant differences in the conditions without vibration in the result of the evaluation questions 9, 10, 12, 16, and 21. Almost all of the evaluation items, except those above, showed significant differences between the with-sound and without-sound conditions at every vibration level. Besides the conditions with or without sound/vibration, there were significant differences among different sound intensities. In addition, there were interactions and simple effects between conditions with and without vibration at each volume. There were also significant differences in high-medium and high-weak strengths of the vibration. However, there was no significant difference in the vibration strength between medium and weak when the volume was in the middle for Q11 and Q19.

Evaluation of the time difference among vibration motors for the illusion of physical penetration of sound

Purpose of experiment: We tried to verify 1) whether the new experience of physical penetration of sound could be created by the emphatic time lags of the starting times between multiple vibration motors, such as forward to backward, and 2) whether the proposed time improves the realistic sensation and the sense of sound image localization.

Hypotheses:

The emphatic time lags of starting times between forward and backward vibration motors 1) generate the feeling of physical penetration of the sounds, 2) improve the realistic sensation and 3) make clear the sense of sound image localization. Participants: Twenty-five university students aged from 19 to 22 years old.

Conditions: We prepared 11 conditions of within-subject design according to a factor of time lag between forward and backward vibration motors: from 0 ms to 1000 ms every 100 ms. The conditions are shown in Table 3. The 11 conditions were randomly arranged for counterbalance.

Procedures and Instructions:

The participants were instructed to listen to the sound of a bass drum for 30 seconds with headphones and the vibrationmotor array device. The forward vibration motor was synchronized with the real-time sound, and the backward vibration motor was operated after the specified time lags for each condition. Accordingly, the backward vibration motors produced stimuli with one of the 11 different time lags. After each 30-second experience, the participants evaluated the impression for the statements of evaluation items that follow. Evaluation Items: The participants made an evaluation using a five-point rating scale of the relevance (5: very relevant, 4: somewhat relevant, 3: even, 2: somewhat irrelevant, 1: irrelevant) of the following statements. Results: The results of the MOSs and ANOVA with repeated measurements among the conditions (p<.05) are shown in Fig. 7 and Table 4. The evaluations except Q2 and Q7 showed significance. In particular, there were significant results in the 100-200 ms and 400-1000 ms lags. The most important results are Q13 and Q14, which showed the highest values in 200ms lags as evaluations for hypothesis 1).

DISCUSSION

The effects of vibration on the realistic sensation of sound

From the results of two factorial analyses of variance, the sound with vibration provided a stronger feeling of immersion and realistic sensation than simple sound by itself. Therefore it is suggested that our proposed method of providing vibration from various directions can elevate the illusion of the physical presence of the sound. Moreover, we conjecture that the realistic sensation of the sound is improved and that the loudness of the sound is strongly perceived, especially when the volume of the sound is low. The harmony between the vibration and the sound may stimulate comfortable feelings; however, if the vibration was too strong, this reduced the level of comfort. Moreover, there was no negative influence on the perception of the sound quality from the vibration. These results confirmed the hypothesis in Section 4.1. For further application for people with hearing difficulties, we should evaluate the musical perception using vibration without sound.

Illusion of the physical penetration of sound

At first, from the results for Q2, there were no perceptional time lags from the musical sound stimuli and the vibration, while the vibration motors required a few milliseconds for perceivable stimuli from the start of the rotation of the motor.

Next, from the results of the factorial analyses of variance, the time lag around 100-200 ms from front to back could provide a feeling as though 1) the sound passed through the user's body without any unnatural or incongruent feeling and 2) a bass drum were being played in front of the user.

Moreover, the participants could perceive the difference in the front and back vibrations even with only 100 ms of time lag. The significant difference in the feeling of physical penetration was confirmed in the statements regarding the penetration of both flesh and bone. Based on the results that the MOS for Q14 was higher than that of Q15, it is possible that the penetration of flesh is stronger than that of bone. This indicates that humans can feel an illusion of the physical penetration of sound.

From these results, the hypotheses in section 4.2 were confirmed. We consider that the illusion of physical penetration of sound elicits both a realistic sensation and localization of the sound with an impression of physical presence. The advantage of multiple sound sources with each different direction of the physical penetration should be discussed in future work.

CONCLUSIONS

In this paper, we proposed a wearable system with a vibration-motor array in synchronization with sound to create an emphasized experience of music that included both a strong bass sound and the illusion of the physical presence of the sound source. The multiple vibration motors expressed not only strength and localization but also fictive penetration through the user's body. The vibration motors were placed surrounding the user's neck, chest, and back.

We evaluated the effects of 1) the combination of vibration with sound in the musical experience and 2) the differences in starting time between the front and back vibrations on creating the illusion of physical penetration. From the results, it is confirmed that 1) the surrounding vibration in synchronization with the sound of music produced both a realistic sensation and a punch of the musical experience and that 2) the appropriate time lags between the front and back vibration motors created an illusion of physical penetration of the sound; at the same time, sound localization and realistic sensation were also elevated.

In future work, we should consider the possibility of the proposed device for musical experience of people with hearing difficulties. It is also suggested that the proposed vibration-motor array could be combined with the visual presentation of sound through head-mounted displays.

  !"#$%&'$()*+,))& -"'$./%0&1%2%$& 34555678!9! :(.+;.<")&& ="'$./%0&1%2%$& >555678Dual-axis vibration motor 6000[RPM] ・・・Micro brushless vibration motor 6000[RPM]

Fig. 2 .Fig. 3 .Fig. 4 .

 234 Fig. 2. Vibration motor array device

Fig. 5 .

 5 Fig. 5. Vibration-motor controls with delay for physical penetration

Fig. 6 .

 6 Fig. 6. MOS results in the experiment of synchronized physical vibrations with music Q17 You felt a sense of overdose on music. Q18 You received a warm impression. Q19 You felt comfortable with the sound. Q20 You felt comfortable with the vibration. Q21 You felt comfortable with the harmony of sound and vibration.

Fig. 7 .

 7 Fig. 7. MOS results in the experiment of physical penetration

Table 1 .

 1 Conditions in the experiment of synchronized vibrations with music We prepared 16 conditions according to two factors: A) four different volumes of sound (s.none, low, middle, high) and B) four different strengths of vibration (v.none, weak, medium, strong). The volume of sound for each condition was set as follows. The "s.none" means without any sound.

		Sound(A) Vibration(B)		Sound(A) Vibration(B)
	pt.01	none	none	pt.09	middle	none
	pt.02	none	weak	pt.10	middle	weak
	pt.03	none	middle	pt.11	middle	middle
	pt.04	none	strong	pt.12	middle	strong
	pt.05	low	none	pt.13	high	none
	pt.06	low	weak	pt.14	high	weak
	pt.07	low	middle	pt.15	high	middle
	pt.08	low	strong	pt.16	high	strong
	Hypothesis: The vibration stimuli covering the user's upper body will elevate
	the realistic sense of the sound's presence and power.
	Participants: Twenty-five university students aged from 19 to 22 years old.
	Conditions:

Table 2 .

 2 Two-factor ANOVAs in the experiment of synchronized vibrations

		loudness(A)	strength of vibration(B)	Interaction	Multiple comparison based on simple main effects
	F(24)	p	multi.comp. F(24)	p	multi.comp. F(24)	p

  Q1The backward vibration was later than the forward vibration. Q2 The forward vibration was later than the sound. Q3 The backward vibration was later than the sound. Q4 You felt as though the sound passed through your body from forward to backward. Q5 You felt the sound was just like the original sound of a bass drum. Q6 You felt the bass drum was played in front of you. Q7 The sound generated from the bass drum hit the front of your body. Q8 The sound generated from the bass drum passed through your body. Q9 You felt that the bass drum was close to you.

	Q10 You felt a punch of the sound.
	Q11 The vibration was natural in connection with the sound.
	Q12 You felt the sound cover you.
	Q13 The vibration was natural.
	Q14 The sound passed through the flesh of your body.
	Q15 The sound passed through the bones of your body.
	Q16 You felt comfortable with the sound.
	Q17 You felt comfortable with the vibration.
	Q18 You felt comfortable with the harmony of the sound and the vibration.

Table 3 .

 3 Conditions in the experiment of physical penetration pt.01 pt.02 pt.03 pt.04 pt.05 pt.06 pt.07 pt.08 pt.09 pt.10 pt.11

	Time difference (ms)	0 100 200 300 400 500 600 700 800 900 1000
	&"
	%"
	! !
	$" " !
	# "
	#"
	!"
	'!"	'#"		'$"	'%"	'&"		'("	')"	'*"	'+"	'!,"
		-./,!"	-./,#"	-./,$"	-./,%"	-./,&"	-./,("	-./,)"	-./,*"	-./,+"	-./!,"	-./!!"
	&"
	%"
	! !
	$" " !
	# "
	#"
	!"
	'!!"	'!#"		'!$"		'!%"	'!&"		'!("	'!)"		'!*"	'!+"

Table 4 .

 4 One-factor ANOVAs in the experiment of physical penetration

	Time difference
	F(19)	p	Multiple comparison (from pt.1 to pt.11)
	Q1 26.049 <0.01*	[3,4,5,6,7,8,9,10,11]-[1,2],2-1
	Q2 1.566 0.1174	-
	Q3 19.402 <0.01*	[3,4,5,6,7,8,9,10,11]-[1,2],2-1
	Q4 6.845 <0.01*	6-11,4-[5,11],3-[5,7,8,9,10,11] ,2-[5,7,8,9,10,11],
	Q5 5.959 <0.01*	4-11, 3-[5,6,7,8,9,10,11] ,2-[7,8,10,11],1-11
	Q6 7.448 <0.01*	4-11,3-[5,7,8,9,10,11],2-[5,7,8,11],1-[5,7,11]
	Q7 1.464 0.1538	-
	Q8 6.021 <0.01*	3-[5,8,10,11],2-[8,10,11],1-[10,11],4-11
	Q9 5.115 <0.01*	3-[7,11],2-[7,8,9,10,11],1-[7,11]
	Q10 5.760 <0.01*	3-[10,11],2-[5,8,10,11],1-[5,8,10,11]
	Q11 9.516 <0.01*	4-11,3-[5,6,7,8,9,10,11], 2-[5,6,7,8,9,10,11],1-[8,11]
	Q12 6.809 <0.01*	3-[4,5,6,7,8,9,10,11]2-[5,7,8,11],
	Q13 7.590 <0.01*	3-[5,6,7,8,9,10,11],2-[7,8,10,11],1-[8,11],4-11
	Q14 8.050 <0.01*	6-1,4-[8,11], 3-[5,7,8,9,10,11],2-[5,7,8,9,10,11],1-11
	Q15 8.643 <0.01*	6-11,4-[5,7,8,11],3-[5,7,8,9,10,11],2-[5,7,8,9,10,11],1-11
	Q16 7.343 <0.01*	3-[5,6,7,8,10,11],2-[5,7,8,9,10,11],1-[5,11]
	Q17 7.600 <0.01*	4-11,3-[5,7,8,9,10,11],2-[5,7,8,10,11],1-[5,8,10,11],
	Q18 10.037 <0.01*	3-[4,5,6,7,8,9,10,11],2-[5,6,7,8,9,10,11],1-[5,8,11]
	Q19 10.270 <0.01* 4-[5,8,11],3-[5,6,7,8,9,10,11],2-[5,6,7,8,9,10,11],1-[5,7,8,11]

Acknowledgments

This research was supported in part by JSPS KAKENHI 25700021 and JSPS KAKENHI 15H01698. The authors would like to thank the participants in the experiment.


Victor Fernandez-Cervantes

Eleni Stroulia
email: stroulia@ualberta.ca

Benjamin Hunter
email: bkhunter@ualberta.ca

A Grammar-based Framework for Rehabilitation Exergames

Keywords: Kinect-Based Gameplay, Interface, Serious Games, Rehabilitation

Numerous serious exergames advocate the use of engaging avatars to motivate a consistent exercise regimen. However, the process of specifying the prescribed exercise, implementing it as avatar animation, and developing an accurate feedback-providing mechanism is complex and requires a high level of expertise in game engines, control languages, and hardware devices. Furthermore, in the context of rehabilitation exergames, the requirements for accurate assessment and timely and precise feedback can be quite stringent. At the same time, the Kinect T M motioncapture sensor offers a natural interface to game consoles, and its affordability and wide availability represents a huge opportunity for at-home exergames. In this paper, we describe our work towards a system that envisions to simplify the process of developing rehabilitation exergames with Kinect T M . The system relies on a language for specifying postures and movements between them, and includes an editor that enables rehabilitation therapists to specify the prescribed exercise, by editing a demonstration of the exercise. This exercise-specification grammar is used to drive the animation of an avatar and the provision of quality feedback, by comparing the player's postures (as captured by the Kinect T M ) against those of the coaching avatar and the grammar.

Introduction

The concept of "serious games" refers to digital games whose purpose is more than entertainment [START_REF] Susi | Serious games: An overview[END_REF]. The core intuition behind the serious-game paradigm is that, when learning tasks are embedded within a gameplay scenario, learners, motivated by the mechanics of gameplay such as "scoring points", "clearing levels", and "getting badges", spend more time learning, which facilitates the acquisition of new knowledge and skills.

A particularly interesting type of serious games are "exergames", i.e., games designed to encourage physical activity towards improving physical ability and fitness, or towards systematizing athletes' training, or towards rehabilitation from injury or other health challenges. The development of exergames is a quite demanding software-engineering task: in addition to requiring the design of engaging avatars, animations and game mechanics, it also demands knowledge about specialized hardware sensors and controllers, game engines, algorithms for processing the sensor signals, producing feedback for the user and controlling the game state. Even more importantly, it relies on the specialized domain knowledge of exercise experts, such as trainers and physical therapists.

Exergames have received substantial attention recently, as controllers that use full-body motion, such as the very popular and relatively inexpensive Wii and Kinect T M , have become increasingly available at-home. The latter, in particular, represents an extremely attractive platform for exergames [START_REF] Obdrzalek | Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population[END_REF] since it enables adequate skeleton tracking, and its SDK and user community offer substantial software-development support. Recognizing this opportunity, a variety of exergames have been developed that cover a broad design spectrum: on one hand, one can find complex systems with no evident gameplay [START_REF] Lin | A kinect-based system for physical rehabilitation: utilizing tai chi exercises to improve movement disorders in patients with balance ability[END_REF]; on the other hand, some games [START_REF] Fikar | The Sorcerer's Apprentice A serious game aiding rehabilitation in the context of Subacromial Impingement Syndrome[END_REF][START_REF] Fernandez-Cervantes | Serious games: Rehabilitation fuzzy grammar for exercise and therapy compliance[END_REF] offer engaging gameplay but with restricted and limited movements, and, therefore, limited potential for physical conditioning.

It is our belief that the Kinect T M -based gameplay for exergames suffers from a feasibility gap: today, there is no toolkit to support the integration of personalized animation, guidance feedback, and valid exercise assessment. As a result, even though the sensor is cost-effective, widely available, and sufficiently accurate, its potential is not yet fully met. Motivated by this position, the key objective of this work is to introduce Avatar Grammar Animation System (AGAS), our Kinect T M -based toolkit for supporting the development of rehabilitation exergames in Unity. Our system offers an editing tool that enables casual computer users, with expertise in physical exercise but no software-engineering expertise, to specify an exercise script, including key postures and the movement between them. This editor, in effect, enables users to annotate a pre-recorded demonstration of the exercise with rules about the angles of the important joints in each step of the exercise. The resulting exercise script, represented in terms of the underlying AGAS grammar, an extension of the grammar reported in [START_REF] Fernandez-Cervantes | Serious games: Rehabilitation fuzzy grammar for exercise and therapy compliance[END_REF], is then used by the Unity game engine to (a) animate a coach avatar that demonstrates the exercise at run-time during gameplay, and (b) provide timely and accurate feedback to the player about his/her posture and movement.

The rest of this paper is organized as follows. Section 2 places our work in the context of related research, reviews Kinect T M as the underlying hardware of our system, and motivates the use of fuzzy logic for assessing the user's movements. In Section 3, we describe the architecture of our system and our experience with it to date. Finally, in Section 4, we close with a summary of the key contributions of our system and our plans for future work.

Related Research and Background

Exergames In 2006, the Wii console evolved the gameplay paradigm with an interactive control-system that established a rapport between the player and a digital avatar [START_REF] Karoussos | Mii & you[END_REF]. This rapport was shown to motivate exercise consistency and perseverance of seniors, who reported increasing levels of enjoyment during the 6-weeks training program with Wii Fit Plus games [START_REF] Nicholson | Six weeks of unsupervised Nintendo Wii Fit gaming is effective at improving balance in independent older adults[END_REF]. Another study reported similar results with overweight children who improved their exercise habits playing a virtual running game, featuring a slim and toned avatar design [START_REF] Li | Wii, myself, and size: the influence of Proteus effect and stereotype threat on overweight children's exercise motivation and behavior in exergames[END_REF].

In late 2010, the affordable motion-capture Kinect T M changed the gameplay once again to a natural body-motion interface. Even though the videogame console, Xbox, did not exploit this new capability -gameplay in games such as JUST DANCE 1 or ZUMBA fitness2 is the same across all the main videogame consoles, and does not take advantage of the unique features of each console-the Kinect T M 's potential for serious games has been amply demonstrated. In [START_REF] Kitsikidis | Dance analysis using multiple kinect sensors[END_REF], the data from multiple tracking devices is combined to recognize dance patterns, using a Hidden Conditional Random Fields (HCRF) classifier. In [START_REF] Lin | A kinect-based system for physical rehabilitation: utilizing tai chi exercises to improve movement disorders in patients with balance ability[END_REF], "seated Tai Chi" is presented as a Kinect-based physical rehabilitation exercise. The system was designed for patients with movement disorders and assessed their ability with a very simple measure: it evaluated whether some angles of interest in the patient skeleton were equal (within a threshold) to some pre-defined values. As a result the system can only give very simple feedback on eighteen postures. The same control system with more innovative game-flow design was presented in the sorcerers apprentice [START_REF] Fikar | The Sorcerer's Apprentice A serious game aiding rehabilitation in the context of Subacromial Impingement Syndrome[END_REF], which was designed for patients with SIS (Subacromial Impingement Syndrome). The prescribed movements are specified as boundaries preconfigured by the therapist. The gameplay allows the patient to focus on playing, while achieving the goals in the rehabilitation exercises. In our own previous work [START_REF] Fernandez-Cervantes | Serious games: Rehabilitation fuzzy grammar for exercise and therapy compliance[END_REF], we focused on a more realistic mechanism for providing high-quality feedback in rehabilitation serious games. To that end, we developed a fuzzy grammar for evaluating the correctness of postures and movements. Our original grammar was, however, limited to analyzing mostly static postures and dynamic transitions through changes of a single angle between three joints. We also developed an intutitive user-interface feedback guide: a fill-up bar, attached to the corresponding joint, which fills up following the movement of the player and presents the correct angle with an arrow. In this paper, we describe an extension of our original grammar that can capture more complex exercises (such as TaiChi) and we develop an editor to enable exercise experts to define exercise scripts in this grammar.

Avatar Animation The topic of avatar animation is quite vast and, in this section, we cannot but mention only an eclectic collection of works, on two broad topics. On the topic of movement animation, there is work on inverse kinematics for maintaining balance [START_REF] Baerlocher | An inverse kinematics architecture enforcing an arbitrary number of strict priority levels[END_REF]; skeleton animation with walking cycles [START_REF] Multon | Computer animation of human walking: a survey[END_REF]; falling and landing with concern for self-preservation [12]; and realistic mass and balance properties of physical characters [START_REF] Kenwright | Real-Time Physics-Based Fight Characters[END_REF]. On the topic of the character appearance, there is work on animating non-human characters based on skeleton semantics [START_REF] Wang | Kinect driven 3D character animation using semantical skeleton[END_REF]; representing avatars as arbitrary 3D mapping points [START_REF] Rhodin | Interactive motion mapping for realtime character control[END_REF] or with visually highly realistic bodies [START_REF] Teran | Creating and simulating skeletal muscle from the visible human data set[END_REF] and facial expressions [START_REF] Kalra | Real-time animation of realistic virtual humans[END_REF]; and animating 3D puppets based on real-time motion capture [START_REF] Leite | Anim-actor: understanding interaction with digital puppetry using low-cost motion capture[END_REF]. In the context of serious exergames for rehabilitation, our interest in avatar animation stems from the need to communicate in an intuitive and unambiguous manner the movements to be mimicked by the player. In effect, our goal is to develop a system that will enable non-computer experts to animate a precise coach avatar demonstrating the prescribed exercise and providing timely and accurate feedback to the player.

A few platforms have been proposed to facilitate this task. The Flexible Action and Articulated Skeleton Toolkit (FAAST) [START_REF] Suma | Faast: The flexible action and articulated skeleton toolkit[END_REF] is a middleware that integrates full body control. Unfortunately, the gestures captured with the Kinect T M are only substitutes for keyboard commands or a virtual mouse. XDKinect [START_REF] Nebeling | XD-Kinect: development framework for cross-device interaction using kinect[END_REF] is a similar toolkit, with an online framework service to recognize gestures. In [START_REF] Abdur Rahman | Multimedia interactive therapy environment for children having physical disabilities[END_REF], a multimedia environment for children rehabilitation is described. The system controls an avatar in Second Life to demonstrate the therapy exercises. The system is limited in terms of the exercises it can cover: the exercise postures focus on a single joint angle and so does the assessment. The motion-tracking evaluation system presented in [START_REF] Alabbasi | Human motion tracking & evaluation using Kinect V2 sensor[END_REF] uses Unreal Engine 4 to compare the player's skeleton as recorded with the Kinect T M sensor with the in-game avatar. This system requires a developer to program the avatar and offers very coarse-grained feedback regarding the correctneess of the player's movements: simply red and green colors over the avatar's joints. This simple feedback makes it almost impossible to discern the specific corrections required in each posture. The Dual-Task Tai Chi game suffers from similar limitations, but with more interesting game play [START_REF] Kayama | Concept software based on Kinect for assessing dual-task ability of elderly people[END_REF], using Kinect T M -based full-body control with a cognitive task, i.e., 4x4 Sudoku. In this game, the user has to reach a specific posture in order to select a number with the right hand or foot, and then place this number on the Sudoku grid with the left hand or foot. The posture sequence during the game are similar to tai-chi.

The Kinect T M Sensor The Kinect T M V2, on which AGAS is based, presents many improvements over its predecessor [START_REF] Butkiewicz | Low-cost coastal mapping using Kinect v2 timeof-flight cameras[END_REF]. It is composed by infrared and color camera, with a resolution of 512×424, and 1920×1080 respectively. The sensing process is orchestrated by a fast clock signal whose strobes an array of three laser diodes, which simultaneously shine through diffusers, bathing the scene with short pulses of infrared light. The sensor also calculates the ambient-lighting, the final image invariant to lighting-changes. The accuracy of the Kinect T M V2 in computing the joints' positions is lower than motion-capture systems, however it is "good enough" for some of the regimen exercises, or therapy. the discrepancy could go from 13mm to 64mm [START_REF] Xu | The validity of the first and second generation Microsoft Kinect for identifying joint center locations during static postures[END_REF].

The Kinect T M SDK version 2.0 estimates the position and orientation of 25 joints, organized in a hierarchy, centered at the spine base (SB), as shown in Figure 1(a). This hierarchy causes difficulties with several postures, due to occlusion or, more generally, lack of information regarding the root of a particular joint and its hierarchical rotation, i.e., the amount of rotation in the 3D space that the joint inherits from its parent joint.

Fuzzy Logic In order to assess the player's exercise style and provide feedback to motivate and improve it, an efficient mechanism for continuously comparing the player's actual posture against the "correct" one is required. In this work, similar to our previous work [START_REF] Fernandez-Cervantes | Serious games: Rehabilitation fuzzy grammar for exercise and therapy compliance[END_REF], we adopt a fuzzy-logic paradigm for that purpose, in order to avoid simplistic angle comparisons (as used in [START_REF] Lin | A kinect-based system for physical rehabilitation: utilizing tai chi exercises to improve movement disorders in patients with balance ability[END_REF]). The fuzzy-logic paradigm considers that uncertainty is unavoidable when concepts are imprecisely expressed in natural language: in the context of the rehabilitation-exercise specification, the concepts of "correct" and "incorrect" posture are expressed with a real value between "1" and "0" indicating a degree of correctness.

3 Software Architecture

The AGAS system consists of three components: (a) a recording component through which an expert demonstration of the exercise is captured in a form that can be viewed and manipulated in Unity;3 (b) an editor, implemented in Unity, through which a exercise expert reviews the demonstration to produce an exercise script, including key postures, their important joints, and the transitions between them; and (c) a game-playing engine, also implemented in Unity, during which the player's movement is compared against the coach avatar, i.e., the exercise script animation, and the rules around the important key-posture joints.

Exercise Demonstration and Recording

The motion-capture-and-recording component records 30 frames per second, each of which includes the complete information regarding all joints shown in Figure 1(a). Each frame record includes (a) the frame timestamp in milliseconds, with the first frame defining time 0; and (b) a sequence of joint records, each one consisting of the joint's identification number, and its orientation in three-dimensional space, centered at the 0 th joint, at the SpineBase.

The Exercise-Script Editor

In our AGAS system, an exercise specialist can review and edit the exercise demonstration, through the editor, shown in Figure 3. The AGAS editor reads as input the demonstration frame sequence and provides a user interface through which an exercise expert, can review the the demonstration and identify the exercise key postures, selecting the appropriate frames where these postures are demonstrated.

Reviewing Avatar Postures Through the AGAS editor, the exercise expert can replay, stop, move to next and previous frames of the demonstration record. When reviewing the avatar posture in a particular frame, the user can inspect the posture from multiple camera views and can review each avatar joint.

As shown in Figure 1(a), the avatar is a rigid body with 25 interconnected joints. The orientation of each joint is relative to its parent joint in the skeleton hierarchy of Figure 1(a). This implies that a change in the orientation of a joint affects the positions of all its descendant joints. For each joint j i , in each frame of the recorded exercise demonstration, the following information is available:

j i = {N ame, θ i x , θ i y , θ i z } ( 1
)
where name is the joint's label (and implicitly its position in the joint hierarchy) in the Kinect T M skeleton, and the angles θ x , θ y , θ z correspond to the orientation of the joint in the specified axes, in the frame under examination.

Specifying Key Postures and Transitions

The main purpose of the editor component is to produce an exercise script, based on the expert's demonstration of the exercise. The exercise script consists of a succession of key postures, starting with the initial posture demonstrated by the expert. The intuition behind the definition of the exercise script is that the player going through the exercise has to reach some key postures, which are defined in terms of their important posture joints, i.e., a subset of joints that have to be at precise orientation angles in this key posture. These major posture joints move from one key posture to the next, according to the demonstrated timing. The exercise expert, using the editor, (a) identifies the key postures; (b) specifies the major joints for each posture; (c) reviews and corrects the precise orientation angles of the major jointsin case the demonstration is not perfect; and, implicitly, (d) records the timing between these major postures. The various elements involved in the specification of an exercise script are illustrated in Figure 1(b). Posture elements specify the static skeleton configurations that the user must reach during the exercise. Movement elements specify the timing of the dynamic transition between two consecutive postures, expressed in the number of lapsed frames between the two postures. For every exercise script, there are two special postures: the initial posture is the first skeleton configuration of the recorded demonstration, and the final posture is the final skeleton configuration, correspondingly. The two may be the same, in fact they often are for exercises that start and finish at the same position, after going through a sequence of other intermediate postures. Each posture is characterized by a descriptive name element, and its unique id attribute. Each posture element is composed of a set of joint elements, a proper subset of the 25 Kinect T M -skeleton elements, whose exact orientations are important for the posture to be considered correctly achieved. For each joint, its name is specified (as shown in Figure 1(a)) as well as some (or all) of the X-axis, Y-axis, Z-axis orientation angles, relative to its parent joint.

<p o s t u r e i n i t i a l =t r u e i d=1> <name>p o s t u r e name</name> < j o i n t i m p o r t a n t=t r u e> <name> j o i n t name</name> <a x i s>a n g l e</ a x i s> </ j o i n t> < j o i n t> . . . </ j o i n t> . . . </ p o s t u r e> The key postures are identified by the exercise expert as he/she reviews the exercise demonstration (see Section 3.1). As the expert moves through the frames of the demonstration record, he/she recognizes a key posture and, through the user interface in Figure 3, selects the important joints, i.e., the defining joints for the posture. If a particular joint is not in a "perfect" position, the expert may also edit its orientation angles. Through this interaction, the key postures of the exercise are collected in the exercise script. The movement elements are inferred implicitly, based on the data between two consecutive key postures. The initial and final key postures are identified, by default, as the postures on the initial and final frames of the demonstration record. Finally, it is important to note that the specification of an important joint in a key posture gives rise to a corresponding fuzzy assessment rule, to be evaluated during game play at run time.

Coach-Avatar Animation

The coach avatar mimics the demonstrated exercise as a prearranged animated sequence, described in the exercise script. The coach avatar is composed of 33 joints. In addition to the twenty-five main joints of the Kinect T M avatar, eight more joints are used for the animation of muscle movement, in key areas such as hips, legs, and shoulders, enabling a more realistic appearance of body movement.

The coach avatar initially appears with the standard T-Posture; in this posture, all Kinect T M -skeleton joints are assumed to be in their corresponding 0,0,0 position. The animation process is controlled by the exercise script, as follows. (c) All major joints of the source key posture move synchronously until they reach their orientation in the target key posture. In principle, there are two directions of movement between any two angles; the direction of the movement is chosen so that the distance the joint travels is minimized. In order to produce a naturalistic animation, corresponding joints in subsequent key postures should not be more than 90 • away (in any angle). (d) Finally, the deltas of the joints' movements, between two frames, in all axes are equal. In effect, the animation component extrapolates a number of intermediate postures, at a rate of 30fps, between every two consecutive key postures, to create the appearance of a smooth and fluid movement.

The animation process starts with the default T-posture which transitions to the first key posture of the exercise, and then iterates through all key postures until the last one.

An interesting feature of the AGAS component for exercise-script animation is that the same exercise script can potentially be applied to different original postures. For example, one could apply a script for upper-arm movement to an original sitting posture, for wheelchair-bound users. As long as the major joints of the key postures in the exercise script do not involve movements of the leg joints, the exercise could be demonstrated by a standing or a sitting coach avatar.

The AGAS editor user interface, shown in Figure 3, enables all the designtime functionalities, i.e., exercise recording and replaying, identifying and reviewing key-postures, and specifying the transitions between them. In Figure 3, subsection 1 shows the recording interface, where the movements are recorded and saved. The playback interface is shown in subsection 2, where the user has the ability to pause the playback, adjust its speed and direction, scrub frame-by frame, and export all key postures created. When playback is paused the user is also given the option to edit the avatar in its current position and save a key posture. This interface is shown in subsection 3, where the user may signify joints as important as well as select them to observe and modify the corresponding X,Y, and Z angles. Camera controls allow multiple views of the posture and a save option lets the user save the current pose as a key posture. Finally, subsection 4 shows the interface for reviewing and animating created key postures.

Here the user can review and edit each posture, or finally preview and specify the transitions between two key-postures to create the animation script for gameplay.

Kinect-Based Gameplay

During gameplay, the coach avatar demonstrates the exercise in the context of the game background, the user performs the exercise and his/her avatar reflects his/her movements right next to the coach avatar, and the assessment component evaluates the correctness of the player's performance and provides appropriate feedback, through highlighting of the user's skeleton's joints (see Figure 1(b)). We have explained the coach avatar demonstration in Section 3.2 and the replay of theKinect T M -captured user movement in Section 3.1. In this section, we describe the assessment component of our AGAS system.

The AGAS exercise-assessment component performs two types of evaluation of the user's performance, according to two measures: (a) the frame-by-frame similarity of the user's skeleton to the coach-avatar skeleton, and (b) its adherence of the user's skeleton to the rules around the orientation of the major joints of each key posture. Conformance to Key Postures: In general, the complete assessment of a key posture relies on a collection of rules, each one corresponding to a major joint of the posture, J 2 , and its angle relative to its parent and child joints, J 1 , and J 3 . The joints J 1 and J 3 form a interior angle with J 2 , and the rule defines the correct angles θ among these joints in the corresponding axes: θ x (lef t -right), θ y (up -down), θ z (f ront -back). The assessment rules are fuzzy: they specify smooth transition between correct and incorrect joint orientation, as opposed to defining a sharp boundary to separate these two states. For each important joint, a fuzzy rule, illustrated by the five overlapping trapezoid functions, shown in Figure 2, defining a range of transitions from perfect to wrong. At run time, the game-playing component continuously evaluates the rules for all major joints for the next anticipated key posture, and identifies the frames at which each of the rules is met "perfectly" or "adequately" or "not at all" and provides feedback by annotating the joints with a color, ranging from green(perfect) to yellow(inadequate) and red(wrong), with a smooth color transition in between functions. Frame-by-Frame Dynamic Angle Similarity: In parallel with the rulebased assessment of the key-posture joints, a second assessment measure is ap-plied to the major arm and leg bones of the player. At each frame, the orientations of these bones in the player's avatar skeleton are compared against the orientations of the corresponding bones in the coach avatar skeleton.

During game play, the feedback component uses the frame-by frame assessment to highlight the arms and legs of the user's avatar. A configurable "difficulty" parameter modifies a threshold which defines what is perfect (green), neutral (yellow), or incorrect (red). For each compared bone, the average of the absolute differences of the three corresponding angles is compared against this threshold to determine the color shown to the user; the closer to 0 the absolute average difference is, the better the player has matched the coach.

Experience and Reflection

The AGAS tool enables the development of the basic elements of a serious exercise game, based on a demonstration of the game exercise. The display in Figure 4 shows the product of this process, placed on a background. At this point, the game is playable, and it enables valid exercise by giving basic feedback on its correctness; yet, further work is needed to make it engaging and motivating. To complete the game, the aesthetic elements of the game have to be developed, including the background visuals, the music, and the body(ies) of the coach and player avatar. This remaining development is primarily the responsibility of designers; in effect, AGAS systematizes and simplifies the developers' tasks, so that an exercise expert can actually accomplish them based on their domain knowledge. To date, we have used AGAS to develop a set of simple upper-body ex- ercises, similar to the ones found at http://eldergym.com/elbow-exercises.html. We have also developed a small TaiChi game, consisting of few simple TaiChi moves. The former game was relatively easy to develop, since the key postures are relatively clear to identify (in fact for most of the above exercises the elderrgym web site identifies the key postures with still pictures), each posture consists of few important joints, and the orientation angles of these joints are simple to describe. In fact for most of these exercises the joints' orientations between key postures change only in one dimension. The TaiChi game is, however, much more challenging: the key postures are much more complex and multiple joints move between any two of them.

In this paper we described AGAS, a tool for supporting the development of serious exergames, that can be used for rehabilitation purposes. These games must communicate in an intuitive and unambiguous manner the movements of the player's rehabilitation regimen, as prescribed by an exercise expert. In developing AGAS, our goal was to enable non-computer experts to animate a precise coach avatar demonstrating the prescribed exercise and providing timely and accurate feedback to the player. At the core of our tool is a simple grammar for postures and trasition movements between them. Specifications of an exercise in this grammar can be constructed by an exercise expert using the AGAS editor to inspect and annotate a demonstration of the exercise. These specifications are then fed into the AGAS game-playing component, which is responsible for observing the player's movements, comparing them to the specification as well as to a coach-avatar simulation and providing feedback to the player so that he can improve his posture and movement.

In the future, we plan to improve the feedback component to provide more accurate feedback with respect to timing, and we will extend the editor to support the exercise experts with hints as to which the best key postures might be.

Fig. 1 :

 1 Fig. 1: (a) Kinect T M Skeleton Tracking SDK 2.0 (Position, Orientation, Hierarchy, and Rotation); (b) Exercise-Script Animation

  <movement> <s o u r c e>< r e f : p o s t u r e r e f =1></ s o u r c e> <t a r g e t>< r e f : p o s t u r e r e f =100></ t a r g e t> <time>f r a m e s</ time> </movement> Fig. 2: The Exercise Script Language

Fig. 3 :

 3 Fig. 3: The AGAS Exercise-Script Editor (a) At each step, the coach avatar moves from one key posture (source) to the next (target). (b) The transition from the source to the target key posture is completed in the number of frames prescribed by the corresponding animation element in the exercise script.(c) All major joints of the source key posture move synchronously until they reach their orientation in the target key posture. In principle, there are two directions of movement between any two angles; the direction of the movement is chosen so that the distance the joint travels is minimized. In order to produce a naturalistic animation, corresponding joints in subsequent key postures should not be more than 90 • away (in any angle). (d) Finally, the deltas of the joints' movements, between two frames, in all axes are equal. In effect, the animation component extrapolates a number of intermediate postures, at a rate of 30fps, between every two consecutive key postures, to create the appearance of a smooth and fluid movement.The animation process starts with the default T-posture which transitions to the first key posture of the exercise, and then iterates through all key postures until the last one.An interesting feature of the AGAS component for exercise-script animation is that the same exercise script can potentially be applied to different original postures. For example, one could apply a script for upper-arm movement to an original sitting posture, for wheelchair-bound users. As long as the major joints of the key postures in the exercise script do not involve movements of the leg joints, the exercise could be demonstrated by a standing or a sitting coach avatar.The AGAS editor user interface, shown in Figure3, enables all the designtime functionalities, i.e., exercise recording and replaying, identifying and reviewing key-postures, and specifying the transitions between them. In Figure3, subsection 1 shows the recording interface, where the movements are recorded

Fig. 4 :

 4 Fig. 4: Run-Time Posture Assessment and Feedback

http://just-dance.ubi.com

http://zumbafitnessgame.com/

The AGAS recorder is based on the examples of https://www.assetstore.unity3d.com/en/#!/content/18708 for how to use Kinect T M V2 in Unity, including examples for how to start the camera, how to record data, and how to control the avatar based on the recording.


André Rattinger
email: andre.rattinger@gmail.com

Günter Wallner
email: guenter.wallner@uni-ak.ac.at

Anders Drachen
email: drachen@hum.aau.dk

Johanna Pirker
email: jpirker@iicm.edu

Rafet Sifa
email: rafet.sifa@iais.fraunhofer.de

Integrating and Inspecting Combined Behavioral Profiling and Social Network Models in Destiny

Keywords: Social Network Analysis, Destiny, Network Visualization, Multi-Player, Profiling, Game Analytics

In this paper two key venues of investigation in game analytics are combined: behavioral profiling and social network analysis. Both venues of research are well developed but combined they permit pattern evaluation across player performance and networks. Here, competitive networks covering almost 3.5 million players of the hybrid online shooter game Destiny are developed and combined with behavioral profiles based on match performance metrics and defined using archetypal analysis. The profiles are embedded in the networks along with other performance indicators for Destiny players. The social behavior of different archetypes is described. Network visualizations are presented which target the problem of making dense networked results actionable.

Introduction

With the introduction of telemetry tracking in digital games, game analytics has become a cornerstone of game development [START_REF]Game Analytics -Maximizing the Value of Player Data[END_REF]. Visualization of behavioral analysis on high-dimensional datasets is challenging because of the typical complex behavioral phenomena in games [START_REF] Bauckhage | Clustering game behavior data[END_REF][START_REF] Cleveland | Visualizing Data[END_REF][START_REF] Drachen | Guns, swords and data: Clustering of player behavior in computer games in the wild[END_REF][START_REF] Drachen | Game data mining[END_REF][START_REF]Game Analytics -Maximizing the Value of Player Data[END_REF][START_REF] Wallner | An introduction to gameplay data visualization[END_REF]. This challenge is a well-described problem in game analytics, where major commercial titles can give rise to thousands of different features [START_REF] Drachen | Game data mining[END_REF]. To counter this problem a variety of methods have been adopted and adapted from other domains. Of these, behavioral profiling plays an established role. The focus here has been on condensing varied, volatile and high-volume data into condensed profiles which encapsulate player behavior and highlight key patterns of use for the specific purpose. The location of patterns in the behavior of players, and how to translate these to business action, using them to inform game AI or informing human behavior research, remains however a major line of inquiry [START_REF] Drachen | Game data mining[END_REF]. The specific purpose of player profiling varies substantially, from top-down explorative analysis to hypothesis-testing [START_REF] Bauckhage | Clustering game behavior data[END_REF][START_REF] Drachen | Game data mining[END_REF]. Multiple examples have been described across a variety of games and using a variety of techniques, with unsupervised techniques such as clustering being the most common [START_REF] Bauckhage | Clustering game behavior data[END_REF][START_REF] Drachen | Game data mining[END_REF][START_REF] Normoyle | Bayesian clustering of player styles for multiplayer games[END_REF]. Cluster models allow segments to be developed which can describe the behavior of players according to specific behaviors and are driven by specific research questions, and they can be translated into behavioral descriptions of the different player segments. Behavioral profiling is notably useful in persistent online games, where the success of a title relies on its ability to keep a population of users engaged longitudinally and thus requires constant monitoring. This is also the situation in Destiny, a hybrid online shooter game which forms the case for the work presented here. Destiny is a persistent online game and the to date most expensive digital title to be developed worldwide, with a player base of roughly 30 million active players. As a hybrid title blending design aspects of multiple genres and featuring varied gameplay, e.g., multiple modes of play, Destiny forms an ideal basis for evaluating new frameworks and models for investigating player behavior.

Behavioral profiling work in games tends to focus on individual players, ignoring the connections between players, which are instead treated separately through social network analysis (SNA) [START_REF] Freeman | Visualizing social networks[END_REF]. From the perspective of SNA in games, the information about the players is generally limited, and the focus is instead on the links that connect players. This means that SNA typically views users as users, and ties minimal contextual information to the nodes of a network (demographics forming a common exception). In SNA, pattern recognition is as important as in behavioral profiling, and cluster analysis is used for defining groups and patterns in networks [START_REF] Van De Bovenkamp | Understanding and recommending play relationships in online social gaming[END_REF][START_REF] Freeman | Visualizing social networks[END_REF][START_REF] Jia | Socializing by gaming: Revealing social relationships in multiplayer online games[END_REF].

For behavioral profiling and SNA, a central challenge rests in the visualization of the results of analysis. In both instances, data and the patterns in them need to be presented to a user such that they are interpretable and actionable [START_REF] Drachen | Game data mining[END_REF][START_REF] Freeman | Visualizing social networks[END_REF]. In this paper, a step is taken towards combining game-based behavioral profiling and game-based SNA. This, however, means that the visualization problem is amplified as the relative scope of analysis is increased. However, combining these two lines of investigation in games research has the advantage of providing a framework for exploring in-game behavior of players along with the connections between players. This permits a more detailed understanding of the individual player, in essence providing a lens that permits the observation of network behavior as well as general in-game behavior. This, in turn, informs the evaluation of design, engagement analysis, monetization decisions and similar factors in games.

Contribution

In this paper, game-based SNA, behavioral profiling, and data visualization is combined for the purpose of investigating the network behavior of competitive Destiny players as a function of the patterns in their performance, and ultimately develop novel visualizations that aim at making the results of combined SNA-profiling analysis interpretable by game designers and the Destiny players themselves. Towards this, competitive networks are constructed based on data from the Player-versus-Player (PvP) game modes, covering almost 3.5 million players. Using archetypal analysis, behavioral profiles are defined for each player, based on a range of performance indicators which includes information about behavior with the different weapon classes in Destiny. Given the shooter-type nature of the game, weapon performance is a key indicator of player skill. The profiles are embedded in the competitive networks and used to analyze the player network as a function of profiles. Furthermore, other metrics are integrated in separate analyses and visualized.

Related Work

The work presented here builds on previous work in three domains: 1) behavioral profiling in games; 2) social network analysis in games; 3) visualization of complex behavioral datasets and networks. For reasons of space this section will focus on key related work.

Behavioral Profiling in Games: Cluster analysis is a method for dimensionality reduction and pattern recognition and has been readily applied across disciplines. As an unsupervised method, it permits the exploration of data and can identify groups of players with similar behaviors or detect features that constitute such behaviors [START_REF] Bauckhage | Clustering game behavior data[END_REF][START_REF] Thawonmas | Visualization of online-game players based on their action behaviors[END_REF]. Behavioral profiling in game analytics has explored a variety of cluster models, including a few comparative studies [START_REF] Bauckhage | Clustering game behavior data[END_REF][START_REF] Drachen | A comparison of methods for player clustering via behavioral telemetry[END_REF]. There is no standard for which model to employ in which situation, but previous work has highlighted that various models are useful for different situations and problems [START_REF] Bauckhage | Clustering game behavior data[END_REF][START_REF] Drachen | A comparison of methods for player clustering via behavioral telemetry[END_REF]. For example, Thurau and Bauckhage [START_REF] Thurau | Analyzing the evolution of social groups in World of Warcraft[END_REF] explored the evolution of guilds in World of Warcraft using matrix factorization which provided condensed views on how guilds change their composition over time. For example, Sifa et al. [START_REF] Sifa | Large-scale cross-game player behavior analysis on steam[END_REF] identified clusters of players based on their relative playtime distribution across games on the Steam distribution platform using k-means clustering. Thawonmas and Iizuka [START_REF] Thawonmas | Visualization of online-game players based on their action behaviors[END_REF] employed multidimensional scaling (CMDS) and KeyGraph to generate visualizations of player clusters. Drachen et al. [START_REF] Drachen | Guns, swords and data: Clustering of player behavior in computer games in the wild[END_REF] employed simplex volume maximization and k-means, developing behavioral profiles for two games. Normoyle and Jensen [START_REF] Normoyle | Bayesian clustering of player styles for multiplayer games[END_REF] used Bayesian Clustering. Moving into spatio-temporal clustering, Bauckhage et al. [START_REF] Bauckhage | Beyond heatmaps: Spatio-temporal clustering using behavior-based partitioning of game levels[END_REF] developed waypoint graphs and adopted DEDICOM for behavior-based partitioning of player trajectories. Archetypal analysis (AA) was adopted by Drachen et al. [START_REF] Drachen | A comparison of methods for player clustering via behavioral telemetry[END_REF] and Sifa et al. [START_REF] Sifa | Behavior evolution in tomb raider underworld[END_REF], who noted the desirable properties of the model for investigating extremal behaviors and permitting soft clustering, i.e. the expression of behavioral profiles in terms of their belongingness to multiple cluster centers. In this paper, AA is adopted as the cluster model.

Social Network Analysis: SNA of relations between people has in recent years become a commonly employed tool, with large-scale online platforms such as Facebook and Twitter providing a direct vehicle for investigation. Prior work has targeted not only analysis of social networks themselves, but also their potential for recommendation and prediction of user behavior. SNA has clear application in games, where the value of the different types of relations between players that exist in multi-player or massively multi-player online games to, e.g., player retention and user experience has been shown in a number of studies (e.g., [START_REF] Yee | The demographics, motivations, and derived experiences of users of massively multi-user online graphical environments[END_REF]).

Due to the hybrid nature of Destiny there is no prior work that is directly comparable with the study presented here. Furthermore, the integration of detailed behavioral features into the network structure is also unexplored. Furthermore, most previous work has focused on analyzing social structures in persistent online communities, rather than competitive team-based games. However, Iosup et al. [START_REF] Losup | Implicit social networks in multiplayer online games[END_REF] examined networks in DOTA and StarCraft with a specific focus on modeling social structure and network robustness toward retention. Similarly, Jia et al. [START_REF] Jia | Socializing by gaming: Revealing social relationships in multiplayer online games[END_REF] introduced networks generated from team-based match data, including networks based on players being in the same match, on the same or different side in a match.

Visualization of Player Behavior : Visualization of player behavior in game analytics has covered a variety of goals but the majority of the work is focused on the spatio-temporal components of behavior. Wallner and Kriglstein [START_REF] Wallner | An introduction to gameplay data visualization[END_REF] provide a recent review of the area, noting the wide range of techniques employed. Drachen and Schubert [START_REF] Drachen | Spatial game analytics[END_REF] reviewed spatio-temporal visualizations in use in game development. In general, the emphasis in the context of games is on pattern recognition and visualization of these patterns in a way that is actionable to the relevant stakeholder.

In network analysis, a wide range of visualizations have been proposed, but with an overall focus on link properties as compared to node properties. There are two common forms of display (cf. [START_REF] Freeman | Visualizing social networks[END_REF]): 1) The most common visualization is based on node-link diagrams where the nodes represent the social actors, and lines connections between them; 2) Matrices, where rows and columns represent social actors and various coal connections linking them. Recent years have seen the convergence of SNA and visualization, combined with interaction, but remains generally focused on network properties, including labeling, rather than node properties [START_REF] Correa | Visualizing social networks[END_REF].

Background: Destiny

Destiny is a science-fiction themed game where players need to defend the Earth from various alien threats, taking on the role of Guardians. Humanity is reduced to one last city, and it is up to the players to make sure the city stays safe, while working for the overall goal of reviving a Deathstar-sized sphere being called the Traveler, who protected human civilization in the past but currently lays dormant. Players journey to different planets, complete missions, daily events, and perform a variety of different tasks to build up their characters and help eliminate the alien threat. Destiny is a hybrid digital game that blends features from a number of traditional game genres but which is first and foremost a shooter. The main components of the gameplay is focused on tactical singleplayer or small-team combat, and the number of weapons, modifications and customizations in the game is staggering.

Weapons are divided into over a dozen different classes, each specialized for specific situations. For example, sniper rifles offer high power at long range, but are virtually useless at close range and require player skill to aim precisely. The utility of different weapon classes varies between game modes and in-game situations. Notably, the amount of damage that players in PvP can absorb can be dramatically different from the damage it takes to eliminate tough aliens in Player-versus-Environment (PvE) mode. In general, players are free to switch between any combination of weapon types, allowing for adjustment to in-game scenarios while at the same time reflecting individual behavior and preferences. Weapon-related performance metrics thus form one possible starting point for behavioral profiling of Destiny players, which targets the core shooter gameplay. Here, weapon class data is augmented with further performance metrics to provide a broad profile for developing profiles focusing on player performance in the game.

For reasons of limited space, the focus here is on those parts of Destiny that relate directly to PvP combat, but it should be pointed out that the game also has multiple PvE modes. PvP in Destiny is played in the Crucible -a central hub for the various PvP match types. These are commonly played in so-called fireteams comprised of 3 players on each side. Typically players can bring three different weapons to a PvP match. In the Crucible, players can earn medals (awards), points, and in-game currency by accomplishing tasks, winning matches, performing specific tasks, or feats of skill. Destiny features three different player classes, each with distinct abilities. Each class is divided into three subclasses. Players can level up their characters until level 40, unlocking new abilities and gradually becoming more powerful along with obtaining new weapons and other equipment through gameplay. In both PvE and PvP game modes, players are rewarded with new weapons and items through random drops or by completing specific tasks.

Behavioral Profiling with Extremes

Considering the large scale nature of today's behavioral datasets and the vast diversity of play-styles, finding profiles gives us the flexibility to manage, understand, and group the common behavioral patterns for the process of informed decision making. In this section we give a brief overview of Archetypal Analysis and it's use for player profiling. Archetypal analysis [START_REF] Cutler | Archetypal analysis[END_REF] is a constrained matrix factorization method that allows for soft clustering and is based on representing the data points as a convex combination of extreme datapoints that are called archetypes. Formally, given a data matrix X ∈ R m×n we aim to find factors minimizing the matrix norms

X -XBA 2 = X -ZA 2 ,
where Z ∈ R m×k represents the archetypes, A ∈ R k×n and B ∈ R n×k are column stochastic and represent the mixing coefficients respectively for archetypes and datapoints that represent the archetypes. Various techniques have been proposed to find appropriate factors for the above decomposition and its approximations. Examples of these include methods using pure alternating least square updates [START_REF] Cutler | Archetypal analysis[END_REF], active set updates [START_REF] Bauckhage | Making archetypal analysis practical[END_REF], or distance geometry [START_REF] Thurau | Yes we can: Simplex volume maximization for descriptive web-scale matrix factorization[END_REF]. In the context of game analytics, archetypal analysis is used to profile players [START_REF] Drachen | Guns, swords and data: Clustering of player behavior in computer games in the wild[END_REF][START_REF] Sifa | Behavior evolution in tomb raider underworld[END_REF][START_REF] Lim | Revealing social identity phenomena in videogames with archetypal analysis[END_REF], build recommender systems [START_REF] Sifa | Archetypal game recommender systems[END_REF], and generate human like bots [START_REF] Sifa | Archetypical motion: Supervised game behavior learning with archetypal analysis[END_REF]. In this study, as proposed in [START_REF] Drachen | Guns, swords and data: Clustering of player behavior in computer games in the wild[END_REF][START_REF] Sifa | Behavior evolution in tomb raider underworld[END_REF], we will concentrate on the use of archetypal analysis for finding player profiles in Destiny, in which we are interested in finding prototypical players that are encoded in Z to define particular player styles and ness coefficients represented in A showing how much each player belongs to the profiles defined in Z.

Player Networks and Characteristics

In multi-player games players interact, play together or against each other, build different in-game groups and are thus forming complex relationships and ingame structures. We can map such relationships to networks to measure and analyze these interactions. To illustrate player interactions in Destiny we can build networks based on different forms of interactions. One way to illustrate in-game relationships is to build an undirected, weighted graph based on the information how often players play matches against or -as done in this paperwith the same players. In the network, players are represented as nodes, and their match relationships are modeled as weighted (based on the number of matches played together) links.

Network Measures

For the following analysis three different networks were created. Links for all networks are built between players that played matches together in the same team. The first network (main network, MN) describes the social structure of all 3,362,636 players based on match information of 930,720 matches. The second network (connected network, CN) illustrates a fully-connected friendship network based on three well-connected players. These players were used as starting point for a breadth-first-search and connected players were added to this network up to a network size of 1,000 nodes. 11 players were removed from the resulting network because of missing weapon information. The third network (random network, RN) was built based on 10,000 randomly picked players. Table 1 gives an overview of the network characteristics of the three player networks. The largest connected component (LCC) describes the largest self-contained subgraph in the network. A large LCC refers to a well-connected main graph. Looking at the number of nodes and links in the LCC the MN is well-connected, CN is fully connected (as designed), and RN is barely connected. The degree (k) of a player refers to the number of links to other players. The average degree refers to the average number of links to other players. The diameter describes the longest of the shortest paths between two nodes and can be used to illustrate the linear size of a network.

Archetype Description

Ranged Elites (AT1) pretty good scores, auto-rifle focus, higher killing spree, unique precision kills, kill/death ratio, and win rate than AT3 Melee (AT2) melee focused, medium performance, win rate similar to AT4 Mixed Weapon Elites (AT3) high scores everywhere, more medals than AT1, better weapon scores except for auto-rifle, slightly lower win rate Short Range (AT4) medium performance, heavy use of shotgun, some melee Newbies (AT5) low performance everywhere

Archetypes

Using archetypal analysis (AA) we identified five player archetypes (see Table 2) based on their most distinguished properties. 15 features from the Destiny dataset were selected (such as precision kills, scores, medals, kill/death ratio, and weapon preference) to be used with AA. Those features describe the in-game behavior and success of Destiny players in PvP matches. Figure 1 illustrates the five archetype profiles across the 15 selected behavioral features. Those features can be split into success-based features and weapon usage features. It further illustrates the varying weapon usage behavior between the archetypes. Table 3 lists the differences in weapon usage between the archetypes for the different weapon ranges (short-, mid-, and long-range). The majority of weapon use for each archetype focuses on short and mid-range weapons, but there are a few exceptions. For instance, some players use long-range weapons almost exclusively. In order to investigate if players belonging to a specific archetype have a tendency to play with the same group of players or if they rather play with random players we tried to map the archetypes to social network metrics. For that purpose, we calculated the weighted arithmetic mean a to determine how likely it is that a player plays with the same group of players, that is sum of weights/k. However, we could not find notable differences between the behavior of players from certain archetypes when matching up with team members, with a being 1.226 (AT1), 1.225 (AT2), 1.219 (AT3), 1.224 (AT4), and 1.233 (AT5).

Visualizations and Network Integration

Figure 2 shows a node-link visualization of the CN network which represents each node as a donut chart. This donut chart denotes the weapon usage (short-, mid-, and long-range) of a player. The size of a node is proportional to the number of matches played, that is, players playing more often are represented by larger nodes. Edges show which players played in the same team. The thickness of the edges corresponds to the number of matches played together, i.e. players who play together more often are connected by thicker edges. Edges are rendered semitransparent whereas the degree of transparency is weighted based on the number of matches in order to accentuate stronger relationships. As can be inferred from Figure 2 players in this particular network prefer short-range ( blue) and mid-range weapons ( red) over long distance weapons ( yellow).

Figure 3 depicts the result of the AA for the random network (based on a subset of 7479 players). The degree of membership to each of the five archetypes is depicted through pie-charts. The spatial arrangement of nodes has been derived using multidimensional scaling (e.g., [START_REF] Kruskal | Multidimensional Scaling[END_REF]) such that players with similar archetype membership are placed in proximity of each other. To this end, the probabilities of belonging to each archetype were treated as a five-dimensional feature vector. The similarity of two players with feature vectors A and B was then measured using cosine similarity (see, e.g., [START_REF] Sifa | Archetypal game recommender systems[END_REF]), that is, s AB = A • B/ A B . From Figure 3 it can be observed that players form three major groups: (i) players mainly belonging to the novice archetype (upper right corner), (ii) players mainly being considered melee focused but also sharing traits of the ranged elites and to a lesser degree of novices (lower right corner), and (iii) players primarily sharing characteristics of the mixed weapon elites and short range archetypes (upper left corner) with some of them exhibiting characteristics of either the novices, ranged elites, or melee archetype. The latter group also accounts for the largest of the three groups in the investigated network. Players sharing traits of four or even five archetypes are more uncommon as reflected by the sparse area in the center of the figure. While this indicates specialization to a certain extent it is also apparent that players can rarely be attributed to a single archetype.

Conclusion and Discussion

In this paper behavioral profiles have been constructed based on performance telemetry data from the PvP activities of 10,000 players, randomly selected from those who had played for at least two hours, of the persistent online FPS game Destiny, using Archetypal Analysis. A five-cluster solution was found, which showcases how performance-based profiling in the PvP component of Destiny can be closely related to the choices players make with respect to which weapons they use, as well as their relative skill, as expressed, e.g., via their kill/death ratios. Each of the players can be expressed in terms of their degree of membership to each archetype. The profiles have been embedded in a competitive player network covering the almost 3.5 million players that the 10,000 are connected to. The competitive network maps the tendency of people in Destiny to play with the same people or random groups across the various PvP modes of the game. In addition, a number of behavioral metrics has been examined in terms of their distribution in the network, notably weapon use distribution, which is a key indicator in Destiny as the main gameplay revolves around combat.

Future work will focus on evaluating player performance and network structure as a function of the number of players PvP matches. Initial dataset analysis indicated a relationship between performance metrics and the number of matches played. Furthermore, players tend to gravitate from using specific weapon classes during early matches to other weapon classes after having played many PvP matches. Both of these patterns will be verified and investigated in future work. Furthermore, the current work developed profiles for 10,000 players in a 3.5 million player network, and it is therefore an obvious next step to scale up the profiling to all the players in the network. This would substantially increase the sample size but also requires algorithms that scale well to such large datasets. Finally, the visualizations presented here will be further evaluated and expanded on, and a web-based interface developed which will target both game designers and players of competitive multi-player matches for performance evaluation.

Fig. 1 .

 1 Fig. 1. Archetype profiles for Destiny across behavioral features. WR = win rate, KDR = kill/death ratio, AME = all medals earned, AS = average score, CR = combat rating, LKS = longest killing spree, UPK = unique precision kills, WKM = weapon kills melee, PK = precision kills with auto rifle (AR), hand cannon (HC), machine gun (MG), scout rifle (SR), shotgun (S), sniper rifle (SN), and pulse rifle (PR)

Fig. 2 .

 2 Fig. 2. Visualization of the fully-connected friendship network with 964 nodes. Donut charts at each node show the weapon usage for the individual players ( short range, mid range, long range). Labels are shown for players who played more than 1000 matches. Player names were anonymized.

Fig. 3 .

 3 Fig. 3. Players' degree of membership to each of the five archetypes ( AT1 -Ranged Elites, AT2 -Melee, AT3 -Mixed Weapon Elites, AT4 -Short Range, AT5 -Newbies).

Table 1 .

 1 Methodological comparison of the three networks, (a) the main network with over 3 million players, (b) a well-connected network, and (c) a random selection of 10,000 players (LCC = largest connected component).

		MN	CN	RN
	Nodes	3,362,636	989	7,479
	Nodes in LCC	3,347,226	989	407
	Links	22,638,062	2,321	1,601
	Links in LCC	22,614,017	2,321	432
	Network size	3,362,596	989	1,991
	Average degree	13	4	0.428
	Diameter	15.0	4.0	23
		MN = main network, CN = connected network, RN = random network

Table 2 .

 2 Overview of the different archetypes

Table 3 .

 3 Weapon use of the five different archetypes

	Archetype	Short-range	Mid-range	Long-range
	AT1	M=0.45, SD=0.38	M=0.45, SD=0.32	M=0.094, SD=0.49
	AT2	M=0.41, SD=0.28	M=0.46, SD=0.23	M=0.13, SD=0.38
	AT3	M=0.42, SD=0.12	M=0.45, SD=0.10	M=0.13, SD=0.15
	AT4	M=0.45, SD=0.17	M=0.46, SD=0.15	M=0.095, SD=0.20
	AT5	M=0.48, SD=0.11	M=0.42, SD=0.10	M=0.097, SD=0.14

Acknowledgments:

The authors would like to extend their sincere gratitude to Bungie for making detailed behavioral telemetry from Destiny available.