Dan Conway

Ian Dick
email: ian.dick01@gmail.com

Zhidong Li

Yang Wang

Fang Chen
email: fang.chen@nicta.com.au

The Effect of Stress on Cognitive Load Measurement

Keywords: Cognitive load, galvanic skin response (GSR), stress

Human physiological signals have been widely used to non-invasively measure cognitive load (CL) during task execution. A major challenge for CL detection is the presence of stress, which may affect physiological measurements in ways that confound reliable detection of CL. In this experiment we investigated the effect of stress on cognitive load measurement using galvanic skin response (GSR) as a physiological index of CL. The experiment utilized feelings of lack of control, task failure and social-evaluation to induce stress. Mean GSR values were shown to be significantly different between CL levels in the "no-stress" condition, but not when including the "stress" condition. On the other hand, features extracted from GSR signals based on peak detection exhibited consistent behaviour under both conditions, demonstrating the usefulness of the features as cognitive load index even when a person"s stress level is fluctuating.

Introduction

The term cognitive load (CL) refers to the amount of cognitive resources required for a person to complete a certain task. CL has been shown to have important implications for learning [START_REF] Sweller | Cognitive load theory, learning difficulty, and instructional design[END_REF], safety in driving [START_REF] Engstrom | Effects of visual and cognitive load in real and simulated motorway driving[END_REF], aviation [START_REF] Wilson | An analysis of mental workload in pilots during flight using multiple psychophysiological measures[END_REF], and user interface design [START_REF] Saadé | First impressions last a lifetime: effect of interface type on disorientation and cognitive load[END_REF]. Being able to accurately monitor CL in real world environment would have important and far-reaching implications. This capability could enable the real-time detection of cognitive overload, when a person"s cognitive resources are overloaded with information, and this could help to avoid dangerous situations, such as an air traffic controller with too many concurrent responsibilities. CL monitoring could also enable the design of better computer interfaces, since interface designs could be rated based on the CL levels they impose on users, and designs with poor performance could be disregarded.

Physiological signals have previously been proposed as a method of quantifying CL. Some notable successes in CL evaluation have been achieved via signals such as respiration, brain wave, and pupillary response [START_REF] Knoll | Measuring cognitive workload with low-cost electroencephalograph[END_REF] [14] [START_REF] Wang | Indexing cognitive workload based on pupillary response under luminance and emotional changes[END_REF] [17] [START_REF] Xu | Pupillary response based cognitive workload measurement under luminance changes[END_REF]. Another physiological signal that has been used successfully to quantify CL, and which is the focus of this research effort, is galvanic skin response (GSR), a measure of the electrical conductance of the skin shown to be sensitive to both CL and stress [START_REF] Healey | SmartCar: detecting driver stress[END_REF] [START_REF] Nourbakhsh | Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks[END_REF].

Any given physiological signal used as an index for CL is likely to be affected by various additional inputs from the human body. Thus, a major task of cognitive workload measurement via physiological means is demonstrating the diagnosticity and construct-validity of the CL index. One of the major contributors to change in human physiological systems is stress [START_REF] Martin | Psychology[END_REF]. Stress has been shown to effect both the sympathetic and parasympathetic nervous systems and, in its more extreme states, results in large changes to physiological function that may well obscure the relationship between a physiological indicator and CL. Furthermore, stress may, in some circumstances, be a confounding factor for CL in that changes in CL may correlate with changes in stress levels. Construct-validity must be established before we can safely assert that changes in physiological indicators are the result of CL and not stress or other confounding factors. This experiment studies the effect of stress on CL measurement using GSR.

Although definitions of stress vary, there is good consensus in the literature regarding conditions where it is likely to arise [START_REF] Dickerson | Acute Stressors and Cortisol Responses: a theoretical integration and synthesis of laboratory research[END_REF] [7] [START_REF] Setz | Discriminating stress from cognitive load using a wearable EDA device[END_REF]. Failure at a task, together with feelings of lack of control, in situations where participants are evaluated by others is a widely used paradigm for stress induction. These conditions could be operationalised in an experimental paradigm developed by Dedovic et al. called the "Montreal Imaging Stress Task" (MIST) [START_REF] Dedovic | The Montreal Imaging Stress Task: using functional imaging to investigate the effects of perceiving and processing psychosocial stress in the human brain[END_REF]. The experiment presented here closely follows the MIST protocol with minor operational adjustments.

Related Work

GSR has been used successfully in the past to index CL. In an experiment involving traffic control management it was illustrated that the mean GSR of test subjects increases as the difficulty of cognitive tasks increases [START_REF] Shi | Galvanic skin response (GSR) as an index of cognitive load[END_REF]. In addition, [START_REF] Nourbakhsh | Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks[END_REF] analysed the time and frequency domains of recorded GSR signals and showed that CL imposed through arithmetic and reading tasks can be indexed by GSR of test subjects. The relationship between GSR and stress has also been examined. In an experiment involving driving tasks, [START_REF] Healey | SmartCar: detecting driver stress[END_REF] were able to successfully classify different driving periods based on the stress levels of the driver. They extracted useful features from GSR signals recorded during the experiment based on peak detection and input these features into machine learning classification algorithms with positive results. [START_REF] Zhai | Stress detection in computer users through non-invasive monitoring of physiological signals[END_REF] combined GSR with several other physiological signals to classify the stress states of test participants through the use of machine learning tools. Stress levels were induced by having subjects complete a "Paced Stroop Test", where the colour of a word that spells a different colour must be identified.

An interesting study on discriminating stress from cognitive load was carried out by Setz et al. [START_REF] Setz | Discriminating stress from cognitive load using a wearable EDA device[END_REF]. However, they did not experimentally manipulate CL and only demonstrate the ability to differentiate between "stress" and "no-stress" conditions, where CL level was consistent between the two conditions. Nonetheless, the feature detection processes they outlined appear promising, and have been further studied in the experiment presented here.

Experiment

11 male students and employees (24-49 years" old, ten right handed and one lefthanded) took part in the experiment. Participants were offered one movie ticket and biscuits as recompense for their participation.

All participants had the voluntary nature of the experiment explained to them and then filled out a paper version of the Kessler K-10 Psychological Distress Scale [START_REF] Andrews | Interpreting scores on the Kessler psychological distress scale (K10)[END_REF] to ascertain that they were unlikely to be vulnerable to ongoing negative effects from the stress condition. Only participants who scored less than 19 (thus fell into the category "likely to be well") were permitted to continue the experiment. Three potential candidates were rejected via this means.

Apparatus

All experimental stimuli were presented on a computer screen using custom software whilst participants were sitting comfortably at a desk. GSR signals were collected using a Thought Technology "ProComp Infiniti" interface and its "SCFlex/Pro" skin conductance sensor. The sensors were attached to the non-dominant hand for all participants. GSR signals were sampled at a rate of 10Hz. The brain wave signals were also recorded with a low-cost EEG device "Emotiv EPOC". However the analysis of the EEG data is out of the scope of this paper. Participants were asked to remain still and only move their dominant hand for mouse control during the experiment.

Procedure

The experiment consisted of a within-subjects, six-way factorial design. There were math questions of three difficulty levels (low, medium and high) administered under two different stress conditions: "no-stress" and "stress". For level 1 problems (low difficulty), three terms were added together. For level 2 (medium difficulty), each problem consisted of four terms, with both addition and subtraction required. Level 3 problems (high difficulty) consisted of five terms, with addition, subtraction and multiplication required. The multiplication terms were in a random position within the problem.

All participants undertook the "no-stress" condition first. Participants were told that they would be completing math tasks but it was emphasized that their performance/accuracy was not important. After submitting some basic demographic information, a two minute baseline period was carried out where the participants were told, via an on screen prompt, that they should just relax and let their mind wander. Then three two-minute blocks of math tasks were presented, each with 4 multiple choice answers. Tasks were not time limited and feedback was not provided. The three blocks in the "no-stress" condition were of level 1, 2 and 3 difficulty in sequential order. The participant was given a two-minute resting period in between each block. After block three, the participants were asked, via on screen prompts, to nominate a "target score" for further tasks based on their estimation of their performance so far. Once submitted, the stress condition ensued. They were told that their performance would be now be monitored. They were also informed of time limits for further trials, and video screens were switched on so that the test subject could see a video of themselves and also of other people observing them (see Figure 1). Now in the stress condition, three more blocks of level 1, 2 and 3 math tasks were carried out, again with two-minute pauses in between each block but with time limits now imposed on each trial. Feedback ("correct", "wrong" or "out of time") was provided for one second after each trial.

Analysis

Fig. 2. Subjective rating of task difficulty.

In order to test the validity of the methods used in the experiment for inducing different levels of cognitive load, one-way ANOVA test of pooled subjective ratings was conducted. Results showed that the difference between the different difficulty levels was significant (F = 82.32, p < 0.05). It can be seen in Figure 2 that the means of each group were increasing with the task difficulty level.

Analysis using mean GSR

The mean GSR values were inspected to study the effect of stress on cognitive load measurement. The distribution of normalised mean GSR values corresponding to the sub-sections of math task difficulty 1, 2 and 3 under both "no-stress" and "stress" conditions can be seen in Figure 3a. To investigate the relationship between mean GSR and cognitive load when no stressful stimuli are present, we conducted ANOVA analysis on these GSR values under "no-stress" conditions and found that there are statistically significant differences between the 3 different levels (F = 10.5, p < 0.05), and there is a noticeable upward trend in mean GSR that corresponds to an increase in task difficulty. The positive correlation between cognitive load and GSR could no longer be observed, however, once the data from the "stress" part of the experiment are included for consideration. Figure 3b shows the distribution of the normalised mean GSR values for math task difficulty levels 1, 2 and 3, with both the "no-stress" and "stress" data included. ANOVA analysis of these values does not produce significant results (F = 0.05, p = 0.95). The results indicate that mean GSR cannot effectively index cognitive load when stress levels are fluctuating, since mean GSR is sensitive to stress and the correlation between cognitive load and mean GSR becomes obfuscated when stress is a confounding factor. To overcome this problem, feature extraction technique has been employed for workload evaluation in the following section.

Analysis using 'peak' features

Similar to [START_REF] Healey | SmartCar: detecting driver stress[END_REF], several features corresponding to the peaks in the signals were extracted from the smoothed GSR signals. The following definitions were made: S D is the distance along the x-axis from the local min preceding a peak to the local max of the peak (i.e. peak duration); S M is the distance along the y-axis from the local min preceding a peak to the local max of the peak (i.e. peak magnitude); S F is the number of peaks divided by the task period (i.e. peak frequency). Figure 4 illustrates these concepts. In this work we used these three peak based GSR features to study the effect of stress on cognitive load measure. Except S M , the other two features, S F and S D , demonstrated their usefulness for indexing CL even when stress is a confounding factor. The S F feature represents the frequency of peaks per sub-section. The "no-stress" and "stress" distributions for this feature are shown below in Figures 5a. For mean GSR, there was no common trend between the "no-stress" and "stress" conditions, and so there is no obvious way of using mean GSR to index CL when stress levels are fluctuating. In contrast to this, the S F feature displayed in Figure 5 behaves similarly in both the "no-stress" and "stress" conditions. We can see that S F is negatively correlated to task difficulty regardless of whether stress is present, although the relationship is stronger in the "stress" condition. Figure 5b shows the distribution of the S F feature for difficulty levels 1, 2 and 3, with both the "no-stress" and "stress" data included. ANOVA analysis was performed on this data to test the significance of the negative correlation. The result exhibited significant difference among the three difficulty levels (F = 3.96, p < 0.05).

Stress No stress

The S D feature corresponds to the peak duration per sub-section. The distribution of normalized S D feature (sum of peak durations divided by the sub-section period) corresponding to math task difficulty 1, 2 and 3 under both "no-stress" and "stress" conditions can be seen in Figure 6a. It turns out that this feature behaves quite similarly to the S F feature, and is negatively correlated to task difficulty under both "no-stress" and "stress" condition. Figure 6b shows the distribution of the S D feature for difficulty levels 1, 2 and 3, with both the "no-stress" and "stress" data included. The downward trend with increasing task difficulty could be observed. ANOVA analysis also generate significant difference among the three difficulty levels (F = 5.14, p < 0.05), indicating the feature could be useful as an index of cognitive load even under the influence of stress conditions.

Conclusion

The experiment in this work helped to reinforce GSR as an index of cognitive load during task execution. Without the impact of stress, it appears that an increase in CL (induced by increasing the difficulty of tasks given to test subjects) results in an increase in mean GSR value. This relationship is, however, obfuscated when test subjects experience fluctuating levels of stress. Stress was introduced into the experiment using an adaptation of the MIST protocol, and this blurred the connection between GSR and CL. GSR may still be useful as an index for CL even when stress is a confounding factor, if we consider peak based features extracted from the GSR signal other than the mean value. Both peak frequency in the signal and peak durations are negatively correlated to task difficulty and hence CL. These features could possibly be used to dissociate CL from stress and develop a stress-agnostic method of CL classification. Our future work in this area include more feature extraction and also machine learning to investigate whether these features may be used to classify a person"s CL under the influence of various confounding factors.

Stress No stress

Fig. 1 .

 1 Fig. 1. Experiment setup.

Fig. 3 .

 3 Fig. 3. Distribution of normalized mean GSR values for task difficulty levels 1,2,3 under the 'no-stress' and "stress" conditions.

Fig. 4 .

 4 Fig. 4. Example of a smoothed GSR signal adorned with S D and S M features.

Fig. 5 .

 5 Fig. 5. Distribution of S F feature for task difficulty levels 1, 2, 3 under the 'no-stress' and "stress" conditions.

Fig. 6 .

 6 Fig. 6. Distribution of normalized S D feature for task difficulty levels 1, 2, 3 under the 'nostress' and "stress" conditions.


Michael Wolf
email: michael@formula-d.com

Lessons Learned from Designing Non-traditional Interfaces for Educational Applications in South Africa

Keywords: Non-traditional interfaces, interaction design, tangible interaction, gesture-based interaction, Locomotion interfaces, interactive surfaces, multitouch, interactive learning environments, HCI examples

As a specialised design consultancy for interactive learning environments and tools, Formula D interactive has gained valuable project experience in designing nontraditional interfaces for digital educational content and tools in the culturally diverse context of South Africa. The aim of this paper is to share the company's experience in the field using prominent examples of their recent work, related research and user testing in order to discuss the merit of largescale interactive surfaces, gesture-based and tangible interfaces in culturally diverse contexts. The company's work includes interactive displays for science centres and museums as well as digital learning tools for classroom environments.

Introduction

The praxis and scope of designing interactive learning environments goes beyond online platforms for learning content or interactive whiteboards. Specifically nontraditional interfaces [START_REF] Kortum | HCI beyond the GUI -Design for Haptic, Speech, Olfactory and Other Nontraditional Interfaces[END_REF] create opportunities for computer supported co-located learning and accessibility of technology tools by audiences with little experience with traditional computer interfaces. In the following project examples of Cape Town based design firm Formula D interactive (www.formula-d.com). the author highlights the key considerations in the design process, challenges and lessons learned when designing nontraditional interfaces for learning applications in South Africa.

Examples of designing and testing nontraditional interfaces and lessons learned 2.1 Multitouch surface at the Two Oceans Aquarium in Cape Town

In 2008 our company designed a 100" rear-projected multitouch wall for the Two Oceans Aquarium in Cape Town (Fig 1). The goal of the exhibit was to create an immersive interactive landscape to encourage visitors of the aquarium to enquire and learn about habitats, calls and threats of local frog species. Users find themselves looking at a photorealistic animated landscape. By touching various hotspots they activate one of the habitats, revealing a 360 Deg panoramic view of a typical area. Now, the challenge is to discover the frog species living in the environment. When a frog icon has been found and touched, a window with description and images, video clips and the call of the frog appears. Screen objects can be moved around by dragging and dropping them, for example, to individual eye-level. Some objects can also be scaled using two fingers or both hands. As multiple frog sounds can be simultaneously activated and deactivated, visitors may collaborate orchestrating "frog symphonies", broadcast via a directional overhead speaker.

The frog wall sported various novel features which had not been seen in South Africa at the time and thus presented itself as a good case study for nontraditional HCI as it was exposed to some thousand users from various cultural backgrounds every day at the Two Oceans Aquarium. • At first, many people of all audiences did not identify the large screen as an interactive surface. In addition to on screen instructions, additional instructional signage was required even though using bold icons as touch points in the GUI (Fig 2).

Touch screens were quite common at the time in South Africa (for example in ATM's which had been in use across all levels of the working society), so we concluded that various contextual elements did not support the user interaction although the GUI was carefully crafted:

The screen size did not correspond with the conceptual model people had of a touch surface at the time. ─ Apart from a photo booth, the screen was the only interactive exhibit in the aquarium. ─ The first impression of the exhibit design was reminiscent of the live exhibits at the aquarium (fish tanks). These exhibits are usually not being touched or touching them is prohibited. • The majority of people in South Africa at the time were unfamiliar with the concept of multitouch (the iPhone had not yet been released in South Africa), so these features were hardly used. The two finger scaling gesture, although indicated by GUI elements (Fig 2), was only slowly adopted by the visitors in 2009, when this interaction entered mainstream through iPhone and iPad. Less privileged communities without access to these devices did not access this feature at all unless they were instructed by staff or observed other visitors. • Users also mostly did not understand that the content windows could be moved around although this feature is very common with desktop computers. • In turn, once the screen was in use by at least one user, other visitors frequently joined in. The size of the display seemed to suggest that multiple users are possible.

In summary, our observations seem to confirm that the context and size of the display was in conflict with the conceptual model of touch screens and desktop computers, which were familiar to at least a part of the audience. This resulted in certain GUI elements common to desktop applications not being identified. On the other hand, this contextual distance from traditional interfaces could have led to users embracing the multi-user functionality almost naturally.

Locomotion interface at Cape Town Tourism

In 2009 we installed our first 6 metre by 2 metre interactive wall at the Cape Town Tourism information centre, which uses visitor's body motions as a means to navigate digital content. The wall projection showcases 40 activities and sights in the Cape region, set against an animated backdrop of interchanging panoramic representations of iconic areas around the Cape. When visitors step in front of the wall, and align themselves with one of the projected icons on the wall, they trigger animations, sounds, and pop-ups with information on the selected attraction (Fig. 3,4).

Fig. 3,4. Interactive wall for Cape Town Tourism Visitor Centre

Limitations of the interaction. At the time we used infrared cameras for blob detection not only for multitouch displays, but also for interactive floor and wall projections (Microsoft Kinect had not been released yet). The overhead 2D-tracking of user activity on the floor only provided x and y positions as well as blob size and grouping as a means of user input. Since the floor in front of the wall is also a passage area in the centre we had to differentiate between intended and accidental users. In order to avoid accidental inputs, we decided to limit the interaction area to a designated floor strip in front of the wall and only worked with object movement on the x-axis, which further limited the user control in the application. We used a vertical colour bar which moved with the user across the wall to make the user aware they are influencing the environment. Next, we decided to trigger content when a user stopped at a certain area as this may suggest an interest in the special area.

Lessons learned.

• Digital "real estate": There are many challenges and limitations when multiple users interact in a shared digital environment which is based on one shared output. Online, or in AR scenarios where users have individual devices or headsets, a subjective render of the environment with personalised content is possible. However, if there is only one shared output, the "real estate" of the digital space needs to be shared just like our physical spaces. • In the design process of this project we had a lot of debate around managing the physical interaction area and its corresponding digital space for an unknown number of simultaneous users. We were concerned about conflicts arising between users who aimed at triggering events in proximity to each other. This resulted in a quite rigid partitioning of the interface area with finite positions of trigger points, so we could be sure there would not be spatial conflicts between interacting users.

• However, this concern has proven to be unjustified since the natural behaviour of people in communal space is to keep a minimum distance between each other unless two people are grouped, like a couple walking arm in arm or an adult with a child. It needs to be noted that this minimum distance between people in physical space differs in different cultural environments. However, we observed that a "spatial etiquette" is maintained when multiple users interact in large interactive environments such as the interactive tourism wall or the Frog Touch wall. People do generally keep distance physically and virtually.

Gesture controlled "point screen" for the Centre for Public Service Innovation

As part of a larger installation of various interactive displays for the Centre for Public Service Innovation (CPSI) near Pretoria, we created several user interfaces for a hand and finger-tracking device developed by HHI Fraunhofer in Berlin (Fig 5). iPoint accurately tracks one or multiple finger gestures, which can then be calibrated to navigate a Graphical User Interface. We developed two applications for this system for an educational exhibition environment, one information kiosk with best practise examples of innovation in public service, one assessment of the users' understanding of innovation principles. • Challenges with the conceptual model of the interaction: Although pointing surely is one of the most archaic gestures as it can already be observed in small infants, users presented with a screen and the invitation to point at it behave all but "natural". Even when instructed to point at the screen "naturally" many users bent their arms in anticipation of triggering a sensor in the black box above or underneath them. • It seems "unnatural" to users to interact with a screen by pointing at it. Thus, the gesture itself needed to be instructed and could only be applied after reflection in the new context (Fig. 6). • The trigger problem: Most computer interfaces separate pointer and input action.

This gives users a chance to reflect on their choice before executing an action. A gesture based point interface does not have a mouse click, so an alternative way of selecting an action had to be invented. Already 10 years ago, I had worked on pointing devices using electric field sensing (EFS) technology at MARS Lab (Fraunhofer Institute for Media Communication) in Germany [START_REF] Strauss | Information Jukebox -A semi-public device for presenting multimedia information content[END_REF]. Various ideas and concepts were thought out at the time, such as multimodal interface solutions which combined finger pointing action with finger snapping or voice commands. However, the added complexity of the interaction always felt unsatisfactory since it undermined the simplicity of the pointing gesture. Thus, for our South African client, we looked at GUI solution that did away with clicking and worked solely with roll-over like the website experiment Don't click it [START_REF]Don't Click It! -www[END_REF].

Our first iPoint GUI (Fig. 7) was a structure of unfolding and collapsing content display areas which open and close according to the position of the cursor. However, what seemed a viable solutions for a Desktop computer interface with a mouse pointer, did not work as a finger-pointing device. After selecting content, users had to leave their arm in the same position as long they needed to absorb the content, which caused great discomfort for our users. A subsequent version was designed in a way that users could lower their arm without triggering other content areas. But even this solution resulted in many accidental selections. The current design comprises of a time-delayed activation, a visual countdown, which activates a link only if a user remains in one position for a few seconds. A similar solution is featured in Microsoft Kinect games. Even with the time-delayed activation functionality, we still faced ergonomic problems. It was uncomfortable and tiring to navigate the interface even if just for a few minutes at a time. We realised that we had spent a lot of time thinking about trigger mechanisms, but had not yet looked closely at the specific ergonomics of pointing with hand and arm. After a few tests we found that although user managed to navigate traditional GUI's with a horizontal menu structure (Fig 9 ), keeping arm and hand on a small target for the necessary 2 seconds until the link activated added a fair amount of discomfort. The second generation of tests was conducted with a checkerboard interface (Fig 10 ), which not only offered larger hit areas, but also used the entire screen space as the interactive area. This made the interaction more fluid and users felt more in control. However, at the same time, the absence of a resting area pressured users to make a quick selection, which made the interaction uneasy. The current interface and best solution to date is a circular GUI (Fig 8,Fig 11), which follows the natural ergonomics for pointing with the hand, since it supports the circular movement of the wrist. The movements feel natural. At the same time, movements can be reduced to an absolute minimum; the interface can be navigated with only the index finger moving. With regard to the trigger area, we found that adding large interactive areas and rollover states added to the experience of control and ease, whilst the trigger areas or buttons needed to be separated from the active areas, so the trigger timer would only be activated if users intended to do so.

The Virtual Chemistry Lab Table (VCLT)

A few years ago Formula D interactive started the development of the Virtual Chemistry Lab Table (VCLT), a tool to help learners understand the basics of Chemistry through hands-on interaction. Once connected to a standard computer and screen, and the software is installed, the Virtual Chemistry Lab Desk allows learners to arrange physical objects on a surface in order control the software (Fig 12). Learners then explore the digitally simulated experiments using an array of tools similar to the ones in a real chemistry lab by adding other objects in proximity to each other. A simple content management system expands the functionality of the lab from a simulator to a documentation and presentation tool. Here, learners embed their own knowledge or test results (from real lab experience or secondary research) and embed content, such as images, video or text within the application. Now, they can share and discuss their findings with peers in a classroom setting. The VCLT is built on top of the reactivision platform (http://reactivision.sourceforge.net/) with custom built hardware and content development system. • A virtual chemistry lab is a great application for tangible interaction through reference objects. Users can easily transfer the mental model from a real chemistry lab to the simulated one. The interactions are similar yet simplified. • Initial considerations for the features of the lab made only provision for various content modules with simulated experiments. Various discussions with teachers and students suggested, however, that the lab needed the possibility for usergenerated content through a simple CMS (Fig 14). At this point the lab would also become a documentation centre and database, which could eventually be shared with other VCLT's online. • The VCLT was originally designed as a Science Centre exhibit. The design included a much larger table with a rear-projected screen of 50". The content animations projected around the objects which were laid on top of the glass surface. Since many South African schools wouldn't be able to afford a large table, we developed a low cost version which was able to plug into existing hardware and was more suited to a classical classroom scenario.We then experimented with a prototype which separated input and output (Fig 13). We expected problems with the interaction and were surprised when young learners of different cultural backgrounds acted very confidently when operating the nontraditional input device in combination with a traditional output device (screen). • For our initial user testing (Fig 15 ) we selected a girl's junior school in Wynberg, a suburb of Cape Town. The pupils at the school come from culturally diverse backgrounds. We observed 8 girls in groups of two for 15 minutes each, and a group session with teacher. Approximately 50% of the testing group has access to a computer at home. The observations confirmed that: ─ The interface was easily understood without instructions by the vast majority of the users including users who have no regular access to computers. ─ Only one girl had to be informed of the possibility of combining different objects, after she had placed single objects onto the lab one by one. ─ Most users were able to memorise the experiments they tried out. ─ Observations of user interaction confirmed that the VCLT invites collaboration since the tangible interface objects can easily be shared and jointly operated. In various interviews the users commented positively on the tool: ─ They liked that they could "see what they are doing" through the strong representation of the current active state in the interface (objects/ingredients on the table). ─ They highlighted that they enjoyed the hands-on approach, as opposed to just listening to the teacher.

Conclusion

A recurring dilemma in designing nontraditional interfaces is that the designer's intention of supporting more meaningful and user-friendly interaction through more "natural" interaction frameworks (like gesture, locomotion or tangible interfaces) is prone to failure when the user's conceptual model of how technology works is predominantly based on traditional interfaces they have been exposed to. These preconceptions can only be influenced if designers use strategies that make provision for the context of the user and the environment in which the tool is deployed. Through project work on the Virtual Chemistry Lab Table it became apparent that using a combination of traditional and non-traditional interface elements can be a good strategy to offer the user a familiar environment on one side, building up the confidence needed to engage with new forms of interaction on the other. Beyond these design considerations and strategies, Formula D interactive's project experiences indicate that non-traditional interfaces such as the Virtual Chemistry Lab Table improve co-located learning and collaboration in the classroom and make interactive digital technology more accessible to audiences with no or little experience with traditional interfaces.

Fig. 1 .

 1 Fig. 1. Frog multitouch wall, Fig 2. Frog wall GUI

Fig. 5 .

 5 Fig. 5. iPoint tracker by HHI Fraunhofer, Fig 6. Problems with the gesture application

Fig. 7 .

 7 Fig. 7. iPoint "Don't click" GUI, Fig 8. Sample of final GUI solution

Fig. 9 .

 9 Fig. 9., Fig 10., Fig 11. Evolution of the point GUI

Fig. 12 .

 12 Fig.12. VCLT GUI , Fig 13. Desktop console concept render


Minna Pakanen
email: 2opakanen@paju.oulu.fi

Leena Arhippainen

Jukka H Vatjus-Anttila
email: jukka.vatjus-anttila@cie.fi

Olli-Pekka Pakanen

Visual Indication While Sharing Items from a Private 3D Portal Room UI to Public Virtual Environments

Keywords: Portals, 3D user interface, virtual environment, user interaction, visual indication, user experience

In this paper, we describe the user experience evaluation results of a 3D Portal Room UI for sharing 3D objects from private space to remote public virtual environments. The user evaluation with 30 participants was conducted with a functional prototype and additional high quality images that were printed on paper sheets. The evaluation indicated that participants liked this way of sharing objects and found it also useful. However, it also raised some privacy concerns, especially if the target virtual environment was perceived as public. Evaluation elicited that the visual indication while sharing objects is important; therefore, designers of 3D virtual environments should prefer a distinguishable glow around the shared object and portal.

Introduction

In collaborative three dimensional (3D) virtual environments (VEs), such as Second Life [START_REF] Lab | Second Life[END_REF] and World of Warcraft [START_REF]World of Warcraft[END_REF], users can see objects and other people's avatars in a 3D space. To share items with other users or players, current solutions must rely on two dimensional (2D) pop-up repository windows, which reduce the view of the VE and weaken the 3D experience. There is a need for a private 3D VE which is located in parallel with public VEs and the possibility to move objects between them. The need is noted in collaborative VE context, where a user wants to have private information in collaborative VE, but at the same time, he/she needs to be able to hide it form other users [START_REF] Butz | Of Vampire Mirrors and Privacy Lamps: Privacy Management in Multi-User Augmented Environments[END_REF]. There is also a need in education context, where a teacher could prepare classes and share material from his/her private 3D space to the pupils in public virtual classrooms, and at the same time, be able to monitor what pupils are doing in collaborative VEs [START_REF] Arhippainen | User Experiences of 3D Virtual Learning Environment[END_REF].

In this paper, we present a 3D Portal Room UI with an ability to have four client server connections in parallel, which are presented to the users as four parallel views to the VEs. The focus of this paper is on visual indication when moving 3D objects from a private 3D Portal Room UI to public VEs. We report the user evaluation of the prototype and additional visual indication image examples with 30 participants. This paper contributes to the 3D VE research and the designer community by presenting user experiences (UXs) [START_REF]Ergonomics of human system interaction -Part 210: Humancentred design for interactive systems[END_REF] with the prototype and participants preferences on visual indication while objects are shared through portals.

Related Research

Portals have been studied in prior research from the navigation point of view [START_REF] Kotziampasis | Seamlessly Integrated Distributed Shared Virtual Environments[END_REF]. Kotziampasis et al. [START_REF] Kotziampasis | Seamlessly Integrated Distributed Shared Virtual Environments[END_REF] found that portals help users to navigate in the virtual worlds, because of the visual presentation of the target space. Portals have also been used as object transfer devices, but only in two dimensional environments, where documents and 2D shapes can be exchanged through portals on a surface display [START_REF] Voelker | Dynamic Portals: a Lightweight Metaphor for Fast Object Transfer on Interactive Surfaces[END_REF]. The limitation of prior research is the lack of user interaction research. Shneiderman [START_REF] Shneiderman | Why Not Make Interfaces Better than 3D Reality[END_REF] argued ten years ago about making 3D UIs to facilitate user tasks with an enhanced 3D design rather than just mimicking reality in all possible ways. To design visually enhanced 3D UIs and games, we agree with El-Nasr and Yan that visual indication is an important factor, especially for novice users [START_REF] El-Nasr | Visual Attention in 3D Video Games[END_REF].

Visual attention process has been studied extensively within cognitive science and psychology, but not from a 3D game perspective [START_REF] El-Nasr | Visual Attention in 3D Video Games[END_REF]. Visual indication in 3D scenes has been studied from the anchor highlighting point of view, where the colour coding on 3D objects was preferred in the usability test and user comments, but the most visually appealing choice was the colour edges [START_REF] Pichler | Interactive Browsing of 3D Scenes in Hypermedia: The Hyper-G 3D Viewer[END_REF]. Also, it is noted in 3D games research that the location of the visual indication is as important as the visual indication itself [START_REF] El-Nasr | Visual Attention in 3D Video Games[END_REF].

Butz et al. [START_REF] Butz | Of Vampire Mirrors and Privacy Lamps: Privacy Management in Multi-User Augmented Environments[END_REF] have studied indication of shared elements in collaborative 3D VEs from privacy perspectives. They introduced two methods for privacy management: a vampire mirror, which shows only shared items, and a privacy lamp, in which a beam of light depicts the private items of other objects in collaborative 3D VE in augmented reality situation [START_REF] Butz | Of Vampire Mirrors and Privacy Lamps: Privacy Management in Multi-User Augmented Environments[END_REF]. Also, it is suggested in prior research that the colour of the shared element should be changed or other visual feedback be given when something is moved from a personal 3D GUI to the collaborative 3D VE [START_REF] Pakanen | Studying Four 3D GUI Metaphors in Virtual Environment in Tablet Context[END_REF].

The Evaluation Prototype

We used the realXtend Tundra 2 [12] virtual world viewer as a setup platform for portals. It is based on a client-server type architecture. We used the following hardware setup in the user evaluation: HP Elitebook 2760p laptop with Ubuntu Linux 12.04 LTS 32bit with WLAN 802.11g public Internet connection. In addition, we had an external 24" Dell P2411Hb 1920x1080 monitor, mouse and keyboard.

The 3D Portal Room UI acts as a private 'log-in screen' to the other VEs. Portals provide an ability to have four client-server connections in parallel, which are presented to the users as four parallel views to the VEs [START_REF] Hickey | User Experience Study of Concurrent Virtual Environments with 2D Tab and 3D Portal UIs[END_REF]. We decided to use a room metaphor, because it creates a feeling of privacy [START_REF] Pakanen | Studying Four 3D GUI Metaphors in Virtual Environment in Tablet Context[END_REF]. As we wanted to increase the feeling of private space, there are no windows in the room and the wall texture was chosen to look sturdy (Fig. 1). In the 3D Portal Room UI, there are the user's personal items such as a microphone, a photo album, a notebook, a trashcan, a game and MS PowerPoint, Word, PDF files. The portals (views) to the public VEs are covered with round doors, when there is no open connection to them. Connection is created by tapping the door, when a door mesh is replaced by a real-time view to the remote VE. We chose four remote public VEs (Music Club, Office, Outdoor Music Club and City) for the prototype. The chosen VEs had different visual styles from each other, because we wanted to study how participants perceive sharing items to them. (Fig. We implemented to the 3D Portal Room UI a functionality to move private 3D objects to remote VEs and destroy objects from the private UI by moving them to a 3D trashcan. The visual indication for the selected object and targets was implemented with turquoise lines and green overlay colour on the object's mesh (Fig. 1).

The idea was to implement the prototype on a touch screen device, but due to limitations of the Linux Ubuntu operating system, it was not a suitable solution. Therefore, we decided to evaluate the prototype with mouse input to get user feedback on the portal idea of moving items through portals and the visual indication while doing it. The main idea with implemented interactions was that a user selects an object by pressing the left mouse button, and then he/she moves the cursor on the VE portal, when an object 'jumps' on the portal. The user then presses the left mouse button and a copy of the object drops on the ground into the VE and the original object returns to its place in the 3D Portal Room UI. We also implemented realistic physics for the shared object when it drops into the VE. Thus, we did not define the position where the shared object should land in the VE; therefore, the objects landed with a random place and orientation.

Example Images for Evaluation

To study what kind of visual indication users would prefer for indicating moving objects from their private 3D UI area to public VE, we prepared fourteen different kinds of indications either on the object or on the target portal (Fig. 2). We used as an inspiration object highlights that we had noticed in games, such as Serious Sam 3 Turquoise dimming on the icon Glow effect on the icons' edge Size: 28,5x16 cm BFE [START_REF]Serious Sam 3 BFE[END_REF]. We wanted to extend the visual indications used in prior research [START_REF] Pichler | Interactive Browsing of 3D Scenes in Hypermedia: The Hyper-G 3D Viewer[END_REF] and investigate suggested solutions [START_REF] Pakanen | Studying Four 3D GUI Metaphors in Virtual Environment in Tablet Context[END_REF]; therefore, we made several versions with different mixtures of colour overlay, colour edges and glow. We decided to use a uniform colour theme in all options, to make them easily comparable with each other. Indications were made in the Adobe Photoshop by adding different kinds of layer styles for the screenshot image of the 3D Portal Room UI. We decided to present visual indication examples as images printed on 28,5x16cm sized paper, because then participants can compare examples easier by taking them into their hands. It is also a good way for collecting user experiences in an early development phase and it is also more costeffective and faster than implementing all possible solutions to the prototype [START_REF] Pakanen | Studying Four 3D GUI Metaphors in Virtual Environment in Tablet Context[END_REF].

User Evaluation

We recruited 30 participants for the user evaluation. The participants' age varied from 20 to 52 years with a mean of 32. 66% were male. 60% of the participants used 3D VEs on a computer, but none on a tablet or a mobile phone. A user evaluation had two parts. In the first part, participants familiarized themselves with the prototype by doing simple tasks with it. In the second part, subjects ranked three best options from given examples (A-N) to represent visual indication while an object is moved from the private 3D UI to a public VE. Subjects were interviewed and their subjective experiences were gathered by observing, writing down their comments and video recording while completing the following tasks: 4) what kind of indication there should be presented while sharing. 3. Select from given alternatives (A-N) the best, 2nd best and 3rd best option for indicating an object moving through the portal to a public VE. The images were placed on the table in random order for each subject. Participants were asked to mark their choice with little Post-it notes and comment each selection.

Findings

For all of the participants, it was easy to understand the logic that by clicking the door a connection opens to the VE. When sharing the objects, 70% of the users interacted directly as intended and 27% did it correctly in the second try. 3% did it right in the third try. The main problem with the interaction during sharing was that participants did not press the left mouse button on the portal, to drop a copy of the object to the VE. As one user commented: "Why I have to press (left button) -it is weird... I would just drag it there". Also the object transition implemented in the prototype caused confusion for a few participants when they were sharing objects. As they moved the cursor slowly, it seemed that the selected object did not follow the cursor at all. As a consequence, users thought that they have not selected the object at all. As one user stated: "(object) should follow (the cursor) smoothly and not just jump on the portal".

Security Concerns while Sharing Objects

In general, participants liked the possibility to share items to the remote VEs and they also found it useful, especially with the case of sharing a PowerPoint file to the Office. However, sharing also raised privacy and security concerns, especially with VEs that participants thought to be open for all people. All of the participants thought that outdoor VEs (Outdoor Music Club and 3D City) were public. Also the indoor Music Club was perceived to be public by 97% of the participants. The Office was perceived public only by 57% of the users. Participants explained that they perceived it either private or semi-private because the environment was quite small and the amount of the chairs in the scene created a feeling that it is meant for a limited amount of attendees. Subjects explained also that in real-life, meeting rooms are meant for private gatherings.

User Experiences with the Visual Indication in the Prototype

Participants liked the realistic physics when a shared object was dropped to the virtual environment. It was found as important but also amusing. They commented it as: "Wow", "Funny" and "visually attractive". One participant commented: "Dropping is a good effect indicating that (an object) goes to the virtual environment". Participants did not like the fact that the objects did not land in a rational location in the scene and they wished to be able to organize and move the objects in the VE after sharing them. The implemented indication while sharing was not perceived to be visible enough. Participants commented that they wished to have clearer visual feedback on the edge of the shared item or/and on the portal, such as "light", "flash of light" or "colour change". Also audio feedback was recommended. What is more, a few wished for a check-up window with text: "do you want to share this item to the ..." to pop-up.

User Experiences with the Visual Indication Example Images

The distribution of participants' preferred visual indications is presented in Figure 4.

To weight the primary choices, we used Borda count [START_REF]The mathematics of voting: Democratic symmetry[END_REF]. We gave three points for participants' 1st choice, two points for their 2nd choice, one point to the 3rd choice and zero points to all other options. A non-parametric Kruskall-Wallis test indicated significant differences between the data points for the fourteen different alternatives (χ2(13) = 27,530, p < 0,02). In the preference ranking task, participants preferred option M (turquoise glow effect around the edge of the photo album) for indicating objects transferring from private 3D UI to the public VE. It received the highest amount in the Borda count (36) and 27% of the participants rated it as the best choice. Participants explained that the indication of the object is the most important thing to be represented. They also commented that when the visual indication is on the object it looks like something is happening; the object is moving to the other VE. The glow effect was also perceived as "visually pleasant", "easily noticeable" and "it really makes the object to pop-up". The second highest was option E (turquoise colour + glow effect around the edge of the portal). It received 30 points and 17% of the first-place votes. Participants explained that the most important thing is now highlighted (e.g. the target). As one participant commented: "I have already chosen the object, so the glow indicates where to drop it". It was also suggested that both the object and the portal could glow.

The third highest was option B (white dimming on the portal view and on half of the photo album, turquoise colour + glow effect around the edge of the portal) with 20 points and 14% of the first-place votes. It was preferred because it gave an impression that the object is already half way through to the other VE, therefore, it looked intuitive. Also the indication on the portal was large enough to be noticed, as one participant commented: "(the indication should be) as visible as possible to preventing unintentional sharing to the other virtual environment". Despite different user views, a pairwise comparison between the three most popular alternatives revealed no significant statistical difference between them.

Conclusion

In this paper, we present a private 3D Portal Room UI for sharing 3D objects to public VEs. We present user evaluation with 30 participants and results on both; interaction with a functional prototype and a preference ranking of high quality example images. The participants found the sharing of objects from a private 3D UI space to public VEs as easy and useful, especially when sharing a PowerPoint file to the Office scene.

The visual indication should be clear when an object is moved out of the private UI area. The evaluation indicated that designers of 3D VEs should prefer a distinguishable glow effect around the shared object and the target portal. Whereas we believe that our results provide useful information for researchers and designers of the 3D VEs, we also recognize the limitations of the study. As we decided to use printed paper sheets as examples in the preference ranking task, we missed the interaction with the examples. However, the method provided us a lot of UX based information in the early development phase and was cost-effective and timesaving. With the feedback, we now know how to proceed with the design and implementation of portals. In the future, we are going to study how the visual indication works with the touch interaction. It would also be interesting to study how shared objects should be indicated to the users of public VEs.

Fig. 1 .

 1 Fig. 1. In the 3D Portal Room UI, the visual indication is showed on the selected object (a) and on possible targets (b, c) when a user is sharing a photo album to the Music Club VE.

Fig. 2 .

 2 Fig. 2. Enlargement of the visual indication effects used in the example images (A-N). Below an example of the original image with the enlargement area indicated with dashed white line.

  White dimming on the portal White dimming on ½ of the icon Colour on the portal edge Glow effect on the portal edge 3D effect on the portal edge White dimming on the portal White dimming on ½ of the icon Colour on the portal edge Glow effect on the portal edge White dimming on the portal White dimming on ½ of the icon Colour on the portal edge White dimming on the portal White dimming on ½ of the icon E F G Colour on the portal edge Glow effect on the portal edge Colour on the portal edge No effects Black dimming on the portal Black dimming on ½ of the icon Black dimming on the portal Black dimming on ½ of the icon Colour on the portal edge Black dimming on the portal Black dimming on ½ of the icon Colour on the portal edge Glow effect on the portal edge Black dimming on the portal Black dimming on ½ of the icon Colour on the portal edge Glow effect on the portal edge 3D effect on the portal edge Turquoise dimming on the icon Glow effect on the icons' edge (turquoise)

1 .

 1 Open connections to the VEs (click doors) & explain if the VE is public or private. 2. Share objects from the private 3D Portal Room UI to public virtual environments: (a) Share Power Point slides to the Office scene (drag PP icon to VE 2) (b) Give feedback to the City council (City scene) (drag W icon to VE 4) (c) Delete an old PDF file (drag PDF icon to the trashcan in 3D Portal Room UI) (d) Share a photo album to the Music Club (drag photo album 3D icon to VE 1). Comment on the following question: Do you find sharing (1) easy, (2) useful, (3) secure, and (

Fig. 4 .

 4 Fig. 4. Preference ranking results for the visual indication.


Maryam Tohidi
email: mtohidi@google.com

Andrew Warr
email: andywarr@google.com

The Bigger Picture: The Use of Mobile Photos in Shopping

Keywords: Mobile, Phones, Cameras, Shopping, Photos

Mobile phones are becoming, if not already, an integral part of our lives. They have a wide range of applications, such as communication, gaming and commerce. Shopping in particular is a rapidly growing domain. Today, shoppers use their phones to make more informed shopping decisions by researching products and merchants, save money using price comparison, mobile coupons and daily deal apps, even purchase products directly on a mobile device. While mobile commerce and shopping apps are in the spotlight, one area that has received little attention is the role of the native capabilities of a mobile phone, such as the mobile camera, in the shopping process. This paper demonstrates the key role mobile photos play in the shopping process, documenting use cases, practices and pain points, and informing opportunity areas for mobile shopping applications and services.

Introduction

In recent years there has been a steady rise in the use of mobile phones. At the end of 2012 an estimated one billion smartphones were in use worldwide [START_REF] Bicheno | Global Smartphone Installed Base Forecast by Operating System for 88 Countries[END_REF]. The adoption of mobile devices and their affordances such as advanced computing capabilities and connectivity have enabled new areas of application; one such area is mobilesupported shopping. Academic research on mobile shopping has primarily focused on mCommerce --shopping for and purchasing items using a mobile device [START_REF] Stafford | Mobile Commerce: What it is and What it Could be[END_REF]. Hillman et al. [START_REF] Hillman | Soft trust and mCommerce shopping behaviours[END_REF] studies the everyday routines of regular mobile device shoppers and identifies a large variety of mobile shopping activities, such as looking for products, comparing prices and purchasing items. Whilst such research has deepened our understanding of mCommerce, little in-depth research has considered the role of mobile phones when shopping in the real world.

According to US census data, eCommerce -a superset of mCommerce -accounted for only 5.5% of all retail sales in the US in 2013 [START_REF]Census Bureau Report[END_REF]. With a large majority of com-merce still taking place offline, it is important to understand the role of mobile devices in supporting shopping in physical stores. Pew Research reported that 6 in 10 mobile phone owners used their phone inside a physical store for assistance or guidance on a purchasing decision [START_REF] Smith | In-store Mobile Commerce During the 2012 Holiday Shopping Season[END_REF]. Nielsen and Smith [START_REF]NielsenWire, Mobile Devices Empower Today's Shoppers In-Store and Online[END_REF][START_REF] Smith | In-store Mobile Commerce During the 2012 Holiday Shopping Season[END_REF] demonstrate the prominence of several common activities such as calling a friend or family member for advice, looking up reviews, looking up price, locating stores and researching products, to name the most common activities. However, some key use cases for mobile phone usage instores were omitted in these surveys, resulting in a lack of awareness of their importance.

One area that has received little attention in the mobile shopping literature is the use of mobile phone photos to support shopping. Häkkilä et al. [START_REF] Häkkilä | Price tags, maps, recipes: mobile phone photos for functional purposes[END_REF] have examined the use of camera phones in general, identifying that the most common functional (i.e. task driven) use for mobile phone photos was to support shopping activities.

In this paper, we explore the application of mobile phone photos in supporting shopping activities and present findings from a quantitative survey as well as a qualitative analysis of photos, identifying motivations, use cases and pain points.

Quantitative Research

To examine the relative importance of shopping-related mobile phone photos we conducted two surveys in November 2012 and February 2013. In the first survey, 2005 respondents, representative of the US population of online users, answered the multiple choice question 'Which of the following shopping activities did you do on your mobile phone in the past 2 weeks?' The choices included the most frequent mCommerce activities identified in previous research [START_REF] Hillman | Soft trust and mCommerce shopping behaviours[END_REF][START_REF]NielsenWire, Mobile Devices Empower Today's Shoppers In-Store and Online[END_REF], as well as 'took a picture to remember a product'. From those who had done any shopping activities on their phones (503), taking a picture to remember a product ranked among the top.

In the second survey, 1339 respondents, representative of the US population of online users, answered a similar question; this time the choices included the most frequent in-stores mobile shopping activities identified in previous research [START_REF]NielsenWire, Mobile Devices Empower Today's Shoppers In-Store and Online[END_REF][START_REF] Smith | In-store Mobile Commerce During the 2012 Holiday Shopping Season[END_REF], as well as 'took photo of a product'. From those who had done any shopping activities on their phones (360), taking a photo of a product ranked among the top again.

Our surveys showed that taking shopping-related mobile photos is comparable to mobile shopping activities previously identified as most frequent.

A Study of Shopping Photos

To understand how mobile photos are used in shopping we collected real examples of shopping-related photos. We posted requests on social and miscellaneous mailing lists at a large US technology company, asking recipients to browse their mobile photo gallery for photos taken related to a shopping activity in the last month, and to email us such photos accompanied by a caption stating why they took the photo. We received a total 285 photos and associated captions from 91 respondents (58% female). Their ages ranged from under 20 to over 50, with a median age range of 30-39 years. Respondents were from across the US, representing a range of jobs, both technical and non-technical.

Furthermore, we asked the participants to elaborate on their practices and pain points around taking these photos in a follow up survey. 80 of the 91 responded to this survey.

General Observations

As suggested by previous research [START_REF] Häkkilä | Price tags, maps, recipes: mobile phone photos for functional purposes[END_REF], many of the photos in our study were taken as a memory-aid. Since their mobile phones are almost always with them, participants used their phone as a repository of to-dos, useful references, reminders and evidence related to their shopping. One participant said "I usually forget the grocery list but not my phone. So I just take a photo of it." Another participant took a photo of a receipt, "so that I had the receipt saved in case I lost it." Since their primary purpose is functional [START_REF] Häkkilä | Price tags, maps, recipes: mobile phone photos for functional purposes[END_REF][START_REF] Kindberg | The Ubiquitous Camera: An In-Depth Study of Camera Phone Use[END_REF], little attention is given to the composition, lighting and other artistic values of these photos. We also saw evidence of 'bursts' [START_REF] Häkkilä | Price tags, maps, recipes: mobile phone photos for functional purposes[END_REF] of photos, taken of similar products, related to the same shopping purpose. For example, one participant had taken 8 photos of floor lamps to show her boyfriend and another participant had taken 7 photos of TV options he had seen at Costco to research later. In our follow up survey 61% of respondents confirmed that they have taken at least one burst of 3+ photos related to the same shopping intention.

Photo Analysis

We took a bottom-up approach to analyzing the photos based on what was pictured (i.e. content) and why they were taken (i.e. intent). Two researchers iteratively developed a coding scheme covering these dimensions. Before coding the entire collection, a random sample of 10% of the photo/caption pairs was coded by two independent judges. The coding scheme had an inter-judge reliability of 80%.

Content

Physical Products: Taking a photo of the actual product was most common, with 54% of photos showing the full product (Figure 3a), and an additional 12% showing a partial product (Figure 3b) -often a close-up focusing on a specific aspect of the product, such as the texture of a scarf. Furthermore, taking a photo of a product already owned accounted for 16% of photos. (Figure 3c) Information about Products: Labels were the 2nd most common content type, accounting for 42% of photos (Figure 3d). Labels included textual or numeric information describing the product (e.g. wine bottle labels, clothing labels and nutritional information). Price and barcodes were also quite commonly pictured, appearing in 24% and 14% of photos respectively (Figure 3e and3f).

We also saw photos of oneself (often trying on, or setting context for a product seen in a physical store), shopping lists, receipts and advertisements.

Intent

The accompanying caption described the intention behind taking a photo. Analyzing these captions, allowed us to categorize the primary motivations (Table 1). Shopping photos were taken as a means to remember (51%), share (44%), consider (36%), or buy a product (34%), or to serve as an inspiration for a future purchase (15%). These intentions often overlapped (shown in Table 2), for example, from the photos taken with the intention 'To Remember', 49% were also intended 'To Buy', and '29% 'To consider' a product.

Discussions

Investigating the intent behind the photos allowed us to drive the common use cases in which mobile photos support shopping. Just over half (51%) the photos were taken with the intention to remember something, helping with the organizational aspects of shopping.

Shopping List: 49% of photos taken with an intent 'To remember', were also intended 'To buy' a product. These photos served as visual shopping list.

Shortlist: 29% of photos taken with an intent 'To remember', were also intended 'To consider' a product. These photos, often taken of similar products, helped keep a shortlist of products under consideration, to research or compare later.

Archive: Another important use case of photos taken 'To remember' was archiving evidence or important references, such as receipts and shipping labels.

The social aspect of shopping is another important dimension supported by photos. 43% of all photos were taken with the intention to share with someone. These photos were often shared through MMS, email, or shown to someone in person.

Second opinion: The primary reason for sharing a photo was to get feedback on a product under consideration (52%). These photos were often shared with a spouse, a close friend or family member, asking for a yay or nay response. There was only one example of sharing with a group to get collective feedback for a mutual purchase.

Hint, hint: 22% of photos taken with an intent 'To share', were also intended 'To inspire' someone else to buy something. For example, one participant sent a photo of her wedding band to her husband as a reminder for her anniversary gift.

Saw this, thought of you: Another reason for sharing photos was to maintain or enhance personal relationships. For example one participant shared a photo of a beer bottle label featuring a werewolf with a group of friends playing a computer game fighting werewolves. Humor also played a role in this type of sharing. One participant took photos of owl figurines to send to a friend who is scared of owls. Another posed for a photo with an infant outfit held in front of her to share with her family for a laugh. This was the only category of sharing, where group sharing was very common.

Other sharing scenarios included expressing frustration with a bad shopping experience, providing proof of a purchase, or setting context for product recommendations.

Mapping of Content and Intent

A mapping of photo intent against photo content illustrates some of the reasons why people took photos of the different aspects of a product. In Table 3, each cell indicates the percentage of photos taken of a content type (row), with a given shopping intent (column). Note than since 'To remember' almost always went hand-in-hand with another intention that was more specific to shopping, we have simplified this graph by focusing on the other 4 intentions.

Inferring Intent from Content

Table 3 highlights usage patterns that could inform our ability to infer shopping intent from the photo content alone. As show above, when the photo shows a full product, the most common shopping intentions are to share the photo with someone else (62%), or to consider the product further (46%). Photos focused on partial products however were more often tied to buying (49% compared to 21% for full product). Taking a shopping-related photo of something they already owned, was very often tied to a buying intent (78%) in order to replace or complement their existing product.

Similarly, when the primary content of the photo was a product label or barcode, there was often a buying intent (43% and 58% respectively). Price tags on the other hand, tended to indicate a product under consideration (49%).

From Inspiration to Consideration to Purchase

The shopping stories (captions) shared with us often spoke of longer term purchases, where the photos connected the dots between different phases of shopping. "I've been shopping online for boots for a couple of months, and I'd been showing my husband a lot of pictures. He sent me several [mobile] photos of boots he saw on two shopping trips, while he was out shopping for something else. We followed up by text/phone. We ended up buying the red boots." Inspiration-related photos were often taken of objects seen in the real world (e.g. "a lamp I liked in a restaurant"), ads or magazines (e.g. "lighting feature in a design magazine I want to DIY"), and collections of products (e.g. "a nursery room in Pottery Barnes"). 61% of inspiration-related photos were to be shared with someone else, either to suggest gift ideas (e.g. "he might like this for Christmas"), or as a selfappointed personal shopper, curating based on the recipient's interests or needs (e.g.

"Cadbury mini-eggs are in stores already! I know you love them").

Consideration-related photos were often taken in 'bursts' to record references to products that the shopper intends to research or compare. Furthermore, consideration and sharing often went hand in hand. For example, a group of bridesmaids took several photos of dresses to send to the bride for her consideration. Apparel and furniture were the 2 most common product categories that participants shared with others to get a second opinion.

Purchase-related photos were centered around specific product(s) to buy later, either online or in-store. About half of these photos were taken in a store to buy online later, "Liked this area rug at Home Depot but in-store inventory was frayed. I went home and ordered it online". In many cases there was a hope to find the product at a cheaper price, "Ski Jacket that was on sale at REI -I wanted to see if I could find a better price online." There were also photos to help future in-store purchases, such as photos of shopping lists and recipes.

Pain Points

Although shopping-related photos are often taken as a memory aid for future actions, it is not uncommon for the photos themselves to be forgotten. "I took these pictures to see whether that item has good reviews. Then forgot about it and took no action." In the follow-up survey, 83% said they have taken at least one photo with the intention to refer to it later, but they forgot to do so. Shopping-related mobile photos tend to stay in the mobile gallery, among other photos, making them harder to remember, find and organize over time. (72% said they don't go back to delete the shopping related photos from their mobile gallery.)

When asked what the biggest challenges regarding these photos were, the primary pain points identified in the follow up survey were remembering to go back to the photos (59% of respondents), finding a photo when you need it (53%), finding the product at a later date (33%).

Areas of Opportunity

Our study suggests that, while there is plenty of usage and applications for mobile photos in the shopping process, there is room for better connecting the dots between the two mediums. The biggest challenge for users is that they forget the photos, or have a hard time finding them when needed. There seems to be an opportunity area for tools to help identify, organize and surface these photos at the right times and places so that they can better support shopping.

Conclusions

This paper highlights the importance of mobile photos in supporting shopping activities, an area previously overlooked in related research. We have proposed a taxonomy for the content and intent of shopping-related photos, identified key use cases, common pain points and suggested an opportunity area for further development.

Fig. 1 .

 1 Fig. 1. Survey 1, top mCommerce activities.

Fig. 2 .

 2 Fig. 2. Survey 2, top in-store mobile shopping activates.

Fig. 3 .

 3 Fig.3. Examples of shopping photos

Table 1 .

 1 Table 1 presents these codes. Coding scheme, frequencies and % of all photos

	Content codes	# (%)	Intent codes	# (%)
	Full product	154 (54%)	To remember	146 (51%)
	Label	121 (42%)	To share	124 (44%)
	Price	68 (24%)	To consider	102 (36%)
	Something owned	45 (16%)	To buy	96 (34%)
	Barcode	40 (14%)	To inspire	44 (15%)
	Partial products	35 (12%)
	Self	10 (4%)
	Shopping list	9 (3%)
	Receipt	7 (2%)
	Advertisement	7 (2%)

Table 2 .

 2 Overlap of intentions, highlighting the most common

		To share	To consider	To buy	To inspire
	To remember	10%	29%	49%	9%
	To share		52%	9%	22%
	To consider			5%	1%
	To buy				14%

Table 3 .

 3 Mapping photo intent against content -highlighting the most common

		To share	To consider	To buy	To inspire
	Full product	62%	46%	21%	16%
	Label	30%	31%	43%	15%
	Price	34%	49%	29%	9%
	Something owned	24%	0%	78%	16%
	Barcode	5%	38%	58%	10%
	Partial product	29%	17%	49%	9%


Eva Garcia-Lopez

Antonio Garcia-Cabot

Luis De-Marcos
email: luis.demarcos@uah.es

Jose-Ramon Hilera
email: jose.hilera@uah.es

Usability guidelines for mobile devices: length of hyperlinks

Keywords: Hyperlinks, usability, guidelines, mobile devices

The use of mobile devices is increasing in recent years, and their characteristics are different from those of computers (keyboard and screen size, interaction method, etc.), so it is important to specifically study the usability of these devices. This paper presents an experiment carried out with users to find the correct length (from the point of view of usability) of hyperlinks in mobile devices. A real mobile device has been used for the experiment, as well as more than 20 users. The experiment compared three different lengths of hyperlinks and it concluded that users prefer hyperlinks as short as possible and they are identified better when the text is not justified.

Introduction

The rise of mobile devices has led to a transformation in the way of creating digital content, not just web pages but any content that these devices can play [START_REF] Gómez | Context-Aware Content Adaptation in mLearning[END_REF]. Some characteristics of these devices, such as the small screen size or the way of interaction, pose a real challenge when generating this content for mobile devices [START_REF] Zhang | Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications[END_REF]. Focusing on web content development from the point of view of usability, there are several efforts to establish guidelines and/or recommendations to generate this content in a usable way [START_REF] Nielsen | Mobile usability[END_REF]. Some important aspects of the web, by its nature, are hyperlinks that allow linking some content with another one. From the point of view of traditional computers, there have been many efforts to establish guidelines and recommendations to create these hyperlinks (their size, significance, etc.); but from the point of view of web content for mobile devices, perhaps these guidelines cannot be used or should be adapted by taking into account the limitations of these devices. Therefore, this paper presents a solution to the problem of which length should hyperlinks have when they are shown in mobile devices, always from a usability perspective.

To solve the problem, this paper presents a set of experiments carried out with users and with different kinds of hyperlinks that allow figuring out the right size for them.

Section 2 of this paper shows the background and the related work to this research, Section 3 explains the details of the experiments carried out, Section 4 presents the results and finally Section 5 shows the conclusions and future work.

Background and related work

Some guidelines and standards establish a concrete length for hyperlinks. In some cases the link length is not explicitly mentioned, but a link should be understandable enough, regardless of its length. Furthermore, some of these guidelines were established for web pages in general, such as [START_REF]Ergonomics of human-system interaction -Part 151: Guidance on World Wide Web user interfaces[END_REF], [START_REF]Research-Based Web Design and Usability Guidelines[END_REF]. However, other guidelines have already been defined specifically for mobile devices [START_REF] Budiu | Usability of Mobile Websites and Applications, Design Guidelines for Improving the User Experience of Mobile Sites and Apps[END_REF], [START_REF] W3c | W3C Mobile Web Best Practices 1[END_REF].

The standard ISO 9241-151 [START_REF]Ergonomics of human-system interaction -Part 151: Guidance on World Wide Web user interfaces[END_REF], designed specifically for general purpose web pages, states in guideline 9.4.13 "Link length" that "Textual link names should be long enough to be understood but short enough to avoid wrapping". The initiative Usability.gov [START_REF]Research-Based Web Design and Usability Guidelines[END_REF] states in guideline 10:11 "Use Appropriate Text Link Lengths" the following: "Make text links long enough to be understood, but short enough to minimize wrapping", which somewhat matches the ISO guideline about creating links long enough to be understood but preventing wrapping. Furthermore, Budiu, R. and Nielsen, J. [START_REF] Budiu | Usability of Mobile Websites and Applications, Design Guidelines for Improving the User Experience of Mobile Sites and Apps[END_REF] propose a set of guidelines focused on web and applications design for mobile devices, and in guideline 103 they state "Use links with good information scent (that is, links which clearly indicate where they take the users) on your mobile pages". The length that links should have is not explicitly shown, but it indicates they must be clear and descriptive. Finally, W3C [START_REF] W3c | W3C Mobile Web Best Practices 1[END_REF] has best practices for creating web pages, and guideline 5.2.6 "Link Target Identification" states that links should clearly indicate the target of the link, but does not mention any length that links should have. Due to the above, this paper presents a set of experiments with real users to help making a recommendation about the importance of the size of hyperlinks in mobile devices. The hypothesis will be taken from the aforementioned guidelines of ISO 9241-151 and Usability.gov, i.e., "Texts of the links should be long enough to be understood and short enough to avoid wrapping" because, although they are not guidelines applied directly to mobile devices, they are concrete and specific.

Planning the experiment

An experiment was designed to test the usability of different link lengths using metrics from the "usability" concept according to ISO 9241-11 [START_REF] Iso | Ergonomic requirements for office work with visual display terminals (VDTs) -Part 11: Guidance on Usability[END_REF] (effectiveness, efficiency and satisfaction). For this, three types of web pages (Fig. 1.) were designed, each of which contained a text with some words or sentences marked as links. These links were colored and underlined, as suggested by [START_REF] Nielsen | Jakob's Nielsen Alertbox: Guidelines for Visualizing Links[END_REF], to maximize the perceived affordance of clickability. The text used was extracted from a chapter of The Theory of Evolution by Charles Darwin. Links in the first web page were short, i.e., each one of them consisted of a single word. In the second web page links were medium, i.e., they had at least two words but without taking up more than two lines. Finally, links in the third web page were long, occupying several lines. Users were asked to count the number of links in each of the three web pages, so that the effectiveness could be measured objectively, using the following formula where n is the number of real links and It is noteworthy that the links were disabled, that is, if the user clicked on it, tion would be triggered. link was composed of a single link or more than one.

Another parameter to measure links. In this way, efficiency taken in counting the number of links in Finally, after the experiment, users completed a were asked about how comfortable w type of links, and if the target of the links was easily understood in each web page of the experiment. The survey had also some "free response" of each web page, so we could infer what users think, in general, about each of the types of links (short, medi

In order that the text of the web page was not the same in all treatments, three di ferent versions of web pages were designed with the three types of links in each, so that there were nine different web pages for distribution to users. for each web page was different. so that each user performed the experiment with a web page with short links, another without taking up more than two lines. Finally, links in the third web page were long, nes.

. Prototype of web pages designed for the experiment: short, medium and long size links, respectively (in Spanish).

Users were asked to count the number of links in each of the three web pages, so could be measured objectively, using the following formula

ݏݏ݁݊݁ݒ݅ݐ݂݂ܿ݁݁ ൌ 1 െ
|ି| is the number of real links and i is the number of links the user counted It is noteworthy that the links were disabled, that is, if the user clicked on it, tion would be triggered. This is to prevent the user found out if, for example, a long link was composed of a single link or more than one.

Another parameter to measure was the time that users took to count the number of efficiency could be measured, taking into account the average time taken in counting the number of links in each of the cases.

Finally, after the experiment, users completed a satisfaction survey in which they were asked about how comfortable was for them the act of reading the text with each and if the target of the links was easily understood in each web page of The survey had also some "free response" questions about the opinion , so we could infer what users think, in general, about each of the , medium and long size). In order that the text of the web page was not the same in all treatments, three di s of web pages were designed with the three types of links in each, so that there were nine different web pages for distribution to users. The number of links for each web page was different. Treatments were randomly distributed among users, ser performed the experiment with a web page with short links, another without taking up more than two lines. Finally, links in the third web page were long, . Prototype of web pages designed for the experiment: short, medium and long size links,

Users were asked to count the number of links in each of the three web pages, so could be measured objectively, using the following formula:

(

is the number of links the user counted. It is noteworthy that the links were disabled, that is, if the user clicked on it, no ac-This is to prevent the user found out if, for example, a long was the time that users took to count the number of taking into account the average time survey in which they reading the text with each and if the target of the links was easily understood in each web page of about the opinion , so we could infer what users think, in general, about each of the In order that the text of the web page was not the same in all treatments, three difs of web pages were designed with the three types of links in each, so The number of links Treatments were randomly distributed among users, ser performed the experiment with a web page with short links, another one with medium size links and finally one with long size links, but never with the same text.

Additionally, other three web pages were created with the same text and with different links (long size), but with justified text instead of left-aligned. This is because when the experiment was performed with "subject 0" (the first user that participated in the experiment), we realized that the justification (or not) of text could confuse the user when counting the links, especially when they take up more than one line, as it is the case of long size links.

Results and discussion

The experiment was performed with 22 subjects, of which 77.27% were male and 22.73% female. 63.64% of the subjects were between 18 and 24 years; 22.73% were between 25 and 34 years; and 13.64% were older than 35 years. Of these, 45.45% was considered expert in the use of mobile devices, 45.45% was considered in an intermediate level, and 9.09% was considered novice. Regarding the highest level of education completed, 13.64% of the subjects had High School completed and were studying a Degree; 31.82% had Associated Degree; 18.18% had a Degree; 31.82% had a Master Degree and 4.55% had a PhD.

Fig. 2. Mechanism used to record user

To carry out the experiment, a mobile device of make Apple was used (specifically model iPhone 4) and also a webcam that recorded continuously the screen that the user was seeing. The webcam was joined to a metal structure, and the structure was joined to a soft housing specific for this kind of device. The webcam was connected to a laptop via the USB cable, so the videos were recorded directly into the laptop.

Four videos were recorded for each user, one for each factor:

• One with unjustified text and short size links.

• Another with unjustified text and medium size links.

• Another one with unjustified text and long size links.

• Another one with justified text and long size links.

To measure effectiveness, as mentioned above, it was measured by the percentage of correct answers of the subjects to count the number of links in each configuration, that is, with short size links, with medium size links, with long size links and unjustified text, and with long size links and justified text. Then the average of successes of all subjects for each configuration and the standard deviation were obtained (Table 1). Later a normality test for data was executed, in order to determine the most appropriate method of analysis (Table 2). *All values are 1 (100% success) for medium-sized links so p cannot be calculated. Data did not pass the normality test (p<0.05) so nonparametric methods should be used. Kruskal-Wallis test returns p=0.009 (<0.05; H(3)=11.46), so the results are statistically significant. That is, the size of the links affects the effectiveness. To distinguish those groups that are statistically better and worse than others, another Kruskal-Wallis test was applied, but this time by pairs of groups (Table 3). In view of the results, short and medium sized links are statistically more effective (i.e., users make fewer mistakes) than long links with justified text. Moreover, one can say, with 90% confidence, that medium sized links are statistically more effective than long links with unjustified text.

Table 2. Normality test for effectiveness

Furthermore, for efficiency, we analyzed the time each subject took to count the number of links with each configuration, and the average, standard deviation and median were obtained for each (Table 4). First of all, a normality test was applied to data, in order to determine the most adequate analysis method (Table 5). Data did not pass the normality test (p<0.05), so nonparametric methods should be used. Kruskal-Wallis test returns p=0.000 (<0.05, H(3)=29.53), from which it follows that the results are statistically significant, i.e., short and medium sized links are statistically more efficient (faster to identify) than long size links. It is not possible to say that there is statistical significance between short and medium sized links; nor between long links with unjustified text or with justified text, so it cannot be said that some are more efficient than others.

Finally, the surveys that users answered after the experiment were analyzed to know the satisfaction of the users. Answers to three questions were assessed, which were the following for each configuration:

• Q1: Is it comfortable to read the text? • Q2: Is the target of the link easy to understand? • Q3: What do you think, in general, you will get if you click on the links?

The answers to the first two questions were measured quantitatively on a Likert scale from 0 to 5. The results for each configuration (average of answers of all users and standard deviation) are shown in Table 6. As for the third question, answers of the subjects were "free", so it is difficult to quantify. However, much of the users (90.91%) think that short links will lead to a web page where the definition of the word is explained, while long and medium sized links usually make to think (by 77.27% of cases) that will lead to a web page that explains in more detail the link sentence.

Conclusions and future work

In view of the results above, the following conclusions are obtained:

• Looking at the comfort for reading a text of a web page from a mobile device (Q1), there is no significant different between justifying or do not the text, that is, some users think that reading the text is more comfortable when it is justified but some others think it is easier when the text is not justified, since there is no a big difference between their averages. However, efficiency is influenced negatively by the text justification, i.e., when the text is justified, users seem unable to differentiate where a link starts and where it ends as well as when the text is not justified. Therefore, one could argue that it is preferable do not justify the text so that the user can clearly distinguish where the links start and end, although this affirmation is based on the average of data and it must be taken into account that it is not statistically significant. • Users prefer clearly short links (according to Q1 average), because they think that is the most comfortable way to read a text containing links. Medium links are not as uncomfortable as long links, and observing an increasing score in reading comfort from long links to short links, it follows that users seem to prefer links as short as possible. This conclusion is further supported because data suggest that users often understand better the target of a link (according to Q2 average) the shorter (there is an increasing score about understanding the target of the link, from the longest to the shortest), and also because effectiveness is higher the shorter the link (users seem to spend less time when identifying the links). As a remark, note that the effectiveness is slightly better with medium links than with short links, but this does not really matter, since only one user was confused when he counted short links. Therefore, the conclusion "the shorter the links the better" is still valid despite this fact.

• Comparing the above conclusions with the initial hypothesis ("Texts of the links should be long enough to be understood and short enough to avoid wrapping"), one could say that in the case of mobile devices this is not true because users seem to prefer links as short as possible, and also the data suggest that they are better and faster identified, both where they start and end, and their target. Therefore, a possible recommendation about the link length in web pages for mobile devices would be: "Texts of the links should be as short as possible and, whenever possible, the text in which they are contained should not be justified".

For future work, the experiment could be repeated with different mobile devices with diverse characteristics (techniques of interaction, screen size, screen orientation), in order to check if conclusions can be extrapolated to other devices aside from smartphones. Also, in addition to the length, the semantics of links should be considered to evaluate the understanding of the links by users.

Fig. 1 .

 1 Fig. 1. Prototype of web pages designed for the experiment: short, medium and long size links,

Table 1 .

 1 Results obtained for effectiveness

	Type of link	Average (% Success) Standard Deviation	Median
	Short size	99.68	0.0152	1
	Medium size	100	0.0000	1
	Long size and unjustified text	98.82	0.0312	1
	Long size and justified text	97.30	0.0465	1

Table 3 .

 3 Results of Kruskal-Wallis test

	Type of link

Table 4 .

 4 Results obtained for efficiency

	Type of link	Average (seconds)	Standard Deviation	Median
	Short links	14.091	6.286	12
	Medium links	15.773	6.164	13
	Long links and unjustified text	26.455	12.054	21
	Long links and justified text	23.273	7.857	22

Table 5 .

 5 Normality test for efficiency

	Type of link	p
	Short links	<0.005
	Medium links	0.009
	Long links and unjustified text	<0.005
	Long links and justified text	0.222

Table 6 .

 6 Survey results

	Type of link	Q1	Q1	Q2	Q2
		(Average,	(Standard	(Average,	(Standard
		from 0 to 5)	deviation)	from 0 to 5)	deviation)
	Short links	4.36	0.726	3.27	1.579
	Medium links	3.27	1.077	3.05	1.090
	Long links and unjustified text	1.41	1.007	2.36	1.733
	Long links and justified text	1.55	1.299	2.59	1.436

Acknowledgement

This research has been co-funded by the Master in Software Engineering for the Web of the University of Alcalá, the projects IPT-2011-0795-430000 and UAH2011/6XP-0043090XT099, and the TIFyC research group.