Soumyadeb Chowdhury

Ron Poet

Lewis Mackenzie

A Comprehensive Study of the Usability of Multiple Graphical Passwords

Keywords: usability, user authentication, multiple image passwords

Recognition-based graphical authentication systems (RBGSs) using images as passwords have been proposed as one potential solution to the need for more usable authentication. The rapid increase in the technologies requiring user authentication has increased the number of passwords that users have to remember. But nearly all prior work with RBGSs has studied the usability of a single password. In this paper, we present the first published comparison of the usability of multiple graphical passwords with four different image types: Mikon, doodle, art and everyday objects (food, buildings, sports etc.). A longitudinal experiment was performed with 100 participants over a period of 8 weeks, to examine the usability performance of each of the image types. The results of the study demonstrate that object images are most usable in the sense of being more memorable and less time-consuming to employ, Mikon images are close behind but doodle and art images are significantly inferior. The results of our study complement cognitive literature on the picture superiority effect, visual search process and nameability of visually complex images.

Introduction

Information security systems must permit only legitimate users to gain access to the system and use its resources. This is done by a two step process. Identification verifies the user's right to access the system and, once the user is identified, they have to prove their identity by authentication. In computer security mechanisms, people are often required to authenticate themselves by using a secret known as a password or authenticator. In current practice, alphanumeric passwords are the most widely used mechanism to authenticate users. According to the studies reported in [START_REF] Adams | Users are Not the Enemy[END_REF][START_REF] Florencio | A large-scale study of web password habits[END_REF] most people find it difficult to remember these passwords. As the number of passwords per user increases, the rate of forgetting them also increases [START_REF] Brostoff | Are passfaces more usable than passwords? A field trial investigation[END_REF]. In order to cope with multiple passwords, users tend to adopt unsafe strategies, which include writing down the passwords, reusing the same passwords and sharing them with others [START_REF] Adams | Users are Not the Enemy[END_REF][START_REF] Brostoff | Are passfaces more usable than passwords? A field trial investigation[END_REF].

RBGSs are an alternative type of mechanism where images are used as passwords. The idea is that humans can remember images better than recalling alphanumeric text [START_REF] Madigan | Imagery, Memory, and Cognition: Essays in Honor of Allan Paivio[END_REF]5] and so this may be a way of devising more memorable passwords. Given the need for alternative usable authentication systems and existing interest in image passwords [START_REF] Brostoff | Are passfaces more usable than passwords? A field trial investigation[END_REF][START_REF] Chowdhury | Comparing the usability of doodle and Mikon images to be used as authenticators in graphical authentication systems[END_REF][START_REF] Davis | On user choice in graphical password schemes[END_REF][START_REF] Dhamija | Déjà vu: A user study using images for authentication[END_REF][START_REF] Renaud | A Visuo-Biometric Authentication Mechanism for Old Users[END_REF][START_REF] Renaud | Web authentication using Mikon images[END_REF] as a potential solution, we identify an important limitation of existing work: most prior studies with images as passwords, except [START_REF] Moncur | Pictures at the ATM: exploring the usability of multiple graphical passwords[END_REF][START_REF] Everitt | A comprehensive study of frequency, interference and training of multiple graphical passwords[END_REF], have focused on the usability of a single password. However, people will need to remember and use multiple image passwords in the same way that they currently use multiple alphanumeric passwords. The usability of multiple image passwords has not been explored except using faces and everyday objects (flowers, food, sculptures, nature etc.). So it is necessary to conduct experimental studies with other popular image types to compare their usability, when multiple image passwords are used. The motivation of our work is to investigate 'which image type (s) performs best in terms of usability, when multiple graphical passwords are used?' We believe, unless such large-scale studies are done, image passwords will always look good (usable) on paper but fail in real life and cannot be considered as a viable alternative to alphanumeric passwords.

This paper presents a study with 100 participants, who used multiple image passwords over a period of 8 weeks. The study compares the usability of 4 image types: Mikon, doodle, art and everyday object, when used as graphical passwords. In this paper: [START_REF] Adams | Users are Not the Enemy[END_REF] we identify the need to study and compare the usability of multiple image passwords using different image types; [START_REF] Florencio | A large-scale study of web password habits[END_REF] we show that the image types used as passwords by the participants significantly affect the effectiveness as well as efficiency of RBGSs, when multiple image passwords are used; [START_REF] Brostoff | Are passfaces more usable than passwords? A field trial investigation[END_REF] We discuss the implications of our findings, regarding the attempts to record image passwords motivating a need for future studies examining the vulnerability (guessability) of these passwords to descriptions given by the users.

Background Work

Existing studies have reported the usability of faces [START_REF] Brostoff | Are passfaces more usable than passwords? A field trial investigation[END_REF][START_REF] Davis | On user choice in graphical password schemes[END_REF], Mikons [START_REF] Chowdhury | Comparing the usability of doodle and Mikon images to be used as authenticators in graphical authentication systems[END_REF][START_REF] Renaud | Web authentication using Mikon images[END_REF], doodles [START_REF] Chowdhury | Comparing the usability of doodle and Mikon images to be used as authenticators in graphical authentication systems[END_REF][START_REF] Renaud | A Visuo-Biometric Authentication Mechanism for Old Users[END_REF], abstract art [START_REF] Dhamija | Déjà vu: A user study using images for authentication[END_REF] and pictures of daily objects [START_REF] Davis | On user choice in graphical password schemes[END_REF]; however, these studies have focused only on the use of single passwords. We are aware of only two pieces of prior work that studied multiple image passwords: [START_REF] Everitt | A comprehensive study of frequency, interference and training of multiple graphical passwords[END_REF] studied the use of multiple facial passwords and [START_REF] Moncur | Pictures at the ATM: exploring the usability of multiple graphical passwords[END_REF] compared multiple picture passwords to that of multiple PIN's.

The work reported in [START_REF] Everitt | A comprehensive study of frequency, interference and training of multiple graphical passwords[END_REF] studied the effects of frequency of facial password usage, the effects of interference resulting from the use of multiple graphical passwords and the effects of different patterns of access while training with multiple facial passwords. The study also demonstrated that long term memorability of the multiple facial passwords issued by the system is low.

The work reported in [START_REF] Moncur | Pictures at the ATM: exploring the usability of multiple graphical passwords[END_REF] compared the memorability of multiple graphical passwords to that of multiple alphanumeric passwords. The images used in this study were photographic and were drawn from different categories such as food, music, sports etc. In the study, there was a single challenge set which contained 4 authenticator images and 6 decoy images. However, this limited set may not be secure and may make the authentication procedure somewhat too easy. In the experimental framework, the participants used the password during training session just after completion of the registration and then three retention tests were carried out with a gap of two weeks between each of them. In such a scenario, the encoding of password information in the long term memory may not be entrenched enough to aid memorability. This is evident from the login success rate reported in the paper, which just after training is almost three times higher than the success rate after 4 weeks. We also find that passwords are issued by the system rather than chosen by the user themselves, and this may hamper the usability of the system.

Given the significant interest of researchers in image passwords, it is clear that additional work is needed using other image types to enhance knowledge of usability, when multiple image passwords are employed. Therefore we designed a usability study to explore the potential of 4 different images types (graphical passwords) Mikon, doodle, art and everyday objects with: (1) Common experimental settings including user tasks for all graphical passwords; (2) Same evaluation parameters; (3) Same design of the authentication system interfaces ; (4) Testing in both conditions i.e. passwords frequently used and less frequently used.

User Study overview

The main aim of the study was to compare the usability of multiple image passwords in RBGSs. A three stage study was designed, which consisted of: (1) a prestudy questionnaire evaluating the participant demographics and current password strategies; (2) an 8-week online study of participants using multiple image passwords; and, (3) a post-study questionnaire regarding participant experiences. In our usability study we used the following image types (See Appendix): (1) Mikon: These are iconlike images which have been drawn by users in studies reported in [START_REF] Chowdhury | Comparing the usability of doodle and Mikon images to be used as authenticators in graphical authentication systems[END_REF][START_REF] Renaud | Web authentication using Mikon images[END_REF] using a tool called the Mikon engine developed by Mikons.com. (2) Doodle: These images are drawn by a user using pen on paper, in studies reported in [START_REF] Chowdhury | Comparing the usability of doodle and Mikon images to be used as authenticators in graphical authentication systems[END_REF][START_REF] Renaud | A Visuo-Biometric Authentication Mechanism for Old Users[END_REF]. (3) Art: These images were collected from a range of free websites and comprised of paintings from different styles like cubism, abstract, modernism etc. (4) Object: These images comprised of pictures of food and drinks, sculpture and buildings as well as sports and leisure activities, collected from a range of free websites.

Experimental conditions and Images

We used independent measures (between subjects) style of experimental design with four conditions (equal number of participants in each condition) namely Mikon (m), doodle (d), art (a) and objects (o). Each participant was assigned to only one of the conditions randomly. The participants in the Mikon condition used Mikon images as their password. They created 4 Mikon passwords and authenticated using them. Each password comprised of 4 Mikon images. So the participants had to remember 16 images in total. Similarly participants in the doodle, art and object conditions used the respective types as passwords. The participants were given a task information sheet which contained the information on the steps to register with the system i.e. select four images to create a password and steps to login to the system after successful registration. However, the participants were neither given any instructions regarding the strategy they should use to select their password images nor the strategy they should employ to remember them. Each condition was associated with a mock online study website that had a distinct logo and address. Each website had 4 hyperlinks (each link corresponding to one password, See Fig 1 1 and Appendix). My status had a collection of 150 images (50 images each from My jokes, My movies and My news respectively). Each hyper link had a distinct name and background color. The participants could post information/content in the link upon successful authentication: This made sure that the participants had a context to use in differentiating their multiple passwords.

Table 1. Image categories in each link

Tasks

Registration

The online study website had 4 links. Each link would correspond to one password (made up of 4 images). The participants could register in each of the links by entering a username and selecting 4 images as their password from the given collection. Figure 1 shows the registration screens of the RBGS developed for our experiment. Each link Upon accessing the online study website, the participants had to select the correct password images (the ones they selected during registration) from a sequence of 4X4 grids at each step of a four-step procedure (Figure 2). All the images other than the password image in the sequence are known as decoy images. The grid consisting of 15 decoy images and 1 password image is called a challenge set. The decoy images for each step were fixed during registration to ensure that the intruder would not be able to guess the correct password image, merely refreshing the web page. The decoy images for each password image were chosen randomly from the collection of 150 images. The decoy images for each of the 4 target images were distinct and never repeated. If the participants at any step during the login procedure selected wrong password image, then they would never get any of their registered password images in the subsequent steps. So in this case, 16 decoy images (without the password image), different from the origi- nal challenge set were displayed to the users. The participants were given feedback on the result of the login only after the last step of the procedure. In case of three continuous failed login attempts, the participants were automatically reminded of their password. The reminder was given only for the first 20 login attempts (week-1) of each password. Once the participants logged in with the correct password images, they could post any information which could be seen by others using the system, who could like it.

Fig. 2. Steps to authenticate in RBGS

Upon successful authentication, the main task of the participants was to post some content in each of the links. The participants were not forced to use the system and the experimental procedure was designed to allow flexibility in the tasks. All sorts of ethical approvals were taken from to conduct the experimental study.

Experimental Framework and Protocol

The essential component of the experimental design is the 8 week online study. We used email prompts to inform the participants about the experimental procedure and tasks. Emails were sent to participants on Day 1, 3 and 5 of each week to notify the progress that they have already made and they are expected to make, to complete the weekly tasks. These emails helped them to keep track of the tasks and made sure they followed the experimental procedures.

There is no standard procedure to design experiments for studying multiple image passwords. In our experiment we tried to vary the frequency of login at regular intervals to simulate a real life scenario. We designed the study, considering the following.

3: Result

Success Failure  Ecological validity: participants had a task and a context. A pre questionnaire was conducted for two reasons: (1) to decide the number of multiple graphical passwords to be used for the experiment; (2) to examine, if our participant sample has the same text password behaviour as those of other participants, reported in existing studies [START_REF] Adams | Users are Not the Enemy[END_REF].  Confounding variable: the authentication systems (user interface, design), tasks and frequency of login were the same for all conditions.

Instructions

The participants had to first register with the system. Each participant in each condition had to create 4 passwords, each password comprising of 4 images. Once they finished registration, they had to practice logging in using the password images. The participants were instructed as follows: Week 1: They must create 4 passwords and login 20 times with each of the passwords (total 80 logins in the week). Week 2: They must login 20 times with each of the passwords (total 80 logins in the week); Week 3-4: They must login with each of the passwords 10 times (total 40 logins in the week); Week 5: In this week participants were instructed to login with each of the passwords 2 times (total 8 logins in the week); Week 6: In this week participants were instructed to login with each of the passwords 4 times (total 16 logins in the week); Week 7: In this week participants were instructed to login with each of the passwords twice (total 8 logins in the week); Week 8: In this week participants were instructed to login with each of the passwords thrice (total 12 logins in the week); The participants were instructed to distribute their login sessions over a period of time, instead of finishing them simultaneously. The participants who did not follow the experimental procedure were left out of the experiment.

Measures -Usability criteria

We evaluated the usability of the different image types used as password in different conditions using the four measures given below:

1. Effectiveness : This examined the average/mean successful login percentage (S) for each of the conditions calculated as, 2. Efficiency: It examined the average/mean registration time (R), and average/mean login time of successful login (L). The registration time for each of the passwords is the time taken to go from screen 3 to screen 6 of the registration process as shown in Fig 1 . The average registration time (R) for each condition is calculated:

The login time for a password is the time taken to go from screen 2 to the success notification screen of the authentication process shown in Fig 2 . The average time of successful login (L) for each condition is calculated as given below, z represents total number of successful login.

3. Satisfaction: This dimension was assessed from the ratings (1-5, 1 being highly dissatisfied to 5 being highly satisfied) given by the participants to the different aspects in the post study questionnaire-(sat1) Ease to register; (sat2) Ease to authenticate; (sat3) Meaningfulness/nameability of the image; (sat4) satisfaction with the type of image used as password. These aspects were based on some of the items in SUS (System Usability Scale) questionnaire [13]. 4. Stress: This dimension was assessed from the ratings (1-5, 1 being least stressful to 5 being highly stressful) given by the participants to the different aspects in the post study questionnaire-(str1) level of mental stress; (str2) level of physical stress; (str3) amount of effort required to choose images during registration; (str4) amount of effort required to successfully login. These aspects were based on the items in the NASA Task Load Index questionnaire [START_REF]NASA Task Load Index[END_REF].

Results

Participants

115 undergraduate participants, 30 female and 85 male, of age 20-24 took part in our experiment. They were studying different undergraduate courses: Mechanical Engineering -22, Electrical Engineering-19, Aerospace Engineering-25, Computer Science-24, Electronics and Communication Engineering-25. Of the 115 participants who took part, 10 participants had a very low participation rate (did not follow the experimental procedure) and 5 of the participants had to withdraw due to some personal circumstances. So the participation rate was 86.9%.

Pre study questionnaire results

Given the number and quality of problems associated with alphanumeric passwords, we conducted an online survey to obtain information about password construction techniques and different issues related to them. A total of 150 participants took part in the survey, which included the 115 participants of our usability study. A web based questionnaire was used to obtain data on different aspects of user behavior and perceptions in context of the use of alphanumeric passwords. The framework of the survey was developed using Grounded theory [START_REF] Strauss | Basics of qualitative research: Grounded theory procedures and techniques[END_REF]. The framework provided a step by step methodology ( The results of the survey revealed that participants used as many as 4-6 passwords in their day to day lives. This result enabled us to make the choice of using 4 graphical passwords with our sample users for the usability study. In context to the password creation strategy, of almost 600 responses: 71.2 % reported using similar passwords for all web accounts; 54.4 % used passwords that could be linked to their personal likings. The users felt that having similar passwords across all accounts aids their memorability. The results also demonstrated that 80% of the participants forget their passwords, either due to the strategy used to aid memorability, or constraints imposed by the system while creating alphanumeric passwords. The results of the survey are in line with the findings of similar research reported in [START_REF] Adams | Users are Not the Enemy[END_REF]. Thus we can confirm that the sample population used for usability studies has the same password behaviour as reported in other studies. Hence, the sample would represent an accurate reflection of the general population.

Effectiveness results

Our first planned analysis compares the effectiveness of the different image types used as password. We analyzed the data from week 2 to week 8. We eliminated the data from week 1, as it was considered a training week, where participants would get used to the system. The dependant variable average/mean login success percentage (for 7 weeks) for each of the conditions (image types) was normally distributed as assessed by the Shapiro-Wilk test. Levene's test indicated that the assumption of homogeneity of variance has not been violated (F (3, 96) = 2.083, p= 0.108). Given the use of the independent measures (between subjects) experimental protocol with four conditions and the normal distribution of the data, we chose One-way independent measures ANOVA to examine statistical significance. The results of the ANOVA shows that there is a statistically significant difference between the conditions F (3, 96) = 129.659, p<0.01. These indicate that the type of images used as password by the participants significantly affect the average successful login percentage. The effect size (r = 0.83, large effect) indicates that the effect of the images used as authenticator on the average successful login percent is substantial. The results of the Tukey post hoc tests revealed significant differences between all conditions (p < 0.001 for all tests) except between Mikon and Object (p = 0.059). Thus there was no significant difference between the effectiveness of Mikon and Object images.

Fig. 4. Weekly login success percentage

We also analyzed the weekly login success percent for each of the image types (Fig 4). We found that the login success percentage for each of the image type falls from week 2 to week 8, as the frequency of usage of the passwords decreases. If we compare the average weekly login success percentage in week 2 and week 8 for each image type (shown by bidirectional arrows in Fig 4), it is found that they fall by 11.44 % in case of Mikon, 12.55 % in case of doodle, 7.74 % in case of art and 14% in case of object. The results of the study show that the average weekly login success percentages for Mikon, doodle and art passwords remain almost the same after week 4 (indicated using black rectangles). In case of object passwords the average weekly login success percentage remains almost same from week 5-7 but drops in week 8. Thus the decrease in memorability for the best performers i.e. Mikon and object are almost the same. Similar characteristic are shown by doodle passwords. In case of art passwords (lowest average login success percent), the difference is comparatively low which clearly suggests that people had problems remembering them in the initial as well as final stages of the study. This reflects that once the participants had used the passwords for a considerable amount of time i.e. at least for a month, memory interference does not hamper the memorability of the image passwords. The memorability of multiple image passwords after a considerable amount of time would depend on the encoding of the passwords in the human memory and frequency of their usage. A two-way ANOVA was conducted with week and image types as the two independent variables. The dependant variable was average weekly login success percent. The week x image interaction was significant, (F (15,576) = 6.102, p < 0.001). This indicated that the average weekly login success significantly varied for each of the image types in different weeks.

Efficiency results

Efficiency is a measure of convenience: since a time consuming process will be a barrier, to the repeated use of the authentication system by the user.

Registration time

The mean registration time of the passwords (in seconds) for each condition is as follows: Mikon (mean: 72.18, SD: 5.48, SE: 1.17), doodle (mean: 75.40, SD: 4.27, SE: 0.88), art (mean: 84.44, SD: 4.91, SE: 0.99), object (mean: 70.61, SD: 3.84, SE: 0.76). We also find that the registration time decreases as users get used to the system in each of the conditions (registration time decreases from p1-first registered password to p4-last registered password) (Fig 5).

Fig. 5. Registration time for each password created in each condition

The mean registration time (of 4 passwords)) for each of the conditions (image types) was normally distributed as assessed by the Shapiro-Wilk test. Levene's test indicated that the assumption of homogeneity of variance has not been violated (F (3, 96) = 1.968, p= 0.127). We used One-way independent measures ANOVA as the statistical test. The result of the ANOVA shows that there is a significant difference between the conditions (F (3, 96) = 41.277, p<0.001). This indicates that the type of images used as a password by the participants affect the average registration time significantly. The effect size (r = 0.78 > 0.5 represents large effect) indicates that the effect of the images used as authenticator on the average registration time is substantial. The results of the post hoc tests revealed significant difference between all groups ( p < 0.05) except Mikon-Doodle ( p=0.091 > 0.05) and Mikon -Object (p = 0.658 > 0.05). In other words, the differences between the average registration time of Mikon and doodle as well as Mikon and object passwords is not significant.

Authentication time

The average authentication time of the successful login (in seconds) for each of the conditions is as follows: Mikon (mean: 19.52, SD: 3.60, SE: 0.72), doodle (mean: 22.16, SD: 3.75, SE: 0.75), art (mean: 24.56, SD: 4.8, SE: 0.96), object (mean: 18.28, SD: 2.84, SE: 0.59). The average login time (of 7 weeks) for each of the conditions (image types) was normally distributed as assessed by the Shapiro-Wilk test. Levene's test indicated that the assumption of homogeneity of variance has not been violated (F (3, 96) = 1.791, p= 0.124). The results of the one way ANOVA shows that there is a statistically significant difference between the conditions (F (3, 96) = 13.199, p < 0.001). This indicates that the type of images used as password by the participants affect the average login time significantly. The effect size (r = 0.61 > 0.5 represents large effect) indicates that the effect of the images used as password on the average login time is substantial. The results of the post hoc tests revealed significant differences between all groups (p < 0.05) except Mikon-Doodle (p=0.091 > 0.05) and Mikon -Object (p = 0.658 > 0.05). In other words, the differences between the average login time of Mikon and doodle as well as Mikon and object passwords is not significant.

Satisfaction results

According to the box plot in Fig 6, the satisfaction scores of objects range from 13-15 which is better than Mikons ranging from 12-14.5. The doodles have a range of 10.5-12 whereas art have a range of 8-10.5. Hence objects have the best satisfaction score distribution followed by Mikons. But the doodle and art images have inferior satisfaction scores.

Fig. 6. Box plot for satisfaction scores

Given the ordinal scale of data (user ratings) and the independent measures (between subjects) experimental protocol with four conditions, we used Kruskal-Wallis test to establish statistical significance of data. The result shows that the satisfaction scores for each of the conditions were statistically significant [H (3) = 52.37, p < 0.001). In other words, the satisfaction of the participants was significantly affected by the type of images used as password. We conducted a Mann-Whitney test to follow up our findings by applying a Bonferroni correction, to report all the effects at a 0.008 level of significance. The results reveal that the satisfaction scores were significantly different in all conditions (p < 0.008 for all tests) except for the Mikon-Object (p = 0.156 > 0.008). So we conclude that users are most satisfied with object and Mikon passwords (no significant difference), followed by doodles and least satisfied with art passwords.

Stress results

According to the box plot in Figure 7, the stress scores of objects range from 11-13 which is same as that of Mikons. The doodles have a range of 12-14, whereas art have a range of 13-16. Hence the art images have the highest stress score distribution closely followed by doodles. The object and Mikon images have the lowest stress score distribution.

Fig. 7. Box Plot for stress scores

Given the ordinal scale of data (user ratings) and the independent measures (between subjects) experimental protocol with four conditions, we used Kruskal-Wallis test. The result demonstrates that the stress scores for each of the conditions were statistically significant [H (3) = 23.314, p < 0.001). In other words, the stress scores given by the participants were significantly affected by the type of images used as password. We conducted a Mann-Whitney test to follow up our findings by applying a Bonferroni correction, to report all the effects at a 0.008 level of significance. The results show that the stress scores were significantly different in all conditions (p < 0.008 for all tests) except for Mikon-Object (p = 0.32 > 0.008). Hence we conclude that art images are most stressful to use, followed by doodles, whereas Mikons and objects (no significant difference) are least stressful to use.

5.7

POST STUDY QUESTIONNAIRE RESULTS

Rate images:

In the post study questionnaire the participants were asked to rate the image types on a scale: will use, not sure and never use. We find most of the Mikon (17/25) and object (20/25) users would like to use these images as passwords in the future. The art users (17/25) disliked these images to be used as password and the doodle users had a split opinion.

Strategy used for password creation: The participants were asked to provide information on the strategy/method they used to create their passwords (Fig 9). The results reveal that most Mikon and doodle users either used a story/pattern to remember their passwords or they chose passwords according to their personal likings (Fig 8). Most art users chose passwords either based on their personal likings-favorite color, objects, scene or visual and aesthetic quality of the images i.e. attractiveness. But, these strategies for creating memorable passwords may either make them guessable to an intruder who knows the user quite well, or could be disclosed and thus shared with ease. Sun Music Happiness Thumbs up Fig. 8. A Mikon password created using pattern strategy. Recording passwords: None of the participants said they made an attempt to record their passwords but almost 84 % of the participants who used art passwords, 68 % of doodle users, 36% of Mikon users and 32% of object users reported that they might use screen captures, sketches or notes (written descriptions) to store their image passwords.

Discussion

This is the first study that compared the usability of multiple image passwords using 4 different images types-Mikon, doodle, art and objects in RBGS. The use of different experimental framework, dependant variables and image types in [START_REF] Moncur | Pictures at the ATM: exploring the usability of multiple graphical passwords[END_REF][START_REF] Everitt | A comprehensive study of frequency, interference and training of multiple graphical passwords[END_REF] makes it difficult to allow systematic comparison of our results with them.

Our findings show that the effectiveness of graphical passwords is significantly affected by the type of images used. In this context, the results show that the mean login success percentage is highest for objects, closely followed by Mikons, then doodles and lowest for art images. According to the cognitive studies, dual coding theory [5] and guided search process [START_REF] Wolfe | Guided search 2.0 a revised model of visual search[END_REF], an elaborative encoding of an image in human memory makes it memorable. Thus, an image which is easily associated with a name (nameable) or can be interpreted in a meaningful way is likely to be more memorable due to superior encoding in human memory. In this context, mean scores of the sat 3 parameter (meaningfulness of the image) is: highest for objects (3.65/5); closely followed by Mikon (3.5/5); then doodles (2.8/5); lowest for art images (2.14/5). Thus we find that the results of the satisfaction parameter are in line with the mean successful login percentage. Hence the higher memorability of the object and mikon images can be attributed to the fact that users find these images meaningful and easy to associate with something. The doodle images are black and white line drawings and do not convey much meaning to aid memorability. Hence these images may not be encoded in an elaborate way in the human memory. The art images were very difficult to remember because according to the users, it was not only difficult to associate them with something meaningful but they were visually complex, containing a lot of information and color which would lead to information overload in memory. This complements the work reported in [START_REF] Szekely | Objective visual complexity as a variable in picture naming[END_REF] which suggested that visual complexity of an image is linked to the ease of associating it with a name. The work has indicated that it is difficult to assign names to visually complex images. The results also reveal that the mean registration time is: lowest for objects; closely followed by Mikons; then doodles; highest for art images. In this context, the mean scores of the sat 1 parameter (ease to register) are: highest for objects (3.24/5); closely followed by Mikons (3.12/5); then doodles (2.85/5); lowest for art images (2.45/5). Thus the qualitative data obtained from the participants through the questionnaire complement the mean registration time obtained from the online study. These results can be attributed to the fact that users find it difficult to choose meaningful images in case of doodle and art, which they could use as passwords. The authentication time follows the same trend as that of the registration time: lowest for objects; closely followed by Mikons; then doodles; highest for art images. The mean scores for the sat-2 parameter (ease to authenticate): highest for objects (3.56/5); closely followed by Mikons (3.36/5); then doodles (2.95/5); lowest for art images (2.45/5). Thus the mean scores of the sat 2 parameter complement the results of the login time data obtained from the online study. The above discussion suggests that the effectiveness and efficiency results complement each other. So we can conclude that images which are meaningful or can be associated with something easily are: effective in the sense of being memorable; efficient i.e. less time consuming to employ. This conclusion is also supported by the mean satisfaction scores obtained through the questionnaires: highest for objects (13.91/20); closely followed by Mikons (13.16/20); then doodles (11.8/20); lowest for art images (9.24/20) and mean stress scores: lowest for objects (11.87/20); closely followed by Mikons (12.2/20); then doodles (13.12/20); highest for art images (14.66/20).

Conclusions and Future work

We have presented the first study with 100 participants over a period of 8 weeks to compare the usability: mean success percentage for 7 weeks; mean registration and login time; the qualitative data i.e. participant's opinion (satisfaction as well as stress), of multiple image passwords using 4 different image types-Mikon, doodle, art and objects. The results of the study revealed that object and Mikon passwords performed best in each of the usability criteria compared to doodle and art passwords. So we conclude that meaningful or easily nameable image types are most usable when multiple graphical passwords are used. The experimental design in our study is: valid, it answers our research question through the data we collected for each measure; reliable, it can be reproduced by the research community; most importantly, such a study for the stated research problem has not been conducted in the past. In terms of improvement, the same study could be reproduced with a user-group other than students. The post study questionnaire results demonstrate that most people chose password images either by making a pattern/story or something which is related to them. These results underscore the need to examine, whether passwords created by using patterns aid memorability, when multiple graphical passwords are used and assess the ease of employing such a strategy these different image types. The results suggest that though meaningful images would aid usability when multiple graphical passwords are used, users may engage in insecure coping mechanisms like recording them through digital or non digital media. So in our ongoing work, we are investigating the vulnerability of image passwords to user descriptions (non digital attempt to describe the images by writing them or verbally communicating them). We also find that the statistical analysis alone does not unambiguously identify the most suitable image type to be used for graphical password. In our ongoing work, we are developing a framework that would help quantify the usability value for each image type, taking into account qualitative as well quantitative data obtained for all criteria from the experiment.

  had a collection of 150 images, presented on the screen as six sets of 25 images in the form of 5X5 grids. The participants could browse from one set to other using the 'change set' button on the web page. The participants could choose all the 4 images from a single set or each image from a different set. The experiment was designed such that each of the participants would use a different collection of the same image type while registering for each of their passwords. For example, each participant in the Mikon condition created 4 passwords, with 1 password selected from the image collection of the first link (My jokes), one from the second link (My movies) and so on. The image archive in each link of the Mikon website consisted of the same image type (Mikon).

Fig. 1 .

 1 Fig. 1. Steps to register with a password in RBGS

2 :

 2 Four-step login 1: Home Step 2... 3.... 4......

Fig 3 )

 3 : (1) identifying the key points of the survey; (2) categorizing the key points depending on the factors influencing them; (3) parameters or concepts to be examined under each category; (4) explanation of the results (qualitative data) to draw conclusions.

Fig. 3 .

 3 Fig. 3. Grounded theory framework for pre-study survey

Fig. 9 .

 9 Fig. 9. Strategy employed to create image passwords

Acknowledgement

We are thankful to SICSA (Scottish Informatics and Computer Science Alliance) for funding this research. We are also thankful as well as grateful to Dr. Balvinder Shukla (Vice Chancellor Amity University, India) for helping us get the participants for the study and permitting us to perform the experiments with them.

Sample Mikon password

Sample doodle password

Sample art password

Sample object password


Julian Seifert

Andreas Bayer
email: andreas.bayer@uni-due.de

Enrico Rukzio
email: enrico.rukzio@uni-ulm.de

PointerPhone: Using Mobile Phones for Direct Pointing Interactions with Remote Displays

Keywords: Mobile phone, pointing, interaction, collaboration. 1

Large screens or projections in public and private settings have become part of our daily lives, as they enable the collaboration and presentation of information in many diverse ways. When discussing the shown information with other persons, we often point to a displayed object with our index finger or a laser pointer in order to talk about it. Although mobile phone-based interactions with remote screens have been investigated intensively in the last decade, none of them considered such direct pointing interactions for application in everyday tasks. In this paper, we present the concept and design space of Pointer-Phone which enables users to directly point at objects on a remote screen with their mobile phone and interact with them in a natural and seamless way. We detail the design space and distinguish three categories of interactions including low-level interactions using the mobile phone as a precise and fast pointing device, as well as an input and output device. We detail the category of widgetlevel interactions. Further, we demonstrate versatile high-level interaction techniques and show their application in a collaborative presentation scenario. Based on the results of a qualitative study, we provide design implications for application designs.

Introduction

Large screens and projections have become part of our daily lives. They support collaboration as they allow multiple users to simultaneously access information. For instance, in a meeting presentation, information is shared with multiple users. However, the control of what is being displayed is usually limited to a single user (e.g., in presentation). Others cannot share, access, or manipulate virtual objects or data on the remote display.

Direct pointing has been investigated for interaction with remote displays [START_REF] Olsen | Laser pointer interaction[END_REF]. Direct pointing interaction is a natural way for users to select and interact with objects on a remote screen (see Figure 1(a)) [START_REF] Myers | Interacting at a distance: measuring the performance of laser pointers and other devices[END_REF]. However, such settings enable only few options for interactions and are limited to basic operations.

As mobile phones are ubiquitously available, they enable users to access remote displays in diverse ways [START_REF] Ballagas | The smart phone: a ubiquitous input device[END_REF]. For instance, downloading information from a remote display to the mobile phone for further inspection [START_REF] Boring | Shoot & Copy: Phonecambased information transfer from public displays onto mobile phones[END_REF] or sharing information on a remote screen with others [START_REF] Alt | Digifieds: insights into deploying digital public notice areas in the wild[END_REF]. Pointing-based interactions offer an easy-to-use way of allowing users to interact with an object by pointing to it [START_REF] Rukzio | An experimental comparison of physical mobile interaction techniques: touching, pointing and scanning[END_REF].

Hence, mobile phones with integrated pointing abilities enable diverse novel options for interaction in a natural and seamless way. Using the mobile phone as a pointing device and general interaction device has not been investigated, thus raising questions regarding how the phone's specific characteristics (e.g., options for input and output) and attributes (e.g., user data context) can be integrated into the interaction process.

In this paper, we contribute the detailed investigation of the novel design space of PointerPhone. PointerPhone uses mobile phones as pointing devices for new direct and natural pointing interactions with remote screens (see Figure 1). We present a classification consisting of low-level, widget-level, and high-level interaction techniques. Further, we show application examples and demonstrate the integration of diverse techniques into a collaborative meeting support application which we implemented based on a prototype system that uses laser-pointer-equipped mobile phones 1 . Furthermore, we present observations and results of a qualitative user study and we provide a catalogue of design guidelines as well as lessons learned that need to be considered when designing applications based on PointerPhone interactions.

Related Work

A large body of research exists in the area of remote interaction with remote information displays such as large screens or projections. These related works can be clas- sified into the following categories: a) general pointing interaction and technologies, b) interaction with remote screens using mobile phones as a pointing device, and c) collaboration support. The review of the related work shows that no previous work investigated the combination of mobile phones with direct pointing for interaction with remote screens.

Pointing Interaction

Early work by Rekimoto investigated the potential of interacting with remote screens in a continuous work space [START_REF] Rekimoto | Augmented surfaces: a spatially continuous work space for hybrid computing environments[END_REF]. Using the hyperdragging technique, users could move data items across multiple devices (e.g., laptops or shared screens). Here the user controlles a cursor with a mouse connected to one device. The cursor would jump to other screens when reaching a display border, allowing to reach out for data across devices. By using a mouse, the pointing is only indirect as users cannot literally point out to the desired target but rather move the mouse on a planar surface. Olsen and Nielsen investigated direct pointing for interaction with virtual objects based on laser pointers [START_REF] Olsen | Laser pointer interaction[END_REF]. They analyzed interaction techniques based on camera tracking of a simple laser pointer on a remote screen (e.g., interaction with widgets, drawing, and navigation). Vogt et al. demonstrated how collaborative multi-user settings can be implemented using several laser pointers [START_REF] Vogt | Tracking multiple laser pointers for large screen interaction[END_REF]. Myers et al. compared the effect of different form factors of pointing devices (a pen-like pointer, a pointer attached to a toy gun, and a pointer attached to a personal digital assistant device) for interaction with large projected screens [START_REF] Myers | Interacting at a distance: measuring the performance of laser pointers and other devices[END_REF]. They observed that it is essential to support interactions which do not cause much jittering of the pointing device and allow users to easily control the making of selections. Shizuki et al. investigated the potential of using pointers to interact with remote screens beyond making mouse-like selections [START_REF] Shizuki | Laser pointer interaction techniques using peripheral areas of screens[END_REF].

Their findings show that simple commands for applications can be directly performed using a standard laser pointer device. Nintendo released the Wii U controller device which enables pointing to a remote screen and interaction through a small touchscreen [START_REF] Nintendo | Wii u official site at Nintendo[END_REF]. Similarly to the PointerPhone approach, this system enables interaction on a remote and a personal screen. Yet, the Wii U game controller does not provide and store users' data such as photos, bookmarks or contacts. Also, a game controller is not always carried by the users and thus is not suited for interacting in diverse settings. Schmidt et al. introduced a pico projector-based approach where users project an interface onto smart objects that are equipped with a specific sensor [START_REF] Schmidt | Picontrol: using a handheld projector for direct control of physical devices through visible light[END_REF]. Through moving the projector and thus the projected interface, users can select and activate different interface widgets. This way of pointing interaction is, however, limited to one user and does not support collaboration.

Pointing Technologies

A large variety of technologies and approaches exist to allow for the sensing of the pointing interaction. They can be classified into direct pointing (e.g., with laser point-ers as in [START_REF] Olsen | Laser pointer interaction[END_REF][START_REF] Vogt | Tracking multiple laser pointers for large screen interaction[END_REF][START_REF] Tangram | Tangram -Smart Dot Pro[END_REF]), and inertial sensing (e.g., using gyroscope and accelerometer sensor data as in [START_REF] Nintendo | Wii u official site at Nintendo[END_REF]). An additional approach was presented by Zizka et al.; they presented SpeckleSense which enables precise interactions with remote screens based on laser speckle motion sensing [START_REF] Zizka | Specklesense:fast,precise,low-costandcompact motion sensing using laser speckle[END_REF]. In contrast to a pointer-based approach, each interaction device handles the sensing internally. This approach also allows for the use of a fixed device which senses the motion of a user based on laser speckle; similar as interaction sensing enabled through a fix installed depth camera (e.g., [START_REF] Morris | Web on the wall: insights from a multimodal interaction elicitation study[END_REF]). With the XWand, Wilson and Shafer introduce a pointing device with the shape of a short stick, which allows users to point to and interact with objects in a smart environment [START_REF] Wilson | XWand: UI for intelligent spaces[END_REF]. This system determines with which object a user intends to interact based on data provided by different integrated sensors such as accelerometers. Hence, no information display is required and user feedback is provided implicitly.

Mobile Interaction with Remote Screens

For interacting with remote screens, mobile phones have been used in many different ways. For instance, different works examined the potential of gestures where the user pretends to throw an object towards the remote screen while holding the mobile phone in their hand (e.g., [START_REF] Chehimi | Throw your photos: an intuitive approach for sharing between mobile phones and interactive tables[END_REF][START_REF] Dachselt | Natural throw and tilt interaction between mobile phones and distant displays[END_REF]). Yet this only allows for imprecise interactions, and selecting items is not possible. Also, different television set manufacturers provide applications for mobile phones which replace the standard remote control: the mobile phone provides an interface for selecting the channel and controlling the volume (e.g., [START_REF]LG TV remote 2011 -Android Apps on Google Play[END_REF]). However, interaction is limited to these simple actions and to steering a cursor through the phone's touch screen. Boring et al. presented a unidirectional mobile phone camera-based approach for selecting and transferring data from a remote display to a personal mobile phone [START_REF] Boring | Shoot & Copy: Phonecambased information transfer from public displays onto mobile phones[END_REF]. Different works investigated the potential to control a virtual cursor on the remote screen, using the phone camera by tracking the optical flow (e.g., [START_REF] Jeon | Interaction with large ubiquitous displays using camera-equipped mobile phones[END_REF]), or the phone's accelerometers (c.f., [START_REF] Boring | Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays[END_REF]). However, this approach is limited in terms of interaction speed. Boring et al. presented TouchProjector, which allows users to perform touch input on their mobile devices; the touch events are then transmitted to a remote screen [START_REF] Boring | Touch projector: mobile interaction through video[END_REF]. Baur et al. investigated interaction based on the metaphor of virtual projection whereby the phone camera captures an external screen to sense the spatial relation of the phone to the screen. Users can exchange data and interact with virtual objects on the remote screen within the camera frustum by performing touch interactions on the mobile phone [START_REF] Baur | Virtual projection: Exploring optical projection as a metaphor for multi-device interaction[END_REF]. The PointerPhone approach presented in this paper, in contrast to the aforementioned works, allows to interact directly with specific targets on the remote screen by pointing in a natural way. Users do not have to hold the mobile phone like a camera as the case when using TouchProjector for remote touch interaction. Also, in contrast to camera-based approaches for steering a cursor, which only allow indirect pointing, PointerPhone allows pointing that is natural for the user, fast, and direct.

Mobile Phone as Pointing Device

Rukzio et al. investigated users' preferences of mobile interaction with smart objects in their environment [START_REF] Rukzio | An experimental comparison of physical mobile interaction techniques: touching, pointing and scanning[END_REF]. They compared the interaction styles of touching (using near field communication), scanning (based on Bluetooth), and pointing. For pointing, they attached a laser pointer to a mobile phone for interacting with smart objects such as a CD player. Pan et al. investigated how inertial sensing of a phone's position and rotation in relation to a projected remote screen can be used to support pointing using a mobile phone [START_REF] Pan | Easypointer: what you pointing at is what you get[END_REF]. In their work, Rashid et al. investigated the differences of proximal and distal selection of targets using a mobile phone connected to a Wii controller [START_REF] Rashid | Proximal and distal selection of widgets: designing distributed ui for mobile interaction with large display[END_REF]. They found that distal interaction is preferred for the selection of targets and results in lower error rates compared to proximal selection. While this work provides interesting insights regarding the selection of targets, they did not focus on other aspects such as interaction with widgets or data sharing interactions.

These works investigated the use of a mobile phone combined with different pointing capabilities. However, they all do not consider the phones' specific attributes and capabilities that allow for novel interactions. PointerPhone considers the characteristics of mobile phones in terms of available sensing, input and output options, and application possibilities which have not been investigated so far.

Collaboration Support

Work on single-display groupware [START_REF] Tse | Avoiding interference: how people use spatial separation and partitioning in SDG workspaces[END_REF] investigated how multiple users can simultaneously interact with a single display, for instance, by using multiple mouse cursors [START_REF] Pawar | Multiple mice for retention tasks in disadvantaged schools[END_REF]. However, in such a setting user collaboration is limited to simultaneously pointing to targets on a shared screen. Sharing personal data is not possible, although feasible through the PointerPhone approach. Connecting personal and mobile devices to a remote shared screen to support collaboration has been investigated in different works. The combination of such different devices enables multiple users to access a shared device while each also controlling their personal device [START_REF] Rekimoto | Augmented surfaces: a spatially continuous work space for hybrid computing environments[END_REF]. Myers et al. connected multiple PDAs to a central workstation [START_REF] Myers | Collaboration using multiple PDAs connected to a pc[END_REF], allowing for data exchange. Additionally, the combination of personal and mobile devices with horizontal tabletoplike displays has been studied [START_REF] Schmidt | A cross-device interaction style for mobiles and surfaces[END_REF][START_REF] Sugimoto | Caretta: a system for supporting face-to-face collaboration by integrating personal and shared spaces[END_REF]. Vogt et al. compared mouse-based and laser pointer-based interaction regarding their ability to support collaboration in small groups [START_REF] Vogt | Exploring collaboration with group pointer interaction[END_REF].

Phone Pointer Interactions

The underlying concept of using the mobile phone as a pointing device for direct interaction with remote displays is simple but at the same time versatile: users point towards targets on a remote screen in order to perform an action that is applied as the user triggers the action (e.g., selecting or editing an item, controlling widgets). The available hardware of the mobile phone and the remote display yield a number of basic attributes and possibilities for interaction. Basic attributes include whether the user is pointing and the location on the remote display where the user is pointing to. Further, each mobile device that is used as a pointing device can be distinguished through its ID. Accordingly, different users can be distinguished.

Each of these basic attributes can be used for interactions. Considering attributes isolated and combinations of different attributes result in interactions of diverse complexity. Hence, simple interactions can be used as building blocks for more complex interactions. For instance, selecting a target by pointing and clicking can be part of any activity such as selecting a file on the remote screen. Again, this activity can be part of many applications such as photo sharing or web browsing.

Accordingly, the diverse interaction possibilities can be classified into three layers of abstraction: low-level input and output, widget-level interactions, and high-level interaction and applications. This classification was chosen as it shows which options for interaction are available and how these can be integrated into application designs.

Low-level Input and Output

Low-level Input Options. The most basic options for performing input on the mobile phone while pointing to a target on the remote display are using software buttons displayed on the phone's screen (see Figure 2 hand or with two hands. One alternative option that avoids pressing hardware or software buttons is to trigger an action by rotating the phone along the pointing axis (see Figure 2(d)). Rotation in different directions (i.e., clockwise and counterclockwise) allows to encode different actions. For instance, a left and right click can be performed. However, rotating the phone could also result in moving the cursor away from the target.

As emphasized by Myers et al. (see [START_REF] Myers | Interacting at a distance: measuring the performance of laser pointers and other devices[END_REF]), using physical buttons causes unintended jitter effects which could lead to input actions on unintended targets. This is potentially also the case when interacting with software buttons or performing gestures on the mobile phone. One approach that enables users to trigger an action without touching and moving the mobile phone uses the proximity sensor in the mobile phone. While users point to a target, they move their hand close to the phone and trigger the action as their hand gets close enough (see Figure 2(e)). Similarly, users could trigger an action without moving or potentially even without touching the mobile phone through snapping with their available hand which could be sensed using the phone's microphone.

Low-level Output Options. Output options for feedback and information presentation are distributed on the users' mobile pointing device and the remote screen. The latter provides visual feedback and optionally, audio output can be provided (e.g., a television set in the user's living room remotely operated through pointing). The output options of the remote display can be targeted to one specific user only to a limited degree. If several users are using the system simultaneously, for instance, audio feedback provided by the remote screen is audible to all present persons. Visual feedback, however, can be displayed on the remote screen close to a user's pointing cursor in order to make clear at whom the feedback is targeted.

In addition, the personal mobile pointing device enables personal feedback which is not accessible to others. That is, the personal mobile devices provide visual feedback and output on their display, which is visible only to the specific user. Also, audio feedback can be provided either via speakers or headphones, which allow feedback that is not audible to others. Third, mobile devices allow for haptic feedback through vibration.

Widget-Level Interaction

Often, interaction with diverse applications requires users to specify specific pieces of information or data (e.g., numeric values, strings) in order to control the state of an application. For instance, users control the zoom level of text or specify the volume of audio data using a slider, or select an option from a list using radio buttons. To facilitate this task, many different widgets are available, including as sliders, radio buttons, and text fields, each of which supports the input of a specific data type. When using mobile phones as pointing device' to interact with applications on a remote or shared display, users require support to interact with all kinds of standard user interface widgets. The given configuration yields up to three different options for interacting with widgets: 1) rotation of the mobile phone (see Figure 3(a)), 2) proxi-mal interaction on the touch screen (see Figure 3(b)), and 3) distal interaction through direct pointing to the widget on the remote screen, selecting it and moving the pointing cursor to change the value of the widget (see Figure 3(c)).

As analyzed by Rashid et al. (see [START_REF] Rashid | Proximal and distal selection of widgets: designing distributed ui for mobile interaction with large display[END_REF]), the performance of proximal and distal selection of targets (e.g., clicking a button) depends on the complexity of the tasks. For complex tasks which involve many and small targets, proximal interaction is superior to distal interaction which was superior in simple tasks (i.e., interaction with few large targets).

However, not all three options for interacting with widgets apply to each widget, depending on the type of data supported. Table 1 offers an overview of standard widgets and how they can be controlled through pointing with a mobile phone. Accordingly, only widgets that are designed for the input of continuous values, that is, sliders or turning knobs, could be directly controlled via rotating the phone while pointing to them in order to change their value. Yet it is also possible to control these widgets proximally on the phone touch screen or distally on the remote screen. Standard buttons can be controlled using all three options, given that the phone rotation is mapped to a selection. Radio buttons could be selected and rotating the phone could change the selection. Check boxes are less suited for this alternative due to their size.

Text fields require the user to interact with a keyboard which is most convenient for the user on the mobile phone.

High-level Interactions and Applications

In this section, we discuss phone-based pointing interaction techniques that build on the previously discussed interactions.

Data Exchange. Users can share data that is stored on the mobile phone by transferring it to the remote screen. To do so, users select one or several items to share, point to a position on the screen, and trigger the transfer. For instance, users could select an image, point to the desired location, and perform a swipe gesture on the phone towards the remote screen (see Figure 4(a)). On the remote screen the image appears at the location of the pointing cursor (see Figure 4(b)).

In order to receive data from the remote display, users point at the intended item (see Figure 4(c)) and trigger the transfer. As illustrated in Figure 4(d), a swipe gesture on the user's phone could be used to pull the item. However, any other low-level input can be applied here. Proximal Context Menus. Pointing-based interaction with a remote display through a mobile phone supports the handling of meta information of items such as files which are displayed on the remote display. For instance, context menus are often used in order to change the name of a file. These provide a list of possible options that can be applied to the selected file. Using the mobile phone as pointing device, users first select a file (see Figure 5 Drawing and Sketching. Through pointing to the remote screen, users can edit and create graphical content such as sketches. Depending on a selected tool and the corresponding parameter settings (e.g., a brush and a selected color) users can create sketches simultaneously (see Figure 6). As different phones are distinguished, each user can select different tools and settings at the same time. Personal Input and Output. Personal mobile phones as pointing devices allow users to receive personal output (e.g., visual or auditory) as well as to perform input on the personal device in collaborative settings. For instance, when multiple users share a view on a web page, a single user who is interested in following a specific link can point to it and open the corresponding web page on their personal device (see Figure 7(a) and (b)). This allows users to look up additional information without interrupting or disturbing the group activity. Additionally, input can be performed on the personal device which, on the one hand, avoids cluttering the remote screen with a large virtual keyboard. On the other hand, input on the personal device allows users to enter sensitive information such as a password. For instance, when a user needs to login to a user account to access some information, the password can be entered on the mobile phone (see Figure 7(c)).

In addition, different types of data such as files, geographical coordinates, contact cards, or appointments can be distinguished and, once selected through pointing to their representation on the remote screen, they can be handled with different applications on the mobile phone. For instance, a user could point to an address and select it, which opens the map application on the mobile phone and displays the given location.

Remote Control. As more television sets support additional diverse applications such as web browsers, one emerging idea is to use secondary display devices to achieve remote control [START_REF] Cesar | Usages of the secondary screen in an interactive television environment: Control, enrich, share, and transfer television content[END_REF][START_REF] Cruickshank | Making inter-active tv easier to use: Interface design for a second screen approach[END_REF][START_REF] Rashid | Proximal and distal selection of widgets: designing distributed ui for mobile interaction with large display[END_REF]. Using the personal mobile phone for this kind of interaction allows multiple users to interact simultaneously, for instance with web pages displayed on a smart TV (see Figure 8).

Collaborative Meeting Scenario

To show how different interaction options can be integrated and used as building blocks for a realistic application context, we designed and implemented a collaborative presentation system that supports users in a meeting scenario.

Scenario: Bob is giving a presentation for some colleagues on a large projected remote screen (see Figure 9(a)). Each meeting participant is equipped with a mobile phone which can be used to point to the remote display and control a curser through phone pointing interaction techniques. Each participant's pointing cursor on the remote screen is distinguished by a different color. Hence, each meeting participant has a visual representation of who and how many users are currently pointing to specific pieces of information on the remote screen. Each projected presentation slide contains diverse pieces of information. For instance, an overview plan may allow specific views on details of the plan through the selection of a corresponding icon (see Figure 9(b)). This allows users to individually explore and access additional information without disturbing others, as pointing to an icon on the remote screen and selecting it results in a detailed preview on their personal mobile phone. Icons next to a person's name indicate that the contact card can be downloaded to the mobile phone by pointing to it and selecting it. Additional background information can easily be accessed, for example, by pointing to an image on the remote display (see Figure 9(c)). In their meeting, the participants also discuss with each other about the presented topic. This discussion and brainstorming is supported through collaborative sketching on a drawing canvas on the remote screen where to each user can contribute using their phones (see Figure 9(d)).

( A video illustrates this scenario in detail: http://youtu.be/qp3pIklYLxo

Evaluation

In order to gain an understanding on how users would use the system and how they appreciate the different interaction techniques, we performed a qualitative evaluation.

The aim was to gain qualitative insights regarding direct pointing-based interaction with a mobile phone and remote display.

Design

The study session consisted of two parts: After participants were introduced to the study, they performed a series of practical tasks. Tasks that involved collaboration were performed by the participant together with the investigator. After the practical tasks, they filled out a questionnaire. During the session, participants were encouraged to think aloud and continuously talk about their actions. Further, the investigator took notes and the task performance was recorded on video.

Practical tasks. Practical tasks were selected to expose participants to a broad variety of different application contexts. Participants used the PointerPhone prototype for the tasks. The following list of tasks was performed by participants in randomized order.

1. Browsing. Participants had to browse through a website that was displayed on the remote screen. The PointerPhone prototype was used for controlling widgets and link selection. They followed text-based instructions, which involved selecting links, downloading images to the phone, and interacting with widgets such as radio buttons through pointing.

2. Photo sharing. Selecting and transferring two photos from the phone's library to the remote display and retrieving photos from the remote screen.

3. Sketching. Collaborative sketching of a simple building on a shared sketching canvas on the remote screen which required different brushes (selected and configured on the mobile phone). Users also performed text entry on the phone and placing the created text on the sketching canvas for labeling the sketched items.

4. Completing a form. Filling in a form on the remote display which included interacting with different kinds of widgets for data input via the mobile phone.

5. Context Menu. Renaming, copying, and deleting files displayed on the remote screen by using a proximal context menu on the mobile phone.

6.

Playing. Playing a simple Pong-like game involving two users who would steer the position and angle of a racket by pointing to the screen to control the translation and rotating the phone to control the angle.

Apparatus. The main components of the apparatus system are mobile phones as pointing devices and a remote display that is connected to a server computer (Figure 10(a)). The mobile devices are connected to the server through Wi-Fi for the exchange of data and commands.

In order to achieve high-level pointing accuracy and low latency, we followed the approach of using laser pointers and camera-based tracking for the pointing task (as previously demonstrated by [START_REF] Myers | Interacting at a distance: measuring the performance of laser pointers and other devices[END_REF][START_REF] Olsen | Laser pointer interaction[END_REF]). That is, a camera is used to capture the remote display. If a user points to the remote screen, the laser pointer creates a bright point on the image which can be extracted through simple image processing. The location of the laser pointer is used to control the user's pointing cursor that is displayed on the remote screen. For distinguishing different laser pointers (and thus different users), a color filter is applied during the image processing step. This tracking approach requires only one calibration sequence before using the system until the camera or display setup is changed (i.e., if they are moved).

We extended a standard mobile phone (a Samsung Nexus S, running Android 2.3) with a laser-pointing module which can be controlled via the mobile phone's software stack (see Figure 10(b)). The laser pointer is turned on and off via a simple circuit with a photodiode that is placed right in front of the flash light LED of the mobile phone. On the Android platform [START_REF]Android developers[END_REF], this component can be controlled via a standard application programming interface.

Based on the system prototype, we developed a number of mobile applications for the mobile phone and a corresponding application for the remote screen which implemented all functionalities that were required for the tasks. To allow participants to experience several possibilities of the PointerPhone interaction, the applications provide different options to perform any single action. For instance, users can make selections distally on the remote display, as well as proximally on the phone display. For the sake of consistency, the activation of the pointing (turning on the laser pointer) is the same in all applications. Short tapping on the hardware button on the bottom right activates permanent pointing. Holding the button activates the pointing until its release. Participants. We recruited 14 participants (7 female) aged between 20 and 31 (Mean=26). Of these, 10 were students (diverse backgrounds) and 4 were employees.

After the study, they were rewarded for their effort with 10 Euro.

Observations and Design Implications

In the following, we discuss the results of the feedback sessions and, where possible, summarize findings as a set of design implications that support application designers when considering pointing interactions for their work.

1. One-and two-handed interaction. We observed that, during their interaction with the different applications, participants switched between using one or two hands to hold the mobile phone depending on the task. For instance, during the sketching task, 7 of the 14 participants held the phone with two hands. An additional 4 participants held the phone in the right hand and supported it with the left. Only 3 participants used one hand to hold the phone and point during the sketching task. In contrast, during the data-sharing task, 13 participants used a single hand to hold the phone, of whom 5 used the thumb of the same hand and 8 used the index finger of their other hand to interact with the phone interface. Accordingly, the manner in which users choose to hold the phone depends on the given task. For tasks that require precise pointing input (e.g., sketching), users tend to use two hands. Tasks that require less precise pointing, however, lead users to prefer one-handed opera-tion. Hence, interfaces for the hand-held pointing device should encourage twohanded interaction when designed for tasks that require precise pointing. Inversely, interfaces for simple tasks should be adapted to one-handed usage.

2. Selecting targets. During the browsing task, participants could select targets such as links either distally on the remote screen or proximally on the phone's display.

We observed that participants preferred to select large targets distally while small targets (e.g., text links) led to a preference to select proximally. This concurs with the findings of Rashid et al. who investigated distal and proximal target selection [START_REF] Rashid | Proximal and distal selection of widgets: designing distributed ui for mobile interaction with large display[END_REF]. We observed that distal selection forced users to switch their focus from the remote display to the phone to ensure hitting the correct button. Accordingly, user interfaces for proximal interaction should be designed to allow users to keep their focus on the remote display -for instance, through the use of hardware buttons on the phone if available or a single large software button, so that the user does not have to look at the pointing device.

3.

Navigating. When users selected an area of the remote screen that should be displayed proximal on the phone, several participants tried to interact with the proximal representation like they were used to interact with smartphone web browsers that allow navigation through dragging and zooming in and out. However, this applies only to small adjustments. Participants expressed that they can select easily an area through pointing to it which is more comfortable than navigating on the phone screen.

4.

Providing output and feedback. We observed several times that users would focus on the remote display while they select a target there, yet they were not aware of the resulting change on the mobile phone. Inversely, this phenomenon was also observed when a user focused on the phone and performed an action which resulted in an event on the remote display. Hence, it is essential to provide cues (e.g., audio feedback or vibration) which notify users regarding resulting actions. Several participants raised the general point that the remote display should not be used to display user-specific information that is not intended for all users. For instance, when interacting with a web page, users could display tool tip information through pointing at an item for a few seconds. These should be displayed on the personal device.

Controlling pointing actions.

Participants had to manually enable or disable the pointing mode through toggling a button (i.e., turning on and off the attached laser pointer). However, when a participant was engaged with performing a task, they forgot to turn the pointer on which resulted in confusion. Hence, if application workflows allow to anticipate when pointing is required, the system should automatically do so. For instance, when pointing to a web page on the remote display in order to transfer a clipping to the phone for further inspection, the pointing should be disabled automatically to prevent unintended updates. All users indicated that they liked that they could see a cursor where they were pointing at. This indicated that alternative implementations (e.g., inertial sensing [START_REF] Pan | Easypointer: what you pointing at is what you get[END_REF]) should provide a visual cursor throughout the interaction.

Discussion & Conclusion

In this work, we analyzed options for interaction when using a mobile phone as a pointing device for interaction with a remote display. Related work on pointing interaction, the use of mobile phones for controlling content on public and shared remote displays, as well as collaboration provides a large body of research on interaction techniques for specific contexts. Using the mobile phone as a personal pointing device provides not only powerful computing and sensing technologies but also the user's personal data context such as photos, calendars, and messages. These rich options for interaction are likely to attract the design of applications in the future. Hence, application designers considering PointerPhone applications should be supported through a design space and corresponding guidelines for using aspects from the design space. Different options exist regarding possible implementations for the application of phone pointing-based interaction outside the laboratory setting. Using inertial sensing to determine the phone's pointing direction is a promising approach, as most available smart phones are equipped with the required sensors (i.e., accelerometer and gyroscope). However, each time a user intends to use such a system, the user needs to calibrate their phone to determine the pointing direction. Moreover, during the interaction the calibration may have to be repeated to maintain the pointing accuracy. Alternatively, direct pointing using a laser pointer can be easily added to standard smart phones. For instance, laser pointers can be plugged into the audio jack of the phone [START_REF] Tangram | Tangram -Smart Dot Pro[END_REF] and operated through an application. This approach would require the remote screen to be equipped with camera tracking. As an alternative, the laser pointer could be based on infrared light which can be sensed by specific screens (e.g., Microsoft PixelSense [START_REF]Welcome to Microsoft Pixelsense[END_REF])

The investigation of the design space provides the list of basic attributes and characteristics. Further, we provide a classification of low-level interaction options regarding input and output options, widget-level interaction techniques, and high-level interaction techniques and applications. This classification into three levels of abstraction is not fixed and can be extended as it includes only selected examples for applications, and technological features for input are likely to be extended. These techniques can be used as building blocks for complex applications.

Finally, we showed how several interaction techniques can be integrated into a presentation application for a collaborative meeting context. We used a prototype implementation of the system based on mobile phones combined with laser pointers to realize a number of applications that we used for a qualitative study. Results from the study support a collection of five design recommendations that should be considered for the design of pointing-based applications.

1 AFig. 1 .

 11 Fig. 1. Using the mobile phone as a pointing device enables versatile interactions with remote screens: (a) Pointing to targets. (b) Performing actions on the personal phone such as selecting and downloading an item. (c) Further interaction with data on the phone.

Fig. 2 .

 2 Fig. 2. Basic selection and input options supported through PointerPhone: (a) software buttons; (b) hardware buttons; (c) touch-based gestures; (d) rotation-based interaction; and (e) proximity activation.

Fig. 3 .

 3 Fig. 3. Controlling widgets for entering data can be implemented in three ways: (a) Rotating the phone to change the value; (b) manipulating the proximal widget representation on the phone screen; (c) distal interaction with widgets on the remote screen.

Fig. 4 .

 4 Fig. 4. Transmitting an item from the mobile phone to the remote screen ((a) and (b)) and vice versa picking up an item from the screen ((c) and (d)).

Fig. 5 .

 5 Fig. 5. Proximal context menus: (a) A user selects a file. (b) The context menu is displayed on the phone. (c) A new file name can be typed in.

Fig. 6 .

 6 Fig. 6. A sketching application. (a) The phone provides a palette of different tools. (b) Tools are applied through pointing. (c) Multiple users can work simultaneously.

Fig. 7 .

 7 Fig. 7. Input and output on the personal device: (a) To avoid disturbing a group activity, the user may point to a link and (b) open it on their device. (c) Entering information on the personal device.

Fig. 8 .

 8 Fig. 8. Using the mobile phone as a remote control for browsing web pages on a TV set.

Fig. 9 .

 9 Fig. 9. Collaborative presentation and meeting support (a). Additional information can be accessed through specific icons ((b) and (c)). Users can share data with others on the remote screen (d).

Fig. 10 .

 10 Fig. 10. (a): System components schema. (b): Mobile phone prototype.

Table 1 .

 1 Overview of the widget control options.

	Widget	Orientation Proximal	Distal
	Turning Knob	Yes	Yes	Click & Drag
	Sliders	Yes	Yes	Click & Drag
	Button	Yes	Yes	Click
	Radio Buttons	Yes	Yes	Click
	Check Boxes	No	Yes	Click
	Text Field	No	Yes	No

Acknowledgements

This work has been conducted in the Emmy Noether research group Mobile Interaction with Pervasive User Interfaces funded by the German Research Foundation.