Abdul Moiz Penkar

Christof Lutteroth
email: christof@cs.auckland.ac.nz

Gerald Weber
email: gerald@cs.auckland.ac.nz

Eyes Only: Navigating Hypertext with Gaze

Keywords: Eye gaze tracking, dwell, navigation

de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

An eye gaze tracker makes it possible to monitor the user's point of gaze, i.e. where the user is looking on the screen. It is considered a promising component of future natural user interfaces. A gaze tracker as an input device is easy to learn and use [START_REF] Stampe | Selection by Looking: A Novel Computer Interface and its Application to Psychological Research[END_REF], and it is the fastest pointing device in some metrics [START_REF] Ware | An Evaluation of an Eye Tracker as a Device for Computer Input[END_REF]. Also, users necessarily look at the task-relevant objects [START_REF] Hayhoe | Eye Movements in Natural Behavior[END_REF], [START_REF] Just | Eye Fixations and Cognitive Processes[END_REF], [START_REF] Land | Eye Movements and the Control of Actions in Everyday Life[END_REF] regardless of the input device used, so intuitively a synergy can be expected if gaze is used as an input device.

Pointing with gaze tracking is relatively straightforward, but using it for performing actions (or clicks) is challenging. Inaccuracy and inadvertent clicking are the main drawbacks of using a gaze tracker as an everyday input device. The inaccuracy can be caused by gaze tracker error, or it can stem from fundamental limitations such as involuntary eye movements (including jitter and drifts) and the lack of pixelpointing precision [START_REF] Mackenzie | An Eye on Input: Research Challenges in using the Eye for Computer Input Control[END_REF]. Inadvertent clicking, also known as the Midas Touch problem, occurs because eyes are sensory (input) organs and this creates a mismatch when using them as a means of communicating intention to the computer.

Techniques devised to perform actions (or mimic mouse clicks) are called click alternatives. There are many such alternatives being researched for encoding an action with gaze. The most obvious and natural one is called dwell or fixation, which triggers an action when the gaze dwells on (i.e. fixates) the same area for a defined time interval. There is a reported preference for simple and natural gaze interaction techniques, requiring minimal deliberate eye movements [START_REF] Jacob | The use of Eye Movements in Human-Computer Interaction Techniques: What You Look at is what You Get[END_REF]. Conscious gazing or blinking as a control mechanism is unnatural and tiring. Consequently, complicated selection techniques are only feasible for limited accessibility uses. Since we consider gaze interaction for all users, the click alternatives being investigated in this study will be based on dwell. The users will not be required to learn or perform anything other than just looking at particular user interface components.

The suitability of gaze tracking as an alternative to the mouse may depend on the application domain. Therefore we decided to narrow the investigation down to a limited but highly relevant use case: gaze tracking for navigating hypertext, i.e. clicking text hyperlinks (or hypertext). We used Wikipedia for our study as it is one of the most visited websites, and it is close to the original vision of hypertext. We consider it therefore representative for hypertext navigation. The specific question being investigated is how inaccuracies and inadvertent clicks can be mitigated when navigating hypertext. The investigated gaze tracking techniques were also compared with the mouse.

Four click alternatives, referred to as Dwell, Single Confirm, Multiple Confirm and Radial Confirm, were developed. All of them use a different way of clicking with gaze, but all are based on dwell, as explained before. After a pilot study, two of the click alternatives (Dwell and Radial Confirm) were discarded from further experimentation as they did not seem to be usable enough. The remaining alternatives were fine-tuned and a full experiment was performed.

The experiment yielded the following main finding: Multiple Confirm, although slower than the mouse, is a feasible click alternative using the gaze and should be explored further. We conclude that eye gaze tracking has the potential to be used as an efficient input method, as long as certain design parameters are considered.

Section 2 summarizes related research. Section 3 describes the experiment design, detailing the click alternatives and tasks performed. Section 4 provides an overview of the findings of the pilot study and the changes made for the full experiment. The results are presented in Section 5, followed by a discussion of the results (Section 6) and limitations (Section 7). The paper ends with a conclusion, which summarizes the findings and identifies further research in this area.

Related Work

The first reported study of how the gaze behaves in a reading task dates back to 1878, and was probably the first to analyze the gaze as a combination of saccades and fixations [START_REF] Javal | Essai Sur La Physiologie De La Lecture[END_REF]. Considerable effort has been invested in finding efficient click alternatives. The most straightforward one is considered to be dwelling or fixating on a clickable area. Formally, this is described by a two-state machine, as shown in Fig. 1. This click alternative primarily suffers from inadvertent clicking, a problem that can be mitigated if it is possible to place the content or labels outside the clickable areas [START_REF] Penkar | Designing for the Eye: Design Parameters for Dwell in Gaze Interaction[END_REF]. Blinks, winks and different muscle sensors have also been experimented with, to minimize inadvertent clicking, especially for limited-accessibility applications [START_REF] Majaranta | Twenty Years of Eye Typing: Systems and Design Issues[END_REF]. Other alternatives are based on recognizing eye gestures [START_REF] Møllenbach | Single Gaze Gestures[END_REF] or anti-saccades [START_REF] Huckauf | What You Don't Look at is what You Get: Anti-Saccades can Reduce the Midas Touch-Problem[END_REF] to indicate a click, or using a physical button for clicking [START_REF] Ware | An Evaluation of an Eye Tracker as a Device for Computer Input[END_REF]. Different techniques for selecting menu options with gaze have also been evaluated (e.g. [START_REF] Kammerer | Looking My Way through the Menu: The Impact of Menu Design and Multimodal Input on Gaze-Based Menu Selection[END_REF], [START_REF] Urbina | Pies with EYEs: The Limits of Hierarchical Pie Menus in Gaze Control[END_REF]). Research has also focused on compensating for the inaccuracy and imprecision of gaze tracking. MAGIC is one such example in which the user relies on explicit commands using other input devices, e.g. the mouse, while also benefiting from the speed of pointing with gaze [START_REF] Zhai | Manual and Gaze Input Cascaded (MAGIC) Pointing[END_REF]. This approach moves the mouse pointer quickly to the area being gazed at, but relies on the mouse for finer adjustments and clicking. Different zooming [START_REF] Lankford | Effective Eye-Gaze Input into Windows[END_REF], [START_REF] Skovsgaard | Small-Target Selection with Gaze Alone[END_REF] and fish-eye methodologies [START_REF] Ashmore | Efficient Eye Pointing with a Fisheye Lens[END_REF] have been evaluated for the same purpose. The EyePoint [START_REF] Kumar | EyePoint: Practical Pointing and Selection using Gaze and Keyboard[END_REF] is another possible solution to cater for lack of accuracy and precision in gaze tracking. This solution involves magnification (of the area being gazed at) on the press of a keyboard button. In the magnified view, the key can be released while dwelling at the object of interest, resulting in an action on that object (e.g. left or right click). However, these techniques either block or distort the screen content, and loss of contextual information can be inconvenient and problematic, especially for tasks involving visual search [START_REF] Greene | Eye Movements and Familiarity Effects in Visual Search[END_REF].

Some recent work has been carried out on gaze-aware or gaze-controlled web browsers. The Text 2.0 Framework [START_REF] Biedert | The Text 2.0 Framework: Writing Web-Based Gaze-Controlled Realtime Applications Quickly and Easily[END_REF] allows setting and using gaze handlers the same way mouse and keyboard event handlers are used in HTML and JavaScript. The primary objective of this framework is to detect and assist in comprehension difficulties while reading. The IntelliGaze Desktop 2.0 [START_REF]Aleatech: IntelliGaze, Desktop 2.0 -Web-Browsing with Gaze Control[END_REF] extension allows web browsing with the gaze by magnifying the area being gazed at if there are multiple clickable options (hyperlinks, buttons or fields) in the gaze area. Dwell is used to click on a link or field in a rectangular magnified view, which always appears in the center of the window. A 'close' button accompanies the magnified view, which can be gazed at to close the magnified view if no click was intended. More controls are dynamically provided in the right margin of the window.

Our aim is to extend this work by evaluating if a more natural click alternative is feasible that supports everyday computing for a general population.

3

Experiment Design

Variables

The experiment was performed using a within-subjects design with "click alternative" as the independent variable, using a nominal scale. The dependent variables measured are "time to click," "number of incorrect clicks" and "number of failed clicks." Furthermore, user satisfaction was measured with a questionnaire.

Click Alternatives Design

In the following we describe each of the investigated click alternatives.

Dwell. Fixating or dwelling on any hyperlink for one second would click that particular link. Visual feedback, using a darkening outline around the hyperlink being gazed at, is provided to indicate the time progression. A state diagram for this click alternative is shown in Fig. 1.

Single Confirm. When dwelling on a hyperlink, a confirm button is activated in the right margin. The user has to look at this button to click the hyperlink. A screenshot of the implementation is shown in Fig. 2 and the state machine in Fig. 3. This design tries to control inadvertent clicking by providing a confirmation step. The confirm button has a darkening border to show time progression during dwell. The initial design had the button appearing right next to the line with the hypertext being looked at, in order to minimize gaze travel. A light grey line extended from the button to the edge of the hyperlink to disambiguate among multiple links present in the same line. This was changed after the pilot, as outlined in Section 4. Multiple Confirm. On dwelling, the confirm buttons of all hyperlinks, in a specified radius around the reported gaze, were activated in the margin. A screenshot of the implementation is shown in Fig. 4 and the state machine in Fig. 5. While avoiding inadvertent clicks, we also expected this alternative to compensate for the inaccuracy and imprecision of gaze trackers, similar to magnification, but in a more natural way. Some changes were made to the user interface after the pilot (outlined in Section 4).

Pointing

Dwell on Link

After 100ms

Dwell on Confirm

Gaze on confirm button

Confirm Buttons Visible After 700ms

Gaze in activated area or margin

After 400ms

Click on link

Gaze on confirm button

Gaze anywhere else

Confirm Button Waning

Gaze not on activated link or margin

Gaze on activated link or margin

Gaze not on link Gaze not on link

Gaze on link

Gaze on same link

Dwell on Confirm Gaze on confirm button

Confirm Buttons Visible After 700ms

Gaze in activated area or margin

After 400ms

Click on link

Gaze on confirm button Gaze anywhere else

Confirm Buttons Waning

Gaze not in activated area or margin Gaze in activated area or margin signed to minimize the gaze travel from the hypertext to the confirm buttons. A screenshot of the implementation is shown in Fig. 6. Instead of appearing in the right margin, the confirm buttons are drawn around the reported gaze point on the web page. Each confirm button is placed in the direction of the corresponding hyperlink from the gaze point. In case the gaze point is near an edge of the browser and drawing a confirm button is not possible in one direction, the button gets drawn in a different direction where space is available. The confirm buttons in this click alternative are connected to the respective hypertext with light grey lines for disambiguation. Mouse. Standard mouse interaction was used as a control condition, while performing the tasks in the same prototype web browser.

Setup

The web browser prototypes were developed in Java. The stereo infrared cameras of a non-invasive remote gaze tracker were mounted below a 15 inch LCD screen running at a resolution of 1280 x 1024 pixels. The gaze direction was determined by the gaze tracker software using the pupil-center-corneal-reflection (PCCR) method. A fully adjustable chair with headrest was used to help maintain the participant's head position, as the maximum freedom of head movement allowed by this gaze tracker was only 2.7 cubic inches. The LCD screen and the gaze tracker cameras were mounted on a movable arm (complete setup shown in Fig. 7). A travel pillow was also used by the participants to help keep their head rested in one position without getting tired. The lab was illuminated with fluorescent lights, and sunlight from windows was blocked during the experiments. The participants were first given an overview of the equipment and the experimental tasks. After responding to the demographics questionnaire, participants were comfortably seated and the chair's height and distance from the gaze tracker were adjusted. The LCD screen and cameras were then adjusted on the movable arm, if needed. The gaze tracker was calibrated, taking 15 to 20 seconds, before starting the tasks for each gaze click alternative.

Task Description

The participants performed the same two navigation tasks using each click alternative (as shown in Fig. 8). Before starting a task, the contents section of the involved Wikipedia pages was collapsed using its 'hide' option and any banner appearing on the top (e.g. for fund raising) was also closed, to maximize the space used for the actual page content. Since Wikipedia content is being updated continuously, specific versions of the pages were used to ensure uniformity. Participants were allowed to familiarize themselves with a gaze click alternative for a few minutes before using it in the tasks.

For each navigation task, a start page was shown and the participants were told to click four hypertext links one after the other. The first navigation task involved clicking four hyperlinks with comparatively few links in the vicinity, while the second navigation task had a higher hyperlink density in the involved pages. The same two navigation tasks were used for training before performing the actual trials.

Participants were allowed to request assistance if they faced difficulty in clicking any hyperlinks. In that case, the experimenter would click the link with the mouse. If an incorrect link was clicked by the participant, the experimenter would use the mouse to go back to the previous page. No scrolling was required for the tasks.

Data Collection

The order of click alternatives was permuted to mitigate order bias and training effects. As the experiment (after the pilot) had three alternatives, six permutations were used. Therefore, a multiple-of-six number of participants were recruited. Events were logged for each web page along with timestamps in CSV log files for each participant's use of each click alternative. The types of events logged included "Gaze on Link," "Button Drawn" and "Click." The data from these log files was analyzed to find the time required for each click, the incorrect links clicked and the mouse clicks done by the experimenter (when assistance was requested by the participant). Six usability questions, derived from the System Usability Scale [START_REF] Brooke | SUS-A Quick and Dirty Usability Scale[END_REF], were answered on a 5-point Likert-scale after using each click alternative. The responses were scored for each participant and averaged to get the usability score of each alternative, with a range from -12 to +12. After completing the experiment, the participants ranked the click alternatives. These ranks were scored (two points for being ranked first, one for being ranked second and zero for being ranked third) and summed up to form a total ranking score for each click alternative. Comments were also solicited in the questionnaire to help explain the rankings.

Pilot Study Results

Five participants took part in the pilot study, one of which had difficulties using the eye tracker and could not complete the experiment. All participants were university students aged 19 to 23. The pilot exposed important shortcomings which are documented in this section. Free Use

Questions

Out of the five click alternatives, Dwell turned out to be very problematic. The first two participants had so much difficulty that it was decided to not evaluate it further. Users found it extremely difficult to hold the gaze on a link for the required 1 second. In case multiple links were present close to each other, an incorrect link would get highlighted or clicked more often than the correct one.

We believed the Radial Confirm alternative to be very promising, but it turned out to have the following problems. The buttons were drawn depending on the reported gaze location. resulted in a different and unpredictable button arrangement every time the buttons were drawn. If the text had four or more hyperlinks in an area, it was difficult to find the right button to click. While looking for the right button, there was a possibility of the buttons disappearing and more buttons being drawn. If the gaze was near the border of the web page, especially in any of the four corners, the buttons ended up appearing in a non-intuitive pattern. This added to the difficulty of finding the right button associated with each hyperlink. Moreover, the buttons and connecting lines occluded the web page content. In short, this design had many usability issues. According to the participants:

 "Radial Confirm is easier to select links as you don't need to look away to select them. However, it is impossible to read other text."  "Radial Confirm was bad as while reading the buttons were easily clicked by mistake."

The Dwell and Radial Confirm were discarded from further experimentation. On the other hand, Single and Multiple Confirm buttons were equally preferred in the pilot. The following are some comments from the participants:

 "Single Confirm is easier to use as there are less chances of making mistakes with it. Multiple Confirm was easiest to use but did have difficulties looking at the link as well as looking for the words shown up on the side."  "Single Confirm was useful, however, having several words on the same line made clicking the button very hard."  "Single Confirm is better than Multiple Confirm as you don't need to look away if the right link has been selected. However, this can be replaced by something that doesn't cover any text (e.g. underline). In Multiple Confirm, need to look away constantly to check if the right link has been selected and also takes time to do so. So it was distracting when trying to read through text."

Many refinements were made in the Single and Multiple Confirm alternatives after conducting the pilot. An underlying objective of these refinements was to minimize confounding factors by making the Single and Multiple Confirm designs more similar to each other. For both alternatives, text was placed outside the buttons to avoid inadvertent clicking and a crosshair was added in the center of buttons to provide a visual anchor, in accordance with earlier findings [START_REF] Penkar | Designing for the Eye: Design Parameters for Dwell in Gaze Interaction[END_REF]. More text was made visible in button labels since some links were not disambiguated easily by only the first five letters. These text labels were placed between the web page and buttons, so that the gaze would not have to travel over the buttons to read the labels. As a result, the con-necting line was removed from the Single Confirm alternative as the text label would already associate the button with a hyperlink. Highlighting of hypertext was also considered but not implemented in order to minimize the design space (e.g. color and type of highlighting) and visual distractions.

The size of the right margin was increased accordingly. This increase in size has the disadvantage of using up more screen real estate, which can be problematic on mobile devices. However, the buttons can be moved off-screen for such devices. The users would then just look at the edge of the device next to the text label to click that link (most gaze trackers can still track the gaze when slightly outside of the screen).

Since the predictability of the button location helps in reducing inadvertent clicks, the Single Confirm button was made to only appear vertically centered in the margin. The Multiple Confirm buttons would always start from the top of the margin. Based on observations, the threshold values were adjusted as well. The first threshold to activate the confirm buttons was reduced to 100ms, while the threshold to click a confirm button was reduced to 400ms. The threshold to deactivate or remove the confirm buttons was increased slightly to 700ms.

Main Study Results

A total of 19 volunteers performed the experiment, out of which 18 (13 men and 5 women) were successful. The age of the participants varied from 20 to 39, including one professional worker, one post-doctoral researcher, two lecturers and 14 students. The participants reported reading English text for between 10 and 80 (averaging 40) hours weekly and using computers for between 2 and 14 (averaging 7.8) hours daily.

Ranking

In the click alternative ranking, eight participants ranked Mouse as the best, five participants ranked Multiple Confirm as the best, while one participant ranked Single Confirm as the best (Fig. 9). The Mouse is the clear (Condorcet) winner; Multiple Confirm is the clear second. Some comments of participants who did not rank Mouse as the best are copied below, with minor grammar corrections:

 "Multiple Confirm is much better than Single Confirm and it's more interesting to use than regular mouse click."  "As I sometime feel RSI, for me this alternative (Multiple Confirm) is good.

Though a more mature solution would be great."  "Multiple Confirm is probably the better one as it gives more options while clicking. Mouse, however, requires using hands which is more physical as compared to eye-tracking single and multiple buttons alternatives."

Score

Usability

Ranking

Out of the 12 participants who ranked Mouse as best, eight indicated that they preferred the mouse over the gaze-tracking alternatives as they had been using mouse for many years and are so well-trained with it. Some comments of such participants are copied below:

 "I only faced occasional issues in using Multiple Confirm buttons. It is unfair to compare Mouse with eye gaze tracking as I am well-trained with the mouse but used eye gaze tracking just once. My response might be different if I had used gaze tracking for longer."  "I found the Multiple Confirm to be better than Single as it was easier to choose between links close by using multiple buttons. Mouse is only the best because it is what I am most used to."  "If I use the mouse too much, I might just end up having CTS in the future. I found Multiple Confirm buttons just right. It is a promising click alternative. I think there's something that can be done about the head movements."

Usability

The usability scores (Fig. 10) confirm the user satisfaction measured by the ranking, i.e. Multiple Confirm is a bit worse than Mouse, and Single Confirm is the worst. As evident from the comments, two participants mentioned Repetitive Strain Injury (RSI) or Carpal Tunnel Syndrome (CTS) as a concern with using the mouse. The equipment's constraints on head movement, which was also mentioned in the responses, likely had a negative effect on the usability of the gaze alternatives.

Performance

Looking at the data from the navigation tasks, Single Confirm does not seem feasible as it was the slowest as well as the most inaccurate (Table 1). While using Single Confirm, 16 participants required clicking with the mouse at least once, as they could not get the required hyperlink activated in the margin. Four users ended up on an incorrect page by mistake. They thought that the button appearing in the margin was for the correct hyperlink and did not read the button label to validate this. The more difficult navigation task of the two had more problems due to the higher hyperlink density. Single Confirm did not take a lot longer than Multiple Confirm, but it did require many 'mouse clicks' and suffered from inadvertent clicking. Both Multiple Confirm and Mouse were fully accurate and did not result in any incorrect click.

There was only one instance with Multiple Confirm where the participant asked for assistance and the link was clicked with the mouse. Pairwise testing of the average time to click a link indicates that the differences between all click alternatives are significant at the 1% level, using the Wilcoxon Signed-Rank test.

The following are some observations from the experiment:

1. Some participants were impressed and excited to use eye gaze tracking with comments like "that's pretty cool," "wow" and "that's so quick". This ef-fect was most prominent when Multiple Confirm was being tested after the Mouse but before the Single Confirm alternative. 2. Single Confirm seemed to be easier to learn and easier to use than Multiple Confirm as it did not require choosing from multiple buttons (less cognitive load). But due to inherent inaccuracy, it was the most irritating and errorprone. 3. Multiple Confirm was easiest to learn and use when it was the last click alternative tested. This learning effect indicates that users can get better at using this click alternative with more practice. This effect was not noted with Single Confirm, probably because it was already relatively easy to learn.

Discussion

The time taken to click during the navigation tasks shows that Multiple Confirm was slower than the Mouse, but the users did not report that as a shortcoming. One user did mention that the speed of the computer was slow, which was evident by the time taken to load webpages (about 5 to 7 seconds depending on page size) in all click alternatives. Single Confirm is problematic and not feasible due to the inherent gaze tracker inaccuracy as well as eye jitter. It was difficult for users to activate the confirm button by looking at a particular link. The problems are similar to those of the "Dwell" click alternative, in which users had difficulty holding the gaze on the correct hyperlink.

Multiple Confirm, as expected, was not as fast as the mouse. But contrary to expectations, not all participants found the mouse to be the best. Even some of those who judged it best believed that with some more practice they would likely change their ranking. The Multiple Confirm buttons were able to solve the two problems associated with gaze tracking, namely inaccuracy and inadvertent clicking. It should be noted that conceptually the Multiple Confirm buttons are performing the same function as magnification techniques, without any visual magnification. The small size of hyperlinks makes it difficult to click with gaze tracking due to the inherent lack of precision, equipment error and involuntary eye movements. Multiple Confirm buttons seem to be able to compensate for these difficulties.

In order to better understand the difference in the average times to click for Mouse and Multiple Confirm, further analysis was performed by looking at the frequency distributions and cumulative percentages of the times to click (Fig. 11 & Fig. 12). The analysis shows that most of the links were clicked within 2 seconds with the mouse and within 6 seconds using the Multiple Confirm buttons. Although not as good as the performance of the mouse, Multiple Confirm worked reasonably well. With further research and better equipment (e.g. allowing for more head movement), we expect that the performance of Multiple Confirm can be improved further.

Multiple Confirm involves two steps for each click: first, looking at the hyperlink; then, activating the correct confirm button. Consequently, we expected Multiple Confirm to be slower than the mouse. However, the time difference is bigger than we expected. It can be explained by considering the following additional factors:  Evaluating the labels next to the confirm buttons to find the right button took some time.  Sometimes the desired button was not among the buttons shown in the margin, and the user had to look back at the hyperlink to activate the confirm buttons again.  Activating the confirm buttons again delayed the clicking time further, due to the time thresholds for removing the previously activated buttons and activating the buttons again.  Occasionally, users had difficulties triggering a confirm button by dwelling on it, especially when the head was moved slightly outside the optimal tracking volume. Some users quickly moved their heads a bit to get the accuracy back, and in some cases this did not help or even decreased the accuracy.  During page loading, some participants already moved the mouse near the expected location of the next hyperlink, before the page was loaded. This somewhat obfuscated the speed advantage of the gaze over the mouse.

Limitations

One limitation was the time it took for the web browser to load and display web pages after a click was performed. These page load times varied from 3 to 10 seconds, depending on the amount of text and the number of images on the web page. Since the log file had only recorded the times between the clicks including the page load times, the page load times were subtracted from the click times later for analysis. The load time for each page was fairly constant, so this did not affect the precision of our measurements much. However, as mentioned before, the page load times favored the mouse as participants had time to move the mouse to the expected areas of the screen before the page was fully loaded. The other major limitation was the restriction on head movement. Not only was this restriction unrealistic, it also added to the inaccuracy of the gaze click alternatives. The navigation task had to be restarted for a few participants as their gaze was not being tracked anymore after they had moved their heads more than the equipment allowed.

The findings cannot be generalized to all users as most participants had similar demographics. Additionally, only two participants had used an eye tracker before, in another experiment. Finally, it is not unreasonable to assume that there was a bias in favor of the gaze tracking alternatives due to the novelty factor.

Conclusion

An important finding is that eye gaze tracking has the potential to be used as an input method for web browsing. The inherent inaccuracy due to eye jitter and equipment error, lack of precision and inadvertent clicking can be compensated for by using the Multiple Confirm click alternative. The confirm buttons should have visual anchors, and button labels (text) should be outside the buttons, preferably between the web page and the buttons. Such a user interface was easy to learn and use, and was preferred by some users over the mouse despite using it only for a few minutes. The biggest challenge was the limited freedom of head movement afforded by the equipment used. Even slight head movement could cause the gaze error to increase and affect the performance. As newer equipment allows more freedom of movement, the usability of the Multiple Confirm click alternative is expected to improve.

Further research is necessary to evaluate if hyperlinks can be highlighted in a nondistracting manner as visual feedback to efficiently identify the hyperlinks that have activated confirm buttons. Underlining the hypertext seems to be a feasible option. The order and location of confirm buttons might also be improved, e.g. to have the multiple confirm buttons centered vertically. There might also be a potential of using 'grab and hold' [START_REF] Miniotas | An Algorithm to Counteract Eye Jitter in Gaze-Controlled Interfaces[END_REF] to compensate for eye jitter and equipment error when clicking a confirm button.

There is also room for improvement in other design parameters. The inactivation, or removal, of confirm buttons (currently after 700ms of looking at other text) could be optimized. The time thresholds to activate and click the confirm buttons (currently 400ms) and the boundary radius of the gaze area from which buttons are activated (currently 30 pixels) may be fine-tuned further. The way confirm buttons are currently drawn in the margin is a potential distraction, which could be minimized in many ways, e.g. by keeping all the buttons visible, possibly with reduced contrast. The labels would only appear and the button would be made fully visible when a confirm button is activated. The labels may also be faded in and out, instead of appearing and disappearing abruptly.

Fig. 1 .

 1 Fig. 1. State machine for the Dwell click alternative

Fig. 2 .Fig. 3 .

 23 Fig. 2. Single Confirm: User navigating to "UNESCO World Heritage Site"

Fig. 4 .Fig. 5 .

 45 Fig. 4. Multiple Confirm: User navigating to "psychological"

Fig. 6 .

 6 Fig. 6. Radial Confirm: User navigating to "Philae temples"

Fig. 7 .

 7 Fig. 7. Gaze tracking setup

Fig. 8 .

 8 Fig. 8. Experimental procedure for each participant

Fig. 9 .Fig. 10 .

 910 Fig. 9. Histogram of click alternative rankings

Fig. 11 .Fig. 12 .

 1112 Fig. 11. Frequency distribution of time to click using Mouse

Table 1 .

 1 Performance of all click alternatives

		Mouse	Single Confirm	Multiple Confirm
	Average time to click a link (sec)	2.1	10.3	7.4
	Number of incorrect links clicked	0	4	0
	Number of times clicked with mouse	n/a	26	1

Acknowledgement

We would like to acknowledge the assistance of Yoonseok Hong and Gen Li in designing the prototypes and performing the pilot study, and the voluntary participation of all users without which this study would not have been possible.


Jayson Turner
email: j.turner@lancaster.ac.uk

Jason Alexander
email: j.alexander@lancaster.ac.uk

Andreas Bulling
email: andreas.bulling@acm.org

Dominik Schmidt
email: dominik.schmidt@hpi.uni-potsdam.de

Hans Gellersen

Eye Pull, Eye Push: Moving Objects between Large Screens and Personal Devices with Gaze & Touch

Keywords: Eye-Based Interaction, Mobile, Cross-Device, Content Transfer, Interaction Techniques

Previous work has validated the eyes and mobile input as a viable approach for pointing at, and selecting out of reach objects. This work presents Eye Pull, Eye Push, a novel interaction concept for content transfer between public and personal devices using gaze and touch. We present three techniques that enable this interaction: Eye Cut & Paste, Eye Drag & Drop, and Eye Summon & Cast. We outline and discuss several scenarios in which these techniques can be used. In a user study we found that participants responded well to the visual feedback provided by Eye Drag & Drop during object movement. In contrast, we found that although Eye Summon & Cast significantly improved performance, participants had difficulty coordinating their hands and eyes during interaction.

Introduction

We are surrounded by out-of-reach digital information. Our private TVs and public shared displays often present URLs, physical addresses, phone numbers, route descriptions, and other information that we wish to 'pull' to our personal devices. Equally, we often wish to add personal content to notices, discussions, presentations and collections on shared screens. Yet we lack fluid mechanisms for moving content between public and personal displays.

We present Eye Pull, Eye Push, a novel interaction concept that allows for the acquisition (pulling) and publication (pushing) of content between personal and remote devices. Using a combination of gaze and touch it is possible to define techniques that enable this interaction style. Gaze is a natural modality choice for selecting objects that catch our visual attention, while touch actions can be performed on personal devices without visual attention. Related work has employed handheld device input, combined with gaze interaction, to assist panning and zooming [START_REF] Stellmach | Designing gaze-supported multimodal interactions for the exploration of large image collections[END_REF] and target acquisition [START_REF] Stellmach | Look & touch: Gaze-supported target acquisition[END_REF] on desktop displays. Our work is distinct in demonstrating the gazesupported transfer of objects across devices.

Fig. 1. Eye Pull, Eye Push: users pull and push objects between remote screens and their personal devices with a combination of gaze and touch. In this scenario, the user selects a form on a public service terminal simply by looking it, retrieves it to their touch device with a swipe, fills it in, and returns it with a swipe while looking up at the terminal.

Figure 1 illustrates our vision: a user selects an object on a public display and, while still visually fixating on it, swipes on their handheld personal device to pull the object down for editing. Once editing is complete, the user re-fixates on the remote target, and returns the object with a further touch gesture. This style of interaction would benefit many contexts of use: group collaboration, classrooms, and public community displays [START_REF] Greenberg | PDAs and shared public displays: Making personal information public, and public information personal[END_REF][START_REF] Myers | Using handhelds and PCs together[END_REF][START_REF] Carter | Digital graffiti: public annotation of multimedia content[END_REF]; our homes for lazy interaction between the TV screen and mobile devices; public terminals that we may find too exposed or too grimy for direct data entry; and anywhere that digital objects exist, that users would like to edit but cannot reach.

This paper makes a two-fold contribution. First, we introduce Eye Pull, Eye Push, a concept for multimodal cross-device content transfer. We define the required input attributes for such interaction and explore application scenarios where it makes a compelling impact.

Second, we define three novel techniques for the transfer of objects between remote screens and personal touch devices, each combines gaze and touch: Eye Cut & Paste (ECP): Objects are cut and pasted using gaze and touch tap events. Eye Drag & Drop (EDD): Objects are moved using gaze and touch hold/release events. Eye Summon & Cast (ESC): Objects are pulled using gaze and a swipe down action, and pushed using gaze and a swipe up action. All three techniques were implemented using a portable eye tracker extended for wider field of view [START_REF] Turner | Extending the visual field of a head-mounted eye tracker for pervasive eye-based interaction[END_REF]. We evaluated these techniques in a user study to understand their strengths and weaknesses in performance and usability. The results demonstrate that users are able to transfer content efficiently using our techniques, thus validating our approach. ECP and EDD performed similarly, with EDD being preferred due to the continuous visual feedback provided by drag-and-drop. ESC was the fastest of the techniques but was rejected by users due to the more complex hand-eye coordination required.

2

Related Work

Cross-Device Information Transfer

The case for moving objects easily between handheld devices and larger screens has been made widely, for group work settings [START_REF] Greenberg | PDAs and shared public displays: Making personal information public, and public information personal[END_REF][START_REF] Myers | Using handhelds and PCs together[END_REF] as well as serendipitous encounters with public displays [START_REF] Carter | Digital graffiti: public annotation of multimedia content[END_REF][START_REF] Aliakseyeu | Bubble radar: efficient penbased interaction[END_REF]. Several works have focused on pushing and pulling content using touch-surfaces as proxies to public displays. Touch Projector [START_REF] Boring | Touch projector: mobile interaction through video[END_REF] demonstrated improvements to work by Tani et al. that enabled the control of remote machinery through live video feeds while maintaining spatial context [START_REF] Tani | Object-oriented video: Interaction with real-world objects through live video[END_REF]. Boring's work made use of a phone camera feed to project touches on to public displays to manipulate and move objects. Similarly, Bubble Radar showed how users could interact at a distance using the representation of a public display on a tablet PC [START_REF] Aliakseyeu | Bubble radar: efficient penbased interaction[END_REF].

Bragdon et al. developed Code-space, a set of techniques focused on interactions between mobile and situated devices in developer meetings [START_REF] Bragdon | Code space: touch + air gesture hybrid interactions for supporting developer meetings[END_REF]. Their work utilised situated depth cameras and inertial sensors embedded in mobile devices to enable intuitive pointing and information transfer for collaboration. Earlier work by Rekimoto et al. entitled Pick-and-Drop [START_REF] Rekimoto | Pick-and-Drop: A direct manipulation technique for multiple computer environments[END_REF] has shown how physical objects can be used to transfer content from one display to another. In this case a pen was used to represent a faux storage device that could pick and drop content.

Several techniques have explored obtaining content at a distance within a single large display. Baudisch et al. investigated different techniques for dragging and dropping objects [START_REF] Baudisch | Drag-and-pop and drag-and-pick: Techniques for accessing remote screen content on touch-and pen-operated systems[END_REF]. Their Drag-and-Pop and Drag-and-Pick techniques used proxies of distant icons to effectively bring them closer to a user. Drop-and-Drag by Doeweling et al. [START_REF] Doeweling | Drop-and-drag: easier drag & drop on large touchscreen displays[END_REF] was a technique similar to traditional drag and drop technique that allowed for interaction to be suspended mid-transfer, thus allowing the user to perform finegrained navigation before dropping an object. The above techniques were all found to be faster than traditional drag and drop for sufficiently distant targets. Finally, Schmidt et al. [START_REF] Schmidt | A cross-device interaction style for mobiles and surfaces[END_REF] described a range of interactions made available by combining a mobile phone with a multi-touch surface. Their techniques allow for fluid content transfer, personalisation of the surface and access-control over publicly visible elements.

Gaze Pointing

Early work on eye-based interaction showed that the eyes could be used as input in desktop environments. However an issue coined by Jacob et al. known as the Midas Touch Problem causes unwanted interactions when trying to explicitly issue commands [START_REF] Jacob | What you look at is what you get: eye movement-based interaction techniques[END_REF]. Dwell-time overcomes this problem by allowing a user to fixate on a control for a set delay before activation occurs. Studies by Jacob et al showed that the delay incurred by dwell-time could be overridden, by using manual input to activate controls interaction can be sped up.

Prior to Jacob et al., Ware et al. [START_REF] Ware | An evaluation of an eye tracker as a device for computer input[END_REF] examined three picking techniques that used gaze combined with dwell, a virtual button and a hardware button for selection. Their experiments found that confirmation via a hardware button was fastest. It was also found that users would attempt to synchronise their eye movement with hardware button presses, causing occasional selection errors as the eyes move away before selection is confirmed. A fully developed alternative to mouse input using gaze and keyboard commands was demonstrated by Kumar et al. [START_REF] Kumar | Eyepoint: practical pointing and selection using gaze and keyboard[END_REF].

Further studies have evaluated gaze as an assistive modality for manual input. Zhai et al. [START_REF] Zhai | Manual and gaze input cascaded (magic) pointing[END_REF] developed MAGIC pointing. In their paper they designed two techniques that combined gaze with mouse input: liberal, the mouse cursor is warped to objects being looked at, the final selection is performed by the mouse, conservative, the mouse cursor is only warped after the user moves the mouse. Their experiment found that users subjectively felt they could interact faster with MAGIC techniques. Their liberal technique was faster than manual input and their conservative technique was slower. Drewes et al. [START_REF] Drewes | The magic touch: Combining magic-pointing with a touchsensitive mouse[END_REF] followed up on this experiment by combining gaze with a touch enabled mouse. They found that warping the cursor based on when the mouse was touched, as opposed to moved, reduced the need for mouse repositioning, thus improving the overall speed. Bieg et al. [START_REF] Bieg | Eye and pointer coordination in search and selection tasks[END_REF] showed however that MAGIC pointing offered no performance boost over mouse only input when used on large displays.

Multi-Modal Gaze Interaction with Public Displays

Gaze-based and gaze-supported interactions with public displays are concepts already explored in the literature. Mardenbegi et al. demonstrated the use of head gestures in combination with gaze to interact with applications on a public display [START_REF] Mardanbegi | Eye-based head gestures[END_REF]. This work followed the same principles as in the previously described work on pointing.

Gaze is used to point, and an additional modality is used to issue commands. Stellmach et al. evaluated techniques in several works that combine gaze and mobile input, i.e., inertial sensing and touch [START_REF] Stellmach | Designing gaze-supported multimodal interactions for the exploration of large image collections[END_REF][START_REF] Stellmach | Look & touch: Gaze-supported target acquisition[END_REF]. Their work developed techniques to navigate large image collections on public displays. Techniques used gaze for pointing while touch and accelerometer values were used to pan and zoom through images [START_REF] Stellmach | Designing gaze-supported multimodal interactions for the exploration of large image collections[END_REF]. Users perceived increased effort and complexity when panning and zooming. This was considered as acceptable however, as it allowed for simultaneous interactions not usually possible with gaze alone. In a later work they combined gaze with touch commands. This work defined five techniques for the remote selection of varying sized targets in a desktop setting [START_REF] Stellmach | Look & touch: Gaze-supported target acquisition[END_REF]. Their findings gave rise to one technique in particular, MAGIC Tab, which allowed users to tab through a series of objects within close proximity to a users gaze, thus overcoming eye tracking accuracy issues. In further work Stellmach et al. evaluated techniques that utilise a combination of eye and head directed pointing with touch interaction for selection and manipulation of distant objects [START_REF] Stellmach | Still Looking: Investigating Seamless Gaze-supported Selection, Positioning, and Manipulation of Distant Targets[END_REF]. Their results highlighted that further improvements are required to allow for more precise distant cursor control with large displays.

Summary

The literature demonstrates success both using multimodal eye-based interactions for remote target acquisition and using touch-based proxies for distant content interaction. Our work joins these areas by using gaze and touch to pull and push objects between public and close proximity devices.

Eye Pull, Eye Push

Here we describe the concept of Eye Pull, Eye Push. 'Pulling' refers to moving content from a public context to a personal one. 'Pushing' refers to the opposite of this, moving from personal to public. The overall concept presents an interaction style whereby these tasks can be completed using a combination of gaze and touch. Below we outline three techniques designed to pull and push objects. We define the stages of interaction required to transfer content between personal and public displays, and explain how each of our techniques provides the required input attributes for each stage.

Input and Interaction Flow

The transfer of an object between a public display and a personal device can be broken down in to four main steps: object location, confirmation of selection, destination location, and confirmation of drop. Each of these requires two attributes to be fulfilled: Locate (the location of the object or target) and Confirm (an action to confirm the location). The three techniques we propose combine gaze and touch actions in different ways. They are able to execute the outlined main steps and fulfil their attributes.

Each technique uses one of three touch commands: Tap, Hold/Release and Swipe and each is performed with a single finger. Tap combines two touch events, touch down and touch up, performed in quick succession. Hold/Release also combines touch down and touch up but they are used in considerably slower succession to confirm actions. Swipe combines touch down, touch moved and touch up, each must be performed in quick succession for the gesture to be recognised. The mappings of touch and gaze for each technique are shown in Table 1. To pull content, the user looks at an object, they then tap on their tablet to select and cut the object from view. A 'paste' is then performed by looking at the target device and a second tap inserts the object at the gaze location. To push content from a personal display, the same steps can be used, i.e., look at an object on the tablet, tap to select, look at the public display and tap again to drop. Alternate semantics are possible for this technique, for example, once an object is cut, many copies can be pasted to a destination.

Eye Drag & Drop. Our second technique, Eye Drag & Drop is likewise inspired by its desktop equivalent. Figure 3 shows, to pull content, an object is located by gaze and selected by a hold gesture. The object follows a users gaze for as long as they maintain holding with touch. As a user's gaze trajectory intersects the personal device, the object appears on the display. Once touch is released, the object is dropped. Similarly to Eye Cut & Paste, the steps of this technique can also be used to push content, i.e., look at an object on the tablet, hold touch, look at the public display and release touch. Eye Summon & Cast. Our final technique Eye Summon & Cast is based on a combination of gaze with a swipe gesture (see Figure 4). Unlike our other techniques, Eye Summon & Cast uses two differing methods (summon and cast) to pull and push content. An object on the remote screen can be located by gaze, and then summoned with a swipe down on the touch device. The swipe serves to confirm the object selection and simultaneously identifies the destination position on the target touch device. A cast is performed similarly: gaze now selects the destination, and a swipe up identifies the object to be transferred and implicitly confirms selection and drop. Different semantics are possible for implicit identification, e.g., selecting the most recently 'pulled' object to be pushed back.

Application Scenarios

In the section we describe six application scenarios that demonstrate the versatility of Eye Pull, Eye Push. Each of our three techniques has been designed to complete the tasks, pull and push. As the flow of interaction differs between techniques, each can also be used for specialised tasks. Here we consider how each technique could be used in real-world scenarios to pull and/or push content. The advantages of this technique can be leveraged when pulling content. Traditional cut and paste allows for objects to be selected and temporarily stored on the clipboard. This allows for two further interactions, first it frees the user to perform other (usually navigation) tasks and second it allows for the duplication of content.

As an example, shown in Figure 5a: A user is typing up a document on a tablet pc in a café, a display above the café counter advertises weekly events. The user looks up at the display and notices a digital flyer about a music night at the café. To acquire a copy of this flyer, while still looking, the user taps on their personal device, the content is then held on the clipboard. Now, the user navigates to their calendar application, looking, taps to paste in the flyer and sets a reminder. Next the user switches to their social networking application and pastes in a second copy of the flyer to share with their friends.

Compared to our other techniques, Eye Cut & Paste is specialised to scenarios such as this, where interaction is required mid-transfer to allow content to be used for different purposes.

Duplicating For Many Users. As shown in the previous example, Eye Cut & Paste can be used for the duplication of content that has been 'cut'. The following example demonstrates how this can be leveraged when pushing content.

A user has cut and pasted a single photograph to a television to show to two other users. The users all like the picture and so want to obtain their own copies. The user performs the paste stage of the technique twice more to create additional copies on the television for the friends to pull to their own devices (see Figure 5b). Digital Form Filling. Eye Drag & Drop is suited to tasks where changing context is part of the natural flow of interaction, where transfer is performed in a slow and continuous manner.

Figure 6a shows an art gallery, where paintings are displayed along a wall. Next to each art piece is a digital comments display containing the thoughts of gallery patrons and empty comment cards. To leave a comment, a user looks at an empty comment card and pulls it to their tablet. This is performed following the steps of Eye Drag & Drop. The user then fills in the card with their thoughts. The card is then pushed back to the comments display by the same method.

As Eye Drag & Drop provides continuous visual feedback, content can be seen to visibly move as it follows a user's gaze. This allows the interaction to become analogous to physical tasks such as filling in and posting comment cards.

Sharing Read-only Content. Users do not always want others to be able to obtain the content they share. Figure 6b shows how Eye Drag & Drop can be used to share in a read-only manner by maintaining control over content as it is displayed.

This technique allows a user to switch back and forth between large and personal display contexts in a steady and continuous manner: A user is in a meeting; they want to show a relevant image temporarily on a projected display without disturbing the current content. First they look at an image on their personal device and perform a touch hold, this attaches the image to the location of their gaze. The user then looks up at the larger display to show the picture, as they maintain holding their touch, the object does not drop. The user then reverts their eyes back to their personal device, removing the image from the large display. They then release their touch to drop the object. Eye Summon & Cast requires the eyes to identify a distant object, drop location is then defined by touch. This mechanism allows for the user to acquire an object without changing their visual context. This allows for the quick acquisition of many objects in sequence while on the go. Figure 7a demonstrates an example: a user has arrived in a busy train station and on the platform is a local information display. The display contains a wealth of tourist centric information on the local area. The user spots a train departure table, a local taxi number and a local map. The user swipes on their mobile device to grab each item in sequence as they pass by the display without having to change context.

Sharing Content.

When browsing media on a personal device, users often want to share their experience with a large group on a bigger display. Figure 7b shows how Eye Summon & Cast can be used to allow for fluid interaction in this scenario: while browsing content, the user holds their finger on an image they wish to share. Now, looking at a larger display, the user can swipe upwards on their personal device to transfer the image for viewing. This interaction allows for a simple and natural method of choosing a public display, in particular in environments where more than one large display may exist.

User Study

In a user study we aimed to compare our three techniques to evaluate usability to understand which was better suited to each task and to users. We analysed performance and usability measures that were recorded as users pull and push a single object between a large display and a mounted tablet device. We recruited 12 paid participants (11 male, 1 female, aged 22 to 41 (M = 25.4 S.D. = 5.1)), all had normal or corrected vision, and one was "colour-blind" but was able to distinguish the colours used in the experiment. Participants stood 150 cm from a 50" plasma display (whose base was 1 m from the floor). A tablet was mounted on a tripod at waist height. This decision was made to ensure eye-tracking accuracy remained constant throughout trials. This prevented parallax error that is inherent in monocular eye-tracking.

Participants and Apparatus

Participants wore a custom eye tracker that was calibrated with each participant at the beginning of the study. The eye tracker is based on SMI's iView X HED system but utilises an additional scene camera to detect personal device screens at close proximity using brightness thresholding with contour detection (see Figure 8) [START_REF] Turner | Extending the visual field of a head-mounted eye tracker for pervasive eye-based interaction[END_REF]. Contours were minimised to four points representing the rectangular surface of each screen. Gaze was then mapped to this rectangle using a perspective transformation to convert scene camera coordinates to on-screen coordinates. Although the system did not use the commercial software provided, the system was accurate to within 1.5 degrees of visual angle, we found this to be sufficient accuracy for the target sizes used in this study. To compensate for parallax error, the system was calibrated twice, once for the public display and once for the tablet. The system switched between calibrations depending on which screen was in view.

Experimental Design and Procedure

The study followed a within-subjects repeated-measures design with two independent variables, technique, with three levels (1) Eye Cut & Paste (ECP), (2) Eye Drag & Drop (EDD), (3) Eye Summon & Cast (ESC) and task, with two levels (1) Pull, (2) Push. The dependent variables were task completion time and error rate. Users were asked to pull and push single objects between displays, this equated to one trial of the experiment.

For each technique participants performed 30 trials: one guided training, five practice, and 24 recorded trials. To begin a trial, participants fixated at a 175 px green circle on the public display and were asked to tap on the tablet. A red target would then appear on the public display. Targets had varying origins but were all located equidistant from the centre of the start point. This was to minimise anticipation when locating the next object. Participants pulled and dropped the object at arbitrary locations on the tablet. Upon dropping the object, its colour changed after a 5 sec delay to blue, prompting the participant to begin the push stage of the task. When pushing, the object had to be dropped within a target area double the size of the object (350 px in diameter) and in the same position from which it had been originally pulled. This was to ensure participants could complete the experiment without introducing a time penalty. All participants used the three techniques (order counterbalanced using a Latin square) and performed all trials with one technique before moving to the next. After completing all tasks with a particular technique, participants provided subjective feedback, including questions from the NASA Task Load Index (NASA-TLX). A final questionnaire gathered preference, task suitability, and general feedback.

All touch and gaze events, task completion times and errors were automatically logged. An error was logged under conditions where, selection failed on the first attempt, an object was dropped out of bounds of a target or an object was dropped out of bounds of a display.

Results

6.1

Task Completion Time Participants completed a total of 864 (24 trials x 3 techniques x 12 participnats) trials.

Figure 9 shows the mean completion times for each task. We compared these values in a 2 x 3 (task x technique) two-way repeated-measures ANOVA with Greenhouse Geisser correction. An interaction effect was found (F 1.721,18.933 =5.178, p=.020). Further tests using a one-way repeated measures ANOVA with Greenhouse Geisser correction showed a significant difference for the pull task in completion time between the three techniques (F 1.992, 21.137 =33.812, p<.0005). Further paired t-tests (Bonferroni corrected, new p-value=0.0083) showed that ESC was significantly faster than EDD (p<.0005) and ECP (p<.0005). ECP and EDD were not found to be significantly different (p=1.000).

For the push task, a significant difference was found across completion time (F 1.704,

6.2

Error Rate Fig. 10. Mean error rates, confidence levels omitted for clarity.

The mean error rates for each technique are shown in Figure 10. In a 2 x 3 (task x technique) two-way repeated measures ANOVA with Greenhouse Geisser correction, we found no significant interaction or main effects. The means are calculated from 288 trials per technique per task. ECP showed a mean error rate of 1.58 for pulling and 1.66 for pushing. EDD had a higher mean error for pushing than pulling, (2.25 and 0.83 respectively). ESC had a slightly higher mean error rate for pushing (0.83) than pulling (1.25) also.

Performance Perception

We recorded participant responses on a 7-point likert scale to questions regarding perceived speed, accuracy, ease of learning, suitability to task and preference. Friedman tests showed no significant differences in perceived speed for pulling or pushing objects, overall speed, accuracy or ease of learning. Participants were asked questions for each technique relating to their suitability and preference for the two tasks. No one technique was significantly suited to or preferred for pulling objects. For pushing objects (X 2 (2)=9.500, P<=.009) ESC was significantly less preferred than EDD (Z=-2.756, P<.006) with no significant difference between other techniques and EDD. Mean responses from NASA-TLX worksheets on a scale of 1-100 are documented in Figure 11. There were no significant differences for any factor. Overall, no one technique was significantly preferred.

Subjective Feedback

Participants provided subjective comments on the techniques they had just used. Participants commented on the perceived slowness of ECP: It felt slow because it felt like I had to do twice as many actions and it required a lot of tapping. Several participants noted the techniques' similarity to its desktop counterpart saying It's similar to copy and paste. In comparison to ESC one participant said I preferred that I didn't have to switch between selection techniques, I was always using my eyes referring to varying swipe events used in ESC.

Participants perceived EDD to offer more control, stating I felt I had more control moving objects and that the continuous feel of contact with the object was something that the other techniques lacked. The sense of control also affected perceived speed and accuracy, it felt slow, but it was definitely much more accurate because I could see the object in place before dropping it. Similarly to ECP one participant found EDD similar to current desktop techniques saying, It's just like moving windows around in an operating system.

ESC was found to be difficult for participants: [it was] much harder than other techniques and I didn't know where to look. One participant found during the push task that it was frustrating that I had to look down to find the object, just out of peripheral vision. Finally the variations of swipe to perform summoning and casting were found to be confusing with participants saying, I didn't really like ESC because it had the addition of swiping in either direction.

Discussion

Results

Overall ESC was found to be the fastest but least preferred technique. Participants disliked ESC for two main reasons: (1) Confusion, the touch command used changed between swipe down and swipe up, this lead to confusion about which to use for each task. (2) Coordination, participants stated that they found it difficult to coordinate their hands and eyes. This result highlights an issue where eye-based input needs to correlate more naturally with a users need to use their eyes, to observe other actions they perform. This issue is specific to the requirements of pushing with ESC. The user must swipe up on a tablet-located object viewed in peripheral vision while simultaneously being required to fixate on a large display. A possible solution for this in further work would remove the need for simultaneous initial selection and targeting, and instead allow these to be performed in sequence, i.e., hold finger on tablet object to select, then look at large display, and finally perform a swipe up to transfer the object.

Participants responded well to EDD. In comparison to ECP, participants felt that being able to see the object moving gave them a greater sense of control. Although the system used a gaze cursor to provide continuous feedback to the user, it is clear that in EDD, this feedback is more obvious and familiar to users thus provoking a positive response. As demonstrated in the example scenarios we outlined in section 4, it is possible to incorporate additional semantics in to our techniques. These can improve usability in more complicated scenarios. Users reported ECP felt slow due to the amount of tap commands required. To improve perceived speed, the paste behaviour can be leveraged in this technique to duplicate a selected object, thus reducing the need for context switching and quicker perception of transfer.

Furthermore, issues outlined above with ESC can be resolved by introducing an 'implicit object identification' semantic. In this case, the most recently pulled object would be pushed automatically, thereby removing the need to redirect visual attention to the touch modality.

Feasibility and Limitations

Eye Pull, Eye Push is dependant on the deployment of eye tracking as a pervasive technology. To realise such a vision there are several requirements and limitations: (1) Embedded or head-worn eye-tracking: it is imperative that users are always visible to the system. Current technology supports both, remote eye-tracking, where systems are embedded or situated below displays, and head-worn eye-tracking where users wear a personal eye-tracker. These are currently in the form of goggles but envisioned to become as small as standard glasses. (2) Calibration: current head-worn and remote eye-trackers require calibration before use. Calibration takes time and must be performed pre-interaction. More modern systems only require calibration that lasts less that 30 seconds but issues can still arise when interacting with displays at varying distances, this is due to a lack of robust parallax compensation. (3) Connection: users require a method to pair with displays as they interact. Do users implicitly pair with each display they look at? Are user's eye-tracking data globally broadcast for use? Or would authentication be required? To create seamless interaction, there would need to be a balance between privacy and functionality so that users are not inhibited by repeated authentication.

Conclusion

In this paper we presented a novel interaction concept Eye Pull, Eye Push, gazesupported cross-device content transfer. In our design we considered transfer between public and personal devices and how gaze and touch can be combined to create interaction techniques for this task. We outlined the following techniques: Eye Cut & Paste, Eye Drag & Drop, and Eye Summon & Cast. We presented and discussed several usage scenarios for these techniques.

Our three techniques were evaluated in a user study. Users were able to complete the basic tasks of pull and push, and responded most positively to our Eye Drag & Drop technique. The results of our user study showed that Eye Summon & Cast outperformed Eye Cut & Paste and Eye Drag & Drop in terms of speed but was least preferred by users due to its hand-eye coordination requirements. Eye Cut & Paste and Eye Drag & Drop performed similarly in terms of speed although Eye Drag & Drop was preferred due the more apparent continuous visual feedback it provided. In our discussion we outlined how additional semantics can be applied to each technique to extend functionality in differing scenarios. Furthermore we discussed the feasibility and limitations of Eye Pull, Eye Push in the real world.

In future work we aim to explore this design space further, to gain a full understanding of factors within it and the implications they have on this style of interaction, i.e., users proximity to content, display sizes and varying content-types.

Fig. 2 .

 2 Fig. 2. Eye Cut & Paste: 1) Look at object, 2) Tap on tablet, 3) Object is selected and cut from view, 4) Look at tablet, 5) Second tap on tablet, 6) Object is dropped.

Fig. 3 .

 3 Fig. 3. Eye Drag & Drop: 1) Look at object, 2) Hold touch on tablet, 3) Object is selected and can be visibly moved, 4) Look at tablet, 5) Release touch from tablet, 6) Object is dropped.

Fig. 4 .

 4 Fig. 4. Eye Summon & Cast. To summon: 1) Look at object. 2) Swipe down on tablet, 3) Object is moved to swipe location, 4) Object is dropped. To cast: 5) Look at destination, 6) Swipe up on object, 7) Object is moved to location of gaze, 8) Object is dropped.

Fig. 5 .

 5 Fig. 5. (a) Mid-Transfer Interaction: A user pulls a flyer, they then switch to a suitable application before tapping to drop it. (b) Duplicating For Many Users: A user pushes three copies of an image using Eye Cut & Paste, two friends now have copies they can pull to keep.

Fig. 6 .

 6 Fig. 6. (a) Digital Form Filling: A user shares their thoughts about artwork on a virtual comments board by pulling, completing and pushing a comment card. (b) Sharing Read-only Content: A user pushes and pulls an image for temporary viewing in a meeting.

Fig. 7 .

 7 Fig. 7. (a) On-the-go: A user acquires many objects in quick succession at train a station. (b) Sharing Content: A user shares content to a display for viewing by a group of people.

Fig. 8 .

 8 Fig. 8. System Setup: (a) Dual scene camera eye tracking system, (1) Additional scene camera (2) Eye camera. (b)(1) The system setup with head-mounted eye-tracker, (2) Touch tablet mounted on tripod (3) and Plasma TV.

Fig. 9 .

 9 Fig. 9. Mean task completion time in seconds with 95% confidence intervals (CI).

Fig. 11 .

 11 Fig. 11. NASA Task Load Index, scale 0-100, confidence levels omitted for clarity. Key: (ECP) Eye Cut & Paste, (EDD) Eye Drag & Drop, (ESC) Eye Summon & Cast.

Table 1 .

 1 Mapping of gaze and touch input to locate objects and confirm actions. Eye Summon & Cast is split in to two rows for clarity. Eye Summon and Eye Cast each involve a single swipe gesture (down or up) combined with gaze.

		Object Selection	Destination Selection
		Locate	Confirm	Locate	Confirm
	Eye Cut & Paste	Gaze	Tap	Gaze	Tap
	Eye Drag & Drop	Gaze	Hold	Gaze	Release
	Eye Summon	Gaze	Swipe	Swipe	Swipe
	Eye Cast	Swipe	Swipe	Gaze	Swipe

Table 2

 2

outlines techniques, tasks and connected examples.

Table 2 .

 2 Example

application scenarios for each technique/task combination. Note for Eye Drag & Drop that the examples involve both tasks. Pull Push Eye Cut & Paste Mid-Transfer Interaction Duplicating for many users Eye Drag & Drop Sharing Read-only Content Digital Form Filling Eye Summon & Cast On-the-go: Acquiring many objects Sharing Content