Sylvain Pauchet
email: s.pauchet@astrolab.net

Catherine Letondal

Jean-Luc Vinot

Mickaël Causse
email: mickael.causse@isae.fr

Mathieu Cousy

Valentin Becquet

Guillaume Crouzet

GazeForm: Dynamic Gaze-adaptive Touch Surface for Eyes-free Interaction in Airliner Cockpits

Keywords: Eyes-free interaction, touchscreens, TEI, shape-changing interfaces, adaptive interfaces, eye-tracking, critical contexts. ACM Classification H.5.2. User Interfaces: Input devices and strategies

Figure 1. GazeForm concept. Left: when eye-hand coordination exists for touch modality the surface is flat. Right: when the gaze is solicited by another task the surface offers a salient tangible control allowing continuity of manipulation and freeing the gaze.

INTRODUCTION

On commercial flights, each of the two pilots is responsible for several activities and often has to perform two or more tasks concurrently in order to meet operational requirements. To do this, pilots interact with aircraft systems through multiple digital displays, using numerous physical controllers and performing a multitude of operations [START_REF] Loukia | Cockpit interruptions and distractions: A line observation study[END_REF]. They ceaselessly look from one screen to another, but also have to make constant visual transitions between inside (instrument panel) and outside the aircraft to anticipate the progression of the flight [START_REF] Glaholt | Eye tracking in the cockpit: a review of the relationships between eye movements and the aviator's cognitive state[END_REF][START_REF] Vinot | Legible, are you sure?: an experimentation-based typographical design in safety-critical context[END_REF]. Airline pilots are thus constantly involved in multiple and parallel activities that imply time-sharing of cognitive resources together with gaze mobility and availability.

The latest certified avionics for airliners involve synthetic, contextual and flexible representation, with more versatile access and direct manipulation of data. They allow pilots to perform better in certain tasks. However, the development of touch technology in the airliner cockpit raises critical research questions for aviation safety. Firstly, interaction with a touch screen is almost impossible in flight when severe turbulence occurs [START_REF] Vinot | Could tangibility improve the safety of touch-based interaction?: exploring a new physical design space for pilot-system interfaces[END_REF][START_REF] Kaminani | Human Computer Interaction Issues with Touch Screen Interfaces in the Flight Deck[END_REF][START_REF] Dodd | Touch Screens on the Flight Deck: The Impact of Touch Target Size, Spacing, Touch Technology and Turbulence on Pilot Performance[END_REF]. Furthermore, it increases the saturation of the visual channel and workload, these resources being already under heavy strain in the cockpit context [START_REF] Loukia | Cockpit interruptions and distractions: A line observation study[END_REF]. As emphasized in studies, touch-based interaction indeed requires a greater use of the visual channel than physical controllers [START_REF] Voelker | Combining Direct and Indirect Touch Input for Interactive Workspaces using Gaze Input[END_REF][START_REF] Tory | Comparing physical, overlay, and touch screen parameter controls[END_REF][START_REF] Harrison | Providing dynamically changeable physical buttons on a visual display[END_REF].

In this paper, we present a system that aims at freeing the gaze during interactions, in particular when parallel tasks are required. To achieve this goal, we have designed a device that enables eyes-free interaction when needed, using physical controls that emerge dynamically when the direction of the gaze leaves the touch interaction area. Our design objective is to facilitate the acquisition of this surrogate interactor, enabling sufficiently continuous actions among the interactive systems as possible. We seek to prevent cognitive interruptions that can induce workload increase and safety concerns. We favored a physical and graspable control rather than a haptic device such as UltraHaptics [START_REF] Carter | UltraHaptics: multi-point mid-air haptic feedback for touch surfaces[END_REF] as haptics may not be adapted to the cockpit context. The latter is a working space exposed to constant vibrations and where pilots sometimes have to be cautious with regard to their own sensations [START_REF] Letondal | Being in the Sky: Framing Tangible and Embodied Interaction for Future Airliner Cockpits[END_REF].

Relevant research questions in the discussion of this system include whether an interface adapting to the position of the gaze does not disturb the visual path while improving ocular movements and the distribution of visual attention, and also whether a shape-changing interface poses challenges regarding continuity of actions [START_REF] Vinot | Concept of continuity, a "strong concept" to design graphical architecture of interactive systems[END_REF]. Such continuity encompasses continuity of the manipulation gesture, and the latency of the transformation. We also analyze the relevance of using the gaze to control the state of the interface. Gaze movements, being either active (conscious) or passive (triggered by another conscious intention), may indeed involve involuntary changes and conform to physical constraints.

Our main contribution lies in the concept of using a dynamic modality adapted to the direction of the user's gaze and in an implementation of this concept into a new tangible device. This device has enabled us to explore our research questions and to perform a controlled experiment to evaluate performance, visual attention distribution, and level and accuracy of control. We also discuss the design of interfaces adapted to gaze direction and applicable to other operational domains.

CONTEXT

Airline pilot crews collaboratively ensure five major activities: aircraft piloting (manually or with autopilot), navigation (managing and tracking the flight route), aircraft system monitoring, communication (with air-traffic controllers or ground support), and management of the company mission. To conduct these activities, pilots interact with aircraft systems through specialized interfaces, grouped in functional units and displayed on different screens, specifically dedicated to each of the crew's main activities. They operate these systems and digital displays through physical controllers: knobs, switches, sticks... [START_REF] Vinot | Could tangibility improve the safety of touch-based interaction?: exploring a new physical design space for pilot-system interfaces[END_REF]. During the flight, pilots are responsible for selectively monitoring, extracting and evaluating relevant information, performing a multitude of operations (e.g. checking procedure) and coordinating all these activities within the crew, but also with external entities working on the flight [START_REF] Loukia | Cockpit interruptions and distractions: A line observation study[END_REF]. In order to ensure Cockpit Task Management (CTM) [START_REF] Funk | Cockpit task management: Preliminary definitions, normative theory, error taxonomy, and design recommendations[END_REF], pilots are constantly involved simultaneously in multiple activities that imply time-sharing of cognitive resources. As a result, they are almost always subject to a heavy workload. In addition, a large number and variety of external factors continuously interrupt pilots. The complexity of multiple task monitoring, exacerbated by the attentional competition of external factors, make the pilot crew particularly vulnerable to errors [START_REF] Loukia | Cockpit interruptions and distractions: A line observation study[END_REF].

New avionics concepts tend to replace current cockpit devices with some touch screens. The challenge for industry is to respond to the growing complexity of systems with greater flexibility and lower costs. Many manufacturers have already implemented touch-based technology in integrated product lines for flight deck interfaces, some of which have received type certification from U. S. and European Civil Aviation Authorities and have already been deployed in operational environments. While raising critical safety issues for airliners, the development of touch-based interfaces in cockpits offers many potential benefits for pilots. Features such as direct interaction with data through graphical representations and greater adaptability of interfaces to the flight context improve usability and performance. Furthermore, the significant reduction in the number of physical controllers allows companies and manufacturers to reduce the costs and improve fleet maintenance and updating.

BACKGROUND

Gaze-adaptive interfaces

Gaze-adaptive interfaces encompass interfaces that take gaze into account in a passive, implicit and indirect way [START_REF] Robert | Eye movement-based humancomputer interaction techniques: Toward noncommand interfaces[END_REF][START_REF] Salvucci | Intelligent gaze-added interfaces[END_REF][START_REF] Göbel | The importance of visual attention for adaptive interfaces[END_REF][START_REF] Qvarfordt | Gaze-informed multimodal interaction[END_REF]. As such, this type of interfaces pertains to contextaware adaptive interfaces, where the user implicitly provides contextual information to the system. An example would be Button+, that takes into account either situational or user information either in a passive or active mode [START_REF] Suh | Button+: Supporting User and Context Aware Interaction through Shape-Changing Interfaces[END_REF]. Oviatt et al. [START_REF] Oviatt | Perceptual user interfaces: multimodal interfaces that process what comes naturally[END_REF] describe this approach as natural multimodal behavior that includes a passive input mode using one or more modalities such as presence, body position or gaze, or that combine active and passive input modes, resulting in a blended interface style. Gaze-adaptive interfaces also include systems that use gaze information more directly to enhance the interaction. Salvucci et al. [START_REF] Salvucci | Intelligent gaze-added interfaces[END_REF] proposes to optionally use the user's gaze as a complement and to interpret it using a model while Serim et al. [START_REF] Serim | Gaze-Adaptive Above and On-Surface Interaction[END_REF] design above and on surface interactions enhanced by gaze information. Göbel et al. [START_REF] Göbel | The importance of visual attention for adaptive interfaces[END_REF] highlight the interest of using visual attention for an adaptive architecture and underline the importance of adaptation timing for coherent design. Closer to our approach, Voelker et al. [START_REF] Voelker | Combining Direct and Indirect Touch Input for Interactive Workspaces using Gaze Input[END_REF] enhance interactions based on a vertical screen as output and a horizontal screen as input using gaze input to add output to the horizontal screen.

Shape changing Interfaces

Shape-changing interfaces use shape changes as output to represent the state of the system or as input to change it. Some shape-changing systems offer the change without modifying the modality of interaction, such as Feelex that actuates a flexible surface [START_REF] Iwata | Project FEELEX: adding haptic surface to graphics[END_REF] or work by Harrisson et al. [START_REF] Harrison | Providing dynamically changeable physical buttons on a visual display[END_REF] that uses pneumatic actuation to achieve dynamically inflated controls or dynamic knobs for mobile devices by Hemmert et al. [START_REF] Hemmert | Dynamic knobs: shape change as a means of interaction on a mobile phone[END_REF]. Other systems, such as the one by Ramakers et al., provide a combination of tactile and physical either using a guide for the finger [START_REF] Ramakers | Paddle: highly deformable mobile devices with physical controls[END_REF], or using a tactile wristband that can be rotated. Emergeables [START_REF] Robinson | Emergeables: Deformable Displays for Continuous Eyes-Free Mobile Interaction[END_REF] also provide a mix of touch and physical controls through a switch from a flat surface to a rotary button or a slider. Unlike these studies, which compare modalities, we study the dynamic modality change of the interface during the interaction with the system.

The design space of how to use the context for shapechanging interfaces has been studied by Rasmussen et al [START_REF] Majken | Shapechanging interfaces: a review of the design space and open research questions[END_REF]. Work such by Kinch et al. [START_REF] Kinch | Encounters on a shape-changing bench: exploring atmospheres and social behaviour in situ[END_REF] studies a bench which changes its shape to foster social interactions, while Suh et al. [START_REF] Suh | Button+: Supporting User and Context Aware Interaction through Shape-Changing Interfaces[END_REF] explore various button shape-changes according to the interactions. In line with these studies we wish to enrich this design space by addressing issues of performance and robustness of interface adaptation to the context of use.

TUI for reducing the visual demand

As highlighted in studies by Voelker et al. [START_REF] Voelker | Knobology Revisited: A Comparison of User Performance between Tangible and Virtual Rotary Knobs[END_REF][START_REF] Voelker | Combining Direct and Indirect Touch Input for Interactive Workspaces using Gaze Input[END_REF], touch interaction, as opposed to physical controllers, requires the visual channel to adjust action and achieve a high enough level of precision. In tasks involving indirect visual control, the performance of touchscreens decreases drastically, while remaining constant with physical buttons [START_REF] Jansen | Tangible remote controllers for wall-size displays[END_REF]. Fitzmaurice et al. [START_REF] Fitzmaurice | Bricks: laying the foundations for graspable user interfaces[END_REF], Tory et al. [START_REF] Tory | Comparing physical, overlay, and touch screen parameter controls[END_REF] and Harrison et al. [START_REF] Harrison | Providing dynamically changeable physical buttons on a visual display[END_REF] demonstrate that visual focusing is required in touch interaction, as opposed to physical systems or hybrid systems combining tactile and physical guides. Using an eye tracking system, we seek to explore these differences quantitatively. Furthermore, Fitzmaurice and Buxton [START_REF] Fitzmaurice | An empirical evaluation of graspable user interfaces: towards specialized, space-multiplexed input[END_REF] have shown that in indirect manipulation situations (feedback zone separated from control zone) tangibles outperform touchscreens. The results are identical if the feedback zone is contiguous [START_REF] Harrison | Providing dynamically changeable physical buttons on a visual display[END_REF][START_REF] Tory | Comparing physical, overlay, and touch screen parameter controls[END_REF]. These previous works do not study questions of continuity and control during the transition from direct manipulation to indirect manipulation; this is an issue we seek to address here.

Limitations of touchscreen interaction in the cockpit

The use of touchscreens has drawbacks that might severely limit their operational use in aeronautics. As mentioned above, pilot activities involve a significant level of multitasking and the use of multiple systems, displays and controllers, thus calling for tools that support multimodal intertwined interactions and enable the operator's gaze to be freed. For instance, changing flight parameter values with the rotary knobs located on the central FCU (Flight Control Unit) panel, requires the pilot to check these parameters on the Primary Flight Display (PFD), located in front of them. High visual demand when using touchscreens [START_REF] Voelker | Combining Direct and Indirect Touch Input for Interactive Workspaces using Gaze Input[END_REF][START_REF] Tory | Comparing physical, overlay, and touch screen parameter controls[END_REF][START_REF] Harrison | Providing dynamically changeable physical buttons on a visual display[END_REF] raises important concerns with respect to piloting activity, that relies on distributed visual attention and on effective and periodic visual scanning. Furthermore, various studies [START_REF] Glaholt | Eye tracking in the cockpit: a review of the relationships between eye movements and the aviator's cognitive state[END_REF] on ocular movements in cockpits have demonstrated, for example, that the viewing time on an area (Proportional Dwell Time) varies according to the phase of flight [START_REF] Brown | Instrument scan strategies of F-117A Pilots[END_REF], that the frequency of fixation depends on the degree of expertise of the pilots and that the matrix of eye transitions changes according to workload [START_REF] Kasarskis | Comparison of expert and novice scan behaviors during VFR flight[END_REF]. The question of touch interaction in cockpits can therefore not be posed without exploring the effects on eye movements and visual attention when performing the concurrent task, which we decided to analyze in this study. On the other hand, interacting with touchscreens is more difficult in airplanes in turbulent conditions [START_REF] Vinot | Could tangibility improve the safety of touch-based interaction?: exploring a new physical design space for pilot-system interfaces[END_REF][START_REF] Kaminani | Human Computer Interaction Issues with Touch Screen Interfaces in the Flight Deck[END_REF][START_REF] Dodd | Touch Screens on the Flight Deck: The Impact of Touch Target Size, Spacing, Touch Technology and Turbulence on Pilot Performance[END_REF]. In particular, Hourlier et al. [START_REF] Hourlier | Testing Touch Screens in Realistic Aeronautic Turbulent Conditions (Light to Severe)[END_REF] question the effectiveness of touch interaction in a context of aeronautical turbulence by presenting a simulation-based evaluation method that faithfully reproduces the physical sensations of a turbulence phenomenon. Noyes et al. [START_REF] Noyes | A Comparison of Speech Input and Touch Screen for Executing Checklists in an Avionics Application[END_REF] also report degraded performance of touch interaction in a dynamic context (turbulence, vibration). Work on the stabilization of touch interaction shows that it is partly possible to minimize the effects of turbulence, however it requires additions of substrates to the surface [START_REF] Cockburn | Turbulent Touch: Touchscreen Input for Cockpit Flight Displays[END_REF]. Our approach on an interface which adapts to the contexts of use enables this usability problem to be partly addressed, as physical and graspable controls could emerge according to the level of turbulence.

GAZEFORM CONCEPT

Our objective in this research is an in-depth study of the potential of the GazeForm concept, where we explore gaze direction changes to trigger a surface shape change, and where our goal is to efficiently and safely increase visual distribution among cockpit screens and instruments. In this section, we detail relevant research questions and specify a set of hypotheses.

Gaze activity (Q1): An important set of research questions related to the concept of a gaze-based adaptive shapechanging interface relates to gaze activity and associated workload. Firstly, does GazeForm reduce the number of ocular movements between some areas of the cockpit, namely the touch-based surface and the flight monitoring displays? Secondly, does it provide an efficient distribution of attention among areas with respect to task priorities? Thirdly, does it preserve performance in terms of errors and time spent? Fourthly, does this concept enable the flexible freeing of visual attention when required? Lastly, does it reduce interruptions within monitoring patterns, and thereby enhance the smooth, homogeneous and controlled rhythm, which guarantees safe execution?

Control in safety-critical context (Q2): Another set of research questions encompasses issues related to control where we need to discuss whether gaze-based control is relevant in a safety-critical context [START_REF] Letondal | Being in the Sky: Framing Tangible and Embodied Interaction for Future Airliner Cockpits[END_REF]. Gaze movements are indeed involuntary or automatic when triggered by higher level objectives. Therefore, we need to assess the level of actual control that is left to the user. We evaluate whether they feel in control of the interface, through an appropriate latency of the detection of gaze direction and the resulting shape change or by various means of tuning the control level. We also check if the system has a reasonable error rate and efficiency, and that it does not feel inconsistent for the user.

Continuity (Q3):

We need to discuss whether a modality based on shape-change built from emerging knobs on a flat surface provides a fluid and continuous enough interaction. In addition to the absence of unnecessary interruptions during task switching and within a given modality (either touch or grasp), we seek for a continuous shape change to facilitate the change between modalities by continuous gestures.

In order to explore these issues, we hypothesize that, with respect to a touch only interface, an interface that changes shape depending on the context would improve humansystem collaboration in the cockpit. For this baseline study, the contextual aspect that we focus on is the direction of gaze, but other contextual aspects such as turbulence or smoke could be taken into consideration. Our research questions and associated hypotheses were examined through a controlled experiment presented later, and completed by contextual interviews with pilots. H1: GazeForm reduces workload (to explore Q1) H2: GazeForm reduces visual attention demand and improves attention distribution (to explore Q1) H3: GazeForm promotes a higher efficiency thanks to a smoother task suspension and recovery (to explore Q2 and Q3) H4: GazeForm does not affect continuity of manipulation (to explore Q3)

GAZEFORM IMPLEMENTATION

Figure 2. GazeForm platform with front touch screen (1), lower transformable surface (2) and eye-tracking system (3).

The GazeForm platform is composed of a Dell 2240T touchscreen located in front of the operator (Figure 2.1) and a touch transformable surface including 3 emerging rotary knobs located on the central and lower parts of the cockpit (Figure 2.2). An eye-tracking glasses (Figure 2.3) recovers the position of the operator's gaze in real time to dynamically change the modality of interaction on the lower central transformable surface (touch modality or salient tangible control) (Figure 1). The various elements are integrated in an ad hoc design and test platform, reproducing an airliner cockpit instrument panel in a very simplified way.

Lower surface shape changing surface

In order to perform a realistic comparison between a full touch-only surface and a transformable one with rising knobs, we have chosen to use different devices. For the touch-only display we use a simple tactilized acrylic glass, while for the touch surface with emerging knobs we use a similar acrylic surface with 3 circular cut-out shapes (Figure 3). The knobs move on a vertical axis with servo motors connected to an Arduino board. In the low position the whole surface is flat, homogeneous and the touch function is activated, in the high position the emerging part becomes a rotary knob and the touch function is deactivated. The dimensions (35cm * 45cm) and the layout of the surface are determined to facilitate integration with the test platform and allow touch interaction with minimal posture fatigue [START_REF] Bachynskyi | Performance and Ergonomics of Touch Surfaces: A Comparative Study using Biomechanical Simulation[END_REF][START_REF] Barbé | Physical ergonomics approach for touch screen interaction in an aircraft cockpit[END_REF].

For both modalities, the area is tactilized with the Airbar® infrared system positioned on the upper part of the surface (Figure 3). While the Airbar device is a multi-touch system, a single touch design met our requirements for the experiment. The projection on the surface is ensured by an Optoma HD20 FullHD video projector mounted above the surface.

Emerging rotary knobs

On the central gaze-adaptive lower surface, the three cylinder shapes of 55mm diameter have been cut by a laser. The button number and layout have been established to mitigate the effect of spatial memory on the different tasks. Linear displacement is ensured by a slider and a crank mechanism which enable displacements on a single axis with precise control of speed and acceleration. The Turnigy™ GTY-R5180MG servo-motors in addition to the connecting rod and crank drive system have been assembled on a 3D printed structure and attached to the back of the PolyMethyl MethAcrylate (PMMA) surface (Figure 4). The structure and transmission are designed to minimize friction and ensure a linear emergence of the rotary knob. The diameter of the cylinder (55mm), the speed of knobs emergence (20mm/s) and the height of emergence (25mm) have been defined after several user tests to ensure optimum usability. The rotary knob is composed of a 3D printed support allowing the integration of a Keyes KY-040 potentiometer and attachment to the transmission system. An upper part, also printed in 3D, is fixed on the potentiometer and allows manipulation in rotation and vertical displacement. We have chosen the KY-040 potentiometer for its unlimited movement beyond 360°. This model has 30 physical notches of 12° each and the direction of rotation, the number of steps and the pressure on the button are retrievable on different ports of an Arduino Uno card.

In order to connect the various elements to the platform, three servo motors are controlled by an Arduino Uno card while the data of the potentiometers are retrieved on a second card of the same type. These 2 cards are linked by a USB cable to the PC running the experimentation software.

Eye Tracking

The eye-tracking system worn by operators has two roles: triggering gaze-based adaptation and gathering data for posttreatment analysis. The position of the user's gaze is required in real time in order to trigger the adaptation. We selected the Pupil-labs device which allows the identification in real-time (7.5ms of latency) of the surface on which the gaze falls (front touch screen or lower transformable surface), using a 5x5 Pupil-labs marker (Figure 3). The open source Pupil Capture software is used to configure and calibrate the headset, publish gaze information on the network and record logs. Gaze data are published in real time on a software bus called Ivy [22] shared by the software components of the platform. After calibration, the gaze accuracy was about 1.2 degrees and the detection of the viewed surface was 98% reliable. In order to avoid the effects of blinking and oscillation of the system, we forced the validation of the exit of an area at the entrance to the contiguous zone.

Software architecture

The core application of the test platform had several roles: user management, counter-balancing task among participants, data recovery from various inputs (Arduino card potentiometer values, eye tracking coordinates, duration of visual movements between areas) and the management of the graphical and tangible user interface (screen and video projector display, changes of form, inputs). This application is developed in Java using the djnn framework [START_REF] Chatty | Verification of properties of interactive components from their executable code[END_REF][START_REF] Chatty | Designing, developing and verifying interactive components iteratively with djnn[END_REF], a free software development framework for highly interactive and visual user interfaces, based on reactive programming.

EXPERIMENT

We conducted a controlled experiment to compare the GazeForm concept of a gaze-based shape change with a standard touch-only surface alternative. As for the design of the platform, the research questions (Q1, Q2, Q3) and the hypotheses (H1, H2, H3 and H4) guided the definition of our experimental protocol. This protocol was built on a simplified version of the following scenario, involving the use of a new cockpit interface using the GazeForm concept.

In Inspired by the scenario, we designed 3 tasks that the participants should carry out using the platform. These tasks encompass a dialing task, a monitoring task and the use of a slider, each alternatively in the touch only input or GazeForm input. This reduced set of task enabled us to be compatible with controlled experiments on one hand, and to generate results applicable to other contexts of use on the other.

Method

There were 3 independent variables:

• 2 inputs (touch only surface (Touch Input) or gaze-adaptive touch surface with rising knobs (GazeForm Input)), • 3 dial task difficulties (easy = target size: 40°, distance to target 60°, medium = target size: 20°, distance to target 100°, difficult = target size: 10°, distance to target 150°), • 2 monitoring subtask difficulties (easy = 3 stimuli, difficult = 6 stimuli) The variation of independent variables was organized through 12 experimental blocks. The tasks were composed of a main task (dialing) and two secondary tasks (monitoring and slider tasks). Thus each participant had to perform 12 blocks (6 in Touch input and 6 with GazeForm Input) each composed of 19 dial tasks, 1 slider subtask and 3 or 6 monitoring subtasks. The input condition was counterbalanced by having half of the participants start with GazeForm Input and the other half with the Touch Input. Trials of each tuning and monitoring difficulty were counterbalanced across participants.

Tasks

The main dial task is inspired from previous studies such as Knobology [START_REF] Voelker | Knobology Revisited: A Comparison of User Performance between Tangible and Virtual Rotary Knobs[END_REF] or Turbulent Touch [START_REF] Cockburn | Turbulent Touch: Touchscreen Input for Cockpit Flight Displays[END_REF] and design scenario. It consists in selecting on lower central screen (Figure 5.1) a needle by pressing (Figure 5.3a) and moving it to a green target area. The task is validated when the needle stays more than 500ms in the target area. Then, the next task is triggered and a new target area is displayed on the lower central screen. Pretests of the protocol showed that selecting the needle is more difficult in touch mode than with a physical button, therefore we increased the selection zone to counterbalance this problem (in pink in Figure 5.3a) [START_REF] Blanch | Semantic pointing: improving target acquisition with control-display ratio adaptation[END_REF]. With the rising knobs, the rotation is not bounded and the increment of adjustment is 12 degrees per notch physically felt by the user. The purpose of the slider subtask on the lower central screen is to constrain the participant to switch to Touch Input from time to time when in GazeForm Input. This subtask is a simplification of a more complex realistic touch task, such as modifying a flight plan on a map that would occupy the entire interaction area. The rationale of this task is to evaluate the ability of the interface to return to a homogeneous and smooth surface enabling a continuous touch interaction. For this purpose, the design of the slider task involves continuous interaction, that includes: 1. selecting the yellow circle by tapping (Figure 5.4a), 2. moving the finger (Figure 5.4b) and 3. validating by entering the green circle (Figure 5.4c).

The monitoring subtask on the front screen reproduces concurrent cockpit tasks, involving eye movements to monitor several displays and instruments. In particular, it reproduces the monitoring of the primary flight instruments that the pilot has to perform throughout a flight. The monitoring subtask consists in validating a target (a cross) displayed on the front screen (Figure 5.2a), and reading aloud the text that is then displayed in place of this target (Figure 5.2c). If the participant does not tap the target within 5 seconds, the cross disappears and the subtask is recorded as failed. The purpose of this subtask is mainly to address hypotheses on task resumption (H3), since it interrupts the main dialing task, and distribution of visual attention over several screens (H2).

Procedure

We recruited 24 adult participants from our aeronautical university (14 staff members and 8 students). Three participants were left-handed but used the mouse with the right hand. 12 were in the age group 18-29, 6 in the 30-39, 2 in the 40-49, 3 in the 50-59 and 1 in the 60-69. Twelve indicated normal vision and 12 minor myopia, astigmatism or mild presbyopia, but all of them were able to perform the experimentation without glasses. Nineteen participants declared themselves experts in the use of touchscreens, 1 declared themself a beginner and 4 declared average expertise.

The participants were greeted by a quick presentation of the experimentation, the platform and the goal of the study. A training session let the researcher explain the technique while the participants practiced both Touch Input and GazeForm Input until they felt at ease. Blocks were supposed to be run without interruption but some rest was allowed between them. Participants had to fill NASA-TLX and SUS questionnaires after each modality. Finally, the participants had to respond to another questionnaire on the GazeForm concept and platform. They were invited to exchange freely about it and to fill in a text area with comments regarding the use of the system, and possible shortcomings or improvements.

Measurements

Performance dependent variables. For the main task: time to position the needle in the target, target overruns. For the monitoring sub-task: time between the display of the cross and the tap, number of crosses without tapping, number of text collation errors, time to resume the main task after the tap on the cross. For the slider sub-task: time to realize the slider. During all tasks measured with eye tracking system: the percentage of time spent gazing at the front and lower central screen and the number of transitions between the two screens (similarly to Dehais et al. [START_REF] Dehais | Mitigation of conflicts with automation: use of cognitive countermeasures[END_REF]).

Subjective data were collected by SUS and NASA-TLX [START_REF] Hart | Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research[END_REF] questionnaires for each input.

Free written comments. The participants were invited to fill a one page comment field describe their user experience of the interaction, or to suggest enhancements. These comments were classified after the study.

Recordings. The experiment sessions were filmed and eyetracking data were recorded. We were thus able to transcribe the participants' oral comments and classify them.

Interviews with pilots

In addition to the controlled experiment, we ran a series of interviews with 5 pilots (4 experienced airline pilots) during four 2-3 hour sessions, that were filmed and partially transcribed. We asked the pilots to perform a reduced version of the protocol (3 trials instead of 6 for each condition). We also let them explore some variants of the system that were not presented to the participants of the first experiment, and they were invited to discuss them.

Statistical analysis

All statistical analyses were performed using Statistica 10 (StatSoft © ) and significance was defined at α = 0.05. For the NASA-TLX, we used the Raw Task Load Index (RTLX) analysis method [START_REF] Hart | Nasa-Task Load Index (NASA-TLX); 20 Years Late[END_REF].

The analysis of the subjective ratings of the 6 NASA-RTLX subscales was done with six different one-way repeated measure ANOVAs. The SUS score variations were investigated also using a one-way repeated measure ANOVA. Finally, behavioral performance and eye tracking data were examined using three-way (2 inputs × 3 execution difficulties × 2 monitoring difficulties) repeated measures ANOVAs to determine the effects of the input, monitoring difficulty and tuning difficulty on the dependent variables. Post-hoc pairwise comparisons were performed using Tukey's HSD.

RESULTS

Subjective measures

NASA-RTLX

The analysis of the subjective ratings of each task load dimension revealed that GazeForm Input generated a significantly lower perceived task load than Touch modality. The six one-way repeated measure ANOVAs were significant for the six NASA-RTLX subscales: GazeForm Input

System Usability Scale

SUS ratings were also in favor of GazeForm Input as demonstrated by the one-way repeated measure ANOVA (F(1, 23) = 29.37, p < 0.001, η²p = .56). Figure 7 shows the mean SUS scores for both modalities, higher values indicated higher perceived usability. Importantly, there was a main effect of the modality on the time spent adjusting the interactor to the target value, with shorter execution time with the GazeForm Input vs. Touch Input (F(1, 23) = 60.37, p < 0.001, η²p = .72). We also found a main effect of the tuning difficulty (F(1, 23) = 65.60, p < 0.001, η²p = .74), with a monotonic increase of the execution time across the three level of difficulties (p < 0.001 in all comparisons), and a main effect of the monitoring difficulty, also with an increase of the execution time when monitoring sub-task was more complex (F(1, 23) = 18.50, p < 0.001, η²p = .45). Interestingly, there was also a significant modality × monitoring difficulty interaction (F(1, 23) = 15.09, p < 0.001, η²p = .40), showing that GazeForm Input was less impacted by the monitoring difficulty (Figure 8).

As a complementary analysis, we examined whether age and experience with touchscreens modulate execution time with the two types of interactors. Bravais-Pearson correlations revealed a significant positive correlation between age and average execution time (irrespectively of the levels of tuning and monitoring difficulties) for both GazeForm Input and Touch Input interactions (r = 0.50, p = 0.012; r = 0.58, p = 0.002, respectively). Also, experience with touchscreen was negatively correlated with execution time using the Touch Input (r = -0.42, p = 0.038) and not when using GazeForm (p > 0.05).

Regarding the specific execution time to the slider task, we found that GazeForm underperformed in comparison to the Touch modality (F(1, 23) = 19.31, p < 0.001, η²p = .46) with an increase of +24.41% of execution time. Again, we found a main effect of the modality on the reaction time to the monitoring subtask, with shorter reaction time (approximatively 8%) with the GazeForm vs. Touch Inputs (F(1, 23) = 22.48, p < 0.001, η²p = .49). We also found a main effect of the tuning difficulty (F(1, 23) = 8.10, p < 0.001, η²p = .26), with longer reaction time in difficulty 3 vs. difficulty 1/2 (p < 0.001 and p = 0.022 respectively), and a main effect of the monitoring difficulty, also with an increase of reaction times when monitoring sub-task was more complex (F(1, 23) = 18.50, p < 0.001, η²p = .45) (Figure 9). GazeForm Input allowed faster task recovery (namely, the time required to switch from the monitoring task, once accomplished, to the tuning task) than the Touch Input (F(1, 23) = 7.13, p = 0.013, η²p = .24), with an improvement of 15.89% (Figure 10).

Reaction time to the monitoring task

Task recovery latency

Eye tracking measurements

Time spent gazing the main front screen and the lower screen

GazeForm Input allowed the participant to free their gaze from the lower central screen and thus they were able to fixate more the screen located in front of them (F(1, 23) = 26.12, p < 0.001, η²p = .53). While they spent only 66.66% (SE = 2.88) of the time on the front screen with the Touch Input, this percentage rose to 81.33% (SE = 1.76) while using the GazeForm Input (Figure 11). Consistently, in the GazeForm condition, participants spent less time gazing at the lower screen on which were located the interactors (F(1, 23) = 24.99, p < 0.001, η²p = .52). We then examined the number of transitions between the two screens. We found a significant impact of the modality with fewer gaze transitions between the two screens using Gazeform Input (F(1, 23) = 112.74, p < 0.001, η²p = .83). There was also a main effect of the monitoring difficulty (F(1, 23) = 66.42, p < 0.001, η²p = .74), with an increased number of transitions in the most difficult monitoring condition, and finally, a modality × monitoring difficulty interaction (F(1, 23) = 55.38, p < 0.001, η²p = .70). This interaction showed that the number of transitions between the two screens remained unaffected in the GazeForm condition whereas the number of transitions increased as the difficulty of monitoring in the Touch Input condition rose (Figure 12).

Number of transition between the main front screen and the lower central screen

Additional observations

In this section, we report additional data gathered from the controlled experiment and interviews. The data, that helped to discuss our research questions, include free written comments from the participants of the experiment, the analysis of the transcriptions from a subset of filmed sessions from this experiment, and exploratory interviews with pilots who also tried the system.

Written comments from the participants of the experiment.

Participants of the experiment were invited to provide free written comments after filling in the questionnaires. Seven of them expressed that they found the concept interesting, and felt the system well designed, operational and pleasant. Several participants provided design ideas regarding the button: audio feedback, size, shape, resistance, notches, acceleration and timing. On the touch-based design, some participants proposed solutions to reach the needle more easily. Participants also suggested enhancements of the concept itself, i.e on how or when to take or not take the gaze into account. In particular, comments on audio feedback, shape and timing pertain to the design of the transition between the touch and GazeForm modalities. We also received comments highlighting the limitations of the current prototype, including the absence of a cursor mirroring the position of the finger on the monitoring front screen.

Analysis of video transcriptions.

Another set of feedback was provided through the videos of the experiment, half of which were transcribed. They enabled us to note that gaze towards the touch surface during the tuning task in touch only mode, generally happened when the needle was lost. We also observed, using eye-tracking recorded data, that the single participant with better touch performance in fact looked constantly at the needle and used their peripheral vision to detect monitoring tasks. Another significant oral comment from the participants was that the system requires a brief training, but that, once understood and learnt during the training phase, the system can be used without further thought. One of the participants did not even remembered that there was something to understand and performed the gestures and gazes in an automated mode.

Exploratory interviews with pilots.

Exploratory sessions with pilots brought insights on aspects related to critical context and operational concerns. In order to discuss research question Q2 (Control), we let pilots vary from fully automatic adaptation to manual adaptation, for instance tapping on the area of the button to raise or flatten it. We also included a semi-automatic mode where both gaze-based adaptation and manual control of the button could be used.

Pilots were also invited to try various delays with respect to the change of modality, with a visual feedback (Figure 13.b). The idea was to either provide some space for quick check glances without removing the button once they were holding it, or not raising the button while in touch only mode to enable them to perform continuous touch interactions (Q3). A control panel (Figure 13.d) also let them tune these delays themselves or change the speed of the button. All the pilots found that having a manual mode is required in order to be able to go back to a safe backup mode (Q2). However, while they all reflected on the semi-automated mode, trying it and looking at potential benefits, they all concluded it was too complicated.

Feedback regarding delays to trigger adaptation were mixed: adding a delay to change the button could allow a space for quick glances, but was in some situations perceived as the system being less responsive. Finally, an observation that we made, analyzing the videos of the sessions, answered questions about using the gaze for adaptation (Q1, Q2). We observed that pilots were able to appropriate gaze-based adaptation thanks to their skills in performing well-known sequences of gestures and thanks to their training in visual scanning (Figure 13.c and f). Some pilots were for instance able to build a sequence of gazes and gestures (e.g a dome gesture in touch mode (Figure 13.a), followed by gaze withdrawal and grasping the raised button) and to apply it throughout the GazeForm mode.

DISCUSSION

In this section, we discuss the results from both the controlled experiment and the interviews with pilots, with respect to our research questions.

Gaze activity.

In accordance with H2, shape changing touch surfaces reduced visual demand and improved the distribution of attention as showed by the eye tracking results. While participants spent only 66.66% of the time gazing at the front screen during the touch condition, this percentage rose up to 81.33% thanks to GazeForm. This result is particularly significant since the monitoring task displayed on the front screen simulated the monitoring of the Primary Flight Display (PFD) that the pilots have to check regularly during a flight. Previous eye tracking experiments confirmed the criticality of this PFD in their visual circuit [START_REF] Lefrancois | The role of Pilots' monitoring strategies in flight performance[END_REF]. Consistently, in our experiment, participant's reaction times to the monitoring subtask on the front screen were faster thanks to GazeForm, confirming the efficiency of this interactor to free the visual resources, which allowed enhanced performance in the monitoring subtask. Another interesting eye tracking result was that the number of times that the participants had to switch from the main front screen to the lower central screen was lower with GazeForm. It confirmed that the use of this interactor required less frequent interruption of visual monitoring of the front screen, which was favorable to the monitoring subtask performance. In accordance with H1, GazeForm allowed a reduction of the subjective mental workload, with all NASA-TLX dimensions in favor of this mode. In addition, perceived usability was also higher for GazeFom vs. touch as showed by the SUS results.

Control in safety-critical context.

The answers to questionnaires and free comments from the participants to the controlled experiment showed an acceptation of the principle of using gaze to change modality. We discovered, contrary to our initial intuitions, that adding control by the gaze does not penalize the mandatory monitoring activities of the system. Moreover, all the participants as well as the interviewed pilots, reported that once understood, the concept could be smoothly used and without particular efforts. One pilot brainstormed on new applications of GazeForm to generic controls associated to the system display pages (Figure 13.e). In addition, the fact that pilots are used to performing specific visual circuits while synchronizing them with gestures in well formalized sequences of actions, makes using such a system easier.

In terms of error rate and efficiency, behavioral results supported the H3 hypothesis, GazeForm Input outperformed touch Input with a higher efficiency (the execution time was 20% faster) and a better robustness to task interruption, with lower latencies to recover the ongoing tuning action after an interruption due to the parallel monitoring subtask. Contrary to the touch input, GazeForm efficiency was not impacted by the tuning difficulty. Quite logically, also contrary to the touch input, GazeForm was not dependent on the participant's level of experience with touchscreens.

The other dimension that pertains to critical context, and that was explored during interviews with pilots, was the issue of the availability of different levels of control. This discussion pertains to balance between complex modelling to trigger adaptive system behaviors and a complex design to enable end-user adaptation, close to Dourish discussion between context as representation and context as interaction [START_REF] Dourish | What we talk about when we talk about context[END_REF]. On the one hand, pilots expressed the need for the system to better adapt to specific situations, such as preserving continuity of the modality within a tactile task (e.g working on an interactive map) as discussed below. Such a perspective actually involves finer modelling of the activity and the cognitive and sensorimotor patterns involved in gesture-gaze coordination. Indeed, as an example, increasing the threshold of fixation duration on the button before triggering its retraction can be either frustrating (loss of time) or useful (enables quick glances without effect, just to check the hand position) depending on specific tasks. On the other hand, in order to increase their sense of control over the behavior of GazeForm, we provided a control panel to the pilots with the possibility to tune: the delay before emergence / retraction; the speed of emergence / retraction; and the modes, fully manual, mixed, or fully automatic. However, this resulted in a mixed feeling about the complexity of the settings and the possible resulting behaviors.

Continuity.

Implications for the design of the GazeForm concept include a reflection on continuity at several levels: Q3 and H3 refers to continuity between tasks (task resumption); Q3 also refer to continuity of manipulation between modalities (the shift from touch to GazeForm or the opposite) and continuity of interaction within a modality (e.g. without undesired shapechanging, for example if a too short gaze delay trigger shapechanging).

As for the continuity of manipulation between modalities, results of the controlled experiment showed that participants were not disturbed by the modality change during the dialing task (20% improvement of execution time). However they demonstrated poorer performances using the slider in GazeForm Input condition (execution time was 20.41% faster with Touch Input condition concerning the slider subtask), because even when the rotary knobs are retracted, the existence of residual discontinuities on the surface does not favor touch action. Regarding continuity within modalities, videos of the experiment showed that GazeForm better enabled a discontinuous gesture (e.g performing the dialing task in several steps) than touch input. In touch input it is indeed safer not to release the needle to perform the task, at least when interacting without the eyes. Another aspect explored with pilots was to try different delays to raise of flatten the button to enable the continuity within a modality with respect to gaze movements. Using a delay indeed enables quick glances to the other screen, in order for instance to perform a check. Another aspect that was not implemented but that was discussed with pilots is the possibility to retain the button and stay in GazeForm Input while staring at it, provided that the button is in hand.

CONCLUSION

In this paper, we have presented the GazeForm concept that modifies the user interface modality according to the direction of the gaze to enable a better distribution of the visual attention. Starting from needs observed in the context of future airliner cockpits, we have implemented this concept as an operational and efficient system that has been validated by pilots, and that enabled us to explore research questions regarding its relevance in terms of gaze activity, control and continuity. We are confident that this concept can be generalized, and that it can be explored further in tangible and embodied interaction research and applied in other domains, notably domains involving complex intertwining of tasks and devices. Future research and design work includes exploring the issue of continuity between modalities, and how to support it through continuous shape changes and associated gestures. It also involves exploring further distributed control between gestures and gaze with the latter as a means of control. Furthermore, we believe that the specific role of gaze direction in triggering interactions deserves further study. Finally, additional work should include refining contextual elements and delays to trigger adaptivity, and designing tools to take into account how users would like to control the behavior of the system.

Figure 3 .

 3 Figure 3. The lower surface: (1) Airbar system, (2) display and touch aera, (3) flattened rotary knobs, (4) eye-tracking marker

Figure 4 .

 4 Figure 4. Rotary knobs movement system: (1) crank, (2) servo motor, (3) connecting rod, (4) movement guide, (5) emerged rotary knob, (6) surface, (7) flattened rotary knob

Figure 5 . ( 1 )

 51 Figure 5. (1) Layout of the areas of interaction with the participants. (2)(3)(4) Different steps of monitoring, dialing and slider tasks.

  generated lower perceived Mental / Physical / Temporal Demands (F(1, 23) = 23.50, p < 0.001, η²p = .51; F(1, 23) = 12.64, p = 0.001, η²p = .35; F(1, 23) = 21.46, p < 0.001, η²p = .48, respectively), a higher perceived Performance (F(1, 23) = 10.44, p = 0.003, η²p = .31), and lower perceived Effort and Frustration (F(1, 23) = 29.37, p < 0.001, η²p = .56; (F(1, 23) = 28.87, p < 0.001, η²p = .56), respectively) see Figure 6.

Figure 6 .

 6 Figure 6. Mean ratings of the six NASA-RTLX subscales scores for GazeForm Input and Touch Input. Error bars represent the standard error of the mean.

Figure 7 .Figure 8 .

 78 Figure 7. Mean SUS score for GazeForm Input and Touch Input. Error bars represent the standard error of the mean.Performance measurementsExecution time

Figure 9 .

 9 Figure 9. Mean reaction times for GazeForm Input and Touch Input across the 3 levels of difficulty and the 2 levels of monitoring difficulties. Error bars represent the standard error of the mean.

Figure 10 .

 10 Figure 10. Task recovery latencies for GazeForm and Touch Inputs. Error bars represent the standard error of the mean.

Figure 11 .

 11 Figure 11. Mean percentages of time spent gazing at the main front screen for GazeForm and Touch Inputs across the 3 levels of difficulty and the 2 levels of monitoring difficulties. Error bars represent the standard error of the mean.

Figure 12 .

 12 Figure 12. Mean number of transitions between the main front screen and the lower screen for GazeForm and Touch inputs across the 3 levels of difficulty and the 2 levels of monitoring difficulties. Error bars represent the standard error of the mean.

Figure 13 .

 13 Figure 13. Explorations with pilots. a) dome gesture; b) feedback before flattening a button; c) FCU controls in visual sequences; d) setting delays with the control panel; e) brainstorming on new GazeForm controls; f) eyes-free controls.

  the new cockpit interface, to avoid a storm cell, the PF (Pilot Flying) starts the setting of the heading target on the lower central In order to check the configuration of the throttle, the pilot releases the rotary knob for a few seconds, places his hand on the throttle and then returns to the heading setting. The rotary knob "falls to hand" and the PF can remain focused on monitoring the PFD [H3 -resumption of task]. Heavy traffic or degraded weather forces the crew to be particularly focused on the flight path while simultaneously monitoring systems and looking outside the aircraft. The PF needs to make the last heading adjustments: he stabilizes his hand on the button, feels the notches of the rotation and makes a precise adjustment while monitoring his flight path and cross-checking with the pilot monitoring [H1workload]. The storm cell is now avoided, the PF has released the button, it retracts, and the touch surface can now be used by the pilot to fully interact on the lower central screen with the flight plan data.

screen in touch mode. As the pilot must turn his gaze toward the PFD (Primary Flight Display), located in front of them, to check that the new parameter is taken into account by the system, the touch-based lower central screen interactor turns into a graspable rotary knob interactor, allowing the PF to continue [H4 -continuity of manipulation] the settings without looking at the interactor [H2visual attention demand].

ACKNOWLEDGMENTS

Special thanks to the pilots who contributed greatly to this research work, to the ANR (French National Research Agency) and the FRAE who funded the AIRTIUS project.