Quan Nguyen
email: quan.nguyen@dfki.de

Michael Kipp
email: michael.kipp@hs-augsburg.de

Where to Start? Exploring the efficiency of translation movements on multitouch devices

Keywords: Multitouch interaction techniques, Fitts' law, 2D translation

Predicting the efficiency of interaction techniques can be crucial for designing user interfaces. While models like Fitts' law make general predictions, there is little research on how efficiency varies under different conditions like in which screen region a movement starts and in which direction it is going, and whether the surface is horizontal or vertical. This study investigates these aspects with regard to translation movements on a touch screen, using an extended Fitts' law setup and considering arm kinematics. The results show that on horizontal displays translation is faster and causes less arm fatigue than on vertical ones. Also, on horizontal displays, we identified screen regions and movement directions that allow significantly faster movement compared to others. Finally, movements that employ shorter kinematic chains (e.g. just the wrist) are significantly faster than those that use longer ones (e.g. wrist, elbow, shoulder). We suggest adjustments to Fitts' original formulation. In the future, our findings can inform or partially automate positioning decisions in interaction design.

Introduction

For many devices multitouch has become the standard interaction technique. Predicting the performance of interaction techniques can be important for various reasons, e.g. if an application is highly time/cost critical (industry, logistics, communications) or needs very high precision (medical, military). Even for applications where performance is not the first priority, it is still a criterion to evaluate the user interface [START_REF] Frøkjaer | Measuring usability: Are effectiveness, efficiency, and satisfaction really correlated?[END_REF]: Nielsen et al. [START_REF] Nielsen | Measuring usability: Preference vs. performance[END_REF] report that in 75% of the 57 studies they evaluated the user preferred the system with the best performance. Ben-Bassat et al. [START_REF] Ben-Bassat | Economic and subjective measures of the perceived value of aesthetics and usability[END_REF] show that the user would choose the system with the better performance and ignore the design, even if they have to expect monetary loss. Other studies could show that users rate a design worse after usage if the usability was low on effectiveness [START_REF] Tuch | Is beautiful really usable? toward understanding the relation between usability, aesthetics, and affect in hci[END_REF] or efficiency [START_REF] Nielsen | Measuring usability: Preference vs. performance[END_REF].

We find multitouch techniques on a broad range of devices, from smartphones and tablets to touch-sensitive tables and display walls. The performance of even basic interaction techniques like the translation and rotation of objects will likely differ depending on the size and orientation (horizontal/vertical) of the device and the position and posture of the user and his or her arm and hand. However, current models like Fitts' law [START_REF] Fitts | The information capacity of the human motor system in controlling the amplitude of movement[END_REF] do not take such conditions into account yet. Although Fitts' law has been confirmed for different input devices like mouse, pen input and multitouch [START_REF] Cockburn | Understanding performance in touch selections: Tap, drag and radial pointing drag with finger, stylus and mouse[END_REF], there have only been few extensions so far [START_REF] Accot | Refining fitts' law models for bivariate pointing[END_REF][START_REF] Bi | Ffitts law: Modeling finger touch with fitts' law[END_REF]. In prior work we have started to look at how the screen region effects the performance of translation and rotation techniques as a first step to explore the conditions under which interaction techniques perform best [START_REF] Nguyen | Orientation matters: Efficiency of translation-rotation multitouch tasks[END_REF].

In this paper, we focus exclusively on translation movements (dragging). However, in our study we include many different conditions like screen orientation (horizontal vs. vertical), screen regions (20 regions where the movement can start) and movement direction (eight directions). Based on existing work [START_REF] Bi | Magic desk: Bringing multi-touch surfaces into desktop work[END_REF][START_REF] Weiss | Benddesk: Dragging across the curve[END_REF] we hypothesized that performance will differ with respect to screen orientation and screen region. We also look at the concept of fatigue as a possible cause for performance differences and as an important aspect in the subjective evaluation of multitouch displays [START_REF] Loi | UltrabooksTM and Windows 8: A touchy UX Story[END_REF]. Various studies have shown that the length of the kinematic chains which were used to execute the task play a decisive role in terms of fatigue and performance. Hincapié-Ramos et al. [START_REF] Hincapié-Ramos | Consumed endurance: A metric to quantify arm fatigue of mid-air interactions[END_REF] for instance showed that working with extended arms, which constitutes a longer kinematic chain, fatigue increases. Other studies found higher performance for shorter kinematic chains [START_REF] Langolf | An investigation of fitts' law using a wide range of movement amplitudes[END_REF].

We hypothesize that there is an increased perceived fatigue for vertical displays [START_REF] Bi | Magic desk: Bringing multi-touch surfaces into desktop work[END_REF]. In accordance with [START_REF] Hincapié-Ramos | Consumed endurance: A metric to quantify arm fatigue of mid-air interactions[END_REF] and [START_REF] Langolf | An investigation of fitts' law using a wide range of movement amplitudes[END_REF] we hypothesize that users will use longer kinematic chains on vertical displays which decreases performance.

Our main contributions are the following: We present significant empirical findings concerning the performance of translation movements on horizontal vs. vertical displays, considering start points in various screen regions and various movement directions. We show that horizontal displays outperform vertical ones and identified various screen regions, on a horizontal screen, which perform better than others. Finally, we prove correlations between performance and kinematic chains based on a manual video analysis of kinematic chains.

Related Work

Most of the research for multitouch deals with the problem of selection/tapping in terms of efficiency and precision or both [START_REF] Lee | The performance of touch screen soft buttons[END_REF][START_REF] Sasangohar | Evaluation of Mouse and Touch Input for a Tabletop Display Using Fitts' Reciprocal Tapping Task[END_REF]. Or they focused on different interaction techniques for manipulating objects (translation, rotation or both) with multiple degrees of freedom in 2D [START_REF] Moscovich | Multi-finger cursor techniques[END_REF] or 3D [START_REF] Kipp | Multitouch puppetry: Creating coordinated 3d motion for an articulated arm[END_REF]. There is relatively little research for the translation/dragging task for multitouch in terms of efficiency in different areas of the display [START_REF] Bi | Magic desk: Bringing multi-touch surfaces into desktop work[END_REF][START_REF] Weiss | Benddesk: Dragging across the curve[END_REF]. Bi et al. [START_REF] Bi | Magic desk: Bringing multi-touch surfaces into desktop work[END_REF] divided the multitouch display into different cells to measure the performance of different tasks for the each cell in relationship to the position of the multitouch display. The displays were placed around the keyboard (left, bottom, right and top) and as a vertical screen. The used task for the study was a one-finger gesture task and two docking tasks with translation, rotation and scaling. In contrast to our experiment they used fewer screen locations (nine) and the direction of the one-finger drag gesture was limited to up, down, left or right. For one-handed tasks cells close to the keyboard performed best. For two-handed tasks the placement on bottom and top had the best performance. The design of Weiss et al. [START_REF] Weiss | Benddesk: Dragging across the curve[END_REF] only included up and down movements.

There are two lines of research where vertical and horizontal displays were at the center of attention. In the first line of research, the two orientations are compared in terms of efficiency and performance [START_REF] Hancock | Improving menu placement strategies for pen input[END_REF][START_REF] Pedersen | An experimental comparison of touch interaction on vertical and horizontal surfaces[END_REF]. In the second line of research, solutions for the integration of both screen orientations are explored [START_REF] Weiss | Benddesk: Dragging across the curve[END_REF]. Hancock et al. [START_REF] Hancock | Improving menu placement strategies for pen input[END_REF], for instance, compared the direct input with a pen input on a vertical and horizontal display surface. With a selection task on menus they tried to find out which regions are faster and easier to reach for the used hands. Based on their findings they suggested an adaptive interface to detect handedness because handedness influences the performance of the selection. For instance, the left hand is faster for upper-left und lower-right regions while the upper-right and lower-left regions are faster for the right hand. Pedersen et al. [START_REF] Pedersen | An experimental comparison of touch interaction on vertical and horizontal surfaces[END_REF] found that tapping was performed 5% faster on the vertical surface, whereas dragging was performed 5% faster and with fewer errors on the horizontal surface. In contrast to our experiment they compared tapping and dragging tasks on large multitouch displays where participants were standing. Additionally, the participants where free to choose the left or right hand for interaction. In contrast, the BendDesk [START_REF] Weiss | Benddesk: Dragging across the curve[END_REF] was constructed as a combined horizontal and vertical display, connected by a curved region. The authors studied this curved area and, among other things, compared down-up movements that cross through all three areas. One of their findings was that dragging on a planar surface is faster and straighter than dragging across the curve. Given that the distances were constant for all dragging tasks Fitts' law would have expected constant movement durations over all areas.

Since Fitts published his formula which predicts that the time to acquire a target is logarithmically related to the distance over the target size [START_REF] Fitts | The information capacity of the human motor system in controlling the amplitude of movement[END_REF], there has been more research on this topic and Fitts' formula has been confirmed for different input devices like mouse, pen input or multitouch [START_REF] Cockburn | Understanding performance in touch selections: Tap, drag and radial pointing drag with finger, stylus and mouse[END_REF].

There has also been work claiming that Fitts' law was unsatisfactory and suggesting to extend it for a 2D task [START_REF] Zhang | Extending fitts' law to account for the effects of movement direction on 2d pointing[END_REF] or for touch input [START_REF] Bi | Ffitts law: Modeling finger touch with fitts' law[END_REF]. Additionally, the authors [START_REF] Nguyen | Orientation matters: Efficiency of translation-rotation multitouch tasks[END_REF] proposed to consider the direction of the movement. Weiss et al. [START_REF] Weiss | Benddesk: Dragging across the curve[END_REF] found indications that the interaction zone has an influence on the task completion time.

Method

Participants

Participants were recruited via noticeboards on university campus and through academic mailing lists. 16 subjects (7 male, 9 female) took part in the study and were paid 10 Euros. The age varied between 19 and 33 with an average of 24 years. In terms of handedness 13 subjects were right-handed and three were left-handed but use the right hand for controlling mouse and touch interfaces.

Apparatus

We used a 22-inch multitouch screen (3M model M2256PW) with 1680x1050 pixels and <6ms touch response time. The application was developed in Java 8 with JavaFX and was run on an iMac. The screen was used in two orientations, vertical and horizontal. In the vertical setup the display stood on a desk, in the horizontal setup it was placed on a low table so that the surface was at a height of 73 cm = 28.7 inches (Fig. 1a and1b). The participants sat on a static chair centered in front of the display. Interactions were tracked by three webcams from the left (side view, cam 3), from above (top view, cam 1) and from behind the participant (shoulder view, cam 2). Figure 1c shows the webcam setup for the vertical display setup.

Tasks

The task required to move a circular cursor (grey circle with a red cross) into a target area marked by a dashed circle (see Figure 2a). The target area was 1.5 cm in diameter. As soon as the cursor was selected by touching it the cursor changed to monochrome colors (see Figure 2b). A trial was rated successful if the center of the cross was located inside the target area when lifting the finger off the screen. Success was signaled by a green check mark (Figure 2c). It was not necessary to achieve a perfect match between cursor and target area. But if the center of the cross remains outside the target area a "sad smiley" appeared to signal failure (see Figure 2d). In this case, the corresponding trial was repeated at the end of the set. The beginning of a new trial and success or failure of a trial were accompanied by distinct sounds.

Material

For a thorough analysis the surface area of the screen needed to be completely covered by movement paths. A movement path was described by a start and an end point, marked by two different circular areas. Start points were evenly distributed through a pattern as broad as possible. The multiplication with end points showed that a high number of start points would have led to an unacceptably high number of trials per orientation. Therefore, the number of start points was set to 20, so that the display was still optimally covered with points in reasonable distances and the study was still feasible in an adequate time frame and with adequate effort (see Figure 3). Possible end points were computed for each start point with four possible distances (2.5 cm, 5 cm, 10 cm and 20 cm) going in eight possible directions (0°, 45°, 90°, 135°, 180°, 225°, 270°and 315°). The combination of distances and directions resulted in 32 different theoretical end points per start point (see Figure 3b). Some of the end points were not reachable because of the screen limits, so such points were removed (see Figure 4a). Figure 4b shows all used configurations for the respective start points. The total number of configurations was thus reduced from 640 to 388. Each configuration occurred once in each set for horizontal and vertical level.

Procedure

The study was conducted in a lab with a supervisor and took about 1:15 h per participant. Each subject was briefed using written instructions while allowing for clarification questions.

Participants completed a pre-test questionnaire for demographic data. To make subjects familiar with the device and its use (strategy and optimal finger movements) each session began with a training phase of 30 randomized trials using the same configurations across subjects.

All tasks were conducted first in the horizontal, then in the vertical condition, or vice versa (setup order was balanced across subjects). Each condition took about 30 minutes to complete with a short break in between. After switching display orientation another training phase was conducted with a different training set. For every orientation condition, the task set consisted of at least 388 trials presented in four blocks separated Every subject received different task sets with 388 distinct configurations. The order of configurations was pseudo-random under the condition that neither start point nor movement direction be the same in two immediately subsequent trials. After completing all tasks, subjects answered a questionnaire with subjective ratings concerning the comparison of both orientation setups.

Design

We devised a within-subject design with two independent variables: (a) display orientation (horizontal, vertical) and (b) configuration (start point, direction). Training phases were not included in the analysis of the trials so that there were 16 subjects × 2 screen orientations × 4 blocks × 97 configurations by block (388 configurations in sum) = 12416 data items were analyzed.

The following data were measured as dependent variables:

Results

We analyzed our data such that concrete design recommendations could be generated. Therefore, we looked at screen areas. First, we analyzed screen halves (top/bottom half and left/right). Second, we defined three functional areas (see Fig. 5) based on the following observations of current UIs:

1. Center (yellow): Translation movements in this area are multidirectional and can be used functionally (e.g. scrollbars, pop up menus etc.) and within different applications (image editing, map navigation etc.). 2. Edges (blue): Screen edges are often used for menu bars (e.g. Windows charm bars) or as storage areas. Here, selection requires translation movements vertical to the respective border directed towards the center of the display. Those movements are also relevant for desktop changes or gestures to control browser menus (tab/side forward/backward etc.). 3. Corners (red): Corners are treated separately from edges because they seem to be quite important. Often, frequently used functions are placed there (start menu, touch and hold menus, which are optionally used in multitouch and mouse combinations).

Translation movements are mainly directed up-or downwards. Gestural interaction in the corners can also require diagonal movements.

We compared the performance of these areas. Performance was measured for horizontal and vertical displays considering start point, movement direction and varying distances. We measured in terms of mean-of-means throughput (TP) after Soukoreff and MacKenzie [START_REF] Soukoreff | Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCI[END_REF]. The difficulty of each configuration was measured using the Index of Difficulty (ID) with the Shannon formula because it always gives a positive rating for the index of task difficulty [START_REF] Mackenzie | Fitts' Law As a Research and Design Tool in Human-computer Interaction[END_REF]. TP combines speed and accuracy into a single dependent measure and is calculated by:

T P = 1 y y i=1   1 x x j=1 ID eij M T ij   Z11 Z12 Z13 Z14 Z15 Z21 Z22 Z23 Z24 Z25 Z31 Z32 Z33 Z34 Z35 Z41 Z42 Z43 Z44 Z45
Fig. 5: Functional areas: center (yellow), edges (blue) and corners (red).

where y is the number of subjects, and x represents the movement condition. M T ij is the meantime over all trials for this condition. The units of throughput are bits per second (or bps).

The advantage of TP is the normal distribution of these data (Shapiro-Wilk-Test: W = 0.9858, p -value = 0.8047) because normal distribution of the data is often a requirement for statistical tests. In contrast, distance times are log distributed (see Fig. 6). Comparing display orientation A two-tailed paired t-test showed that translation movements on a horizontal surface (M = 8.67, SD = 2.02) outperformed the ones on a vertical surface (M = 7.53, SD = 1.46); t(15) = 3.62, p < 0.002, Cohen s d = 0.907 (see Fig. 7) which is also clearly visible in the heat maps in Fig. 8. Comparing functional areas For the horizontal condition, we found differences between areas (see Fig. 8). A pairwise t-test with Bonferroni-Holm correction between the corners shows that the left bottom corner of the horizontal display is the best corner in terms of performance (see Tab. 1). For vertical displays there were no significances.

An additional pairwise t-test with the same correction for the edges shows significant differences with large effect sizes in performance (see Tab. Comparing movement directions The most interesting finding from the analysis of the movement directions was the higher performance of upward movements (270°) from the bottom half (M = 9.00, SD = 2.06) compared to downward movements (90°) from the top half (M = 8.61, SD = 2.16, t(15) = 2.21), p < 0.05, Cohen s d = 0.553 in the horizontal condition. For the vertical condition there were no significances for these movements. Although we found several statistically significant differences between directions in the horizontal and vertical condition it was difficult to find systematic patterns. Figure 9 illustrates both the best and worst directions. The stars indicates the level of significance: * p < 0.05; * * p < 0.01; * * * p < 0.001;

Comparing movement axes We slightly abstracted away from actual directions by collapsing two opposing directions into one axis (for the labels see Fig. 10). A pairwise comparison of axes in different areas showed, in some areas, a highly significant difference between the performance of the axes. Figure 11 shows only significant pairs. For instance the V axis showed significant differences to the H axis and the D2 axis in the zones Z21, Z22, Z31, Z32, Z33.

Kinematic chains

For the analysis of the kinematic chains the sessions were recorded by three cameras (see Fig. 1 for the setup and Fig. 12 for an example screenshot of the video). The recorded material was annotated by two independent coders using the ANVIL video annotation tool [START_REF] Kipp | ANVIL: The Video Annotation Research Tool[END_REF]. For each translation movement the coders annotated the employed kinematic chain in the user's arm. A kinematic chain is a series of connected joints, e.g. finger-hand-forearm. The more joints there are involved, the longer the kinematic chain. To categorize a user movement we defined four kine- matic chains: finger, finger-hand, finger-hand-forearm and finger-hand-forearmupper arm. Each chain type is denoted by the topmost joint in the chain (printed in bold). For instance, if the target was moved by the finger without moving the hand, elbow or shoulder, then we categorized it as "finger". If the elbow was involved, then "forearm" was annotated.

A chi square statistic shows that the the orientation of the display influences the usage of the kinematic chain χ 2 = (3, n = 12335) = 423.210, p < 0.001. On horizontal displays, users employed shorter kinematic chains. Table 3 illustrates the usage in percent. We can see that the values for finger, hand and forearm increases up to 200% in comparison to the vertical display.

T-Tests for the horizontal condition show that shorter chains (finger, hand, forearm) (M = 9.36, SD = 2.27) have significantly higher performance than longer chains (upper arm) (M = 8.47, SD = 1.95); t(15) = 6.593, p < 0.000. The same applies for Fig. 12: Screenshots of the three camera perspectives. the vertical orientation: shorter chains (M = 8.27, SD = 1.66) compared to upper arm (M = 7.47, SD = 1.46), t(15) = 3.884, p < 0.001.

Adjusting Fitts' law

We adjusted Fitts' law to include the factors of screen orientation, start point and direction. We performed a multiple (linear) regression with the mentioned factors using the Shannon formulation for index of difficulty (id). For the computation we indexed the real values for start x-point (startX), start y-point (startY), screen orientation and direction (angle) to calculate the regression coefficients. We used values from 1 to 5 for the five possible starting x-coordinates, 1 to 4 for the starting ycoordinates and 1 to 8 for the angles in clockwise direction (1 = 0°... 8 = 315°). For screen orientation we used 1 for horizontal and 2 for vertical. We randomly divided our data into a training set and a test set, each of which contained 50% of vertical and 50% of horizontal data points. We used the training set to generate the model and derived the following parameters (startX, orientation and angle) and regression coefficients after the multiple regression (the starting y-coordinate was not significant for this model, p > 0. The first line is the regular Fitts' law formulation (from now on called Fitts' model). We used the training data to compute the coefficients for the regular Fitts' model, too, and derived y = 0.1605x + 0.089.

Comparing the actual test data (green) with the predictions of the regular Fitts' model (red) and of our adjusted Fitts' model (blue) over all ids (Figure 13a), both models achieve a similarly high correlation (Fitts' model R 2 = 0.98, adjusted model R 2 = 0.92). However, if we take the other factors into account like orientation, starting point and direction we get a better picture of the performance of the two models (see Figures 13b,13c,13d). Here, the adjusted model is more precise than the the predicted constant time of Fitts' model. It is clearly visible how the adjusted model accounts for our findings: The prediction for surface orientation for instance shows the different performances for horizontal and vertical displays (see Fig. 13b) -horizontal outperforms vertical. In Figure 13c the prediction shows that the left side is faster than the right side of the display. And as we can see in Fig. 13d the different directions have different performances.

Questionnaire and interviews

The direct comparison between horizontal and vertical display orientation shows an explicit preference for a horizontal multitouch display over all categories among the subjects (find all results in Table 4). Most of the subjects perceive the horizontal orientation as faster, simpler and operable with less effort compared to the vertical orientation. Furthermore, subjects feel safer and even for the correction of errors like re-adjusting due to missing the target area the horizontal display is preferred over the vertical display.

Discussion

Our results first of all show that the performance of a translation movement depends on the orientation of the display, the point where the interaction starts, the direction in which the translation moves and which kinematic chain the user used.

The horizontal display yielded the best performance and lowest perceived fatigue compared to the vertical one. Additionally, users used shorter kinematic chains. Our results show that kinematic chains influence performance: shorter kinematic chains (involving finger, hand, forearm) yield higher performance than longer ones (involving the upper arm) which is in accordance with earlier findings [START_REF] Langolf | An investigation of fitts' law using a wide range of movement amplitudes[END_REF]. However, how do kinematic chains relate to the concept of fatigue? A recent study by Hincapié-Ramos et al. [START_REF] Hincapié-Ramos | Consumed endurance: A metric to quantify arm fatigue of mid-air interactions[END_REF] showed that what they call the "bent arm position" is the least tiring of all positions they tested for a selection task on a 2D plane. This clearly corresponds to our notion of a short kinematic chain. Therefore, we can assume that kinematic chains correspond not only with performance but also with fatigue: shorter kinematic chains cause less fatigue than longer kinematic chains. Taken together, these results imply that it may be desirable to train users accordingly, i.e. to use shorter kinematic chains to reduce fatigue and increase performance.

On horizontal displays, performance also differs across the screen. The left half yields higher performance than the right half. Also the bottom half outperforms the top half. The bottom edge and the bottom-left corner seem to be particularly good areas for high performance. This confirms UI decisions like placing a start button in the lower left corner or putting a dock bar along the bottom edge. It contradicts the UI decision of placing a menu along the right edge (e.g. Windows charm bar). However, this only applies to horizontal displays. Vertical displays have more homogeneous performance characteristics.

In terms of movement direction we found some patterns for high performance. Upward motion is performed faster than downward motion (in the horizontal condition). This was partially confirmed when looking at motion axes where the vertical axis and the diagonal from lower-left to upper-right are particularly good performance-wise. This is relevant if virtual objects (UI elements, photos, documents) have to be dragged to a target area. According to our results source and target positions should be located in the lower and upper screen regions respectively and ideally be on the vertical or the mentioned diagonal axis.

In prior work we found that combining the movement of translation and rotation achieves higher performance when directed to the right than when directed to the left [START_REF] Nguyen | Orientation matters: Efficiency of translation-rotation multitouch tasks[END_REF]. Our current study shows that this does not hold for translation-only movements. It can be concluded that the found direction preference must be due to the rotation part or the combination of rotation and translation which is an interesting refinement of our earlier result.

Regarding Fitts' law, we found contradictory data since there are significant differences in performance depending on the the start position and movement direction. However, Fitts' law would have predicted constant movement durations over all areas and direction. This confirms earlier works [START_REF] Weiss | Benddesk: Dragging across the curve[END_REF][START_REF] Nguyen | Orientation matters: Efficiency of translation-rotation multitouch tasks[END_REF]. So we suggest to adapt Fitts' law with additional parameters to factor in the start position, the screen orientation and the direction of the movement. Our adaption of Fitts' law gives a better approximation and prediction of the expected performance although the measured time data were not involved in the modeling of the prediction formula. In our case the adapted version predicted the different performances between the directions, orientations and x-coordinates. We are aware of the fact that our extension is only the simplest approach to adapt Fitts' law, but this should demonstrate the necessary of the extension of Fitts' law to achieve usable predictions.

Finally we would like to point out that the conditions have to be extended to have a higher generalizibility of the outcomes. Possible extensions are outlined in future work (Section 5).

Conclusion

We presented a study to systematically explore the performance of translation movements on a multitouch display. Our results show that the performance varies significantly and with large effect size depending on surface orientation (vertical vs. horizontal), movement start point and movement direction.

We showed that horizontal screens yield the highest performance and are subjectively preferred over vertical ones. We also found specific areas where the performance outperformed other areas of the display in both conditions, vertical and horizontal. Most differences occur on the horizontal display which means that the optimization potential is higher with horizontal displays. And we could show that the direction of the movement influences performance. These findings contradict Fitts' law which predicts constant movement time over all areas and along all directions. Therefore, we suggest to extend Fitts' law using a simple linear combination that factors in display orientation, start point and direction.

In terms of ergonomics we could show that orientation influences how the user executes movements. On horizontal displays user employ shorter kinematic chains com-pared to movements on a vertical screen. We interpret this as a possible cause for the higher performance and the lower perceived fatigue on horizontal screens.

Future Work

Future work should investigate more in-depth the correlation between performance, perceived fatigue and the usage of shorter kinematic chains. Our experimental design could be used to examine a wider array of conditions. It would be interesting to include smaller and larger form factors like smartphones and phablets, which are often operated by thumb, or multitouch tables and large touch walls where users may have to stretch or even walk to reach certain areas. Other important aspects to consider are different postures (sitting, standing) and different screen angles (45°and others).

Additionally, we need to check whether and how handedness affects our results. It would also be interesting to look at cultural differences (e.g. different reading and writing directions).

Ultimately, our results should make it possible to generate recommendations e.g. in the form of heat maps for specific devices, and to automatically evaluate user interfaces. Future interfaces may even constantly adapt the UI layout depending on user characteristics based on results such as ours.

Fig. 1 :

 1 Fig. 1: (a) and (b) show the setup for horizontal and vertical screens, (c) shows the top view (cam 3) in the vertical screen configuration.

( a )

 a Task is shown (b) Subject starts motion by touch (c) Success (d) Failure

Fig. 2 :

 2 Fig. 2: Tasks

Fig. 3 :

 3 Fig. 3: (a) shows all start points, (b) shows all generated directions and distances between start points (black) and end points (colored dots).

Fig. 4 :

 4 Fig.4: Since some end points are off screen we removed several directions and distances, (a) shows an example where the red points were removed from the configuration set by removing the distance or the angle to this corresponding point, (b) shows all finally used directions for each start point in the experiment.

Fig. 6 :

 6 Fig. 6: Execution time is log-distributed (a) . TP is normal distributed (b).

Fig. 7 :

 7 Fig. 7: Mean throughput (TP) for horizontal screen (blue) which significantly outperforms the vertical screen (red).

Fig. 8 :

 8 Fig. 8: Performance heat map of the horizontal (a) and vertical (b) screen. Red means high performance, blue low performance.

  2): the bottom edge Table 1: Results of pairwise comparison of the corners: Left-Bottom (LB), Left-Top (LT), Right-Bottom (RB) and Right-Top (RT).

  = 9.073, SD = 2.07) LT (M = 8.11, SD = 2.07) < 0.004 1.062 LB RB (M = 8.21, SD = 2.05) < 0.01 0.925 LB RT (M = 8.15, SD = 2.03) < 0.05 0.718 allows for better performance than the top and right edge. The left edge outperforms the top edge. For vertical displays there are no significances. Table 2: Results of a pairwise comparison with Bonferroni-Holm correction of the areas on the edges. Edge 1 Edge 2 p-Value Cohen's d Bottom (M = 8.70, SD = 1.97) Top (M = 8.26, SD = 2.03) < 0.02 0.878 Bottom (M = 8.70, SD = 1.97) Right (M = 8.40, SD = 1.88) < 0.03 0.780 Left (M = 8.95, SD = 2.17) Top (M = 8.26, SD = 2.03) < 0.01 0.946

Fig. 9 :Fig. 10 :

 910 Fig. 9: The best and worst directions for each area. The stars indicate the level of significance between the corresponding direction vectors ( * p < 0.05; * * p < 0.01; * * * p < 0.001). Note: To make the difference between the vectors visible the view was zoomed in by clipping the start part and enlarging the rest by a factor of 200.

Fig. 11 :

 11 Fig. 11: All axes with significance were drawn, all others were omitted.

  1): M T = 0.0100 + 0.155 * id + 0.008 * startX + 0.053 * orientation -0.006 * angle with R 2 = 0.92

Fig. 13 :

 13 Fig. 13: Comparison of two models (Fitts' model, red, and adjusted Fitts' model, blue) against the test data (green): (a) for all ids (b) with fixed id for both orientations (c) for different x coordinates (d) and different angles.

  1. Distance time (DT): time from start point to end point minus tolerance. 2. Correction time: time from entering the target area to lifting finger off screen 3. Error rate Correction time and error rate have to be measured for the effective ID (ID e ) in the mean-of-means throughput (TP) formula after Soukoreff and MacKenzie [21] .

Table 3 :

 3 Distribution of kinematic chains in percent.

	Horizontal		Vertical
	Distance Finger Hand Forearm Upper arm Finger Hand Forearm Upper arm
	2.5 2.02 % 6.69 % 24.29 %	66.99 % 0.23 % 1.09 % 15.09 %	83.59 %
	5 0.06 % 1.14 % 23.26 %	75.54 % 0.00 % 0.11 % 10.69 %	89.20 %
	10 0.00 % 0.06 % 18.33 %	81.61 % 0.00 % 0.00 %	6.31 %	93.69 %
	20 0.00 % 0.00 %	8.54 %	91.46 % 0.00 % 0.00 %	0.33 %	99.67 %
	at all 0.52 % 1.97 % 18.61 %	78.90 % 0.06 % 0.30 %	8.10 %	91.54 %

Table 4 :

 4 Results of questionnaire and interviews.

	Assessment category	horizontal neither nor vertical
	Faster and safe handling	62.50 %	25.00 % 12.50 %
	Simpler operation	75.00 %	18.75 % 6.25 %
	Less effort in operation	81.25 %	12.50 % 6.25 %
	Easier correctable (precise working)	50.00 %	31.25 % 18.75 %
	Felt faster	81.25 %	12.50 % 6.25 %

Acknowledgements

We thank Markus Pospeschill (Saarland University) for his competent advice in the user study and the data analysis. We thank Johanna Nguyen and Dat Ba Nguyen for the help in the execution of the experiments.

Parts of this research have been carried out within the framework of the Softwarecampus sponsored by the German Federal Ministry of Education and Research (BMBF), grant "01IS12050".


Kyoko Shibata
email: shibata.kyoko@kochi-tech.ac.jp

Yoshio Inoue
email: inoue.yoshio@kochi-tech.ac.jp

Hironobu Satoh
email: satoh@ee.kochi-ct.ac.jp

Intelligent Ankle-Foot Orthosis by Energy Regeneration for Controllable Damping During Gait in Real Time

Keywords: Medical and Welfare Assistance, Ankle-foot Orthosis, Semi-active damper, Energy Regeneration

diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

In natural normal gait, as shown in Fig. 1, with the appropriate braking force generated by working dorsal and planter flexors alternately, the rotation of the ankle joint changes. In view of energy, it is applied during toe-off timing, and it is consumed during other timing because the braking force is required. On the other hand, in the swing phase, hemiplegic patients cannot be taken the clearance between the foot and the ground (foot-drop), and be leading to stumble and fall, because they are difficult to generate a braking force by muscular weakness, etc. Therefore, conventionally, many hemiplegic patients mount ankle-foot orthosis (AFO) which are possible to suppress the movement by a fixed ankle joint. This braking force of plantar flexion is compensated, so foot-drop can be prevented. The use of conventional AFO is effective to improve their gait. These are clinically well known. However, since conventional AFOs have large rigidity, cannot be freely rotated ankle, appropriate braking force is not generated, in other words, the patient performs an unnatural and high energy consumption gait. This causes hyperextension of the knee joint during heel ground. So, because hemiplegic gait differ significantly from healthy gait, even using conventional AFO, rehabilitation effect cannot be obtained to reacquire a smooth gait.

Here, if an AFO that is intelligent that can variably control the resistance (damping) of the ankle joint according to the gait state, it is considered that the patient's gait motion becomes smoother, and more so that continue to use to lead to the restoration of gait function by learning the natural gait. However, when the function of the orthosis becomes higher, secure energy source is required. Therefore the increase in weight and size and the restriction of use time or space become new problems. For AFO with a variable damping ankle, so far, the orthosis with a passive non-linear hydraulic damper [START_REF] Yamamoto | Development of an Ankle-Foot Orthosis with Dorsiflexion Assist, Part 2: Structure and Evaluation[END_REF] , semi-active orthosis by MR fluid 2 , active orthosis by a servo motor [START_REF] Blaya | Adaptive control of a variable-impedance ankle-foot orthosis to assist drop-foot gait[END_REF] , etc. are commercialized or studied. In these, effect of gait improvement have been reported, however, it is difficult to be satisfied both to ensure the natural gait and energy sources.

For these problems, in this study, new AFO system (iAFO) is developed by the semi-active damper using the DC motor and the energy regeneration with the step-up chopper circuit. It can be controlled the damping of rotated ankle joint depending on the gait state, and driven a long time. This iAFO allows for active rehabilitation of continuous long-term. A single iAFO can be applied for of course rehabilitation at hospital, home rehabilitation (Fig. 2) and life support. In Fig. 2, time-series data of the ankle joint angle are transmitted in real time to the server from the controller including iAFO. This data can be monitored to doctors or physical therapists in a hospital, and patient can be instructed in remote by them. Also, if necessary, it can be derived that the patient gait becomes more natural gait by automatic adjusting the strength of the controllable damping by real time remote feedback. Since this is become selftraining, it is considered that effect of gait functional recovery is large. Further doctors or patients can check the progress of the recovery of gait ability at any time with cloud service. In this report, first, the overview of iAFO is described. Then, from gait experiments using iAFO, the results of investigation is shown about the damping variable control, charging of a battery, gait improvement, and the power recovery ratio.

Basic characteristics of the prototype

In this study, the DC motor and the set-up chopper circuit are combined, whereby rotation damping is controllable and energy is regenerated.

The prototype iAFO is shown in Fig. 3. The controller which is consisted of the microcomputer and the set-up chopper circuit [START_REF] Okada | Energy Regenerative and Active Control Suspension[END_REF] is attached to the plastic AFO with an ankle joint can be rotated freely. The DC motor is rotated through links and gear (159:1) depending on the motion of the ankle joint. The gait state (Fig. 1) is determined by the output of the two pressure sensors attached to the foot, and the rotation damping ankle is controlled. Specifically, the damping magnitude is smaller in order to soften the movement of the ankle in the stance phase, and is larger in order to restrict the movement of the ankle in the swing phase. Depending on the gait state, ON circuit without battery and OFF including battery of the set-up chopper circuit is switched at high frequency by the controller. The average current per unit time flowing through the DC motor by controlling duty ratio of the PWM control is changed, therefore, the appropriate braking torque is generated. That is, it is possible to obtain the semi-active damping for plantar flexion or dorsiflexion. At this time, the energy due to the rotation of the DC motor is converted into electrical energy, it is charged in the battery by the step-up chopper circuit, and the driving force required for the control is obtained. The total weight of iAFO including the battery is 1.110 [kg].

Consideration of gait improvement

A healthy subject is wearing iAFO on his right leg, and imitates a hemiplegic gait. He is continuously gait at a velocity of 3[km/h] using a treadmill. In this experiment, duty ratio is set to 0.4 to slightly soften the rotation damping of the ankle joint at the initial contact, 0.1 to soften at the mid-stance or 0.9 for tightly at the toe-off timing and swing phase.

Fig. 4 is time history of the ankle angle per one gait cycle when (1) wearing the conventional AFO, (2) wearing iAFO, (3) normal gait (bare foot).

At results, since it is not open large ankle angles when fitted either AFOs, it can be seen that it is possible to prevent foot-drop. Also, because the ankle of iAFO can move freely, the range of ankle angle change of iAFO is larger than conventional AFO. In other words, it is considered that gait difficulty due to the rigidity of AFO has been improved by the proposed method. In addition, gait time is longer, that is, gait improvement is shown since the step length is increased.

Consideration of electric power recovery ratio

Using two 5[V] batteries, one connects to the microcomputer and sensors for power supply and other connects to the step-up chopper circuit for charging. Fig. 5 shows a part of the current flowing from the power supply for the battery, Fig. 6 shows a part of the current flowing into the charging battery. In this paper, power recovery ratio is defined the percentage of charging power to the power consumption. By an average value of five measurements, power consumption is 57.36[mW], charging power is 49.60 [mW]. Therefore the power recovery ratio in this gait is 86.5%.

Conclusion

In this study, it is proposed to assist a hemiplegic gait using the AFO with the DC motor and the set-up chopper circuit. Developed iAFO can be controlled to the semiactive damping of ankle joint depending on the gait state. At the same time, iAFO is capable of driving long time using by regenerated energy that is absorbed by the ankle joint motion into the small battery.

In this paper, prototype iAFO are produced, and gait experiments with healthy person are performed using iAFO. As a result, it is confirmed that it is possible to charge the battery, it is possible to prevent foot-drop, further, it is possible to change the ankle joint smoothly compared to the conventional AFO, and the power recovery is high ratio. Thus, effectiveness of this proposed system is shown.

In the future, based on the ankle joint angle from the encoder of the DC motor, this study deploy to the cloud services, such as remote monitoring by orthopaedists or physical therapists and real-time damping adjustment.

Fig. 2 .

 2 Fig. 1. Normal gait

Fig. 3

 3 Fig. 3. iAFO

Fig. 4 .

 4 Fig. 4. Ankle angles at gait experiments

Fig. 5 .

 5 Fig. 5. Consumed current (One gait cycle) Fig. 6. Charged current


Takakuni Iituka

Kyoko Shibata

Yoshio Inoue
email: inoue.yoshio@kochi-tech.ac.jp

Estimation of Radius of Curvature of Lumbar Spine Using Bending Sensor for Low Back Pain Prevention

Keywords: Bending sensor, Herniated disk, Wearable sensing system, Radius of curvature

Estimation of the disk load in order to prevent low back pain is useful. However, the conventional methods of measuring disc load are invasive and their use is limited due to measurement environments. This study proposes a new method of estimating the lumbar disc load to measure curvature of the lumbar portion and to estimate the lumbar disc load safely using a bending sensor. The radius curvature can be measured relatively easily and without damaging the body by using this method. The bending sensors are attached along vertebra end of five vertebras and the curvature of the lumbar portion is estimated by reading the change in output voltage. The lumbar disk load with static posture was estimated by proposed method. The result shows the same tendency as the previous method. The proposed method has a possibility of developing a new system using the biofeedback based on the lumbar disc load.

Introduction

In recent years, people suffering from low back pain increase by changes in labor environment and living environment. But the obvious way for preventing low back pain is not yet introduced. So it is useful to develop of wearable sensor system for preventing low back pain. Although there are various causes in low back pain, this study targets a herniated disk. Herniated disk works as cushioning material that exists between the vertebrae of the lumbar. Pain and numbness come out when excessive force is applied to herniated disk. Therefore we can make use for treatment, rehabilitation and prevention if it is possible to measure the load on the disk.

Useful method of measuring the load on the disk has been reported up to the present time. One of them is a method of inserting a direct pressure sensor into the disk by Nachemson an orthopedist in Sweden. This method is a highly accurate, and measurement can be done in various positions but it has a risk of damaging the body, sometimes putting a patient into surgical operation. The other previous research [START_REF] Andersson. | LUMBAR DISC PRESSURE AND MYOELECTRIC BACK MUSCLE ACTIVITY DURING SITTING",I.Studies on an Experimental Chair[END_REF][START_REF] Rohlman | Analysis of the influence of disc degeneration on the mechanical behavior of a lumbar motion segment using the finite element method[END_REF] suggests that there is a way to measure the intervertebral disc load by using a nonlinear three-dimensional finite element model of the vertebrae disk based on the CTs and to estimate the joint moments and muscle tone by adapting the inverse kinematics. But it is difficult to measure the mechanical properties of the dynamic characteristics because it is a partial model links. To measure with high precision and the procedure of measurement is complicated because there are various factors to be considered. The present study group focuses on the vertebral edge which exists near skin surface of the back. Since lumbar system is curved due to the attitude change [START_REF] Kyoko | Sutudy on Noninvasive Estimate Method for Intervertebral Disk Load at Lumbar Vertebrae[END_REF] , a bending sensor is attached along the five vertebral edge portions that constitute the lumbar part and the curvature of the lumbar part is estimated by reading the change in the output voltage. With this method measurement can be done relatively easily without damaging the body. Since the bending sensor is thin, cheap and wearable, it can be worn in everyday life. The proposed method has a possibility of developing a new system using the biofeedback based on the lumbar disk load, and in the future, this system could be used for preventing low back pain and improving the posture. In this paper, the radius of curvature of lumbar spine is estimated using the same position that Nachemson did in his experiment. The validity of the proposed method is considered by comparing the intervertebral disc load estimation result of Nachemson and the proposed method.

METHODS

Fig. 1 shows a schematic diagram of spine, lumbar part and vertebrae. As the Fig. 1 indicates, disc exists between the vertebrae and has the role of cushioning material. Upper and lower vertebrae are connected with a pin at the facet joints. Vertebrae should be considered as a rigid body 3 because the Young's modulus is larger than the disc and the human body. The elastic deformation in the zygapophysial joint is contravened because it is very small. If attitude changes, the gap between the vertebral bodies changes as vertebrae move. That is to say, change of the disk load is caused by the gap change between the vertebral bodies. Let us consider that the lumbar spine moves uniformly, and that the internal pressure in the lumber disc is also uniform. The radius of curvature is obtained by the approximation by arc of the surface shape of body. The average intervertebral disk load is derived 3 by estimating the gap variation between the average vertebrae using the changes of the radius of curvature in the lumbar system with the posture change. Bending sensor is a variable resistance. Resistance value changes according to the force applied to it. The radius of curvature is estimated from the change in the resistance value of the bending sensor attached on the skin surface of lumbar. Bending sensors are attached along vertebral end from the first lumbar spine to the fifth lumbar spine (L1-L5). The resistance value is calculated from output voltage and the radius of curvature of the lumbar spine is estimated. Figure2 shows calibration curve which represents the relationship between the known radius of curvature and the resistance value.

Fig. 2. Calibration curve

The resistance value obtained from the bending sensor is substituted into the calibration curve to estimate the radius of curvature when the bending sensor is attached to the lumbar part of the human body. The test is administered in four different positions, two standings and two sittings (Fig. 4). These are the same as Nachemson used in his experiment, where the electrodes were inserted directly [START_REF] Andersson. | LUMBAR DISC PRESSURE AND MYOELECTRIC BACK MUSCLE ACTIVITY DURING SITTING",I.Studies on an Experimental Chair[END_REF] . A measurement was done four times with each position to all three subjects.

RESULTS

Fig. 4 shows comparison of the lumbar load estimation of experimental results by Nachemson and the proposed method. The same tendency as Nachemson was obtained in all subjects that the lumbar disk load is large when taking the bending forward positions.

DISCUSSION

The some variations intervertebral disk load are found. This is because body movements of the subjects were slightly different and also because a curve of the data sheet is approximated by a quadratic function. It could be considered that the estimating of the radius of curvature of lumbar spine is possible with static posture using the measurement method proposed in this study. The proposed sensing system can be applied to car seats and their use is not limited due to measurement environment. The proposed method has a possibility of developing a new system using the biofeedback based on the lumbar disk load, and in the future, this system could be used for preventing low back pain and improving the posture.

Fig. 1 .

 1 Fig. 1. Lumbar vertebrae

Fig. 3 .

 3 Fig. 3. Measurement static postures

Fig. 4 .

 4 Fig. 4. Comparison of the lumbar load estimation of experimental results

Table 1 .

 1 Information of subjects

	Subject	Age	Height[m] Weight[kg]	BMI
	A	22	1.67	66	23.70
	B	22	1.69	54	18.90
	C	22	1.72	70	25.40


Sidney Fels

Michael Lyons

How to Design and Build New Musical Interfaces

de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

How to Design and Build New Musical Interfaces

Sidney Fels and Michael Lyons

University of British Columbia and Ritsumeiken University Abstract. This half day course introduces the field of musical interface design and implementation. Participants will learn and practice key aspects of the theory and practice of designing original interactive music technology with case studies including augmented and sensor based instruments, audio-visual instruments, mobile, and networked music making. Digital technologies offer powerful opportunities for the creation and manipulation of sound, however the flexibility of these technologies implies a confusing array of choices for musical composers and performers. Some artists have faced this challenge by using computers directly to create new musical forms. However, most would agree the computer is not a musical instrument, in the same sense as traditional instruments, and it is natural to ask 'how to play the computer' in a way appropriate to human brains and bodies.

Intended Audience

Our primary objective in presenting this tutorial is to bring participants upto-speed on the application of interactive techniques to music technology. No specific technical background is required, though some familarity with interactive systems may be helpful. In particular, we do not assume any prior knowledge of sensors, microcontrollers, audio synthesis methods, but rather aim to provide participants with an introduction to these areas, as well as pointers to resources for further study. As such, the tutorial should be well suited to beginning graduate students or advanced undergraduates who are interested in this area of research; composers and performers who wish to expand their repertoire of methods and tools; member of the general public curious about recent developments and ongoing research in music technology.

Reading list

The course notes will be available for attendees to read prior to the course. Currently, a version of the notes can be found at: DOI: 10.1145/2659467.2659470

Fig. 1 .

 1 Fig. 1. Example Case Studies from the New Interfaces for Musical Expression (NIME) conference.


Bernhard Maurer

Roland Buchner

Martin Murer

Manfred Tscheligi

Actuated Shear: Enabling Haptic Feedback on Rich Touch Interfaces

Keywords: Rich Touch, Shear Force, Actuated Displays, Haptic Feedback

We present an approach of a shear force based touch interface that provides a way of actively changing the possible shear-based input (force tangential to a screens surface) by physically locking the corresponding axis of the device. This approach of actuated shear aims at using shear not only as input, but to create a new form of output modality that changes the input affordance of the device itself. It enables a new channel of incorporating physical information and constraints into touch-based interaction (i.e., by changing the input affordance of the device and using shear as a feedback mechanism). With this actuated shear approach, we create a coupling between the digital context created via touch and the actual physical input affordance of the device. Based on the implementation of a prototype, we discuss the design space of actively changing the input affordance of a shear-based touch device, sketch interaction ideas as well as future application scenarios and domains.

Introduction

Within recent years touch-based interfaces have become more and more pervasive and with the introduction of multi-touch capable touch screens a new level of interaction was possible. However, the resulting interactions were mainly based on using the X and Y coordinates of a user's touch. To further improve the interaction and to create new forms of touch interaction, much research effort has been going on to enrich touch-based interaction by using more data related with the user's touch (e.g., [START_REF] Heo | ForceDrag: using pressure as a touch input modifier. Using pressure as a touch input modifier[END_REF], [START_REF] Huang | Touchsense: Expanding touchscreen input vocabulary using different areas of users' finger pads[END_REF]) This research stream was coined by Harrison et al. as "Rich Touch", as these touch interactions go beyond traditional ones. To further study the design space of rich touch Harrison et al. [START_REF] Harrison | Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction[END_REF] introduced the concept of using shear (the force tangential to a screens surface) as a supplemental 2-dimensional input channel. Thereby, a user exerts a force on the surface of an object (e.g., touchscreen of a mobile phone) into a tangential direction, which in turn moves the object physically. Shear-based interactions on surfaces consist of two forms of forcethe tangential force to move the object and friction force that is created by applying pressure on such an object. Previous concepts of incorporating shear force into touch-based interactions, however, only use shear for input purposes. Also, the challenge of how to provide feedback for shear force interactions is still an open issue [START_REF] Harrison | Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction[END_REF]. The goal of our research was to address this problem of giving feedback in shear-based touch interactions by utilizing actuated shear and explore the resulting design space for touch interaction. Therefore, we built a prototype that extends a touch interface with actuated shear for physically limiting the shear-based input, i.e., locking the devices corresponding X and Y-axis. Actuated shear allows us to create a new way of physical output -namely changing the shear-based input affordance of the device itself to regards of the digital context created via the user's touch (i.e., area of touch changes shear input behaviour). This approach provides an opportunity to give feedback during shear force based touch interaction, and to create a coupling between the digital context created via the user's touch and the device's shear input affordance. Our concept of actuated shear extends prior work (see [START_REF] Harrison | Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction[END_REF]) as we are able to actively change the possible axis of shear-based input and use this for feedback purposes. In this paper we present our approach of actuated shear for touch-based interaction and our exploratory prototype. We describe several use cases of how the concept of actuated shear can be utilized as a supplemental feedback and output channel. We finish with a discussion on the resulting design space and point out future work in this area.

Related Work

Rich touch approaches do not only use X and Y coordinates of a user's touch: new interactions are possible by using e.g., different areas of the fingers [START_REF] Huang | Touchsense: Expanding touchscreen input vocabulary using different areas of users' finger pads[END_REF] or thumb gestures [START_REF] Roudaut | Microrolls: Expanding touch-screen input vocabulary by distinguishing rolls vs. slides of the thumb[END_REF], therefore enabling an additional layer of input. Harrison et al. [START_REF] Harrison | Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction[END_REF] investigated how shear force (force applied tangential to the screens surface) could be applied as touch input on a tablet by linking common touch gestures (e.g., double tap) to this new layer of input. Shear force input was further applied to the field of smart watch interaction by creating a device which is capable of being physically tilted, panned, clicked, and twisted for interaction purposes [START_REF] Xiao | Expanding the Input Expressivity of Smartwatches with Mechanical Pan, Twist, Tilt and Click[END_REF]. However, rich touch interactions still lack proper feedback [START_REF] Harrison | Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction[END_REF]. There is much research effort on how feedback on touch interfaces can be improved, e.g., near-surface feedback [START_REF] Antolini | A haptic direction indicator using the gyro effect[END_REF], or tactile feedback using air [START_REF] Sodhi | Aireal: Interactive tactile experiences in free air[END_REF]. These approaches are mostly concerned to guide the user's touch input. The area of research on actuated displays shows how dynamic, on-demand control of the physical properties of interfaces can improve interaction [START_REF] Poupyrev | Actuation and tangible user interfaces: The vaucanson duck, robots, and shape displays[END_REF]. Previous work on actuated displays was done by researching shape changing displays [START_REF] Hemmert | Dynamic knobs: Shape change as a means of interaction on a mobile phone[END_REF] or by creating displays that are actuated to dynamically present different physical forms as feedback to the user [START_REF] Alexander | Tilt displays: Designing display surfaces with multi-axis tilting and actuation[END_REF], [START_REF] Roudaut | Morphees: Toward high "shape resolution" in self-actuated exible mobile devices[END_REF]. Nevertheless, as shear-based touch interfaces still lack proper feedback, we want to address this issue by extending the concept of shear force interaction using actuated shear to enable dynamic changes of the physical input affordance presented of a touch device. In that sense, we use shear not only as input but for actively locking the X and Y axis as a new form of output and supplemental feedback modality. We utilize the digital context created via the user's touch to change the devices shear-based input behaviour. This creates a coupling between digital context and the device's shear be-havior. With our work, we contribute to the existing research efforts towards contextually aware rich touch interfaces.

Prototype Device

To explore the design space of using actuated shear, we built a prototype of a touch device that follows the concept of actively constraining the possible shear input axis (see Fig. 1). The prototype was built using a 7" tablet and a set of Maker Beam1 as a frame to hold the device. The frame holds buttons on each side of the tablet that are activated when shear force is applied via the user's touch. Each side of the device is actuated via Arduino2 controlled solenoids that can actively lock the different axis of the shear input. For instance, activating the solenoids on the left and right locks the device's shear input in the corresponding axis. We used solenoids because they are capable of changing their state rapidly. This enables us to quickly adapt the possible shear input based on the users input, resulting in a more seamless interaction. The communication between the tablet and the Arduino hardware was done via Spacebrew 3 , which is a websocket-based prototyping framework that allowed us to quickly connect the prototypes hardware and software parts.

Utilizing Actuated Shear

In the following, we describe different aspects of how our approach of actuated shear enables new interaction concepts. These concepts are the result of exploring the design space with our prototype and applying it to different contexts and domains by building a range of application prototypes. The current prototype allows us to explore the following types of interactions: actuated shear as haptic and directional feedback, embodied interaction by coupling digital context with matching actuated shear, and multimodal interaction based on constraining shear input to specific areas of the screen.

Actuated Shear as Haptic and Directional Feedback

Since the lack of feedback on touch screens is still an important issue, we see actuated shear as a further step to improve the feedback on such devices. When designing shear-based touch interaction, the question of how to design for visual feedback is also still an open question [START_REF] Harrison | Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction[END_REF]. We want to address this lack of feedback on shearbased devices via haptic feedback created with actuated shear as it can be used to support the visual feedback presented on screen or to set tangible "boundaries". For instance, when the maximum of a slider is reached, the corresponding shear input axis can be locked as a physical indicator. Further, our approach can be utilized to enable directional feedback. By activating a specific axis of the touch device (e.g., using solenoids as actuators) the device can be moved a short distance within the boundaries of the prototype's frame. Therefore, we can support the visual "snapping" of digital objects e.g., during a drag and drop task with the corresponding directional feedback (slightly pushing the device in the target direction) to create the perception of two items "snapping" to each other (see Fig. 2).

Fig. 2.

Utilizing actuated shear as directional feedback in a drag and drop task. The user drags a digital object while actuated shear feedback is applied to create a tangible "snapping effect

Changing Shear Input Affordance

By constraining the device's shear input capabilities, we create a channel of physical output that can be used to change the device's input affordance. Hence, the digital context (e.g., location of the user's touch) and the device's "shear affordance" can be coupled. For instance, this can be used to create an affordance coupling between physical movement attributes of UI elements (e.g., a slider moving horizontally) and the device's affordance itself (i.e., shear force constrained to X axis). The touch input is used to set the digital context (e.g., the selection of a number value), whereas the shear input changes the value itself. In this example, adjusting a number value corresponds to a slider element, thus, the device locks the corresponding Y axis to make the device moveable only in the X axis. Hence, the properties of a slider, i.e., being able to move left and right, are applied to the shear input affordance of the device (see Fig. 3). In contrast to that, touching on areas related to boolean values (i.e., switching something on or off) conveys the affordance of a toggle, i.e., the ability to move up and down, by locking the corresponding shear input axis of the device.

This mapping between shear input affordance and the digital context created via touch input, can be used to support (semi) blind interaction scenarios in safety critical application domains like the car. In-car scenarios are often based on peripheral interaction with a touch screen in the middle console, while visually focusing on the main driving task (interaction remains the secondary task). Touch screens become more and more present in nowadays cars, however, the interaction with such interfaces lack the haptic nature of physical interface elements making peripheral interaction potentially harder. Actuated shear can integrate haptic guidance, that would be conveyed by a physical slider or toggle, into touch interaction (e.g., touch screen incorporates movement affordance of a slider).

Constrain Shear Input to a Specific Screen Area

By actively locking a specific side around the device, we are able to constrain the possible shear force input to specific screen areas (see Fig. 4 a andb). Locking a single side of the device, enables "rotational" shear input (i.e., device moves in rotational manner around the locked axis that defines a rotation point). One exemplary use case is to consider actuated in-situ manipulation of objects, e.g., selecting an object via touch and rotating it via shear-based input (see Fig. 4 a). The digital context (i.e., position of the user's touch and type of object to be manipulated) defines the shear input behaviour and which area of the screen is locked correspondingly. For instance, when the user touches the corner of a digital object (e.g., via long-press), the device locks its axis at the corresponding side by activating its solenoids. That creates a rotation point in the center of this axis enabling a rotate interaction via shear input.

There are more such object manipulation interactions possible like flipping an object horizontally/vertically or turning it for 90 degrees. Such interactions can be utilized for touch interactions where exact positioning and adjusting is necessary (e.g., touchbased image manipulation). Further, actuated shear could be used for game-based interactions on touch devices. For instance, Fig 4 b illustrates another concept of limiting the possible shear force to a specific screen area. By using a typical overlay of a physical game controller and limiting the shear movement to one side of the device, we create two different parallel interaction modalities on the same touch device.

Design Space

Based on the exploratory prototype and the above mentioned aspects of how to utilize actuated shear, we describe three levels of information that can be conveyed by actuated shear. We do this by applying a framing introduced by Antolini et al. [START_REF] Alexander | Tilt displays: Designing display surfaces with multi-axis tilting and actuation[END_REF] that also applies to actuated shear as a feedback modality.

Information redundancy: Actuated shear can be used to create feedback redundant to visual on-screen information (i.e., coupling of possible shear movement and digital context). This additional channel of information can be used in (semi) blind/peripheral interaction scenarios or to provide feedback in in-situ touch manipulation tasks.

Information transposition: Actuated shear provides a way of giving feedback beyond what is felt via touch. The sensory substitution provided by our approach can be used to provide e.g., directional cues or physically perceivable virtual "boundaries" for touch-based games.

Information balance: Actuated shear can be used to reduce the amount of visual on-screen information needed, based on providing information via this new layer of output. We address the lacking visual feedback of other shear-based approaches with haptic feedback created by actuated shear. Therefore, this haptic feedback channel can be used instead of providing visual feedback in order to save screen real estate.

Findings and Limitations

First feedback from our explorations showed that the shear "state changes" our approach provides, should be synchronous with the touch input in order to enable seamless interaction between the user's touch input and the feedback conveyed by actuated shear. Otherwise the potential lag in feedback can easily be misinterpreted by the user. Further, an open issue to consider is that by providing feedback via actuation during shear-based touch interaction, the user's touch itself is also influenced. This in turn may lead to potential unintended inputs or loosing the initial focus of touch. The presented prototype acts as a proof of concept and provided a base for our design space explorations of using actuated shear for rich touch interactions. As we focused on the aspect of actively locking specific axis of the possible shear input, the buttonbased detection of the actual shear force was simplified in our prototype (only stepwise, on/off). Further, the locking of the device's axis movement can thus far only be switched on or off. In future iterations of the hardware we aim at continuously varying the strength of the actuated shear feedback (i.e., pressure applied against the user's shear force vector) to enable more diverse feedback possibilities. For future iterations of the hardware we aim at making the device smaller to be truly handheld in order to explore the resulting design space of utilizing actuated shear on small touch displays (e.g., smart watches).

Conclusion

With actuated shear we presented a supplemental 2D feedback channel to change the shear-based input affordance of a touch device. By sketching a range of use cases that extend the possible interactions and feedback on a shear-based touch device, we showed how this approach can be applied to different shear-based interactions. The concept of actuation extends and complements the existing concept of using shear force for rich touch interaction. This new modality allows a coupling between the digital context, created by the user's touch and the device's movement affordance, which results in richer touch-based interactions. Existing shear-based rich touch approaches (e.g., [START_REF] Harrison | Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction[END_REF], [START_REF] Heo | ForceDrag: using pressure as a touch input modifier. Using pressure as a touch input modifier[END_REF]) enable a wide range of new input possibilities, whereas our approach extends them by enabling new feedback and interaction mechanisms with the opportunity to lock the device's X and Y shear input axis actively and independently. Combining our approach of actuating the possible shear input with existing approaches of dynamically processing shear force (with more detail compared to our prototype) would result in a substantial extension of what is currently possible on shear-based touch interfaces. We will further conduct a series of user studies to gain insights on the feedback provided by actuated shear and how it affects the user's experience. At the moment only the digital context -created via touch input -is used to influence the shear actuation behaviour. In our future research we aim at incorporating the physical context (e.g., physical surroundings influencing shear input behaviour) to enable couplings between the physical context and potential physically related interactions (e.g., collaborating with other co-located humans). The fact that the affordance changes that actuated shear conveys are not visually perceivable, but have to be sensed via touch, bears potential for interesting touch-based interaction that use "touch" beyond what is currently the case (i.e., not only as input but as a means for sensing) as well as for embodied interaction design to further enrich touch-based input modalities.

Fig. 1 .

 1 Fig. 1. Picture of prototype device. The device captures 2D touch location and 2D shear input. Each solenoid on the side of the tablet can be activated independently to lock a specific axis of the shear input (e.g., based on the touch input of the user).

Fig. 3 .

 3 Fig. 3. Translation of the physical movement attributes of the UI elements into changes of the device's possible shear input. The touch input, i.e., area of screen, sets the digital context to actuate/lock the corresponding axes.

Fig. 4 .

 4 Fig. 4. a) Actuated shear-based in-situ manipulation of objects. b) Game controller overlay on touch screen. Left side with shear-based "joystick" overlay; shear movement on right side is locked.

http://www.makerbeam.eu

http://www.arduino.cc

http://docs.spacebrew.cc

Acknowledgments.

The financial support by the Austrian Federal Ministry of Science, Research and Economy and the National Foundation for Research, Technology and Development is gratefully acknowledged (Christian Doppler Laboratory for Contextual Interfaces).