Alexander Ng

Stephen A Brewster
email: stephen.brewster@glasgow.ac.uk

John Williamson

The Impact of Encumbrance on Mobile Interactions

Keywords: Encumbrance, Mobility, Mobile interactions, Target acquisition

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Mobile devices such as smartphones play a vital role in our everyday activities as they allow users to perform common tasks such as talking to friends, emailing documents to colleagues and searching for nearby services while on the move. As a result, mobile devices are being used in a wide range of different contexts and it is important to examine if users experience interaction and usability difficulties when faced with potentially demanding multitasking situations. One context that has not been explored in great depth is studying users when they are encumbered: holding typical objects as such handbags, umbrellas, shopping bags and boxes while engaging with their devices simultaneously. People frequently carry these objects while walking from one place to another and as a result, users often need to use their phones at the same time to send messages, look at maps or refer to other services. Therefore, encumbrance and mobility is closely linked with each other. Using a mobile phone in these kinds of situations while physically hampered is challenging as it can be awkward to see the screen and make input. It would be valuable to understand how encumbrance affects interactions with touchscreen mobile devices so that better input techniques and error detection could be developed to negate the issues and problems it causes. Also, carrying different types of objects may have a different impact on the user thus it is beneficial to be able to categorize the different types of encumbrances by the way they affect the user's performance when interacting with mobile devices.

Since users are likely to carry objects between places, it is also imperative to examine how walking and encumbrance together affect input performance on touchscreen mobile devices. The physical motion required to walk naturally causes the user's arm to move and swing in phase with each foot step. However, this natural stance is disrupted when attention from the mobile device is required as the user attempts at steadying their constantly moving arm and hand to input. The effects of encumbrance is likely to cause interaction to become even more physically demanding and put significant pressure on the user's mental ability to multitask as they walk and navigate the surrounding area while avoiding nearby obstacles. The issues caused by walking alone on mobile interactions have previously been investigated in great detail as researchers have developed various solutions to assist the user to input more accurately [START_REF] Goel | WalkType: using accelerometer data to accommodate situational impairments in mobile touchscreen text entry[END_REF] and [START_REF] Kane | Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces[END_REF]. However, it is unclear what further issues the introduction of encumbrance causes on interaction when the user is walking. Therefore, it is important to understand if performance is further worsened by encumbrance and to detect new usability problems that previously have been overlooked. It would also be interesting to see if encumbrance and mobility causes distinctive interaction issues therefore better applications could be developed to assist the user to maintain a good control of their devices in realistic multitasking situations.

To examine the effects of encumbrance and mobility on mobile interactions, we defined a set of common objects that users frequently carry to be assessed in our experiment. There is a large number of possible objects that people hold in their daily activities and the issue is further complicated the object's characteristics such as its size, shape, heaviness and quantity. Also, a particular object could be held in various different ways which normally depend on personal preference and the context the person is currently in. As a result, it is important to focus on the typical strategies that users perform while encumbered in order to simultaneously use their mobile device. An observational study was carried out to find the most common types of objects and the way they were being carried in a range of public settings. Based on the results collected from the study, the encumbrance experiment examined two types of objects: holding a bag in hand and carrying a box underarm. A target acquisition task was used on a touchscreen mobile phone to evaluate the effects of the two types of encumbrances and mobility. The participants performed target selections on a touchscreen mobile phone and holding the different types of objects at the same time while either standing still or walking to simulate realistic encumbrance scenarios. To investigate how holding the different types of bags and boxes cause targeting difficulties, motion capture cameras were used to track hand and arm movements during interaction. The assumption was that carrying different objects while walking will cause more physical movements and instability to the user's hand and arm which makes it more challenging for the user to maintain a steady position to input accurately on the touchscreen device.

Background

This section of the paper will review the related literature with the first part discussing research that has examined encumbrance and the second part reviewing studies on mobile interactions while walking.

The Effects of Encumbrance

There has been little research that specifically examined the interaction problems caused by encumbrance due to holding common objects. Ng et al. [START_REF] Ng | The Effects of Encumbrance on Mobile Gesture Interactions[END_REF] studied the topic of encumbrance by examining wrist rotation gestures as a novel hands-free and eyes-free interaction technique to reduce the issues caused by holding a bag and box when interacting with a mobile phone. A small bag and a rectangular box were evaluated to replicate some of the effects of carrying cumbersome objects. The main motivation behind the study was to illustrate that holding common objects does have an impact on mobile interactions and different types of objects have a different impact on the user's performance. The findings from their Fitts' Law targeting experiment suggested holding the bag caused users to become less accurate while the performance of carrying the box underarm was similar to unencumbered. The swinging motion caused when holding the bag made it difficult to steady the arm for input while the users were able to stabilize their forearm when gripping the box in place which helped them to perform wrist gestures more easily. Target movement times between the two types of encumbrances were not significant. The study also showed that sensor components such as gyroscopes that are found in most modern mobile devices could be used to detect precise wrist orientation and movements and therefore support the user when they are physically hindered.

Oulasvirta and Bergstrom-Lehtovirta [START_REF] Oulasvirta | Ease of juggling: studying the effects of manual multitasking[END_REF] studied the relationship between holding a group of smaller-sized objects and input accuracy on different computing devices. Twelve different multitasking situations were examined which covered a range of hand grips and arm postures while holding a variety of everyday items such as a writing pen and a beverage cup. Participant performed the various types of encumbrances while selecting targets on a laptop computer via mouse and trackpad input and text entry by different forms of keyboards on a mobile phone. The results showed that holding objects such as a pen while performing the target selection task on the laptop by mouse input caused performance to decline more than the trackpad. One-handed text entry on the mobile device while holding an object (such as a pair of scissors) caused a decrease in performance when compared to two-handed input especially with single-handed stylus targeting since it normally requires both hands to input. Interestingly, text entry via a virtual keyboard exerted better results than the physical equivalent as more finger pressing pressure is required. The study examined a set of interesting manual tasks and included activities that require a pushing action rather than the common holding grip in hand. One final point that the study makes is the notion of safety as there are situations where it may not be possible to prevent hindrance when using mobile devices (for example, holding small children). Better context detection systems and more effective input techniques are required to help users during interaction when confronted with these challenging situations. Wolf [START_REF] Wolf | Microinteractions beside ongoing manual tasks[END_REF] assessed how people performed manual tasks by examining hand grips and positions in order to explore the areas that are free to perform secondary interaction activities.

Mainwaring et al. [START_REF] Mainwaring | Living for the global city: mobile kits, urban interfaces, and ubicomp[END_REF] conducted an ethnographic study across three major cities to examine the personal connection between the items that people carried and how these items were used in their daily context with surrounding people and interfaces. Items were classified into various categories such as those which distracted the user from the environment (music players, phones and books), personal belongings (wallets, keys, make-up) and professional tools (laptop, PDAs). The findings from the study suggested different items have its own unique personal value and as a result different objects may have a varied impact on interaction with mobile devices. The issue of a particular object's personal value to the user creates an unusual viewpoint on encumbrance. For example, dropping a wallet could be more disastrous and frustrating to the user than a mobile phone while on the move. It would be worthwhile to investigate if this is the case and if objects can also be grouped by personal value rather than the standard categories of the object's size, shape or how it obstructs the user.

Tamminen et al. [START_REF] Tamminen | Understanding mobile contexts[END_REF] observed how outdoor environments constantly compete for the user's attention and discovered repeated instances where the user's hands were busy performing activities ranging from holding a newspaper while travelling between locations to clutching a cigarette packet while searching for money placed in the person's pockets. Performing several activities at the same time is likely to cause the user difficulty in dividing its visual attention to complete each individual task successfully as discussed by [START_REF] Oulasvirta | Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI[END_REF]. There needs to be more efficient interaction methods and alternative techniques to assist the user when they are physically impaired and visually distracted. People carry personal belongings and objects from one place to another therefore it is also important to examine the impact mobility has on mobile interactions and its relationship with encumbrance.

The Effects of Mobility

The interaction difficulties caused by walking while using mobile devices have been well documented. Bergstrom-Lehtovirta et al. [START_REF] Bergstrom-Lehtovirta | The Effects of Walking Speed on Target Acquisition on a Touchscreen Interface[END_REF] examined the relationship between target selections on a touchscreen mobile phone and the user's preferred walking speed (PWS) on a treadmill. The results showed that when users walked approximately between 40 -80% of their PWS, performance began to level as users were able to stabilize the dominant hand more successfully to input more accurately on the touchscreen mobile phone. Mizobuchi et al. [START_REF] Mizobuchi | Mobile text entry: relationship between walking speed and text input task difficulty[END_REF] recorded an average walking speed of 1.77 km/h when examining the relationship between mobility and button size. This finding is much lower than the walking speed of an average adult human being [START_REF] Knoblauch | [END_REF]. It is important to see if holding different types of objects cause the user to reduce their walking speed even further and if the slowdown in pace meant that users were able to keep a reasonable level of input accuracy and performance.

Kane et al. [START_REF] Kane | Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces[END_REF] attempted to compensate some of the problems caused by mobility by developing Walking User Interfaces (WUI) which dynamically change the characteristics of interface widgets to support the user to input more effectively while walking. The results showed a trade-off between button size and the amount of effort and time required to scroll the screen to make the appropriate selection. This was also discussed by Schildbach and Rukzio [START_REF] Schildbach | Investigating selection and reading performance on a mobile phone while walking[END_REF]. The performance of their WUI prototype was comparable to an equivalent static interface although it was not as effective as a fixed layout with larger sized buttons. Goel et al. [START_REF] Goel | WalkType: using accelerometer data to accommodate situational impairments in mobile touchscreen text entry[END_REF] and Nicolau and Jorge [START_REF] Nicolau | Touch typing using thumbs: und understanding the effect of mobility and hand posture[END_REF] have also studied the issues of walking and developed better text entry systems to help typing on touchscreen mobile devices. It would be useful to examine if these applications and similar interfaces are still as effective and could solve some of the interaction problems caused by encumbrance.

Brewster [START_REF] Brewster | Overcoming the lack of screen space on mobile computer[END_REF] showed that button pressing on a PDA was more accurate and subjectively easier when the user was sitting down compared to walking outdoors. A drop in performance of approximately 30% was recorded and one possible cause could have been due to the increased mental attention required to walk and navigate the environment while engaging with the mobile device at the same time. The effects of encumbrance may result in error rate to increase further between standing and walking. The other aspect that is worth considering is evaluating walking-based experiments in laboratory settings and outdoor environments. Our experiment was conducted inside a quiet room due to the restrictions of the motion capture hardware. Consequently, this may have had an undesired effect on targeting performance when carrying the different types of objects since the indoor environment is much calmer than a real world setting therefore making the context less challenging for the user. However, Lin et al. [START_REF] Lin | How do people tap when walking? An empirical investigation of nomadic data entry[END_REF] suggested using an artificial route with obstacles to increase the user's cognitive workload to a level similar to walking in outdoor settings.

Moving away from HCI literature to examine how walking affects the user's mental performance, Lajoie et al. [START_REF] Lajoie | Attentional demands for static and dynamic equilibrium[END_REF] discussed attentional demands for static and dynamic tasks. The dual-task procedure was used where subjects reacted to auditory stimuli in three different mobility positions (sitting, standing and walking). The results showed walking required more cognitive attention than standing and sitting down due to the additional motion of balance needed to walk. Pellecchia [START_REF] Pellecchia | Postural sway increases with attentional demands of concurrent cognitive task[END_REF] examined the relationship between mental demands and muscle motor movements and indicated that by increasing the difficulty of the cognitive task resulted in greater body movements in terms of postural sway. This suggests there is a close relation between the person's cognitive abilities and human motor performance. If walking alone can cause increased mental stresses on the user, it is important to examine if encumbrance creates similar cognitive difficulties therefore affecting the user's ability to engage with mobile devices effectively when interaction is required.

The limited research on encumbrance suggests the topic is at its early stages and a better understanding into the usability problems it causes can be beneficial to mobile device users. Although many studies have examined the issues of walking during interaction and solutions have been developed to enhance the user experience, the effects of mobility combined with encumbrance may result in much greater problems therefore it is crucial to examine if such issues do occur. The next section of the paper will discuss the methodology of our experiment which was conducted to investigate the impact of carrying different types of objects on targeting performance on a touchscreen mobile phone.

Methodology

The section will be split into three parts to describe the procedure taken for the encumbrance experiment.

The first part will discuss the initial observational study conducted to classify the most common objects that users were seen to carry and to choose two types of those objects to be evaluated in the experiment. The second part will describe the use of a motion capture system to detect hand postures and body movements and how users interacted with the mobile phone when encumbered and walking. The third part will describe the target acquisition task and the design of the experiment.

Common Types of Encumbrance Objects

The main purpose of the observational study was to examine the objects that users were seen to carry regularly while using a mobile device and to group those objects into suitable categories. Since there is a vast amount of possible objects that could encumber the user, it is essential to identify and concentrate on the most common types of objects that are likely to be held during interaction. In order to define a set of encumbrance objects to be assessed in the experiment, three different types of public locations were observed (main street, transport station and supermarket) to examine the wide variety of objects that people held and carried. The experimenter would observe the general public during peak times for two hours at each type of location (for example, early commuting hours between 8am to 10am at a railway station and lunch period between 12pm and 2pm in a supermarket) since there will be a great influx of people and the probability of seeing a range of different objects is increased. Two different sites for each type of location were observed which resulted in six set of data. Each object seen being held or carried was noted down in terms of the following characteristics: type, shape (rectangular or round?), size (length, width & thickness), quantity, input hand (non-dominant, dominant or both), hand action and grip required, and arm posture.

Once the observational study was completed, the data collected was firstly grouped by object type and then sorted by how often it was noted down during the study. The results showed that different types of bags were the most frequently held objects as it account for 49% of all the items recorded. Boxes were the second most common object with 35% while the remaining 16% of objects documented ranged from beverage cups, umbrellas, specialized equipment to children, prams and pet leashes. The objects recorded were also separately categorized based on the arm movement and hand actions required by the user. There were four main categories: 1. swinging -objects such as bags and holding a child's hand were placed into this group as it caused the arm to swing somewhat unpredictably; 2. bulky -boxes were grouped into this class as people were seen to hold different types of boxes normally underarm which required an awkward but assuring grip from the arm to prevent the object from falling to the ground; 3. push and pull -objects such as prams and trolleys were put into the push class while people were seen pulling wheeled luggage; and 4. complexobjects in this category included keys, wallets and hot beverages which require more careful and intricate finger action and grip.

Based on our observations of people in the public environment, different types of bags and boxes were chosen as the encumbrance objects to be evaluated in the experiment. Since there was a great variety of bags and boxes noted during the observation, the decision was made to evaluate two different types of each object based on its size and shape. Therefore, the experiment assessed two bags (small and medium sized) and two boxes (thin and thick broadness). The small bag represented a handbag while the medium bag simulated people carrying a shopping bag. The dimensions (width x height x depth) of the small and medium bags were 35 x 25 x 17 cm and 45 x 55 x 25 cm respectively. The thin and thick boxes measured 37 x 30 x 15 cm and 39 x 30 x 29 cm respectively. The bags were held in hand while the boxes were carried underarm as people were seen to adopt these strategies during the observational study. All bags and boxes weighed 3kg each to keep the object's heaviness consistent. The weight of 3kg was chosen to simulate the effects of carrying realistic objects that would make interaction physically difficult yet limit the amount of physical straining on the participants. The objects and the method that they were held during the experiment are shown in Figure 1.

Tracking Hand and Arm Motions

To understand how holding the different objects encumbered the user and to analyze the level of hand and arm instability, a Vicon motion capture camera system was used (http://www.vicon.com). Twelve infrared cameras sampling at 120Hz recorded body movements (to a thousandth of a millimetre) in three-dimensions by tracking reflective markers which were firmly attached to each participant at specific body locations to track their movements while performing the experimental task in the capturing volume. The reflective markers were placed on the front and back of the upper torso and the hand and arm areas. A total of 15 markers were attached to each participant and Figure 2 illustrates their location.

The marker on the neck was used to calculate the total distance walked and the average walking speed for each participant. The right and left shoulder and left thumb markers (all participants were right handed therefore the mobile device was held in the left, non-dominant hand) were used to determine the relative position between the device and the user to calculate the amount of hand movement along each dimension. One marker was attached to the right index finger (intermediate phalanx section) to track the motion of the input finger. It would have been more appropriate to place the marker on the tip of the index finger but due to the size of the markers it would have obscured part of the touchscreen and made targeting more difficult. The remaining markers were used to define sections of the body. Participants were asked to avoid wearing loose clothing and long hair was tied up in a head cap provided to avoid excessive marker movement and prevent the markers from being occluded.

Experimental Task

The experimental task was to select a series of crosshair targets one at a time on a touchscreen mobile phone. The participants had to select the current target crosshair (colored green) as quickly and as accurately as possible. The screen also showed the location of the next target (colored red) so that the participants always knew where to input next until they reached the last target. There were one hundred targets aligned in a 10 x 10 grid across the screen which were randomly ordered for each condition. Each target border measured (width x height) 40 by 60 pixels with the central crosshair measuring 10 pixels in each direction (1 pixel = 0.1mm). A gap was created between the last row of the targets and bottom of the touchscreen to prevent participants accidentally tapping the soft keys. The task ran on a Google Nexus One Android 3.1 smartphone as shown in Figure 3.

Experimental Design

A within-subjects design was used for the experiment and each participant completed the target acquisition task while unencumbered (holding nothing) and carrying each of the four objects in either the dominant or non-dominant side. As a result, there were nine encumbrance levels and each level was evaluated either standing still or walking which gave 18 conditions in total. The participants stood at the centre of the capturing volume for the standing conditions and navigated a pre-defined rectangular route (the outer and inner borders were 2.8 x 3 meters and 1 x 1.4 meters respectively) for the walking conditions as shown in Figure 4. The dimensions of the route were limited to the position of the cameras. Participants were instructed to keep within the path and walked in a clockwise direction. Each participant's preferred walking speed (PWS) was calculated before the experiment began. The participants were asked to navigate the route for two minutes at a pace that they normally would if they were walking on a quiet street.

The conditions were randomly ordered for each participant to reduce learning effects. 18 participants (4 males, 14 females) aged between 19 -38 years and all right handed were recruited from the university to take part in the experiment. The Independent Variables were encumbrance and mobility. The Dependent Variables were targeting accuracy, targeting speed and the level of movement in the non-dominant hand holding the device. The hypotheses were: H1. Targeting accuracy significantly decreases when the user is encumbered and walking compared to unencumbered and standing still.

H2. Targeting speed is significantly slower when the user is encumbered and walking compared to unencumbered and standing still H3. Targeting is less accurate when the dominant hand or arm is encumbered compared to the non-dominant hand or arm carrying the objects.

Results

To eliminate unintentional target selections, the recorded target positions that were greater than 70 pixels horizontally and 110 pixels vertically from the centre of the target crosshair were removed from the final data analysis. The reason for choosing the specific limit was to permit one target size margin of error. Target accuracy was measured as the distance (in pixels) from the centre of the target crosshair to the position recorded on the touchscreen. Speed of input (in seconds) was the time taken to select the current target.

Target Accuracy and Target Speed

A two-factor repeated-measures ANOVA with type of encumbrance and mobility as factors was calculated to examine both target accuracy and target speed.

The x-axis and y-axis were analysed independently for accuracy to assess if there was more error in a particular direction. For target accuracy on the x-axis, there was a significant main effect for stance, F(1,17) = 69.358, p < 0.05 and for encumbrance, F(8,136) = 7.131, p < 0.05. The interaction was also significant F(8,136) = 2.658, p < 0.05. A pairwise comparison for encumbrance with Bonferroni corrections showed that unencumbered was more accurate than holding the bags in either hands and carrying the thin and thick boxes under the dominant arm. However, carrying either the thin or thick box under the non-dominant arm was not significantly less accurate than holding no objects. Also, carrying the thin box under the non-dominant arm was more accurate than holding the wider box under the dominant arm. Table 1 illustrates the pairwise comparisons that were significant for encumbrance on targeting accuracy along the x-axis. Similarly for input accuracy along y-axis of the screen, there was a significant main effect for mobility, F(1,17) = 25.901, p < 0.05 and for encumbrance, F(8,136) = 4.022, p < 0.05. The interaction between the two factors was also significant F(8,136) = 2.334, p < 0.05. The participants were more accurate while standing than walking. A pairwise comparison with Bonferroni adjustment showed that unencumbered was only significantly more accurate in the vertical direction than holding the medium bag in the dominant hand (mean difference = -3.927, std. error = 0.933 and p =0.021). All other encumbrance comparisons were not significantly different (p > 0.05). Figures 5a and5b illustrate the mean target accuracy for the x-axis and y-axis of the touchscreen respectively. Based on the results for accuracy, hypothesis H1 is rejected since holding the boxes in the non-dominant hand was not significantly less accurate than unencumbered. Also, carrying the medium bag in the dominant hand was the only hindrance that caused accuracy to be significantly worse than holding no objects in the y-axis. Standing was significantly more accurate than walking.

The results for targeting speed from conducting an ANOVA indicated a significant main effect for encumbrance; F(8,136) = 13.239, p < 0.05 and mobility; F(1,17) = 12.230, p < 0.05. The interaction was also significant F(8,136) = 3.257, p < 0.05. Unexpectedly, holding the medium bag and both boxes in the non-dominant side caused selection times to be significantly lower than unencumbered. Hypothesis H2 is rejected based on this finding. Also, targets took significantly less time to select when carrying each of the four objects in the non-dominant hand or arm than the dominant side. Figure 6 illustrates the mean targeting times for each condition. The color representation for the conditions is the same as Fig. 5a.

Fig. 6. The mean targeting speed (seconds) for each condition (95% CI). The color representation for the conditions is the same as Fig. 5a.

Preferred Walking Speed (PWS)

Table 2 shows the drop in the participants' preferred walking speed (PWS) for each type of encumbrance while performing the task compared to walking alone and no interaction.

Holding the broader box under the non-dominant arm caused the highest decline in PWS of approximately 41%. Each encumbered condition's drop in PWS was greater than the results reported by [START_REF] Bergstrom-Lehtovirta | The Effects of Walking Speed on Target Acquisition on a Touchscreen Interface[END_REF], who found a 24% decline in PWS when performing a similar pointing task to ours on a treadmill. This suggests that carrying different types of objects during interaction may further affect the user's walking pace.

Analysing Hand Movements

The motion capture data was processed to examine the movements of the nondominant hand which held the mobile phone to see if carrying the different types of objects chosen for our experiment caused the user difficulty to steady their hands for input.

The markers placed on the non-dominant hand (left thumb and left index finger) and both shoulders were used to calculate the relative position between the device and the participant to determine the change in movement on each axis. The x-axis, y-axis and z-axis of motion represents leftright, forwardbackward and upwardsdownwards movements respectively. These notations will be used when discussing hand movements to avoid confusion with describing target accuracy.

Encumbrance was grouped by the hand or arm carrying the object (i.e. dominant vs. non-dominant) rather than the actual object itself when analyzing hand movements. This was a more appropriate method to understand how holding different objects impacts handedness and to identify where the problem is occurring. A twoway ANOVA with mobility and handedness (non-dominant vs. dominant) as factors was calculated to examine its effect on input accuracy. The results showed there was a significant main effect for mobility; F(1,71) = 133.369, p < 0.05 and for handedness; F(1,71) = 19.309, p < 0.05. The interaction was also significant F(1,71) = 12.753, p < 0.05. Pairwise comparisons with Bonferroni adjustment showed the participants were less accurate when the dominant hand was encumbered compared to the non-dominant hand. The mean target accuracies for the dominant and nondominant sides while standing and walking are shown in Table 3.

Table 3. The mean target accuracy (pixels) when the non-dominant and dominant hand/arm was encumbered. The amount of movement of the non-dominant hand holding the mobile phone was assessed in each direction separately to see if encumbrance caused more movement and instability on a particular axis. The mean hand movement (mm) was calculated by processing the amount of motion between three tenths of a second prior to a target being selected and the instance the tap onscreen was recorded.

Target Accuracy (pixels)

Non

A two-factor repeated-measures ANOVA with mobility and handedness as factors was conducted in each direction to assess if there was a significant difference in mean movement on the hand or arm carrying the bag and the box. For left and right direction of movement, there was a significant main effect for mobility; F(1,27) = 791.591, p < 0.05 and encumbered hand; F(1,27) = 240.090, p < 0.05. The interaction was significant F(1,27) = 51.422, p < 0.05. For forward and backward movement, ANOVA showed a significant main effect for mobility; F(1,27) = 855.252, p < 0.05 and encumbered hand; F(1,27) = 188.013, p < 0.05. The interaction was significant F(1,27) = 24.424, p < 0.05. The results for upward and downward movement indicated a significant main effect for stance; F(1,27) = 1255.995, p < 0.05 and encumbered hand; F(1,27) = 61.908, p < 0.05. The interaction was significant F(1,27) = 459.970, p < 0.05. Table 4 shows the mean movement in each direction of the non-dominant hand while encumbered and either standing still and walking the route. When the participants were standing still, carrying the objects in the dominant hand caused the other hand (holding the device) to move significantly more in each of the three directions when compared to the non-dominant hand being encumbered. The difference in mean hand movement between the non-dominant and dominant side being encumbered was very similar while walking. However, walking caused a significant increase of movement in both the non-dominant and dominant sides when compared to standing still. The results support hypothesis H3 as holding the bags and boxes in the dominant side resulted in significantly more movement in the hand holding the device and caused input to become less accurate when compared to the nondominant side being hindered.

Discussion

The results from the experiment have shown that holding different types of bags in hand or carrying boxes underarm caused a negative impact on targeting performance on a touchscreen mobile phone. The participant's input performance has illustrated that holding the objects chosen for our experiment caused targeting to become less accurate particularly when the object is being held in the dominant hand which was also targeting at the touchscreen to input at the same time. Interestingly, there was no significant effect for accuracy between unencumbered and carrying the two differently sized boxes under the non-dominant arm. Furthermore, performing the experimental task while unencumbered took significantly longer to input than holding the medium bag and both types of boxes. [START_REF] Ng | The Effects of Encumbrance on Mobile Gesture Interactions[END_REF] reported the performance of wrist gestures while carrying a box under the dominant arm was similar to unencumbered since users were actually able to use the box to assist their input. In terms of assessing the effects of mobility, walking while encumbered in general caused input to become less accurate and required more time to select the targets onscreen. The non-dominant hand which held the mobile phone was tracked to examine the level of movement caused by carrying the different types of objects. There was significantly more movement and instability when the dominant hand or arm was hampered compared to the non-dominant side. The difference in the amount of movement between encumbering the dominant and non-dominant sides was very similar when walking compared to standing still which may suggest walking alone could have an overwhelming effect on interaction.

Observing how the participants held the bags and boxes provided a valuable insight of some of the physical difficulties that users may experience when interacting with mobile devices while encumbered. The expectation was that carrying the thinner box underarm would be physically less challenging than the wider box. However, watching the participants suggested that the thinner box caused more problems to hold in place to avoid it being dropped to the ground especially when walking the route. Comments from the participants revealed the thinner box kept slipping down from their arms and it was difficult to find a comfortable gripping position while performing target selections, especially when carrying the box under the dominant arm. Surprisingly, the broader box did not cause as much physical issues as the thin box once a secure carrying posture was found. But, a few participants did find it uncomfortable to carry the wider box over a long period of time due to the length of their arms which may have caused selecting the targets more difficult and performance to decline.

The majority of participants had little trouble holding the two types of bags during interaction as the physical issues were more due to fatigue and tiredness. A number of participants commented that it was physically unpleasant to carry the bags due to the handles causing discomfort to their hands which may have caused targeting performance to decline. Holding the medium bag required a more restrictive upright arm position to prevent the bag from touching the floor. The participants mentioned that it was more demanding to hold both type of bags in the dominant hand and input at the same time compared to carrying the objects in the non-dominant hand. Furthermore, when standing still, it was evident that the dominant hand started to move downwards as the arm began to tire which would have made viewing the touchscreen more awkward and caused input to become more challenging. Remarks from the participants suggested that holding both types of bags in the dominant hand while walking the route required considerably more effort to control and maintain the forearm in a steady posture to target at the screen as the bag unpredictably swings from side to side. Also, it was fascinating to examine the different strategies adopted by the participants to carrying the different types of bags and boxes. Although we instructed the participants to carry the boxes underarm and the bags in hand, each participant's method of carrying the objects was slightly different and unique to the individual. This was more apparent when holding the box underarm as the participants with shorter arm lengths found it more challenging to grip the box and had to adjust their arm position at regular intervals to make input easier.

A simple recommendation that may improve input performance of two handed interaction while encumbered is to avoid carrying objects in the dominant hand or arm. However, in realistic situations, it may not be possible to switch holding the objects to the non-dominant hand only as the user could be occupied by carrying multiple objects. We need to design and develop effective techniques and applications to help users input more accurately when encumbering the dominant hand is unavoidable. One possibility is to increase target size to give the user a bigger margin of error. However, this would require careful interface design as it limits onscreen space for other widgets as discussed by [START_REF] Kane | Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces[END_REF]. We also propose that future studies should revisit previous research which have developed enhanced methods to assist users to input more efficiently while on the move. For example, Goel et al. [START_REF] Goel | WalkType: using accelerometer data to accommodate situational impairments in mobile touchscreen text entry[END_REF]'s WalkType application has shown to improve text entry while the user is walking. A repeated study that examined both encumbrance and mobility would indicate if the application is still as effective and could be used to reduce some of the issues caused by encumbrance. A final suggestion on future studies is to assess the impact of carrying a wider range of objects and include different hand gripping positions such as those that require a pushing action to extend our knowledge into the effects of encumbrance on mobile interactions.

Conclusions

The main purpose of the study was to examine the impact of encumbrance and make researchers aware of the usability issues and difficulties when users are physically hampered during interaction with mobile devices. An observational study was conducted to investigate the typical objects that people frequently held and carried in their everyday activities. Based on the results, we evaluated different types of bags and boxes to simulate realistic cumbersome and physically challenging situations that users are likely to encounter. The results from our main experiment showed that encumbrance caused targeting accuracy on a touchscreen phone to decline compared to holding no objects. However, targeting while carrying the thinner box under the non-dominant arm was significantly quicker and more accurate than unencumbered or holding the box in the dominant side. Moreover, targeting was less accurate when the dominant hand or arm was physically hampered during interaction and tracking the motion of left hand (which held the mobile phone) showed there was significantly more movement compared to the nondominant side was carrying the objects. This suggests that we should focus on assisting the user to input more accurately especially when the dominant hand and arm is encumbered.

Encumbrance also affected the user's preferred walking speed and although input was significantly less precise when the user was walking compared to standing, the performance difference was not as great as expected. Further investigation is required to see if there is a tradeoff between walking speed and encumbrance and assess input performance when the user has to maintain their PWS while encumbered. We hope the study presented in this paper has motivated researchers to develop better interaction techniques to aid users to input more effectively on touchscreen mobile devices.

Fig. 1 .

 1 Fig. 1. The four objects evaluated in the experiment. From left to right: small bag, medium bag, thin box and broader box. The images show how each object was held in the nondominant hand or arm.

Fig. 2 .

 2 Fig. 2. The location of the reflective markers (red dots) placed on each participant.

Fig. 3 .

 3 Fig. 3. The image on the left side illustrates the layout of the targets while the experimental task is shown in the right side image.

Fig. 4 .

 4 Fig. 4. The left image shows the actual capturing area. The right image illustrates the dimensions of the pre-defined path. Participants stood at the centre for the standing conditions.

Fig. 5a .

 5a Fig. 5a. Mean target accuracy along the x-axis of the touchscreen (95% CI). The red and blue bars represent the standing and walking conditions respectively. The lighter and darker shade of each color indicates holding the objects in the non-dominant and dominant sides respectively.

Fig. 5b .

 5b Fig. 5b. Mean target accuracy along the y-axis of the touchscreen (95% CI). The color representation for the conditions is the same as Fig. 5a.

Table 1 .

 1 The table shows the pairwise comparisons for encumbrance that were significantly different for accuracy on the x-axis. *Adjustment for multiple comparisons: Bonferroni.

	Comparison	Mean Diff.	Std. Err.	Sig.*
	Unencumbered	Small bag (non)	-2.718	0.690	0.038
	Unencumbered	Small bag (dom)	-3.490	0.874	0.034
	Unencumbered	Medium bag (non)	-3.366	0.461	0.000
	Unencumbered	Medium bag (dom)	-5.248	0.973	0.002
	Unencumbered	Thin box (dom)	-4.463	1.048	0.019
	Unencumbered	Thick box (dom)	-4.952	0.928	0.002
	Thin box (non)	Thick box (dom)	-4.063	0.966	0.021

Table 2 .

 2 The decrease in PWS (%) while holding the different types of bags and boxes compared to no interaction.

	Condition	Drop in PWS (%)
	Unencumbered	16.68	SD = 9.46
	Small bag (non-dominant)	25.17	SD = 11.96
	Small bag (dominant)	27.64	SD = 12.09
	Medium bag (non-dominant)	30.48	SD = 13.17
	Medium bag (dominant)	31.49	SD = 13.17
	Thin box (non-dominant)	36.88	SD = 13.45
	Thin box (dominant)	33.48	SD = 14.76
	Thick box (non-dominant)	41.06	SD = 11.64
	Thick box (dominant)	36.48	SD = 11.67

Table 4 .

 4 The mean movement of the left hand holding the mobile phone in each direction while encumbered. (S) and (W) represent standing and walking respectively.

	Encumbered	Movement Direction	Mean Movement	Std. Dev.
	Hand/Arm		(mm)
	Non-Dominant (S)	Left/Right	1.411	0.262
	Dominant (S)	Left/Right	4.781	1.11
	Non-Dominant (W)	Left/Right	5.338	0.922
	Dominant (W)	Left/Right	6.648	0.611
	Non-Dominant (S)	Forward / Backward	0.697	0.232
	Dominant (S)	Forward / Backward	2.505	0.414
	Non-Dominant (W)	Forward / Backward	4.044	0.667
	Dominant (W)	Forward / Backward	4.862	0.760
	Non-Dominant (S)	Upward/Downward	0.991	0.273
	Dominant (S)	Upward/Downward	3.286	0.499
	Non-Dominant (W)	Upward/Downward	5.939	0.550
	Dominant (W)	Upward/Downward	6.365	0.664


Christopher Martin

Matthias Freund

Henning Hager

Annerose Braune
email: annerose.braune@tu-dresden.de2henning.hager@monkey-works.de

Conception of Ambiguous Mapping and Transformation Models

Keywords: Model-based User Interface Design, Model Transformations, Mapping Model, Transformation Model, Iterative Development

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Developing user interfaces is a cost-and time-consuming task, especially if multiple platforms in terms of hard-and software have to be supported. Model-driven development processes for user interfaces (UIs) try to handle this problem by allowing a designer to specify the functionality of a user interface by means of models. These models abstract from implementation details and can ideally be used to generate executable user interfaces for different target platforms. In general, the models used in such model-driven development processes are defined on different layers of abstraction. The generally accepted Cameleon Reference Framework (CRF) constitutes four layers of abstraction [START_REF] Calvary | A unifying reference framework for multi-target user interfaces[END_REF][START_REF] Vanderdonckt | A MDA-compliant environment for developing user interfaces of information systems[END_REF]: The Task & Concepts (T&C) level allows defining the tasks that the user shall execute with the user interface. An Abstract User Interface (AUI) model describes the UI in a modality-independent way. In contrast, a Concrete User Interface (CUI) model is already bound to a specific modality but still independent of an underlying computing platform. A Final User Interface (FUI) is an executable UI dependent on a specific (class of) computing platform(s).

Model transformations either convert more abstract user interface models into increasingly concrete models (cf. the vertical transformations in Fig. 1) or they transfer models of a certain level of abstraction to another context of use (cf. the horizontal transformations in Fig. 1). In any case, transformations constitute the connection between the different layers. Therefore, their quality and reusability is a key factor for the efficiency of the entire model-based approach. [START_REF] Calvary | A unifying reference framework for multi-target user interfaces[END_REF] as introduced in [START_REF] Vanderdonckt | A MDA-compliant environment for developing user interfaces of information systems[END_REF] Model transformation rules incorporate two basic tasks (comp. [START_REF] Gruhn | MDA -Effektives Softwareengineering mit UML2 und Eclipse[END_REF]): (1) the description of mappings from source elements onto target elements and (2) the description of the execution of these mappings including the generation of the target model(s). While standard (hard coded) model transformations usually do not separate the mappings from their execution but combine them in the transformation code, there are already some approaches that favor the definition of formal mapping respectively transformation models. These models describe which model element or meta-model element of the source language can or shall be mapped onto which element of the target language. Thereby, they are independent of the technology that is used afterwards for the implementation of the transformation execution. Consequently, as shown in Fig. 2, they separate the functionality of the transformation (the description of mappings) from the implementation details (the execution of these mappings). The advantage of mapping models is that the correlations between elements of the source and the target (meta-)model can be configured instead of having to be programmed. In this case, the creation of a model transformation only requires knowledge about the application domain and not about implementation language details.

Fig. 1. Simplified version of the Cameleon Reference Framework

Existing mapping model specifications essentially only allow unambiguous (1-on-1) mappings that map one source element onto exactly one target element. For example, a mapping model from AUI onto CUI elements would describe that -in case of a graphical user interface -every modeled AUI-level "selection element" has to be mapped to e.g. the drop-down list element at CUI-level.

Fig. 2. Separation between description and execution of mappings by means of a mapping model

However, Puerta and Eisenstein show that not unambiguous (1-on-1) but ambiguous (1-on-n) mappings are frequent for the model-driven development of user interfaces [START_REF] Puerta | Towards a general computational framework for model-based interface development systems[END_REF]. That means, that the "selection element" instruction at AUI-level in general could be visualized by means of a drop-down list as well as by a set of radio buttons or a vocal selection. Consequently, mapping models for user interface design require multiple instead of unique target model elements. During the design process these ambiguities have to be resolved to allow the transformation of the model. Which of the (ambiguous) mappings is used depends, among others, on the target platform, the developer's preferences and on company guidelines. It is important to note that the resolving of the ambiguities has to be performed for each element of the source model as, in general, not every instance of a source meta-model element shall be mapped to an instance of the same element of the target meta-model. For example, depending on the number of the possible choices of a "selection element", either a drop-down list or radio buttons could be reasonable.

The model-based user interface development has to support iterative development processes to be widely accepted by UI designers because user interfaces are usually not developed in a single linear development process but rather in multiple development cycles. Therefore, the proposed mapping model should not only be able to specify ambiguous mappings but also save the information about the chosen mappings for each element. This allows for an easy reuse of transformations and the chosen mappings for every iteration step of the development process.

Hence, this paper presents a meta-model that (a) supports the definition of ambiguous mappings for the transformation of UI models and (b) stores the chosen mappings of the executed transformation.

Related Work

In regular transformations, both the mapping of a source element onto a target element as well as the generation of the target model are hard coded for one specific pair of UI description languages in an implementation technology such as ATL [START_REF]The Eclipse Foundation: ATL -a model transformation technology[END_REF], QVT [START_REF]Object Management Group: Meta Object Facility (MOF) 2.0 Query/View/Transformation[END_REF] or XSLT [START_REF]W3C: XSL Transformations (XSLT)[END_REF]. The occurring possible (ambiguous) mappings are usually either not defined so that only one of the possible target elements is used or only programmed directly into the transformation in an informal way [START_REF] Hager | Modelltransformationen in nutzerzentrierten Entwurfsprozessen der Automation[END_REF]. Bézivin et al. show in a fundamental work that, in contrast to the hard coded transformations, it is also possible to formally specify the mappings in so called Transformation Models [START_REF] Bézivin | Model transformations? Transformation models![END_REF]. Therefore, the mappings of the meta-model elements are described by formally assigning source and target elements to each other. This approach is used by the UI description language UsiXML, which specifies a transformation model that allows the description of a model transformation based on graph grammar [START_REF] Limbourg | USIXML: a language supporting multi-path development of user interfaces[END_REF]. The MARIA1 description language family [START_REF] Paternò | MARIA: a universal, declarative, multiple abstraction-level language for service-oriented applications in ubiquitous environments[END_REF] also allows the definition of unambiguous mappings with the help of the graphical editor MARIAE [START_REF] Lisai | Supporting transformations across user interface descriptions at various abstraction levels[END_REF]. However, in both cases the unambiguous mappings apply to every element of the same type, i.e. every instance of the same meta-model element. This means that e.g. every "selection element" can only be mapped to a drop-down list for the entire user interface. An alternative mapping to a radio button or a vocal selection is only possible in another transformation for a different context or by manually changing the target model. Consequently, there is no reusable information about the ambiguous mappings.

First approaches for ambiguous mapping models are made by Hager et al. [START_REF] Hager | Modelltransformationen in nutzerzentrierten Entwurfsprozessen der Automation[END_REF] and by Aquino et al. [START_REF] Aquino | Transformation templates: adding flexibility to model-driven engineering of user interfaces[END_REF]. Hager addresses this problem by implicitly describing all possible ambiguous mappings for one pair of description languages, namely DISL2 [START_REF] Schäfer | Model-based Development of Multimodal and Multi-device User Interfaces in Context-aware Environments[END_REF] at the AUI level and Movisa [START_REF] Hennig | Sustainable Visualization Solutions in Industrial Automation with Movisa -a Case Study[END_REF] at the CUI level, in (hard coded) transformation rules. The UI designer can choose an unambiguous mapping in an interactive process. With this approach, a UI designer can map a DISL element type (AUI level) onto various Movisa element types (CUI level) even in a single user interface. The choices made are then stored in a Persistent Transformation Mapping Model (Petmap) and are available for further iteration steps. However, the ambiguous mappings for this pair of UI description languages are also not formalized as they are implicitly included in (hard-coded into) the transformation rules.

In [START_REF] Aquino | Transformation templates: adding flexibility to model-driven engineering of user interfaces[END_REF], Aquino et al. propose Transformation Templates that extend the UsiXML Transformation Model defined in [START_REF] Limbourg | USIXML: a language supporting multi-path development of user interfaces[END_REF]. They allow defining (possibly ambiguous) mappings by the specification of a selector that is applied to the target meta-model. Furthermore, weighting factors can be specified that reference the UsiXML context model and that are used if multiple target elements for one source element are possible. However, both [START_REF] Hager | Modelltransformationen in nutzerzentrierten Entwurfsprozessen der Automation[END_REF] and [START_REF] Aquino | Transformation templates: adding flexibility to model-driven engineering of user interfaces[END_REF] specify their transformation models imperatively: They include implementation details of the transformation. Because of the imperative description of the mappings in this approach it is possible to use variables, conditions and loop constructs. For example, the specification of the selectors requires deep knowledge of the syntax and semantics of the transformation templates and thus is not applicable by UI designers but only by transformation language experts.

In contrast to imperative modeling, declarative modeling of transformations simply states which elements shall be mapped onto each other but not how these mappings are implemented by the transformations: It describes the "what" and not the "how". The advantage of a declarative modeling is the simple and clear notation suitable also for a UI designer without detailed knowledge of implementation technologies. The disadvantage of this approach is the limited language scope that e.g. does not allow expressing dependencies between different mappings.

Nevertheless, the creation of imperative transformation models requires advanced knowledge about the syntax and semantics of the implementation technology and therefore a transformation expert. Consequently, the approach presented in the following makes use of a declarative mapping model.

Conception of a Meta-Model for the Description of Ambiguous Mappings

This section introduces the concepts used to realize the requirements for ambiguous mappings for transformations of UI models and for storing chosen concrete mappings derived in Section 1. The realization of these concepts as the Persistent Ambiguous Mapping and Transformation Model (PAMTraM) will also be described. The workflow of this solution including all involved models and meta-models as well as the role of the PAMTraM is shown in Fig. 3.

Fig. 3. Proposed workflow and involvement of the PAMTraM

During step 1, all ambiguous mappings from elements of the source meta-model onto elements of the target meta-model have to be defined. These mappings are stored in the PAMTraM. This step has only to be performed once for every language pair. Afterwards, step 2 of the workflow consists of choosing the concrete mapping for every element of the source model. Therefore, the ambiguous mappings defined in step 1 are read from the PAMTraM and evaluated by a selector. This selector can e.g. be a UI designer or additional constraints (e.g. platform constraints) that allow an automatic evaluation.

Finally, step 3 constitutes the actual execution of the transformation. The execution uses the unambiguous mappings defined in step 2 to create the target model from the source model. Furthermore, the performed transformation including the selected unambiguous mappings is stored in the PAMTraM so that it can be reused for the next iteration cycles. In addition, the information about chosen mappings can e.g. be evaluated if statistical selection mechanisms are to be applied during step 2.

Steps 2 and 3 can be repeated to allow an iterative development process. The following sections describe the realization of the two parts of the PAMTraM by means of its meta-model.

Definition of Ambiguous Mappings

The mapping rules are defined on the meta-model level [START_REF] Gruhn | MDA -Effektives Softwareengineering mit UML2 und Eclipse[END_REF] because the allowed mappings of elements are mainly based on the UI description languages. They specify mappings from a source to one or more target meta-model elements. The mapping model must therefore allow the definition of those mappings and thus reference the source and the target meta-model. To ensure a comfortable creation of the mapping models, a declarative description will be implemented, such as e.g. already used in MARIAE's transformation editor for the definition of the unambiguous mappings. This approach allows the definition of mappings at a high level of abstraction. This way, a domain expert can create the mapping model and a transformation expert is not required.

The declarative description of the mappings, compared to an imperative approach, requires additional effort for the implementation of the mapping model into an executable transformation rule. But this additional effort is put into perspective, since such an implementation is only required once per UI description language pair. Another positive effect of the declarative description is the capability to easily define ambiguous mappings that assign one source element to multiple target elements. Fig. 4(a) shows the structure of a declarative, ambiguous mapping (Mapping). It enables the assignment of a source meta-model element to all eligible target meta-model elements.

As it is not reasonable to define multiple Mapping elements (each in turn defining multiple target meta-model elements) for one source meta-model element, additional constraints or validation steps should ensure that only one mapping is defined for every element of the source meta-model. It is important to note that the element MetamodelElement can be any element of any meta-model -it is not limited to classes (if a UML-like structure is used for the meta-models) but can also be a class attribute. This is necessary because -depending on the meta-models used -it cannot be ensured that merely classes are mapped onto each other. This is best explained by a simple example: In DISL (see Section 2), the different widgets that can be used to realize a user interface are not represented by different classes of the meta-model but by different enumeration literals of the enumeration GenericWidgetType. Consequently, DISL uses a simple meta-model class Widget with an attribute genericWidget of the type GenericWidgetType. Fig. 5 shows the relevant excerpt of the DISL meta-model.

Fig. 5. Excerpt of the DISL meta-model: realization of widget types

Consequently, when using DISL as source or target meta-model, the mappings do not have to be specified for the class Widget but for the different enumeration literals like textfield or command.

3.2

Storage of Concrete Mappings

Modeling ambiguous mappings allows the explicit definition of all possible mappings of source to target element types. Before a concrete transformation (as an instance of the ambiguous mappings) can be executed, the ambiguities need to be resolved. This resolution depends on numerous factors such as the specific context of a user interface or industry-or company-specific guidelines. Furthermore, the ambiguities usually have to be resolved for every element individually, because it cannot be assumed that a source element type should always be mapped to the same target element type (cf. [START_REF] Hennig | User Driven Evolution of User Interface Models -The FLEPR Approach[END_REF]). This process may be time-consuming -especially if a user is involved in the resolving of ambiguities -and has to be repeated for every iteration cycle even if only minor changes have been applied to the UI model. Consequently, the design decisions, which are made during a transformation, should be saved, so that they can be reused for iterative improvements of the user interface: In that case, an unambiguous mapping has only to be determined for the model elements that have been added or modified during the last iteration cycle whereas the stored concrete mappings of a previous transformation can be reused for the unaltered elements.

For this purpose, the meta-model element Transformation (see Fig. 4(b)) represents a concrete transformation that was executed for a specific user interface in a specific context, which can be saved in the PAMTraM. Gruhn et al. define a transformation as an instance of a mapping on the model level [START_REF] Gruhn | MDA -Effektives Softwareengineering mit UML2 und Eclipse[END_REF]. Therefore the unambiguous mappings that are determined for each model element during the transformation are saved as TransformationMappings (see Fig. 4(b)). Analogous to the Mappings, a declarative structure is used again. In comparison to the Mappings, TransformationMappings, however, map concrete (model) elements onto each other. Otherwise they have similar characteristics like the involved element types. Hence, an inheritance relationship between TransformationMapping and Mapping was used.

It should be noted that not necessarily only a single target element must be selected: Especially in multimodal user interfaces, it is often useful to map an AUI element to multiple CUI target elements, so as to combine a graphical output of an alarm with an auditory output of the same alarm. This relationship is represented by the Trans-formationMapping that allows the mapping onto multiple target model elements.

Those two (basically independent) requirements -modeling of ambiguous mappings and saving of performed transformations -were jointly implemented in a metamodel, since they are both specific for each pair of meta-models. In addition, when retransforming a user interface, both, the basic mappings as well as the specific performed transformations, are needed. A common storage of all performed transformations in one model also allows for a good starting point for further developments on the reduction of the ambiguities like the statistical analysis of the determined mappings.

The resulting meta-model PAMTraM (cf. Fig. 4) is applicable for any combination of target and source meta-models, because there are no restrictions or statements made about their structure and only general concepts were implemented. Beyond that, it allows the use of different methods to resolve the ambiguities as it only stores the resulting mappings.

Tool Support for the Application of the PAMTraM

In order to make the workflow (depicted in Fig. 3) -and therewith the concepts introduced in Sections 3.1 and 3.2 -applicable, tools supporting the UI designer in performing the three steps of the workflow should be created. Therefore, we have implemented the PAMTraM by using the Eclipse Modeling Framework (EMF) [START_REF]Eclipse Modeling Framework Project[END_REF].

During the first step of the workflow, the UI designer has to be supported in the definition of the ambiguous mappings. Although the EMF automatically creates simple, generic editors (e.g. a tree editor), the usability of these editors is not very high as they are not tailored towards the needs of the user. Instead, the design of a specialized concrete syntax 3 supporting the modeler to think in his domain is necessary [START_REF] Stahl | Model-Driven Software Development: Technology, Engineering, Management[END_REF].

In general, concrete syntaxes can be divided into textual and graphical representations, both with different advantages and disadvantages. For example, whereas a textual concrete syntax may enable a faster specification of models, a graphical concrete syntax may help in visualizing relationships. As multiple concrete syntaxes can be used simultaneously, the advantages of both types can be combined. As a first prototype, we have implemented a textual concrete syntax for the PAMTraM by means of the tool EMFText [19]. In order to assist the modeler, a content assistant and code completion are integrated.

Regarding the second step of the workflow, the necessary tool support can be divided into two parts: the implementation of the selection mechanism and the reuse of previous transformations in case of an iterative development.

As our approach does not dictate the used mechanism for the selection of concrete mappings during the second step of the workflow, the required tool support cannot be generalized. For example, if the UI designer shall choose the concrete mappings, a tool supporting this user interaction has to be implemented. This could e.g. be a dialogue that presents the different possibilities to the user and asks him to select one or more of the alternatives. If, however, statistical algorithms or platform constraints shall be applied, these mechanisms have to be implemented.

In case of an iterative development, a previous version of the user interface may already have been transformed once or repeatedly. In this case, the chosen (unambiguous) mappings for those parts of the source model that have not been altered during the iteration should usually be reused to keep the resulting parts of the target model that are not involved in the iterative changes consistent. Consequently, the selection mechanism should only be applied to those parts of the source model that have been changed during the iteration step. The described behavior has been implemented by means of a tool that reads previous transformations from the PAMTraM. Afterwards, the user is asked if one of the previous transformations (and with those the results of the used selection mechanism) shall be reused or if the selection mechanism shall be applied for every model element.

The tool support necessary for the third step of the workflow consists of the transformation executing the selected concrete mappings (and thereby generating the target model) and storing them in the PAMTraM for further iteration cycles (cf. Sec-tion 3.2). However, as this transformation is dependent on the used source and target meta-model, its implementation cannot be generalized either.

An exemplary implementation of the tools supporting the workflow is presented in the following case study.

Case Study

This section demonstrates the application of the presented approach based on the development of a user interface for the process plant shown in Fig. 6. The process plant consists of three tanks that are connected via pipes. For this case study, the UI shall enable the user to select one of the three tanks resulting in the illustration of the fill level of the selected tank. Two possible final user interfaces are presented in Fig. 7: The UI on the left makes use of a set of radio buttons to select the tank and of an animated image to display its fill level. In contrast, the UI on the right uses a dropdown list for the selection of a tank and a textual representation of its fill level.

The starting point of the case study is an AUI model expressed in DISL. By application of the PAMTraM, a concrete model of the UI, based on MARIA, shall be generated. Therefore, the XML schemata of both languages have been converted to EMFbased meta-models for easier processing in the EMF environment used for the implementation of the tools described in Section 3.3. The (model-to-text) transformation generating the final UI has only been implemented prototypically to demonstrate possible representations of the resulting UI. Fig. 6. Depiction of the process plant used in the case study DISL is a dialogue descriptions language, which is situated on the AUI level of the Cameleon Reference Framework [START_REF] Schäfer | Model-based Development of Multimodal and Multi-device User Interfaces in Context-aware Environments[END_REF]. DISL offers ten Widgets, represented by Enu-merationLiterals (cf. Fig. 5), for the description of user interfaces. In the DISL model (AUI model) of the case study, the structure of the realized process visualization is described. For this purpose, a widget to display the currently shown tank and its level (textfield), a selection element for the three tanks (choicegroup) and a widget to confirm that choice (command) are modeled. MARIA is a description language for graphical, vocal and multimodal user interfaces, which covers the top three abstraction levels of the CRF [START_REF] Paternò | MARIA: a universal, declarative, multiple abstraction-level language for service-oriented applications in ubiquitous environments[END_REF]. For this case study, only the CUI level is used, in which MARIA, among others, defines the Multi-modalDesktopCUI that is used hereinafter. Table 1 defines possible mappings of the Widgets that are used in the case study onto elements (Classes) of the Multi-modalDesktopCUI of MARIA. These constitute the basis for the formulation of the Mappings of the PAMTraM. Table 1 shows (in excerpts) the occurrence of ambiguities, for example for the possibilities to define an output variable (textfield). For further steps, these (ambiguous) mappings were stored in an instance of the PAMTraM that was presented in Section 3. Fig. 9 shows the prompt for user interaction during the initial transformation of the textfield-Widget that displays the fill level. The possible target elements correspond to those that were specified in Table 1 and defined in the ambiguous mapping model. It should be noted that the alternative(s) that was/were chosen by the user are stored as TransformationMappings as described in Section 3.2. Fig. 10 shows the corresponding part of an instance of the PAMTraM (cf. Fig. 4(b)): The upper half displays a stored transformation consisting of several transformation mappings for the various UI elements. In the bottom half, one of the stored transformation mappings is presented in more detail showing that the source model element "fill_level" as instance of the meta-model element "textfield" has been mapped to an instance of the target meta-model element "image".

When re-transforming the UI model, e.g. in case of an iterative development, the user can select one of the stored, previously performed transformations. In this way, the corresponding unambiguous mappings are reused and a user interaction is only necessary for the transformation of elements that were added or changed during the last iteration step (comp. Sections 3.2 and 3.3). As a designer can make different choices when selecting the mappings, different UIs can be the result of the transformation. Two possible user interfaces for the process plant that were created using the PAMTraM approach have already been shown in Fig. 7. They are the result of a model-to-text transformation applied to the CUI model resulting from the previous steps. In the UI on the left, an image was chosen to represent the fill level of the tank, while textual output was chosen on the right. Furthermore, a group of radio buttons respectively a drop-down list was chosen to select the tank, whose fill level should be shown. These decisions could e.g. be made based on platform restrictions as the second UI is more compact and thus fits better onto smaller screens.

Conclusions

The proposed Persistent Ambiguous Mapping and Transformation Model meets the requirements that were developed in Section 1. It implements (a) the definition of ambiguous mappings and (b) the storing of executed transformations in one combined meta-model. By making use of a declarative structure, domain experts without any special knowledge about the execution of a transformation should be enabled to define ambiguous mappings. Indeed, this has yet to be proven by an empirical evaluation In order to hide the structure of the underlying meta-model from the domain expert, a textual editor has been created that supports the user in the definition of mappings. However, a graphical editor, like the one MARIAE offers, could be useful to handle the often very large models and to support the user's actions. Furthermore, it has to be elaborated if mappings between groups of elements (instead of single elements) need to be supported and how this support could be realized.

The structure of the defined meta-model allows the integration of different methods to resolve ambiguities: In the case study, we have demonstrated an interactive transformation. Furthermore, the utilization of the context of use or an integration of statistical algorithms (e.g. the evaluation of previous user choices based on stored transformations) is possible.

PAMTraM supports the iterative development of user interfaces by storing the chosen, unambiguous mappings. By reusing these stored mappings, the selection mechanism (e.g. a user interaction) only has to be applied to elements that have changed since the last iteration cycle.

Moreover, as PAMTraM uses universal structures, it is not limited to transformations between AUI and CUI (as demonstrated in the case study) but can be applied to all model(-to-model) transformations within the CRF. This extends to a transformation that makes only use of unambiguous mappings. Thus, a continuous methodology including a continuous tool support can be applied. However, as PAMTraM makes use of models respectively meta-models of the source and the target language, it cannot be applied to languages that are not represented by a formal meta-model but e.g. by a description in natural language.

The transformation execution that evaluates the ambiguous mappings defined in the PAMTraM and stores the chosen, unambiguous mappings has to be implemented for every language pair. However, the achieved separation between the domain knowledge stored in the PAMTraM and the actual transformation reduces the necessary effort for this implementation: The transformation can be implemented by an expert with in-depth knowledge of the creation of transformations who does not need to be a domain expert. Furthermore, the once implemented transformation can be used for every user interface that is described using the same UI description language pair. Future work will cover the integration of the context of use into the decision process to reduce ambiguities. Therefore, possibilities to define conditions have to be created. Besides that, the user has to be able to attach these conditions to ambiguous mappings. By combining this with a subsequent user interaction, it can be ensured that the user may only choose between alternatives that are reasonable (for a certain context of use and a certain user interface). Compared to the exclusive user interaction realized in Section 4 the user is relieved of useless target elements.

Finally, an extensive empirical study has to be conducted in order to prove and classify the advantage of our approach. Therefore, the specification of transformations for various pairs of UI description languages as well as the development of specific user interfaces by means of the specified transformation has to be evaluated and compared to existing approaches concerning its effectiveness and usability.

Fig. 4 .

 4 Fig. 4. Resulting implementation of the PAMTraM for the storage of ambiguous mappings (a) and performed transformations (b)

Fig. 7 .

 7 Fig. 7. Two versions of the final UI based on different results of the selection of unambiguous mappings (cf. Fig. 3)

Table 1 .

 1 Excerpt of the possible mappings from DISL elements onto elements of the MARIA MultimodalDesktopCUI

Fig. 8

 8 displays the resulting code for the definition of the mappings that has been created with the textual model editor (comp. Section 3.3). By means of the editor capabilities (e.g. the content assistant), a UI designer can define the mapping model without deep knowledge about the underlying meta-model of the PAMTraM.

Fig. 8 .

 8 Fig. 8. Definition of the mappings with the textual editor

Fig. 9 .Fig. 10 .

 910 Fig. 9. User interaction for selecting the desired mapping

Modelbased lAnguage foR Interactive Applications

Dialog and Interface Specification Language

The term concrete syntax denotes the concepts used for the representation of models.