Georgios Christou

Panayiotis Zaphiris

Christoforos Christoforou

Cyprus  R K I Leaders Ltd

Gilbert Cockton
email: gilbert.cockton@northumbria.ac.uk

Panagiotis Germanakos

Andri Ioannou

Despina Michael

Matthias Neubauer

Philippe Palanque

Mark Springett

Panagiotis Zacharias

Lai-Chong Law

Carmelo Ardito
email: ardito@di.uniba.it

Maria Francesca Costabile

Rosa Lanzilotti
email: lanzilotti@di.uniba.it

Panagiotis Zaharias

Marios Belk

George Samaras

Costin Pribeanu
email: pribeanu@ici.ro

Alexandru Balog

Fernando Loizides
email: fernando.loizides.1@city.ac.uk

George Buchanan
email: george.buchanan.1@city.ac.uk

Leena Norros

Paula Savioja

Stanislaw Zabramski
email: stanislaw.zabramski@im.uu.se

Dimitrios Gkouskos
email: dimitrios.gkouskos@im.uu.se

Mats Lind
email: mats.lind@im.uu.se

Asbjørn Følstad

Andrew Saxon

Shane Walker

David Prytherch

Jeroen Vanattenhoven
email: jeroen.vanattenhoven@soc.kuleuven.be

Andrea Wilkinson
email: andrea.wilkinson@khlim.be

David Geerts
email: david.geerts@soc.kuleuven.be

Proceedings of the 1st European Workshop on HCI Design and Evaluation

Keywords: E-learning system design, e-learning system evaluation, usability evaluation techniques ACM Classification Usability evaluation, perceived quality, augmented reality, e-learning, hierarchical models ACM Classification H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. See [3] for help using the ACM Classification system Document Triage, Eye-tracking, Information Seeking ACM Classification H.1.2 User/Machine Systems Human Factors, Experimentation ACM Classification Information Interfaces and Presentation interaction styles, evaluation/ methodology Ontologies, adaptive websites, fuzzy relations. ACM Classification H5.2. [Information interfaces and presentation (e.g., HCI)]: User Interfaces---User interface management systems (UIMS), User-centered design; I.5.1. [Pattern recognition]: Models---Fuzzy sets Design, Human factors, Management Games, safety-critical systems, usability evaluation, development processes. ACM Classification H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous Human-Factors, Reliability ACM Classification ACM Classification Mouse, stylus, touch, freehand, tracing, comparison. ACM Classification H.5.2 User Interfaces: Evaluation/methodology ACM Classification User Experience, Experience Reports, Text Analysis, non-reactive Research Method, Automotive Context User-Centered Design, design practice, industry, userexperience, context, communication ACM Classification H.5.2 User Interfaces: User-Centered Design

E-learning is becoming very important in fields where access to learning materials needs to be brought about effectively and efficiently. Our experience in dealing with e-learning systems of different types has shown that several aspects influence the design and the evaluation of such systems, primarily the educational aspect, since the main purpose of such systems is to achieve educational goals. We are well aware that specific conceptual models and practical protocols are needed, capable of considering all the aspects that influence design and evaluations of e-learning systems. As a consequence, in the last few years we have developed the TICS (Technology, Interaction, Content, Services) framework and the eLSE (e-Learning Systematic Evaluation) methodology that, taking into account the most important aspects of e-learning system quality, aim at guiding designers and evaluators.

Limassol, Cyprus -April 8th, 2011

A B C D E F G H I J K L M N O P Q R S T U

Introduction and Acknowledgements

These are the proceedings of the 1 st European Workshop on HCI Design and Evaluation. The broad theme of the workshop is the influence of domain on HCI design and evaluation. We have tried to include a variety of different domains in which HCI design and evaluation plays an important role, and we have had many submissions with topics ranging from E-Learning to Web and Information Technology, and from Safety Critical and Emergency domains to Industrial and Small to Medium Business Software domains. We even had submissions for User Evaluation Methodologies. For a detailed list of the themes covered in the workshop, take a look at the table of contents of the proceedings.

Naturally, as in any event, a long list of people and organizations are involved to create something that runs smoothly and works well. Such is the case here as well. Thus, as chairs, we owe a big thanks to the many people who have made this workshop possible. We firstly thank the two organizations that have made the workshop possible, ESF COST Action IC0904 -Towards the Integration of Transectorial IT Design and Evaluation, and SIGCHI Cyprus. We thank of course our session chairs, Andri Ioannou, Jan Gulliksen, and Ebba Thora Hvannberg. We thank our invited speakers, Alistair Sutcliffe and Davide Rocchesso. We thank our sponsors, European Science Foundation, Cyprus Tourism Organization, Cyprus University of Technology, University of Leicester, and European University Cyprus for giving us the support required for the smooth hosting of this workshop. A special thank you to Marco Winkler who designed the front and back cover of the proceedings, and more! Finally we thank our program committee. The program committee is the following:

Table of Contents Session: User Evaluations of e-Learning Domain

Guiding design and evaluation of e-learning systems

Introduction

In the e-learning context, a major challenge for designers is to develop software tools and environments that can engage novice learners and support their learning even at a distance. Towards this end, there should be a synergy between the learning process and the learner's interaction with the software.

As for any interactive system, usability is a primary requirement. If an e-learning system is not usable, the learner spends more time learning how to use the software rather than learning the contents. Besides being usable, an e-learning system addresses other challenges, e.g., it must be effective in meeting pedagogical objectives. System evaluation should thus integrate an assessment of the educational aspects.

Despite the large number of e-learning systems now available, one of the barriers to successful deployment of technology-based learning is the lack of high quality systems tailored to the needs of individual users and groups. Various definitions and frameworks for the quality of e-learning systems are reported in literature, but the identified solutions appear as a short blanket, able to cover only some of the multiple aspects that characterize the complexity of the problem [2,3,5].

Quality of e-learning systems has been primarily considered from the point of view of education experts, focusing on the educational content with little attention to how it is presented. With respect to this point of view, we emphasize the interaction dimension and, specifically, the interaction between the user (teacher or learner) and the overall system, not only its content (the learning materials). On the basis of our experience, we believe that, in the e-learning context, proper conceptual models, as well as design and evaluation techniques and methodologies, have to be considered in order to foster a user-system interaction that facilitates the learning process. It is not possible to neglect the influence of the user interface on the learners' activities [4,10]. Many e-learning systems provide various functionalities and services, but it is often difficult, if not impossible, for users to find and use them in an effective and efficient way. Thus, we have proposed a definition of e-learning systems quality, which highlights the importance of the user interface in designing and/or evaluating the overall quality of e-learning systems [6]. If designers are fully aware of the high value of the user interface, they will not neglect this aspect of e-learning systems and will create products of better quality, which will contribute to the success of technology-based learning.

Looking at e-learning systems from the point of view of Human-Computer Interaction (HCI) experts, we have to acknowledge the great influence of the educational objectives of such systems in their design and evaluation. With the intent of balancing the above points of view in design and evaluation of e-learning systems, we have developed the TICS framework and the eLSE methodology described in the next sections.

TICS framework

By considering the literature on e-learning systems quality, the experience of HCI experts and the results of observing real users interacting with e-learning systems, a new framework, called TICS (Technology, Interaction, Content, Services), has been developed. It addresses the most important aspects to be considered when an e-learning system is designed or evaluated. TICS primarily focuses on the quality dimensions recommended by the Italian Ministry of Education in [9], integrating them with dimensions that refer to the interaction aspect that, as we have discussed above, plays a crucial role in the fruition of the e-learning material. In the following the TICS aspects, and their specific dimensions, as shown in Figure 1, are described.

Technology

The technology aspect focuses on the dimension Hypermedial Technology, since this is the technology on which e-learning systems are usually based. This aspect refers to the technological problems that can obstruct or make difficult the use of the e-learning system, namely compatibility with different operating systems; performance; monitor resolutions; hardware and software for system accessibility.

figure 1 Quality aspects of e-learning systems

Interaction

The explicit inclusion of the interaction aspect is one of the novelty of our approach with respect to the quality dimensions usually addressed in the e-learning literature. We believe it is crucial for technology-based learning. This aspect involves Presentation of the educational material and of the provided services and tools, and User Activity performed during the interaction with the system. In particular, user errors should be prevented as much as possible. If an error occurs, the e-learning system should provide appropriate support to manage it.

Content

The content aspect refers to the Appropriateness of the educational material that must be learned and to the Educational Process that involves the person learning through the system. Appropriateness and quality of the educational material could be achieved through an accurate learner-centered design [12]. This aspect also refers to the way the material is taught and to the capability of the e-learning system to propose study activities to the learner, who should also be free to autonomously choose his/her learning path (e.g. alternating moments of study, exercises, and verifications). Besides, the learner should have the possibility to go more deeply into the course content and the system should provide concrete, real, and helpful examples to facilitate understanding of the educational content.

Services

The services aspect refers to the Application Proactivity, which involves the tools that facilitate and support the user during the navigation through the system and the fruition of the educational material. It is necessary to provide users with communication tools, autoevaluation tools, help services, search engines, references, scaffolding, and so on. Ease of learning and ease of use of such tools allow users to concentrate their efforts on the learning paths without being required to spend too much time trying to understand the way the e-learning system works.

Some authors already stated that the user interface must be designed and evaluated on the basis of well defined criteria and methods, specific for e-learning [13]. The TICS framework has been developed in order to further emphasize that e-learning systems have to consider several aspects that influence their design and evaluation. TICS dimensions and corresponding guidelines should guide designers to focus on what is required for creating good quality systems. On the basis of TICS, we have also developed an evaluation methodology, described in the next section.

eLSE Methodology eLSE (e-Learning Systematic Evaluation) methodology proposes to evaluate e-learning systems with the support of appropriate evaluation patterns [6,8]. Here we briefly summarize the eLSE methodology; its goal is to increase the reliability and the effectiveness of elearning system evaluation by proposing a structured and systematic approach to it, which considers all the important aspects that influence the quality of such systems, as addressed by the TICS framework.

eLSE suggests analyzing an application along specific dimensions (those highlighted by TICS), which address the appropriateness of design with respect to the peculiar nature and purposes of e-learning systems. eLSE proposes an inspection technique based on the use of evaluation patterns that are specifically defined for e-learning systems. For this reason, it is called Patter-Based inspection or (PB inspection).

For each TICS specific dimension (i.e. Hypermedial Technology, Presentation, User Activity, Educational Process, Content Appropriateness, Educational Process, Application Proactivity), general usability principles are identified and then decomposed into finer-grained criteria. By considering the literature on e-learning, results of users studies, and the experience of HCI experts, a number of specific guidelines have been identified and associated to these criteria, to be taken into account during the design phase. Then, a set of evaluation patterns, addressing these guidelines, has been defined [6,8].

An evaluation pattern is a description of what an evaluator has to do when inspecting an application. Evaluation patterns guide the evaluator's activities by describing which elements of the application to look for, and which actions the evaluators must perform in order to analyse such elements. In this way, even novice evaluators, with lack of expertise in usability and/or application domain, are able to come out with more complete and precise results.

Evaluation patterns make possible to maximize the reuse of evaluators' know-how, by capturing usability inspection expertise, and by expressing it in a precise and understandable form, so that it can be easily reproduced, communicated, and exploited. They therefore allow evaluators to take "... advantage of any of the efforts done in previous works, to reduce the effort needed to achieve a new one" [11]. Evaluation patterns are precisely formulated by means of a template that provides a consistent format. More details on the library of evaluation patterns defined for evaluating e-learning systems can be found in [1].

eLSE methodology organizes the evaluation process activities into a preparatory phase and an execution phase. The preparatory phase is performed only once for each analysis dimension; its purpose is to create a conceptual framework that will be used to carry out actual evaluations. The output of the preparatory phase can be easily shared among different evaluators, or different evaluation laboratories that have similar interests and evaluate such applications from similar points of view. The preparatory phase consists of the identification of guidelines to be considered for the given dimensions and the definition of a library of evaluation patterns. The execution phase is performed every time a specific application must be evaluated. It mainly consists of a PB inspection, performed by some evaluators. If needed, the inspection can be followed by user testing sessions, involving real users. At the end of each evaluation session, the evaluators must provide designers and developers with a proper evaluation feedback.

eLSE methodology also proposes an important distinction between the e-learning platform and the educational modules. The e-learning platform is the software environment that usually offers a number of integrated tools and services for teaching, learning, communicating, and managing learning material. The educational modules, also called learning objects, are the specific learning material provided through the platform. Design guidelines and evaluation patterns defined for the platform differ from those ones defined for e-learning modules, since different features and criteria need to be considered [1].

The eLSE methodology can be easily adapted to different types of applications in different domains, by identifying specific analysis dimensions and the proper set of evaluation patterns.

Validating the PB inspection

The main novelty of the eLSE methodology is the PB inspection. This technique, particularly suited for novice evaluators, has been empirically validated through a controlled experiment, which compared PB inspection to heuristic evaluation and user testing [7]. The study involved 73 novice evaluators and 25 end users, who evaluated an e-learning application using one of the three techniques. The comparison metric was defined along six major dimensions, covering concepts of classical test theory and pragmatic aspects of usability evaluation. The study showed that evaluation patterns, capitalizing on the reuse of expert evaluators knowhow, provide a systematic framework which reduces reliance on individual skills, increases inter-rater reliability and output standardization, permits the discovery of a larger set of different problems and decreases evaluation cost.

Introduction

Background and Related Work

Discussion

References [1] Design Journal, 4 [2] Behaviour & Information Technology 25: 2 [3] Taylor & Francis [4] Ext. Abstracts CHI 2008 [5] Ext. Abstracts CHI 2005 [6] Proc. British HCI 2002 [7] Aesthetic Computing [8] Proc. ICIS 2004 [9] British Journal of Educational Technology 41 [10] Virtual Reality 10

[11]
Virtual Reality 10

[12]
Transactions on Edutainment I 5080 [13] Computer-Supported Collaborative Learning 3, [START_REF] Teo | Development and validation of the e-learning acceptance measure[END_REF] Online Communities 5621 [START_REF] Wind | Spinnstube®: A Seated Augmented Reality Display System[END_REF] Proc. CHI 2009 [16] . Information

Introduction

Educational systems based on desktop augmented reality (AR) technologies are integrating real life objects into computer environments. Touching and holding real objects is an appealing user experience that is increasing the students' motivation to learn and could better support active and collaborative learning [9,11]. As AR technologies become more wide-spread, there is an increasing interest in their usability. Designing for usability is not an easy task in emerging AR technologies where the focus is mainly on designing and implementing new interaction paradigms [5,7].

The quality of an e-learning platform should satisfy three kinds of requirements: usefulness for the learning process, ease of use, and increased motivation for the learner. Quality could be seen as general concept that is manifested by specific dimensions. As such it could be modeled as a higher-order factor. In general, the higher-order factors have proven to be successful in increasing theoretical parsimony and reducing model complexity [6,12].

In this paper we explore a hierarchical model for the perceived quality of a learning scenario for Biology implemented onto an augmented reality teaching platform (ARTP). This educational platform was developed in the framework of the ARiSE (Augmented Reality for School Environments) research project. A "3D process visualization" paradigm was implemented that was targeted to enhance students' understanding and motivation to learn the human digestive system. Firstly we will present a hierarchical model that is more suitable to integrate three core components of the perceived quality: perceived ease of use (PEOU), perceived enjoyment (PE) and perceived usefulness (PU). Then we will explore the relation between the perceived quality and the intention to use ARTP for learning.

Perceived quality of educational platforms

ISO standard 9126:2001 defines quality in use as the extent to which specified users accomplish specified goals with effectiveness, productivity, security and satisfaction in a given context of use [10]. Quality of educational platforms could be discussed from different perspectives (learner, teacher or school).

Perceived quality of e-learning systems is an important research topic in recent years. Understanding extrinsic and intrinsic factors that are influencing the acceptance of new technologies helps designers to focus on critical aspects. There are several influencing factors and many ways to model the quality among which the most frequently used are the perceived ease of use (related to usability), the perceived enjoyment (related to user experience) and perceived usefulness [1,4,13,[START_REF] Teo | Development and validation of the e-learning acceptance measure[END_REF].

In a previous work [3] we developed a measurement model for ARTP that was grounded in the technology acceptance models (TAM) theory in order to explain the causal relations between various factors influencing the intention to use. Although the model was useful to test various hypotheses, the variance explained was small, which suggests looking for an alternative model. This study extends the research by specifying the perceived quality as a hierarchical, reflective construct and examining its impact on the intention to use.

Case study

ARTP is a desktop AR environment: users are looking to a see-through screen where virtual images are superimposed over the perceived image of real objects placed on a table [START_REF] Wind | Spinnstube®: A Seated Augmented Reality Display System[END_REF]. The real object is a flat torso of the human digestive system that is used by two users staying face to face (see Figure 1). A pointing device having a colored ball on the end of a stick and a remote controller Wii Nintendo as handler has been used as interaction tool that serves for three types of interaction: pointing on a real object, selection of a virtual object and selection of a menu item.

The model specification and estimation is done on a sample collected in 2008. A total of 139 students 8 th grade participated. Each of them tested the Biology scenario by performing four tasks: The participants have been assigned 4 tasks: a demo program explaining the absorption / decomposition process of food and three exercises: (1) asking to indicate the organs of the digestive system, (2) asking to indicate the nutrients absorbed / decomposed in each organ, and (3) asking to indicate the organs where a nutrient is absorbed / decomposed. After testing they filled in a questionnaire (five points Likert scale). Scale reliability was 0.94 which is acceptable (Cronbach's alpha).

We hypothesized that the perceived quality of the Biology scenario is associated with two extrinsic motivational factors (PEOU and PU) and an intrinsic motivational factor (PE).

We specified the construct "Quality of ARTP" (Q-ARTP-B) as a second-order, hierarchical reflective model which comprises three first-order reflective constructs: perceived ease of use (PEOU), perceived enjoyment (PE), and perceived usefulness (PU). Q-ARTP-B represents a general concept that is manifested by specific dimensions measuring the perceived quality of the application implementing the Biology scenario. In this respect, Q-ARTP-B is a superordinate construct [6].

We analyzed the data collected for the Biology scenario for outliers and normality, 9 cases were eliminated, thus resulting a final sample of 130 observations. Then we carried on an Exploratory Factor Analysis (EFA) using SPSS 18. Principal axis factoring with Promax rotation was used to extract factors. Eigenvalue and scree plot were used to determine number of factors extracted. The initial result revealed a three-factor solution, which accounted for 51% of variance. To achieve a more meaningful and interpretable solution, it was necessary to delete some items with low loadings or those loaded on more than one factor [8]. The final results with these three factors accounted for 69.38% of variance.

Table 1. Constructs and items

Perceived ease of use O1 Observing through the screen is clear O2 Understanding how to operate with ARTP is easy O3 Superposition between projection an the real object is clear O4 Reading the information on the screen is easy Perceived usefulness U1 Using ARTP helps to understand the lesson more quickly U2 After using ARTP I will get better results at tests U3 After using ARTP I will know more on this topic Perceived enjoyment E1 The system makes learning more interesting E2 Performing exercises is captivating E3 Overall I like learning with this system E4 Overall I find the system interesting Intention to use I1 I would like to have this system in school I2 I intend to use this system for learning I3 I will recommend to other colleagues to use ARTP From a learner's point of view the ease of use is likely to be influenced by the devices used to perform specific tasks. In an AR setting, the real objects used are also influencing both extrinsic and intrinsic motivational so is worth to identify particular ease of use aspects that are critical for a given AR application.

The results of EFA highlight three items related to the quality of visual perception as perceived by users: observing through the screen, superposition between projection and the real object, and reading the information on the screen. The descriptive statistics also showed that the two items got the lowest mean value (M=3.76, SD=1.07 for O1 and M=3.62, SD=1.02 for O2) while most of the items were rated over 4.00.

Then we applied Confirmatory Factor Analysis (CFA) using Structural Equation Modeling (SEM) with AMOS 17. A two-step approach [2] was adopted for examining the measurement model and following the structural model.

The results of the first-order factor model (the factors were free to correlate) did not provide a satisfactory result with a chi-square value of 68.60 (df=41), which was significant at the p<.05 level. Other fit indices revealed an acceptable fit (the chi-squared per degree of freedom was 1.67, SRMR=0.043, TLI=0.946, CFI=0.960, RMSEA=.072 with 90% CI 0.04-0.10 and pclose=.113). All of the items have statistically significant relationships with their factors. All loadings were above 0.67 and a good majority of them above 0.70. The first-order factors are highly correlated: 0.68 (PEOU, PU), 0.86 (PU, PE), 0.71 (PEOU, PE) thus showing evidence for convergent validity.

As expected, the fit indices of the second-order factor model were identically with those of the first-order model. The degree of explained variance (R 2 values) of hierarchical model is reflected in its components, that is, PEOU (.56), PU (.82), and PE (.90), indicating acceptable reliability for all factors. The Composite Reliability (CR) exceeded the recommended level of 0.70. Also, Average Variance Extracted (AVE) exceeded 0.50 (Table 2). All the path coefficients from Q-ARTP-B to its components are significant at p<0.001.

Note that the loadings of the first-order factors to the second-order factor are all above 0.75 and that the tvalues were significant (p<0.001).

PEOU has a significant contribution to Q-ARTP-B (γ=0.75). Critical quality attributes for this scenario are related to the clarity of the visual perception and the ease of understanding how to operate with ARTP.

PU has a significant contribution to Q-ARTP-B (γ=0.91), showing that ARTP is useful to understand the lesson more quickly, to get better results at tests and to learn more on this topic.

The strong relationship between first-order factors and the second-order factor indicates a convergent validity of the posited second-order model. As such, the second-order model is very effective in representing the data. Both the magnitude of path coefficients and variance explained by the model showed higher values than those previously obtained [3].

Discussion and conclusion

There are some inherent limitations in our study. The sample used in this study is small and the data was collected with an evaluation instrument that was intended for another research model. As shown in a previous work [3] there are also several limitations of the evaluation instrument due to the novelty of the platform (neither specific questionnaires nor similar models available), the target user population (young and not happy to answer a long questionnaire), and the mixed character of the e-learning system (utilitarian and hedonic).

However, the hierarchical model proved to be a better alternative to predict the intention to use (79% vs. 20%) and is a first step in modeling the perceived quality of an AR-based learning scenario. This study confirmed the previous finding that PE had a slightly higher influence than PU on the intention to use.

The fact that this study is targeting a specific learning scenario implemented onto ARTP is both a weakness and strength. On the one hand, the generalizability is not possible, not even at ARTP level. On the other hand it enables a better understanding of strengths and weaknesses of each learning scenario in terms of ease of use, usefulness and enjoyment. The Fast Lane: Rapid Document Triage Using an Eye-tracker.

Abstract

Document triage is the process by which an information seeker goes through a set of documents and decides their relevance to his or her information need. In this paper we examine this process with a particular focus on how a limited time frame (simulating fast decision making) affects behavior and relevance judgments. We present results using an eye-tracking methodology, showing document features viewed by information seekers during a time constrained document triage process.

Introduction and Related Work

Document triage takes place after the automatic information retrieval process, and before a relevance judgment occurs. Currently, there has been limited research of the manual human behavior into this area as well as a lack of specialized software tools. Conceptualization of information seeking is mostly limited to the process in its entirety (e.g Marchionini's model for information seeking [6]).

Our current work has shown areas of importance during document triage to the information seeker [2,4]. This research corresponds to, and complements earlier work which looked at how document features influence relevance decisions [7]. As further research 1 is performed, focusing directly on the document triage process, it is becoming clearer that the overall pattern of behavior in document triage is more generalized rather than task oriented. This is a positive insight for software developers and designers. By deciphering the inner workings of document triage, software tools can be effectively created to assist this, so far, mostly unaided process. Currently, there have been limited attempts at creating specialized tools for supporting the document triage process [3,5]. In order for this process to be better understood it is vital to identify and test the limitations that may constrain the users. One such constraint that has previously been looked at is the effects of display configurations in the document triage process [8]. Another constraint has been the reading and organizing of document sets [1].

In this paper we explore a natural constraint to information seekers when performing document triage. Document triage is by nature a fast part of the information seeking process. This fact, coupled with individual user time constraints, often leads to relevance decisions being made quickly. Evidence of low tolerance for manual searching can be seen in how information seekers rarely triage past the second page in an on-line search results list [9]. Users would rather modify their query term, thus relying on the automatic process to do the work. Furthermore, it was seen in [2,4] that information seekers can be biased against long documents and documents with a bad structure, which hinders their locating information quickly enough. It is also commonly known that users do not spend much time (a few seconds at most) scrutinizing a website before making a decision of whether or not it is useful. This is ported to the academic document triage field in which users often make a relevance judgment early on, but may continue to triage further for reassurance [4].

The work presented here explores document triage when performed at a fast pace. We attempt to answer the following research questions under the constraint of time:

1. What parts of a document do information seekers feel are vital, important and unnecessary to look at during a fast search?

2. At what point in the document triage process, when given a whole document, are information seekers consistent with their relevance ratings?

3. How much would information seekers deviate from their regular document triage behavioral pattern given a limited time-frame?

Study Description

We chose 20 participants, 7 female and 13 male, for the study. The first 2 participants were chosen as pilot study subjects while the remaining 18 participated in the main experiment. All participants were at postgraduate level or above in an HCI (Human Computer Interaction) related discipline. Their ages ranged from 21 to 50 years of age. The criteria for the selection of appropriate participants was their familiarity with academic documents; namely to have read academic documents before and are therefore accustomed to the general format. Further to this, the participants needed to have a minimum understanding and experience in using electronic document readers and a basic foundation in HCI. Special care was taken however, in order that the participants were not familiar with the specific documents presented to them in the study.

We briefed the participants on the definition of the term document triage. We did not give out information on how users are expected to act during document triage that may bias the behavior of our participants In the main study, participants were given two information needs. For the first information need, they were given six documents to triage and place a relevance score from zero to ten (zero being extremely irrelevant and ten being extremely relevant to the information need). The second information need included ten documents for the participants to triage in the same fashion. Participants were not given an openended time frame but were assigned a specific time limit per document. A within-subjects design allowed for participants to perform triage in one of three time limits (30, 60 and 90 seconds). These results could then also be cross examined with the data presented in [4] where the same documents were used but participants were permitted an open ended time frame. There is one limitation to this study however. In the original study presented in [4] no eye-tracker was used. Thus, we can only compare the participants' relevance scores directly. The documents chosen to be scrutinized by the participants were all rated by 3 experts as highly relevant and selected specifically for their distinct features (such as length, structural clarity and Main title relevance).

The documents were presented one at a time on a 19 inch screen in Portable Document Format (PDF). The participants' eye gaze was tracked using the nonintrusive Tobii x50 eye-tracker. The Tobii x50 eyetracker allows 0.5-0.7 degree accuracy, a spatial resolution of 0.35 degrees, a drift of < 1 degree and a frame rate of 50 Hz. The participants' freedom of head movement was 30 x 15 x 20 cm at 60 cm from tracker. The participants' screen was recorded using BB Flashback Screen Recorder Standard Edition. A dictionary was supplied for the clarification of unknown words and questions could be asked at any point in the study provided they were related to clarifying participant goals. Guidance regarding the relevance of the documents themselves however was not given.

Following the main study a semi-structured interview was performed in order to gain qualitative feedback from the participants. Parts of the study were played back to the participants and they were asked to explain their thoughts during interesting or confusing behaviors to the investigator. This approach was deemed more appropriate than interrupting participants during their triage process. Participants were rewarded with £10 (GBP) for their time.

Results

In this section we begin by presenting the pre-study questionnaire feedback which shows natural subjective bias of the users before initiating document triage towards certain document features. We then continue to present the actual behaviour and participants' navigational patterns and visual focus, explaining which document features were viewed most and those which were scarcely or not viewed. We also look at the participant relevance ratings for the three groups and compare the results with our open-time frame study [4]. Finally, we summarize and discuss the most distinct findings.

Navigational Patterns. When users perform document triage, they will navigate through a document following a specific pattern. These patterns were recorded and reported on in earlier work [4]. When users triage while under pressure from a time constraint, they are found to replicate some of these patterns also.

The most common navigational behavior was that of 'step up navigation'. This linear skimming of the document and stopping at areas of interest was performed by all participants in all three groups 92% of the time. The remaining 8% of the time, participants would produce a hybrid behavior between 'Flatline navigation' and 'Begin and End navigation'. 'Flatline navigation' is a term used to describe no scrolling beyond the initial page of a document. 'Begin and End navigation' indicates that a user looks at the initial page and scrolls to the conclusions section only. User Attention. The areas of participants' visual focus between the three groups were analyzed and reported on (See Figure 1). Participants were allowed to examine every document for a limited amount of time; up to 5 seconds extra was allowed after the limit was reached for the decision to be orally given to the investigator. Participants with a 30 second limit, 60 second limit and 90 second limit will be referred to as Group 1, Group 2 and Group 3 respectively. According to our study document features accounted for 18.8% of the total document area. Participants focused on these features for 68.5% of the time. This fact, including scrolling time, suggests the remaining plain text area was skimmed, but not scrutinized, therefore unlikely to be of benefit to the participants.

Document Relevance Ratings.

Figure 2 shows the average rating and standard deviation for the three groups in our study, as well as the average rating from a fourth group taken from [4]. This fourth group underwent the same document triage process on the same documents but was permitted an open-ended time frame. As we can see from the results, there is no significance between time taken to triage documents and relevance scores.

Discussion

It was the intention of this paper to report on two primary findings. First, we researched the behavior (in terms of navigational and viewing areas) of users, while performing document triage with a limited time constraint. Second, we compared and contrasted the relevance judgments made between our participants' groups and studies from our previous work.

Behavior. We saw behaviors from all three participants groups which coincide with findings from previous work. One example is the visual importance of the initial page [2,4]. The navigational patterns used by our participants, replicated those performed by users with no time limit. We did however, discover an overwhelming bias in the number of participants choosing to use the 'step up' navigation rather than any other. A contributing factor to this behavior is the attraction of headings to users' focus during skimming. Headings receive increased attention when an information seeker performs within-document triage with a short time constraint. This is especially noticed from our 30 second group. Headings, Diagrams and Figures, as well as the conclusions section receive the most visual attention for the information seeker, relative to their viewing area on a document.

Relevance Judgments. Interestingly, we saw no significant change in the average relevance ratings given by participants between any of the three groups presented here. The same statement holds when comparing the results with users from a previous group. This group of participants performed triage on the same set of documents with no time restrictions [4].

Overall, we have identified that time is not a largely contributing factor in the relevance decision process. Furthermore, we have identified (and cross referenced our results with previous studies) document parts which are used to make the relevance decisions. These findings can also be used as heuristic guidelines into informing supporting software designs. We can inform document triage software by targeting visually important document features and bypass features which do not contribute substantially towards a user's decision process.

Future Work

As more data is gathered regarding document triage, we see patterns evolving in information seekers' behaviors. These patterns, being generic, can be standardized into models for librarians, software developers and human computer interaction scientists to use in understanding, conceptualizing and facilitating the document triage process further. A prototype software tool is currently under development which aims to provide rapid triage support to information seekers in searching through a document set effectively.

Conclusions

In this paper we have explored the effects of limited time on document triage in a process we label rapid triage. We assigned an information need to participants and gave them a set amount of documents to triage in three different time frames. By comparing their subjective feedback with their actual behavior, using an eye-tracker, we were able to distinguish areas of interest and level of importance for several features on a document empirically. We were able to determine that time does not produce a significant factor in the document relevance ratings of information seekers. We have also produced further questions for research and progressed in the formulation of a standardized general model for the document triage process. Apply for construction of a 24 m 2 garage.

Introduction Method

2

Apply for a 54 m 2 addition to the house containing extra rooms.

Introduction

The term adaptive Web site refers to sites that automatically adapt their design and interface,

Copyright is held by the author/owner(s).

Dimitrios K. Kardaras

Athens University of Economics and Business.

76, Patission Str.

Athens, GR10434 GREECE kardaras@aueb.gr

Xenia J. Mamakou

Athens University of Economics and Business.

76, Patission Str.

Athens, GR10434 GREECE xenia@aueb.gr

Bill Karakostas

HCI Design Center City University

Northampton Square

London, EC1V 0HB, UK billk@soi.city.ac.uk according to a user"s profile. The adaptation process of data-intensive web sites operates on three main components: content, navigation, and presentation [5].

It is important to design adaptive web sites, since a poorly designed web site may result in a loss of 50 per cent of potential sales, due to users being unable to find what they want, and a loss of 40 per cent of potential repeat visits, due to initial negative experience [4], [11].

Domain knowledge and ontologies

Only by capturing domain knowledge can automatic web site adaptation be realized. Domain knowledge that characterizes the context as a source of information [7], needs to be captured and formalized as ontologies. Such ontologies define the context factors relevant to the user, his or her environment and situation.

Adaptive websites

Today, many online businesses are adaptive to each user or user group preferences and profiles.

Approaches to adaptation vary as some of them introduce new content, adapt the structure of links and change the presentation of page elements. Various adaptation techniques are used, depending on the component that is targeted, the degree of prior knowledge about the user (user profile) and the method used to elicit knowledge dynamically, i.e. as the user interacts with the web site [9].

Knowledge mining approaches such as Bayesian networks and decision [6] are used to identify patterns in web site usage and use such patterns to assist users navigate between web pages and/or rearrange pages based on their relevance to the user or according to the user"s expectation of where the page should be.

Collaborating filtering relies not only on a single user, but on groups of users with similar profiles/interests to select and customize content. One such approach, based on declarative rules, relies on both profile and configuration knowledge [5].

Adaptation can be carried either on the server side or the client side. Server side adaptation implies that knowledge is stored on the web site, and adaptation takes place on the server, prior to delivering content to the user. Client-side adaptation approaches employ agents that are responsible for collecting user information and interests and customize the site accordingly [1]. Attentive agent-based information systems do not intrude in the user"s ongoing activity, but work on the background and gather information about the user"s behavior, model the user and provide content that is relevant but not critical. Attentive systems agents may for example monitor eye gaze to determine what the user is reading and look for relevant information. The results of their inquiry are displayed discreetly on the side of the page.

The effectiveness of adaptation approaches in general, can be improved by using formalized knowledge of the domain, context and even of the adaptation process, in terms of ontologies [3]. The most important advantage of using an ontological description of such knowledge in languages such as OWL-S is that it can be augmented, enriched and synthesized using suitable reasoning mechanisms, and also exploited in reasoning on context ontologies. To overcome the problems of partial and imprecise context information, fuzzy knowledge representation approaches can be used.

Composition of Fuzzy Relations

Fuzzy relation is an extension of relations in conventional set theory. Classical relations are given by sets, while fuzzy relations are given by fuzzy sets [10]. Like classical relations, fuzzy relations also map elements of one universe, say X, to those of another universe, say Y, through the Cartesian product of the two universes. However, the fuzzy relation possesses stronger expressive power while relating x and y [2], since the relation between ordered pairs of the two universes is measured with a membership function expressing various "degrees" of strength of the relation on the unit interval [0,1]. Hence, a fuzzy relation R is a mapping from the Cartesian space X x Y to the interval [0,1], where the strength of the mapping is expressed by the membership function of the relation for ordered pairs from the two universes, of μR(x,y) [10]. Composition of fuzzy relations can be defined as following [8]. Suppose two fuzzy relations

The proposed approach

The generation process of adaptive web sites assumes the availability of ontologies that capture the domain knowledge and the modeling of their interrelationships using fuzzy relations. The composition of fuzzy relation provides the inference mechanism for adapting web design alternatives to the user types. The proposed approach consists of the following steps:

1)

Construct the User Ontology that specifies the user styles (T={type-1, type-2, … type-n}) in terms, shown as i u "s in figure 1, that represent the user cognition, the demographics such as age, sex, income, attitude towards innovation, ability to articulate needs and priorities, educational level etc. User types are associated with the perceived for each type importance of services and service characteristics.

2)

Develop the Service Ontology that represents the characteristics ( ci s "s in figure 1) that constitute the service described in the web site, and captures knowledge that pertains to the contribution of each characteristic to service quality. For example, in the case of a hotel web service, these characteristics could be price, view, room amenities, beach etc.

3) Construct the Web design Ontology, which specifies the web design features ( Fi w "s in figure 1)

and represent web usability heuristics. Design features can be, for example, text, images, videos etc., which are associated with size, location on the web page, font types, etc. The composition takes into account the degree to which a user fits a user profile, and then adapts the design of the web page, according to the user requirements for service and the suitability of alternative web design features.

Figure 1. Integrating Ontologies with Fuzzy Relations for web adaptation

Illustrative example

The following example illustrates the proposed methodology. Let us consider three different user types (type1, type2 and type3). In the case of a hotel web site, showing hotel features such as price, location and amenities, and using web page elements such as text, image and video we have: T={type1,type2,type3}, Si={low,medium,high} and Wj={low,medium,high}, where i is the specific service feature and j is the specific web design element. As a result, the corresponding fuzzy relations Ri and Pi,j for i=price and j=text will be: which shows how strongly the service feature "price" is required by each of the three different customer types. For example, customer type 1 is price sensitive with a low degree (0.2), medium (0.4) and high (0.1). which indicates the suitability of presenting price with text, taking into consideration the importance for the hotel characteristic price. For example, if price is considered to be a low requirement, that means that it should be presented by text 0.3 low, 0.2 medium and 0.1 high (where low, medium and high for text indicate the font size). Other web page elements and properties such as size, location etc. can be represented in a similar manner. Then, the resulting fuzzy composition which relates elements of user types (T) to elements of text (Wtext) will be found by the type: Consequently, the fuzzy composition R • P , is: which shows that a type 3 user should view the price presented by text 0.5-low, 0.3-medium and 0.3-high. Thus, price when represented as text on the web page, should probably be shown using small size fonts for this type of customer.

Consider now a customer with a profile that is closer to a type 2 user:

The decision for web adaptation comes from the composition T o ( R o P ), which is:

The composition shows that for the specific customer, the service price, when presented by text, should probably be presented by small fonts. The same composition is calculated for each service characteristic, web feature, and user type, until the most suitable design is chosen among all alternatives. Thus, customers having different profiles will view different web pages that dynamically adapt to their service requirements and corresponding design choices.

Conclusions

The use of fuzzy knowledge representation allows web page designs to be dynamically adapted to user profiles. The proposed approach suggests that the integration of ontologies and fuzzy relations provides the means to adapt web design along a continuum that is defined by the user profile and preferences as well as the suitability of web design alternatives. In the near future we are planning to validate the above approach by applying it to the domain of tourism web sites, by carrying out a number of experiments involving several user groups. The profiles of participants will be captured and codified using the ontologies described above. Then, the users will be asked to evaluate the usability of the automatically adapted web pages. The results of such experiments will allow us to assess the effectiveness of the proposed approach.

Usability Evaluation: Commonalities and Discrepancies in Games and Safety Critical Systems.

Abstract

This paper presents the basic principles of widely agreed upon practice for usability evaluation in two different domains: Games and Safety Critical Interactive Systems. The user interfaces (and their associated interaction techniques) proposed in the applications deployed in these two domains are usually very different while the objects and functions to be manipulated might be very similar. For instance in a flight simulator game a rear view of the aircraft is usually proposed (see Figure 1), this rear view is also proposed in flight simulators (see Figure 2) while in a real cockpit such representation is not possible (see cockpit capture Figure 7). This is surprising as the key point in flight simulator games is to propose an asrealistic-as-possible cockpit interface.

Based on an overview on the development processes and the respective usability evaluation methods used in these two domains, a comparison of the approaches showing commonalities and discrepancies is presented. Concluding remarks focus on how these two domains could fruitfully cooperate, by adopting and enhancing methods used in the two domains. Additionally, such comparison could provide foundations for trans-.copyright notice to be completed.

Introduction

Since early work on software quality from McCall in 1977 [8], usability has been explicitly identified (even within the software engineering community) as a key factor for the quality of software. However, for the evaluation of usability of any interactive system it is important to take into consideration the application domain as the application domains can heavily influence how the usability is evaluated. For instance, when evaluating usability in games it is important to take into account that games typically are not task based and that playing a game is (by definition [4]) non-productive and oriented towards entertainment. On the other hand, in domains where safety plays a more important role than usability, this preponderant factor influences usability evaluation and the interpretation of results: detected usability problems in a given user interface might not be resolved by improving the user interface, but by training the user to simply learn how to use the problematic interface in a safe and error-free manner.

Goal of this research is to compare the domains of safety-critical systems and games in terms of usability evaluation methods. We argue that both domains can benefit from each other by incorporating the knowledge available in the other domain into their usability evaluation approaches.

First, the two domains are described, giving an overview on the development processes used, the applied usability evaluation methods (including their need of technical infrastructure). Next the approaches in the domains are compared showing commonalities and discrepancies. Concluding remarks highlight potentialities for the two domains to learn from each other. Lastly, we outline a draft research agenda for a more global approach on trans-sectorial usability evaluation methods.

Games

For the development of games there is a common agreement in the community that successful games rely on an iterative development approach. Usability evaluation is an important aspect in games development: if a game is not usable (e.g. the interaction technology does not allow easy learning how to play the game), a game is typically not successful. There is a broad range of game development processes used, the majority of them having special evaluation phases (including usability evaluation).

Games Development Processes

When looking at the development of games, standard software engineering approaches like the waterfallmodel or Boehm's spiral model have been described as useful for game development. In the majority of cases a successful game development company does not Figure 6 presents an abstract view of a game development process. We have not represented the iterations that are of typical HCI practice (in the prephase), typical software engineering practice such as presented in the V model (in the main-phase) and closer to the agile processes (in the post-phase).

The following phases typically guide a game development process [9]:

(1) Concept: This phase is dedicated to the initial game idea and is devoted to producing a first concept document describing the game. The development team in this phase is typically small (e.g. consisting of designer, programmer, artist and producer).

(2) Pre-Production Phase: This phase includes the development of art style guides, production plans and first description of the game design and the technical design document.

(3) Prototype: Goal of this phase is a first working piece of software allowing to demonstrate key characteristics of the game and enabling to understand basic concepts related to the general user experience of the game ("Is the game fun to play?"). ( 4) Production: The production phase can range from few weeks development to years of programming. This phase can be structured additionally, following approaches like an increment to completion approach, a cascade approach or an "iterative-until you drop" approach [6].

(5) Localization: an important phase for games that will be delivered to different markets (countries) is the localization phase. In this phase game-play can be adjusted to suit the tastes of the market, to allow language translation and modifications due to local regulatory authorities. ( 6) Alpha-Phase: This is the phase when a game is playable from start to finish, allowing different evaluation methods to be applied to better understand aspects like fun, playability and user experience. (7) Beta-Phase: Main goal during this phase is normally to fix bugs. In terms of user experience in this phase lots of fine-tuning is necessary to improve the overall user experience. The beta-phase includes steps like certification or submission (the hardwaremanufacturer of the proprietary platform will test the Games development has some additional special milestone (depending on the product developed). Especially the release candidate milestone is important for games that are produced for special consoles, as the release candidate is then tested and evaluated by the game console manufacturer.

Usability Evaluation in Games Development Processes METHODS

For the usability evaluation there is a broad range of methods used, that are applied in the different phases of the game development. At the early stages of the development (concept phase) these methods are: focus groups, interviews, informal play-testing, questionnaires; at later stages, especially during the production phase these are play testing (including biometrical measurements), (semi-structured) interviews, observation, video coding, quantitative comparisons of gamers behaviours, questionnaires focusing on users attitudes or experiences or heuristic evaluation (including the usage of heuristics that focus additionally on the playability of the game).

Two usability evaluation methods from the entertainment and games domain have become a focus of attention in other areas (1) play testing and (2) beta-testing. Play tests are used during all phases of a game development, with the majority of usability and play testing performed once the alpha phase is accomplished until the beta-phase is finished. (Alpha is the milestone in games development were the game can be played the first time from the beginning to the end, still missing some branches in the game that are not finalized, or graphics that are not ready; Beta is milestone in the game development when no further developments and changes are made for the game play, and the only goal is to get rid of bugs). Betatesting is a form of game-play evaluation that also includes usability evaluation. During Beta tests the game is distributed to (a selected community of) end users that are playing the game. These users (sometimes thousands of users) provide feedback in terms of usability (e.g. mapping of the input to the various game elements) as well as in terms of gameplay (allowing for example the fine-tuning of gamerules based on player-death logging). EQUIPMENT Usability evaluation of games is conducted both in the lab (Figure 4) and in the field (Figure 5). To enable the evaluation of usability at early development stages prototypes are developed and used. To understand how usability and overall user experience and fun are related usability evaluation for games development today rely on sophisticated usability labs, including biophysiological measurements, eye-tracking or motion sensors.

Safety-Critical Interactive Systems Development Processes

In safety critical systems several quality factors deeply influence the development process such as reliability, fault-tolerance or security. However, usability could/should be seen as a critical factor especially when [10] report that 80%-90% of accidents in industry are attributed to human error.

One key element of the development process of safety critical systems is the certification phase which involves an external body who is in charge of assessing the process that has been followed during the development of the system. Figure 8 presents such a process in which the last phase precisely concerns certification activities. It is important to note that, however, such phase is more prominent when the system involves "normal" citizens. Another typical aspect of the safety critical domain is the fact that standards processes are available and should be carefully followed. Figure 8 is defined in the DO 178-B standard from [1] while Figure 9 is from another standard IEC 61511 dealing specifically with safety issues [5].

Usability Evaluation in Safety-Critical Interactive Systems

Usability evaluation in safety critical systems is typically conducted as part of the development processes presented above. However, it is usually not an explicit phase as usability concerns are taken into account in more or less every phase.

From the scientific perspective, usability evaluation is scarcely described or even reported. From our experience, for instance in the case of civil aircrafts cockpit designs, usability and operability concerns are dealt with by means of expert users (called test pilots) who are involved from the beginning (architectural design of the cockpit) to the very late final phases (design of the training of airlines pilots). This could be compared to participatory design approaches.

Using a simulator as a test-bed for design and testing of concepts and interfaces can be seen as a specific equipment or supporting usability studies. For instance, while designing the elements of a head-up display, scenarios can be defined and tested to assess the impact of a display freeze of the head-up display on the operations (i.e. the pilot's tasks).

To summarize a broad range of usability evaluation methods are used in this area but they typically do not refer and relate to the standard method descriptions advocated in the field of HCI.

Equipment for Usability Evaluation

As for games, usability evaluation for safety critical systems is conducted both in the lab and in the field. But given the necessary focus on safety and reliability, evaluations in the lab are based on the extensive usage of very complex and realistic simulators (e.g. force feedback systems to simulate the behavior of physical equipment in the cockpit, vibrations, 3D movements of the cabin …). Figure 3 presents a realistic simulator for a UAV (unmanned aerial vehicle) control and command system embedding a physical model of the aircraft, weather, gravity ….

When usability evaluation is carried out in the field, very specific equipment has to be designed and embedded to gather information about the operations and the pilot. As for games, they can collect biometric measures about the crew but in addition, they have to be very robust to variations as the evaluation has to be as close as possible of the real operations. For instance, eye tracker devices must be able to track eyes movements both in the dark (while landing at night) and while facing sun (while departing East in the morning or landing West in the evening) [1].

Lastly, usability testing in safety critical systems exhibits the following constraints:



Summary of Similarities and Discrepancies

Usability is usually evaluated according to the time spent in training (costly training is perceived as poor design of the UI).

Games -Process

Concluding Remarks

Looking at the specificities of the two domains it is surprising to find a set of similarities: both domains share the approach that usability as a factor of the interaction with the system might be less important than other software quality factors. Both domains have a set of additional development phases and usability evaluation is typically quite heavy in terms of infrastructure. Being interested in the domain dependent adaptation of usability evaluation, the domain of safety-critical system and the domain of games are promising candidates to understand how usability evaluation methods must be adapted to better fit the domain.

We believe that safety critical systems and games represent the two extremes of a continuum of domains where usability evaluation methods have to be customized and adapted to fit their specificities.

In a nutshell and to provide a second adaptation of Landsburg's words as stated in [2]: "Usability wants us to die rich; UX wants us to die happy", we propose a new vision of it by saying "Usability wants us to die fit; UX wants us to die happy; Safety want us to die still alive".

oculometry: a preliminary study in real flight conditions. In proceedings of HCI Aero 2010, ACM Digital Library.

Background

the core core task demands

Flow of HF research in COPE

Methods used in human factors work

Methods of gathering data about the end user activity

Introduction Irene-Angelica Chounta Nikolaos Avouris

Introduction

Research on input methods and their influence on human input has been focused mainly on the performance aspects. Many input devices have been tested on their effectiveness in pointing, dragging, crossing and path steering navigation tasks and this knowledge is used for different analyses and comparisons [7,4,6]. These standard navigational tasks became the subjects of mathematical modeling. Fitts' Law is a proven method that models linear pointing and clicking tasks but it appears to be not as well suited for modeling two dimensional tasks [5]. This has been supplemented by the Steering Law, which is a more suitable predictive model for investigating two dimensional navigation tasks by considering them as a constrained motion within predefined tunnels of error [1].

The recent popularity of hand held devices equipped with displays capable of sensing multiple ways of human-input created new opportunities for creative users. Any surface-based human input can be broken down to a time-series of 2D coordinates. Therefore we can use the analogy of line tracing to describe the output of the continuous user's action that takes place e.g. on a touch-sensitive surface.

Line tracing can be an example of a task which might be negatively influenced by the low accuracy of the input method used for drawing lines but also by any kind of constraints imposed on the user. However, we have been unable to find a model specifically designed for unconstrained drawing in freehand input with initially unpredictable user error. Therefore, we decided to investigate the shape-based approach to assess the potential of small screens for unconstrained free-hand tracing tasks.

In order to compare the three most commonly used input devices, namely mouse, pen and touch input, in a drawing task, we performed an experiment with 16 participants.

In the experiment, the participants had to complete a tracing task of tracing a simple, random shape with all three input methods. The purpose of this experiment is to find out if there is a clear difference in performance when performing the tracing task between the three input-devices that are being investigated.

Method

In our study, users were asked to trace over, in one stroke, a greyed-out shape that was displayed on the screen using one of the three input methods in question in a randomized order. Additionally, we controlled the visual feedback of tracing that imitated drawing with black or invisible ink. We have also measured the accuracy and time it took the participants to perform the task.

The shape that was given to the participants was designed with use of a modified version of Method 4 described by Attneave [2]. We created asymmetrical non-sense contour shapes that did not bear any resemblance to well-known shapes. The modification of Attneave's method was limited to making the shapes consisting of at least two of each kind of perceptually meaningful properties like: convex corners, concave corners, straight line segments, and curve line segments. These segments of the shape did not cross at any point and their parameters like length or angle were randomized.

There are multiple factors that can describe the differences between two shapes: general shape, translation, rotation, and scale [3]. In order to calculate the user's accuracy of each trial of the tracing task, we decided to calculate the error score based only on the pixel-wise difference in general shape between the shape given and the shape that each participant produced.

We extracted multiple pixel-based values from each generated shape and used the following equation to calculate error scores for each task:

(

Where:

 CP is the number of common pixels between the participant-generated shape and the original shape  DP is the total number of the participant-generated pixels  TP is the total number of pixels of the original shape.

The error value is represented by the length of line from points A to B as seen on the plot below: While performing such a tracing task it is theoretically possible to achieve maximum accuracy. This would mean that a user has traced over a shape perfectly and created the same amount of pixels in the exact same position as the original shape that was presented.

Experiment design

The experiment has a mixed design. The input methods have a within subjects design and were assigned in a randomized order for counterbalancing. The visual feedback has a between subjects design. 8 randomly selected participants performed the task with visual feedback and 8 without it. An HP Touchsmart TM2 Tablet PC with a 12.1 inch screen and a resolution of 1280*800, with stylus and finger input, as well as a Logitech basic optical mouse were used. The HP TM2 was used in tablet mode with the stylus and finger input, parallel to the desk, and in laptop mode with the mouse. Morae version 3 was used in order to acquire time data from the trials. Participants had to fill in a pre-test questionnaire and were offered a minute long introductory session for the stylus and touch input in MS Paint. Then, they were presented with the shape and instructed to "trace over the shape in one stroke, starting from the top right corner". Afterwards they were asked to fill in a post-test questionnaire regarding their preferences and opinions for the input devices that were tested. The following figures display the shape that was presented to the participants along with a high scoring and a low scoring participant attempt.

Results

First the shape error data were analyzed for deviations from normality by means of the Kolgomorov-Smirnov test. A liberal decision criterion of 0.1 was used in order not to use parametric test unduly. However, none of these tests showed any such deviations and the further analyses were performed using ANOVA using a decision criterion of 0.05. The ANOVA for shape error revealed neither significant differences between the input methods (F(2,28)=0.876, p=0.427), the visualization of feedback conditions (F(1,14)=0.26, p= 0.61), nor any interaction between these factors (F(2,28)=0.317, p=0.731). The grand mean error value was 0.741. Then we turned to the timing data. The ANOVA was performed on the logarithm of the time in seconds because reaction type data are known to be nonnormally distributed otherwise. There was no effect of visualization of feedback (F(1,14)=0.0613, p=0.808) nor any interaction between this factor and input method (F(2,28)=0.275, p=0.761). However, there was a clear difference between the input methods (F(2,28)=49.535, p<0.0000001). The mean times for each input device were mouse=23.00s, pen=15.10s and touch =9.81s.

Conclusion

Results show that all input methods have comparable error scores for the shape that was used. There are, however, large differences between time scores for each input method. Touch outperforms mouse by a factor of 2.3 and pen by a factor of 1.54.

Discussion

The precise line-tracing task might be representative of multiple tasks related from creative graphics design and free-hand drawing to complex linear selections of multiple graphical elements. Therefore the results of our study show that for at least moderately complex drawing tasks touch input is much more efficient than pen or mouse (what was also confirmed in tasks with no visual feedback) and might be the preferred input method in graphics design applications. This was supported by the qualitative post-test data that indicates that our users preferred touch input with pen input second and mouse input last. More of complex shapes need to be investigated, as it is expected that the results may vary from the one produced for this experiment. Finally posture must be considered as well, when using touch input, especially since there might be additional muscle strain due to the lack of proper support for the user's arms during the use of touch devices. The contribution of this paper is the presentation of a novel framework for systematically categorizing and evaluating HCI methods. It is anticipated that this framework will ease the process of selecting appropriate HCI methods for particular design and evaluation tasks. An overview of the framework is provided, its structure and qualities are discussed and directions for its future development are proposed. The discussion highlights areas of improvement for similar efforts, including providing effective overview of methods, and extensions that support the study of recent trends and needs in HCI, such as the shift of emphasis to user experience (UX) and the focus on specific application domains.

Introduction domain exclusiveness

Domain exclusiveness as a function of specialization and required training

Domain exclusiveness specialist domains limited training domains generalist domains popular domains

Level of training

Level of specialization

Usability evaluation in domains with low levels of specialization

Usability evaluation in domains with high levels of specialization

Overview of methodological approaches

An apparent way of categorizing HCI methods can be based on the development stage that the method is applied to, be it analysis, design, implementation, formative evaluation, or summative evaluation. However, such a categorization does not provide any direct appreciation of the different kinds of results and insights provided by a method. Nor does it highlight the resources required, or the fact that some methods can be effectively used in more than one development phase. Fitzpatrick and Dix proposed an alternative schema for categorizing HCI methods according to their strategic application [3]. In this approach, four strategies are proposed, based on the resources at human and system level (i.e. real or representative users, and real or representative system) thereby creating a 2 by 2 matrix with top level categories of real world, virtual engineering, soft modeling and hard review. However in this framework, methods are then classified at a second level based on their type and the way they are used, resulting ultimately in a usage analysis table. The result of this process is that the toplevel categorization does not reflect the underlying goal of classifying the methods and, as such, does not offer useful insights. A third approach involves categorizing methods by the type of participants featuring in the UX evaluation [6]. This leads to a top-level breakdown into categories of lab tests, field studies, online surveys, and expert evaluations without actual users. Another recent approach classifies usability evaluation methods into data gathering & modeling methods, user interactions evaluation methods, and collaborative methods [4].

An HCI methodological framework

The first step in the development of the methodological framework presented in this paper was to gather a comprehensive corpus of HCI methods. Overall, 41 HCI methods were sourced from the literature and studied, analyzed and systematically described (for the complete list, see [8]). The set of methods included both traditional usability methods and those that take into account experiential aspects of a system.

In the framework described in this paper, a top-level categorization of the methods is achieved according to the way that they are used, resulting in four categories: inquiry, prototyping, inspection and testing, with 14, 8, 12 and 7 methods in each category, respectively.

Inquiry methods refer to the study of the use of an existing system or the characteristics of a system under construction by querying users and other stakeholders. Methods of this type typically require a considerable number of participants and, since they are based on participants' opinions and feelings about a system, provide subjective results. Examples include questionnaires, interviews and focus groups.

Prototyping methods involve the construction of prototypes, which enable the study of the characteristics of a system at an early stage in the development process. Prototypes can be further classified according to their fidelity to the final system, level of detail, and scope, into high or low fidelity, and horizontal or vertical prototypes. Examples of prototyping methods include paper prototyping and storyboards. Inspection methods involve the close examination of a system by one or more HCI professionals. Typical inspections can be based on heuristics or guidelines, and can be driven by scenarios (i.e. walkthroughs). In testing methods, experiments on the system or a prototype are conducted with the participation of users. Typical examples include thinkaloud protocol, wizard-of-Oz, and retrospective testing.

This categorization was selected as it highlights both the usage of a method and the type of resources that are required. In general, methods in different groups were found to exhibit high complementarity and low overlap in their results, focus and required resources. For example, inquiry methods tend to provide subjective results, as opposed to testing methods, whereas inspection methods do not require the direct participation of users, as inquiry and testing methods do.

The key to the methodological framework is the comparative evaluation of the HCI methods, which is based on a set of measurable criteria. This comparative evaluation, in combination with a good understanding of the workings of each method from their description and analysis, and a good apprehension of the needs of the system under development, can facilitate the selection of the most suitable combination of methods for each project.

The criteria for the comparative evaluation were selected so that they also enhance the general overview of the available methods. The HCI methods and evaluation parameters are tabulated in order to enable quick and effective overview and comparison. An excerpt of this Results are subjective and not easily documented or analyzed. Employee participants may be biased.

Heuristics limit significantly the scope of the evaluation.

A walkthrough covers only a small part of the system. Inherent bias due to the selection of tasks/scenarios to be evaluated.

Provides answers to how people use a system, but not why. Does not take into account experiential aspects of the use of the system. The results require statistical analysis. Caution is needed as in not to breach users' privacy.

Discussion

This framework integrates characteristics from similar efforts [3,6,7] to provide a complete and comprehensive catalogue and comparative tool.

The main advantages of the framework and the points of differentiation from similar attempts are enumerated below: § Systematic. A lucid and eloquent overview of methods is achieved by categorizing and positioning them in a single table. § Critical Review. Inclusion of a descriptive analysis of the main advantages and disadvantages (strengths and weaknesses) of each method. This brief overview will effectively assist HCI professionals in incorporating and applying a method in their projects. §

Comprehensive. The framework covers 41 HCI methods in total, embracing aspects of system development from initial conception to final testing. This compares well to previous attempts (e.g. [6,7]). §

Extensible. The framework is template-based, so it can be updated by appending new methods, and allows for the revision of the characteristics of each method in a collaborative way from the HCI community. New parameters and possible ways of categorization can be included to address emerging needs of the HCI community, such as the shift to UX or the focus on a specific domain.

The main disadvantages and limitations of the framework are as follows: § Subjective. The characteristics and the values for each method's parameters have been elicited either from the relevant literature, or from the personal experience of a small number of HCI researchers and practitioners. § Non-experiential. The framework does not explicitly address the experiential aspects of interaction with a system.

Suggested enhancements

In order to address the shortcomings we propose a number of enhancements to the framework. These ideas can also prove to be useful enhancements to similar efforts undertaken in this area.

First, an interactive online version of the methodological matrix should be developed. This should make use of visualization elements and techniques to simplify the overview and review of methods and support the decision-making process. Drawing from examples of information visualization used in decision making (e.g. [1]), visual elements such as color, saturation, shape, size, texture and orientation can be used to convey information. Interactivity can support the decision making process by dynamically altering the visibility and visual elements in the online version of the framework. An online interactive version of the framework will further enhance the clear overview of the methods, which is a problem in large and detailed collections of methods (e.g. [4]), and is also expected to contribute to further disseminating the methods to the community of HCI practitioners, researchers, students, software developers and other stakeholders.

Second, an interactive online version of the matrix can be enhanced to support participation from HCI researchers and practitioners. This should yield similar benefits, in terms of exposure to a wide range of opinions, as those frameworks derived from survey data (e.g. [2,5,6,7]). However, integration into an online platform will be a more streamlined and efficient way, which is expected to contribute to the dissemination of the results. Of course, this is expected to be one step further than mere collaborative editing, as done in wikis, for example. The data from many participants, including some basic profile information, can be analyzed statistically and allow the dynamic provision of different views of the framework. For instance, it will be possible to find which methods can be used when one does not have access to users, or which methods are most applicable to a specific application domain.

Introduction

The TwinTide COST Action (www.twintide.org) has a focus on the transferability of design and evaluation methodologies between application sectors and domains, with an aim of developing cross-sectoral methodologies. Transferability is a physical metaphor that involves the movement of one entity from one place to another. For example, footballers transfer between clubs, students transfer between courses and universities, money is transferred between bank accounts, and files are transferred between computers. In all of these examples, something is transferred whole from one place to another.

Two questions follow for the transferability of method(ologie)s between domains:

 Is a method(olog)y transfer whole or partial?

 How are the places in the transfer characterised?

To answer the first question, we need to know what a method(ology) comprises. To answer the second, we need to be able to characterise application domains or sectors in a way that is relevant to the transferability of method(ologie)s. For example, the UK's creative industries grew at 6% p.a. from 1997 to 2005, when they accounted for 7% of GVA. While both are key economic attributes, it is not clear that they have any automatic relevance for design method transferability. We could consider further well established sector attributes, and still struggle to identify relevant ones.

To avoid a fruitless domain-centred search for relevant attributes, a method-led approach is developed to identify potential interactions between method attributes and domains. These interactions give rise to conjectures that can be explored in research programmes. Domains are thus explored through the lens of methods, rather than vice-versa, where it is not clear where to focus, especially with so many alternative classifications of economic sectors, e.g.,  Primary, secondary, tertiary, quaternary, quinary.

 Agriculture, Mining, Construction, Manufacturing, Transport/Utilities, Wholesale, Retail, FIRE (Finance, Insurance, Real Estate), Services

In short, if we want to reach an understanding of crosssectoral transfer, we may never get there if we start with the nature of domains. A more tractable approach starts from the nature of methods.

This paper considers methods rather than methodologies, since the former comprise the latter, and we need to identify the parts of a methodology that can be transferred, and with what costs. The next section outlines a framework for a method-led analysis (instead of a less promising domain-centred analysis).

A Framework for Transferability

The opening conjecture is that a method is compatible with domains with which its demands are compatible. A demand is any requirement for a method that may not, or cannot, be met in a domain. For example, in healthcare, medical interventions (especially the use of medicines) need to be approved by national review bodies. Thus co-design approaches that let patients design medical interventions cannot always be used in healthcare. Transferability would depend on the nature of patient ideas, which cannot be predicted in advance. Some suggestions would require formal review, but not others (but may still require doctors' consent).

A method-led framework for transferability thus hinges on the nature of method demands. It is possible to distinguish between demands on the basis of their origins in the focus, roles and resources of design work.

Design Situations and the Foci of Design Work

HCI has advocated user-centred design. If all design was user-centred, then the term would add nothing to design. Not all design is focused on users. Even within user-centred design, we cannot always focus on users. For example, dependencies on, and availability of, the back button for web site navigation must be carefully considered. Issues and options here are primarily technical, and have little if any bearing on user needs and preferences. Also, reliability and avoiding bias often guide decisions on evaluations. These are technical concerns within a human sciences context, not user issues. An example of a genuinely user-centred design choice would be avoiding use of the word 'abort' as a command name (e.g., on a GUI button) or in the text of a prompt (e.g., in a modal dialog box), due to its potential to cause distress to various social groups.

From these examples, we can see that there are three distinct types of design choice here:

 Choices about users and other beneficiaries (e.g., pregnant women should not be asked to abort now)

 Choices about artefacts (e.g., disabling back button)

 Choices about evaluations (e.g., Likert scale sizes)

Even at this very high level of abstraction, we can anticipate method demands that are incompatible with domains. For example, incompatibilities can arise if a type of design choice is not valued within a domain. All domains must value choices about artefacts, since choices of form, features, content, capability etc. must occur in all design settings (otherwise nothing gets designed). However, design settings within traditional applied arts or engineering may not value choices about beneficiaries. Indeed, this is why user-centred design came into existence, i.e., as a methodology that was claimed to be superior to the designer-centred applied arts and technology-centred engineering design. Methods that commit considerable resources to understanding users on the basis of primary research (as opposed to stating an unstudied audience) create demands that are not valued when, e.g., designing a bridge or a poster for a play. Conversely, in engineering design, precise actionable verifiable specifications and requirements are more highly valued than in user-centred design. Engineering methods for requirements specification address a type of choice that receives limited attention in user-centred design, i.e., design purpose. Choices of design purpose can also be supported by design briefs for the applied arts, although without emphasis on precision and verification. However, no HCI methods support option generation, option communication and option choice for design purpose (apart from some experimental worthfocused approaches [3,4]).

One way to capture the different types of design choice that are valued in different domains is through the concept of an Abstract Design Situation (ADS [2]). An ADS abstracts over concrete settings to isolate the types of design choices that are made, whether simple (e.g., choice of evaluation) or complex (e.g., aligning evaluation with design purpose and beneficiaries). We can distinguish four ADS classes:

 Applied Arts: explicit choices about artefacts and evaluations, co-ordinated through reflection/critique  Engineering: explicit choices about artefacts, purpose and evaluations, aligned via requirements  User-Centred: explicit choices about artefacts, beneficiaries and evaluations, aligned via rationales  Design for Human Outcomes: explicit choices about artefacts, beneficiaries and evaluations, aligned with explicit choices of purpose (e.g., worth focus)

Each type of explicit design choice (simple or complex) creates demands that may not currently be valued in a domain or sector. We can refine our initial conjecture:

(1) a method is transferrable to domains that sufficiently value and prioritise its associated design choice types.

However, this does not imply that design methods associated with 'unwelcome' choice types cannot transfer. This will only be so if the key influencers within a domain or sector cannot be persuaded to alter their perspectives. Such changes of heart are not uncommon. For example, in the UK, some areas of public service design have recently embraced usercentred co-design approaches. Similarly, the CHI-MED project in the UK (www.chi-med.ac.uk) aims to enable the adoption of user-centred and safety critical methods for medical device development. Given this, transferability is unlikely to be a property of method(ologie)s alone, but instead also results from interactions between human communities.

Methods give rise to demands through their association with types of design choices. Further demands are revealed as we drop down a level of abstraction.

Meta-Principles and the Roles of Design Work

Methods rarely fully support the making of design choices. Instead, different methods perform different roles in relation to making choices. To be able to choose, designers need a 'menu' of well expressed realistic options. Some methods thus focus on the generation of options, others on their communication, and others on their credibility. Methods thus need to be combined to support high quality choices. For example, personas are primarily a resource for expressing potential beneficiaries. Not all will be primary personas, so choices have to be made. However, for personas to be credible, they need to be grounded in field data.

Also, for a set of personas to be comprehensive, field research methods need to be receptive to a broad range of possible stakeholders.

Design and evaluation methods thus have different roles relative to different types of design choices (e.g., choices of beneficiaries). Roles correspond to different types of design work, which in turn are instances of meta-principles for designing [1]:

 receptiveness: openness to options  expressivity: clarity of options  credibility: realism of options Note that a method often only adequately performs one primary role, and thus needs to be combined with other methods to ensure that a wide range of realistic options are clearly expressed to enable high quality design choices. Methods typically also often support one type of design choice, e.g., expressivity for beneficiaries, credibility of evaluations, or receptiveness to creative opportunities for digital products or services.

As with design choice types, the values required from a meta-principle may make inappropriate demands for some domains. For example, receptiveness to creative opportunities for artefact is more strongly favoured in applied art settings than in engineering design settings, and even less so in user-centred design settings. Also, rich expressivity is favoured more in applied arts and user-centred settings than in engineering, which values precision over inspiration. Despite the scientific foundations of user-centred design, standards of credibility tend to be higher in engineering design. applied arts traditions value designers' judgements over objective credibility.

A method's transferability thus also depends on the work that it supports. We can add a second conjecture:

(2) a method is transferrable to domains that sufficiently value and prioritise it role in design work for its associated choice types.

If associated choice types are not valued in a domain, then neither is the work supported by a method.

Receptiveness, expressivity and credibility are three simple meta-principles that associate with a single design choice type. Complex meta-principles involve multiple design choice types, but are mostly associated with design settings that design for human outcomes, where four choice types need to be co-ordinated. In engineering design, co-ordination tends to be achieved via verifiable requirements, and in user-centred design via design rationale methods (albeit often tacitly and inconsistently). For reasons of space, the fit between domain values and the three complex meta-principles of committedness, improvability and inclusiveness [1] is not considered in this paper.

Methods thus give rise to demands through their association with types of both design choices and design work that may not be valued in a domain.

Further demands are again revealed as we drop down a further level of abstraction to look inside methods.

Inside Methods: Design and Evaluation Resources

Methods support design work, but they are not themselves design work. A gap between generalised methods and actual work practices has to be filled with project specific resources. For example, for user testing, participant recruitment criteria, test briefings, instructions, activities and debriefings must all be decided upon within specific projects. There are no generic criteria here. Every user test setting is unique.

Given the loose coupling between methods as ideas and work as a reality, the word approach better indicates that much work is needed in response to choosing some approach. Workable methods only result when resources for an approach have been complemented, parameterised and/or applied in a specific design setting. The effort required to get from an outline approach to a practical method may not be feasible for some domains. We can thus add a third conjecture:

(3) an approach is transferrable to domains that can afford the effort required to produce a practical high quality method when using the approach's resources.

The nature of the resources that comprise approaches is an open question, but examples of resource types for design and evaluation methods include [8]:

(i) scoping resources that indicate the extent of applicability, e.g., specialised heuristics for computer games [5] (ii) expressive resources that communicate the output of a method via specifications, reports, templates and formats etc., e.g., The ISO Common Industry Format (CIF) for evaluation reports [6] (iii) instrumentation resources that collect issues and measures for evaluations, e.g, physiological measurement instruments [7] Such resources give rise to further demands in addition to those associated with design work and choice types.

For example, domains that value scientific validation approaches may find heuristics unacceptable. Similarly, domains that value expert judgement may see no value in CIFs. Also, domains where user experience is not a primary consideration are unlikely to value physiological measurement instruments. We can thus add a fourth conjecture:

(4) an approach is transferrable to domains where its supplied resources are valued Further conjectures can be derived from more detailed analysis of the interactions between design choice types, design work types, and design and evaluation resources, but there is not space here to extend the analysis. Clearly, there is more to method transferability between domains than the four conjectures derived above, but these do offer a basis for more systematic derivation of testable hypotheses.

Conclusions

A compact method-led framework (Figure 1) of choice types, work types, and resources within approaches has more promise as a basis for investigating the bases for method transferability than an approach based on demarcating domain types and their attributes. This paper has focused on presenting the framework and deriving conjectures as a contribution to the new research focus on method transferability. Space has not allowed a thorough and systematic defence of the framework.

Introduction & Motivation

According to Wright and McCarthy [8], people want to share significant experiences with others, and others often expect to hear people's stories of such experiences. This process might be thought of as "recounting" or "storytelling" (see also [1][8]). Thus, communicated, remembered experiences and the way they are reported in texts can be considered as relevant sources for UX. However, such stories can never catch the totality of peoples' lived experience as they are always edited versions.

We aimed at getting a deeper understanding of car owners' remembered experiences as a initial step within a large seven-year research project on UX in the automotive domain. The central focus of our work is on how people describe experiences with their cars and where these descriptions are embedded in and linked to. By investigating written experience reports in online car fora with the means of qualitative text analysis, we made an initial step towards getting a richer understanding of car owners' experiences. The study results build the basis for the second step, which aims at identifying relevant components of UX for in-car interfaces, that should guide the experience prototyping, design and UX evaluation processes.

Related Work

The presence of a researcher in a specific context can easily disturb and influence the users' experiences. Thus, several self-reporting methods such as probing, experience-sampling (ESM), or Experience Reports [5] have been introduced in HCI. All these methods provide participants with tools and materials for collecting and reflecting their experiences for themselves. Although the collected data have proved to be valuable, the participants are still aware of being part of a study and may be influenced by this fact. One way to reduce this bias and reveal more realistic data are long-term studies [4]. Another way is to apply non-reactive, unobtrusive methods (e.g., content analysis) [1]. When applying such methods, the person is not aware of the object of a research project and no specific action is requested on his/her side. The challenge is to achieve a differentiated understanding of UX by combining data from reactive methods with insights from non-reactive methods, such as peoples' descriptions of experiences (e.g., reviews, narrations [8] etc.) initiated and produced by themselves.

Stories and narrations are a vehicle that people use to condense and remember experiences, and to communicate them to an audience [3]. This happens extensively through web blog entries, discussion fora, product test reports, experience reports, etc. As Blythe and Cairns [1] showed in their study, such material provides a rich resource to inform research and design, but also requires new methods. Qualitative research is useful to explore, discover, understand, and describe phenomena that have already been identified but are not well understood yet, such as the UX concept [6].

Text Analysis of Car Forum Entries

A qualitative analysis was conducted for uncovering peoples' car experiences based on written experience reports collected from a public online car forum. We are aware that experience descriptions in car fora only illuminate a facet of the whole experience of a car owner (without linkage to a particular car type), but we are confident that the selected approach represents an initial step towards understanding UX in the automotive context. The selected car forum "Autoplenum.de" is an appropriate source for data because it provides written descriptions of personal experiences by car owners, and serves as a place for communicating these experiences. The newest entries from the car forum were selected (using theoretical sampling) and analyzed by two coders together using theoretical coding in the tradition of the grounded theory [9]. Theoretical saturation was reached after the coding of 21 experience stories (meaning that new data did not require the creation of new categories). The average text length was 337 words; all stories were from different authors.

Text analysis by Strauss is chosen as an appropriate method to analyze narrations about remembered experiences. Our central research question was: How do people experience their car? In order to investigate this question, we focused on the descriptions in the car forum. We believe that remembered experiences and the way they are reported in texts provide a first foundation for formulating car experience types. By the use of text analysis, we identified categories in the text and the relations between these categories. The basic activity in the analysis was the coding of the data, i.e. isolating different concepts in the text, with three successive types of coding: 1) open coding, 2) axial coding, and 3) selective coding. This text analysis process was applied in this initial exploratory study as part of a seven-year research project on Contextual Interfaces and UX research, amongst others, focusing on the automotive domain. Based on the coding process, the following four categories were identified:

1. Attributed qualities: characteristics car owners attribute to their car (e.g., fuel-efficient, fast). 2. System characteristics: car related features mentioned by car owners (e.g., gas consumption, engine performance). 3. Needs and motives: arguments car owners use to describe their needs and motives for owning a car (e.g., practicability, social belonging). 4. Car perceptions: experiences describing the car owners' main focus and values toward their car (e.g., car as tool, car as status symbol).

By using a matrix, the above revealed categories were further linked to each other using the analysis software NVivo 1 . Thus, we identified commonalities and differences between the sub-categories. In a next interpretation step, four main car experience types were extracted based on these initial analysis steps.

Initial Car Experience Types

The four revealed car experience types are presented below, summarizing the initial analysis results from the text analysis based on the selected cases.

Type 1: Car as Tool

This type mainly sees the car as a "means to an end", using it for the purpose of transport. Reasons for owning a car are to facilitate everyday activities and routines. Thus, practicability and suitability for everyday life is in the foreground. Attributed Overall, the car descriptions can be characterized as objective descriptions, listing both positive and negative attributes in a rather neutral tone. In sum, people belonging to this type are satisfied with their cars, they are aware that they do not have high-end cars but take negative attributes into account. However, the positive attributes outweigh the negative ones.

Type 2: Car as Fascination Object

For this car experience type, the car represents a positive entity. Interacting with the car induces positive feelings or memories, indicated by predominant positive associations in the texts. Motives found for this type are nostalgia (thinking of the good old times) or identity management (upgrading one's identity by driving a rare/scarce car). When reading such car descriptions, one has the feeling that the author is a fan of his own car, praising it and writing about the car in a very positive way. Exaggerated affirmations and rather car-untypical attributions can be found in the texts, such as top class, amazing, awesome, genial, impressive, and incomparable. Negative aspects of the car are only mentioned marginally or are not mentioned at all. Positive associated feelings with the car are the main focus within these reports (e.g., robust, snappy, smooth, melodious). Overall, the car owners seem to have a special relationship to their car, often describing it as something reliable they are proud of.

Type 3: Car for Self-Appreciation For this third type, the car represents a means to enhance the persons' social status/identity and to facilitate social affiliation. Moreover, having an enjoyable and pleasurable experience while driving the car (enjoyment of driving) can be found throughout these reports, and becomes a repeated key element in the persons' experiences. Another characteristic of this type is the desire to be outstanding or to belong to a special social group. This seems to be achieved by owning a special and unique car. The authors often mention attributes such as rare, expensive, and authentic for underlining the uniqueness of their car. The car seems to represent a fixed point in their lives, cars being stable in their value for the owner and driving the car is needed by the owner (addictive tendencies in the examined reports). Overall, the car seems to affect the personality of the car owner in a way that is considered as positive and valuable for their life and personal/social identification.

Type 4: Car as Frustration Object

This type differs from the previous three types mainly by characterizing the car as an object of frustration.

Only negative characteristics are attributed to the car (e.g., car causes costs, deficient, expensive, fuelinefficient); no positive attributes can be found in the forum reports. By listing negative aspects of the car, the car owner seems to communicate his/her frustration to the rest of the world. A clear motive for owning a car cannot be identified. Reports of this kind are highly emotional and subjective, revealing the author's disappointment and anger about the car. An overview on all four identified car experience types and a summary of their main characteristics can be found in table 1. The main characteristics are linked to relevant UX components, which we identified in an extensive previous literature review.

Conclusions and Future Work

Our primary goal was to identify relevant categories of "car experiences" from the written car experience reports by means of text analysis and theoretical coding following a grounded theory approach. By trying to understand how people describe their experiences with their car, main attributed qualities, system characteristics, needs and motives, and peoples' perceptions of their car were identified and further interpreted, and grouped into four main car experience types. It is essential for designers to understand these different types. When we know what people remember and recount from their experiences to others, then we can reveal domain-appropriate design recommendations and experience types later on as well as develop, for instance, measuring tools for exploring UX in consideration of the context in which the experiences occur. Currently, for instance, we develop a first version of a car experience questionnaire based on the attributes revealed from the text analysis. This questionnaire will be further extended with more data sets and validated in follow-up user studies in the automotive context to collect further insights on car experience types using additional data sources as well as considering different car types and owners' experiences. In addition, the grouped car experience types can be further developed and used as an inspirational tool for experience-centered design (similar to personas, however, with a special focus on relevant UX components to be considered in the design process). Moreover, we intend to explore our approach towards other domain areas.

Although the text analysis used in the presented qualitative study helps to gather rich descriptions of UX, it has also limitations. Text analyses focus on written reports of remembered experiences produced by people and thus, the analysis misses the experiences not remembered by people or not communicated in the texts. As stated at the beginning of the paper, it is essential to link such data from nonreactive studies with reactive data collection methods. We are planning further user studies using narrative methods (e.g., explicitation interviews [2]) to extend our car experience type descriptions.

Introduction

The academic community has developed many different methods that can be used in UCD. An important issue with these methods is that more often than not, these methods are created from an academic viewpoint, and are difficult to implement in industry practice [3][4][6] [8]. It is therefore essential to understand how UCD is used in practice in order to develop or adapt methods that work in an industry context. The main difference between the academic (scientific) context, and the industry context is that the former can invest a significant amount of time and effort into controlling every aspect of their scientific studies, whereas the latter are confronted with several constraints such as limited budgets, short timeframes etc [3][6]. The latter situation has been described as "wicked problems" [1] [5]. Whereas [2][8] focus on usability evaluation methods, this paper investigates how SMEs appropriate aspects of UCD in general into their industry context for product improvement and/or innovation.

This study was carried out in the Usewell research project, which is aimed at stimulating UCD practice in Flemish SMEs. To achieve that goal we aim to develop a toolkit for SMEs starting out with UCD. The explorative study in this paper forms the first step the project: we want to understand UCD practice in SMEs from an industry point of view, in Flanders. The research question is how contextual factors in which these SMEs operate daily, influence and shape their application of UCD principles and methods. The insights obtained from this study form essential input for the design of the roadmap and toolkit.

The paper is structured as follows: first, we describe the method we used, followed by the organizations that took part in our study. Then, we discuss the main results of the study. Finally, we formulate the main conclusions.

Method

To gather data about the organizations involved, we used an informal interview/information-gathering technique called "mapping". Instead of asking participants to present their experiences and past projects in a standard powerpoint presentation or document, we, along with the participants, used the mapping session to map the trajectory of the players in the project: who was involved, the role he/she played, the various stages of the project, timeline, etc.

The benefit of mapping is that it shares, records and documents insight into a particular organization"s working process and allows the moderator and the participants to have conversations about the project and its key components instead of following a linear structure. Using a limited icon set to map the conversation and experience, these icons act as touch points that allow researchers to gather and analyze information across organizations. In this study the mapping technique was mainly used to stimulate the conversation; the icon sets, the layers of transparent paper, and their annotations, form an external representation which helps participants in dynamic problem solving [7].

In the following instances, the mapping kit used was loosely based on the existing Map-It toolkit (www.mapit.be). The icon set for these mapping sessions included the following key icons: duration/phases, people/group roles, opportunity, achievement, tool, problem, method, role-change and documentation, lock and bomb.

figure 1. Mapping a project using the toolkit Mapping allows for collective group reflection on a given project. Mapping gives respondents a chance to "lock" elements that were/are successful (allowing them to discuss why they are successful and to consider how it can be implemented in other projects) and "bomb" aspects that should be reviewed before being implemented again (aspects that did not go well). Instead of mapping a project from left to right, the mapping technique also uses transparency and layering, so that it is possible to visualize how aspects of the project are "built upon" instead of seeing a project as a linear succession.

Part workshop and part conversation, mapping requires a moderator to lead the mapping activity and participants from the organization that are/were actively involved on the project being mapped. In our study there were one or two moderators and one researcher observing and taking notes. In three cases the mapping was carried out with one person on behalf of the organization, in one case there were two people, and in the last case, there were four people involved.

For data analysis we relied on the researcher"s notes made during the mapping study with each organization. These notes are the researcher"s summaries made of participants" arguments when they placed "bombs" and "locks" on the mapping of their project; the notes describe the contextual difficulties encountered by the organizations, and the respective ways of dealing with these difficulties.

Participants

In order to gain insight from different perspectives, we selected two SMEs (the main focus of the project), two UCD consultancy firms, and one research department inside a larger company. All organizations (or specific teams) involved less than 10 people. The perspective of a UCD consultancy is relevant since their main activities involve helping out other companies without UCD knowledge or experience. Involving a research department of larger company helped us to get a view on the role of UCD in innovation. In sum, obtaining insights from different viewpoints seems advantageous given the explorative nature of the study.

The different organizations involved will now be described by relying on the descriptions they gave of themselves during the mapping study.

The first organization (which we will name C1 in this paper), is an SME that started as a university spin-off. Their main focus is creating software solutions for the social economy with the goal of improving the lives of people with disabilities. They operate in Belgium.

The second organization (C2) is also an SME working on information and audiovisual technologies for a broad, diverse range of clients in the educational, cultural and training sectors, in Belgium as well as abroad.

The third organization (C3) is a UCD and innovation consultancy firm offering the whole range of UCD activities and innovation, for public and private sector clients in Belgium and abroad.

The fourth organization (C4) is also a UCD and innovation consultancy (the only Dutch partner in the project), carrying out design research and service design, for a broad and diverse range of Dutch and international clients, in the public and private sector.

The fifth organization (C5) is a research and development (R&D) department inside a larger, international technology company, which is involved in "everything that goes on in the network". The role of the R&D department is to create "disruptive innovations" in the domain of visual communications for the larger company.

Results

Uncertainty

An important theme recurring in all conversations with the organizations is uncertainty on the part of their clients or other stakeholders, or on the part of the organization that is working with the client.

One form of uncertainty on the part of the organizations occurs before the start of the project when decision-makers have to be convinced to go ahead with the project. Oftentimes such a decisionmaker is not the same person as the one a company talks to about the scope of the project on the part of the client (C2, C3, and C4). Furthermore, a lot of new funding can be at stake for the organization when trying to acquire new projects; it is therefore a delicate balancing act in how much effort to spend in this preproject phase (C1 and C2).

A second form of uncertainty occurs during the projects. This uncertainty is the changing project scope.

In that regard, C4 mentioned that they created a detailed plan for themselves to anticipate on possible outcomes of each phase of the project. At the same time they argued that this detailed plan was always changed significantly at the end of project. Nevertheless, they still considered the exercise of making a detailed plan and evaluating possible directions a project could take as essential.

Communication

In order to manage the above mentioned uncertainties, good communication is vital. The question is: what does good communication actually entail?

All organizations considered visual design documents (personas, user interface sketches, logical schemes…) absolutely necessary. Textual design documents were considered less powerful in order to communicate ideas to and discuss them with the client. In the project discussed with C5 videos were created in order to show a design concept in which they showed people using their application concept in scenarios. The team members found that the video conveyed their ideas better than written personas and scenarios, when they showed them inside their larger organization. The downside was the time that was needed to make these video productions. C4 also used video in order to show design concepts, or to illustrate certain aspects of their ethnographic findings. One of the tools C1 extensively used during client meetings was a whiteboard which allowed them to show, map and draw schemes and plans for discussion with the client. C3 mentioned that using mockups and prototypes they created made their client realize and appreciate the work that is needed to generate these prototypes. Due to severe time constraints C2 used screenshots (very easy, fast and cheap to make) of work-in-progress to communicate the state of the project and discuss its direction.

This does not mean that textual documents were never used; but their use was mainly for documentation i.e. to look up certain decisions (and/or their rationale) made during the project afterwards.

The above statements show us that the organizations decided in each situation what UCD method was the best course of action (based on the available time and budget), and adapted it to their needs.

PERSUASION, INVOLVEMENT

There are different kinds of communication that occur during the course of a project. Which kinds were considered important by the organizations?

Before the start of an actual project, C3 argued that persuasion is important in order to get the client on board. C3 used storytelling to make a compelling case based on their prior cases with other clients to convince the client of the outcome and benefits the proposed UCD approach. There was no standard way of doing this; based on what they knew of the client, they decided on the spot what and how to show their clients what they considered to be valuable approaches.

During projects all organizations met with the client on a regular basis in order to steer the project. C4 considered it very important to involve the client and different stakeholders directly in the project. Stakeholders were invited to workshops in which they would analyze the ethnographic research (maps, photographs, quotes…) results together. The results would not be completely analyzed before going into the workshop. This way they would avoid going into these workshop with their minds made up, and increase the involvement of the client themselves by having them work with the material.

Communication was considered an essential project activity during the discussion of the "locks" and "bombs".

To illustrate its importance, C4 stated that half of the project work consist of communication with the client.

Conclusions

The main theme that came forward by analyzing the results of the mapping study was uncertainty on behalf of the client which is consistent with what in literature is described as practice (as opposed to academic context) and industry [2][3][6] [8]. Before a project could start, the organization had to convince the client that the proposed UCD approached justified the investment. During the project the organizations showed how they mainly used visual means of communication. To conclude, communication was a very important topic that came forward during the discussion of the "locks" and "bombs" on the mappings.

The main implication of these results for the design of our roadmap and toolkit is that in addition to providing useful UCD content (methods, principles), a great deal of attention will have to be paid to communicating this content to SMEs without UCD experience. We can hereby think of use cases which illustrate how a certain UCD method was adapted to the specific company context, and what the value was of the applied approach for the client.

In future work, we will perform a more thorough analysis based on the complete transcriptions of the interviews (as opposed to only using researcher"s notes), an analysis of the mapping artifacts, and a comparison of these two data sources.

Figure 1 .

 1 Figure 1. Students testing the Biology scenario in 2008

Figure 2 .

 2 Figure 2. Hierarchical model of the perceived quality

Figure 1 :

 1 Figure 1: Average document area per feature Vs. Average percentage of visual attention between Groups.

Figure 2 :

 2 Figure 2: Average ratings and standard deviation for Group 1 (30 secs), Group 2 (60 secs), Group 3 (90 secs) and Group 4 (open time frame).

  R and S are defined on sets X, Y and Z. That is, R is a fuzzy relation on the Cartesian space XxY and S is a fuzzy relation on YxZ. The composition R • S = T is expressed by the fuzzy relation of XxZ and this composition is defined as: μΤ(x,z)=Max[Min(μR(x,y),μS(y,z))]. If the relations R and S are represented by matrices MR and MS, the matrix MR•S is obtained from the product of MR and MS which is: MR•S=MR•MS

Ũ

  is the fuzzy relation "defines style" associating each user characteristics ( i u ) with each user types (T), where: Ũ = i u xT ,  i u , T R is the fuzzy relation "requires" which shows how strongly each service characteristic ( ci s ) is required by each user types (T), where: R =Tx ci s ,  T, ci s P is the fuzzy relation "is suitable", which indicates the most suitable presentation of each service characteristic ( ci s ) by comparing and contrasting each alternative web design features ( Fi w ), where: the web design adaptation alternative that is most suitable for a given domain instance (user type, service and web characteristics). The adaptive web design decisions are made by the following fuzzy compositions: Ã W = T o ( R o P ), where T is the fuzzy relation "closer to type (i)" and indicates how close a considered user is to each of the user types of the set T of user types.

  NarbonneToulouse, 31062, France regina.bernhaupt@irit.frMarco WincklerIRIT, Groupe ICS 118 Route de Narbonne Toulouse, 31062, France winckler@irit.fr sectorial usability evaluation methods identifying both ground for common shared tools and techniques and also identifying idiosyncrasies.

Figure 1 .

 1 Figure 1. Screen shot from piloting game (outside view of the aircraft)

Figure 6

 6 Figure 6 an example of game development process model (showing the 3 main phases)

Figure 3 .Figure 4 .Figure 5 .

 345 Figure 3. Test bench for UAVs command and control system (Copyright © 1999-2011 by Aero-News Network, Inc. All rights reserved)

Figure 7 .

 7 Figure 7. F-16 A/B fighter cockpit -inside view of the aircraft

Figure 8 .

 8 Figure 8. DO 178B [1] standard (no usability -no users)

Figure 9 .

 9 Figure 9. IEC 61511 [5] Process Safety (operations are part of the process)

figure 1 :

 1 figure 1: Error measurement plot

figure 2 : 3 : 4 :Figure 5 :

 2345 figure 2: Original shape

Figure 1 .

 1 Figure 1. Method-led Outline Framework for Transferability

Table 2 .

 2 Convergent validity

	Construct	CR	AVE
	PEOU	0.800	0.501
	PU	0.821	0.606
	PE	0.874	0.636

Table 1 :

 1

	Data Collection

10 Average time 18:14 20:30 Highest time 24:46 30:15 Lowest time 13:22 13:30Table 2 :

 2

		System A	System B
	Participant	A1	A2	B1	B2
	1
	2
	3
	4
	5
	6
	7
	8
	9
	10
	Avg. rating	17.4	18.9	23.6	21.9
	Highest rating	22	24	27	25
	Lowest rating	7	8	14	13

Table 3 :

 3 Distribution of Rating Types

			System A			System B
	Participant	R1	R2	R3	R1	R2	R3
	1
	2
	3
	4
	5
	6
	7
	8
	9
	10
	Total	26	34	34	19	33	2

Table 4 :

 4

  Expensive users (retired pilot, current pilots from airlines, test pilots working for the manufacturer),

		Expensive real tests for a new designs/systems
		(50k€ one hour flight test to be included in the
		budget),
		Complex tests scenarios that are usually based
		both on normal and abnormal conditions (previous
		incidents/accidents, emergency situations, partial
		failure of systems …).

  table, comprising only 4 of the 41 methods studied is depicted in Table 1. The evaluation parameters are explained below. §

		Name	Focus groups	Contextual interviews Heuristic walkthrough	Automatic logging of use
		Type	inquiry	inquiry	inspection	testing
		Dev. phase	req. analysis, design,	req. analysis	design, formative and	formative and summative
			formative evaluation		summative evaluation	evaluation
		Type of results	qualitative	qualitative	qualitative, quantitative	quantitative
		Bias	subjective	subjective	objective	objective
		Cost	low (recruitment of	medium (recruitment of	low		low (recording and logging
			participants)	participants during work			equipment, data analysis)
				time, trip to the
				workplace)
		HCI experts	yes	yes	yes		no
		No. of users	6-9	5-10	-	Type of method: This refers to the classification of -(no users recruited specifically)
		Level of detail	high, low	high, low	methods as inquiry, prototyping, inspection or high, low low
		Immediacy	yes	yes	yes	testing.	no
		Location	lab, work place	work place	§ lab	Life-cycle stage: One or more of the following: requirements analysis, design, implementation, lab
		Intrusiveness	yes	yes	no	formative evaluation, and summative evaluation. no
		Strengths	The users' preferences and ideas come from spontaneous reactions.	Takes into account the context of use of a system. Can focus on	§ Inexpensive, flexible, Type of results obtained (quantitative -qualitative). Demonstrates how a system is structured, quick and really used. Allows data collection This is a particularly important parameter, since repeatable evaluation from a large number of real users,
			Group dynamics come into effect. Easily repeatable. Can focus on specific	specific issues and aspects of the system in detail. Most effective for	method. Can be applied on methods that provide different types of results in an automatic and systematic low fidelity prototypes. Easily way. Allows for a longitudinal usually exhibit high complementarity and low documented results. Can approach to studying users'
			issues.	exploring an application domain.	focus on specific parts of a overlap, thereby leading to a more efficient behavior. Provides easily development process. Quantitative results are system. documented results.
		Weaknesses	Results are subjective and not easily documented or analyzed. The presence of			easily analyzed statistically, presented and interpreted in reports, and can be used for
			a group moderator is imperative to keep the group on track and make			comparison of products or ideas in a straightforward way. On the other hand, qualitative
			sure that participants do not influence each other.			results are not easily documented, but can provide important insights that can be easily missed in a
		This is something that should be taken into account		quantitative approach. user recruitment costs and the cost of usability
		in the interpretation of the results and the selection	§	Bias (subjective -objective results). The results experts (if required).
	§	of methods. Cost. Includes items such as required equipment (e.g. a usability lab), prototype development costs,	§	derived from the application of a method may be Need for recruiting HCI experts. Boolean parameter influenced to a significant extent by a personal referring to whether the method requires HCI predilection in part of a user or a usability expert. experts for correct execution.

Table 1 .

 1 Excerpt

	from the HCI
	Methodological Framework depicting
	4 of the 41 methods studied. The
	tabular presentation of the methods
	enables a quick and effective
	overview and comparison. The brief,
	descriptive analysis of the main
	strengths and weaknesses provides
	further insights aiding the selection
	of the most appropriate
	combination of methods.

table 1 .

 1 Overview on the four main identified car experience types with their main characteristics and linkages to UX components.

	Appreciation
	Type 4: Car as Frustration
	Object

Type 1: Car as Tool Type 2: Car as Fascination Object Type 3: Car for Self-

Acknowledgements

The research reported in this paper has been supported by the following grants: COST Action 294 MAUSE (Towards the MAturation of Information Technology USability Evaluation), COST Action IC 0904 TWINTIDE (Towards the Integration of Transectorial IT Design and Evaluation), the Italian MIUR project L4ALL (Learning For All).

Citations

Acknowledgements

This work was supported from the European research project FP6 027039 and the national research project TEHSIN 503/2009.

Acknowledgements

This work is partly funded by Airbus under the contract CIFRE PBO D08028747-788/2008, R&T CNES (National Space Studies Center) Tortuga R-S08/BS-0003-029 and ruwido.

Acknowledgements

We thank The Swedish IT-User Centre (NITA) for partial financing of this study.

Acknowledgements Citations

Acknowledgements

Citations

Acknowledgements

The work reported in this paper is supported by FCT grant SFRH/BD/65908/2009.

Acknowledgements

The financial support by the Federal Ministry of Economy, Family and Youth and the National Foundation for Research, Technology and Development and by AUDIO MOBIL Elektronik GmbH is gratefully acknowledged (Christian Doppler Laboratory for "Contextual Interfaces").

Acknowledgements

We would like to thank FlandersInShape for their coordination of the project and fruitful discussions relating to the issue of how to promote UCD in Flanders. Furthermore, we would like to thank all involved organizations in the project, that helped us during the mapping study by providing access to their experiences in UCD practice.

Discussion

Summary and conclusions

Future Plans [1] The Good Research Guide: For Small-scale Social Research Projects [2] 'Investigating attractiveness in web user interfaces [3] In: Product Experience .

[4] Emotional design: Why we love (or hate) everyday things, [5] i-com, ,

Usability Engineering [7] The psychology of personal constructs [8] [9]

British Journal of Psychology 117 [10] A manual for repertory grid technique [11] British Journal of Psychology 62 [12] An Overview and Tutorial of the Repertory Grid Technique in Information Systems Research.

An Integrated Approach Towards the Construction of an HCI Methodological Framework

Abstract

We present a methodological framework aiming at the support of HCI practitioners and researchers in selecting and applying the most appropriate combination of HCI methods for particular problems. We highlight the need for a clear and effective overview of methods and provide further discussion on possible extensions that can support recent trends and needs, such as the focus on specific application domains.

Keywords

User experience, usability, methods, framework, HCI, domains.

ACM Classification Keywords

H5.2. Information interfaces and presentation: User Interfaces -evaluation/methodology.

General Terms

Human factors.

Introduction and background

The use of human-centered design models and methodologies facilitates software development processes. However a plethora of approaches exists and it can be challenging for developers to appropriately match tools to problems. One way to address this issue is via methodological frameworks,

Tasos Spiliotopoulos

Department of Mathematics & Engineering

University of Madeira 9000-390 Funchal, Portugal tasos@m-iti.org

Ian Oakley

Madeira Interactive Technologies Institute

University of Madeira 9000-390 Funchal, Portugal ian@uma.pt

Domain Values and Method Transferability: an Initial Framework

Abstract

The transferability of design and evaluation methods between domains is an interesting research focus that needs to be given firmer foundations. This paper presents an initial framework that can generate useful conjectures about factors that (dis)favour transferability of methods.

Hidden in the Text: Exploring User Experience for the Automotive Domain

Abstract

Within the HCI community there is a need for rich descriptions of User Experience (UX) from a real life perspective in order to guide future experiencecentered design processes. Most UX insights are revealed in lab or field studies where people are directly involved in the data collection process. Within this paper, we make an initial step towards getting a richer understanding of car owners' experiences by investigating already available, written experience reports in a selected online car forum with the means of qualitative text analysis. We want to show the potential of qualitative analysis for getting less biased insights on UX. Following a grounded theory approach four main categories are identified and interpreted from the reports. They are further grouped into four car experience types and described in more detail with regard to their main characteristics and linked with relevant UX components to guide future experiencecentered design processes of in-car interfaces.

European Science