Ernesto Arroyo
email: ernesto.arroyo@upf.edu

Ted Selker
email: ted.selker@sv.cmu.edu

Attention and Intention Goals Can Mediate Disruption in Human-Computer Interaction

Keywords: Disruption, Interuption, Adaptive Interface, Software Managers, Human Computer Interaction

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The increase of information overflow and continuous request for user attention [START_REF] Horvitz | Models of attention in computing and communication: From principles to applications[END_REF] makes interruptions a common occurrence in computing environments. Users might delegate another person or a computerized intelligent agent to perform tasks on their behalf to avoid cognitive overload and successfully perform multiple tasks. With enough delegation, management of it can also cause interruption [START_REF] Maes | Agents that Reduce Work and Information Overload[END_REF]. This paper focuses on the disruption caused by interruptions presented to users by computer technologies as the result of a request for synchronous or asynchronous communication. Careful design of interrupting notifications might help reduce disruption effects on people's performance on ongoing activities. But in order to keep up with the ever-increasing demands for user attention, future computers environments will require active mediators that are able to interpret and recognize the value of communication. Indeed, mediating approaches have been helpful in developing successful software agents such as spam filters that act on behalf of the user. Our work deepens such systems into the area of mediating people's communications with friends and colleagues so that people are able to maintain concentration amidst their busy different lifestyles. While it is possible to restrict all interruptions so that they do not disrupt the ongoing task, sometimes interruptions represent important corollary tasks, opportunities you never want to pass up, or being responsible for tasks of even a higher priority such as health. We argue that user goals and motivations should take precedence over micro-task benefits in considering which interruptions to block or attempt to block. To illustrate this argument, we present a Disruption Management Framework designed as a modeling method for designing computer-based disruption managers. The framework outlines some of the factors needed to mediate disruption in computing activities regarding the interruption context and its relationships to the user's goals.

Interruptions, Disruption, and Distraction

Interruptions are an everyday and normal part of human behavior. People frequently interrupt communication dialogue, such as an unanticipated request for topic switching while having a conversation [START_REF] Mcfarlane | Coordinating the interruption of people in human-computer interaction[END_REF]. An interruption can be defined as an unanticipated request for task switching from a person, an object, or an event while multitasking [START_REF] Mcfarlane | Coordinating the interruption of people in human-computer interaction[END_REF] [START_REF] Mcfarlane | The scope and importance of human interruption in human-computer interaction design[END_REF][24] [START_REF] Gillie | What makes Interruptions Disruptive? A study of length, Similarity and Complexity[END_REF]. Interruptions typically request immediate attention and insist on action [START_REF] Covey | The Seven Habits of Highly Effective People[END_REF], and reduce productive focus. If an interruption is allowed to distract the user into action, it escalates into disruption [START_REF] Jackson | The cost of email interruption[END_REF]. Thus, disruption is defined as the negative effects on a primary task from interruptions requiring transition and reallocation of attention focus from one activity to another. This paper will use the term disruption to accommodate a situation in which a request has been accepted from another task, causing a negative effect on the ongoing activity. Although the main focus of this paper is disruption it is important to mention that distractions and interruptions are similar in that they can both occur while the user performs a primary activity. For instance distraction conflict theory represents a research stream investigating how distractions can be ignored or processed at the same time as the primary activity [START_REF] Groff | Distraction, attentional conflict, and drivelike behavior[END_REF]. Unlike distractions, interruption-disruption shares the same sensory channel as the primary activity and encompasses a task that could be performed. Our main interest is in on interruptions that result in capacity and structural interference and that disrupt the ongoing activity; negatively affecting human performance [START_REF] Hess | Training to reduce the disruptive effects on interruptions[END_REF].

Interruption Management

Coordinating interruptions involves one or more person's modes of activity: cognition, perception, or physical action. Interruption coordination is defined as the method by which a person shifts their focus of consciousness from one processing stream to another [START_REF] Mcfarlane | Coordinating the interruption of people in human-computer interaction[END_REF]. How efficiently and effectively interruptions are handled by users might depend on characteristics of the notification itself and characteristics of the ongoing activity [START_REF] Burmistrov | Do interrupted users work faster or slower? The microanalysis of computerized text editing task[END_REF]. However, people's reactions to interruptions and perceived disruption are principally affected by goal oriented strategies adopted to evaluate incoming interruptions in order to accomplish their goals [START_REF] Latorella | Effects of Modality on Interrupted Flight Deck Performance: Implications for Data Link[END_REF], [START_REF] Mccrickard | Attuning Notification Design to User Goals and Attention Costs[END_REF], [START_REF] Bailey | Iqbal Understanding Changes in Mental Workload During Execution of Goal-directed Tasks and Its Application for Interruption Management[END_REF]. For example, a person who works at an office is more likely to take an incoming phone call from a co-worker while at work than when he is on his way home. It is common for people to juggle several competing goals at one time, while their priorities might change depending on various factors. This is exemplified in a diary study of office work, which reported frequent and deliberate task-switching activities [START_REF] Czerwinski | A diary study of task switching and interruptions[END_REF]. Similarly, residential interviews and self-reports revealed that willingness to handle interruptions varies across individuals with current location, as well as with current activity [START_REF] Nagel | Predictors of availability in home life context-mediated communication[END_REF]. Can a computing system recognize and mediate relative to the underlying factors that influence people when dealing with interruptions? Our work addresses this question and shows how a disruption manager utilizing context and relationships to user goals and tasks can reduce disruptive qualities of interruption requests.

Disruption Management

One of the key questions for the understanding of human disruption is identifying the factors that play a role in people's decision process regarding interruptions. Previous research has focused on identifying task complexity and its influence on user performance [START_REF] Bailey | The effect of interruptions on task performance, Annoyance, and Anxiety in the User Interface[END_REF] [START_REF] Cutrell | Notification, Disruption and Memory: Effects of Messaging Interruptions on Memory and Performance[END_REF], the coordination method used to handle interruptions [START_REF] Mcfarlane | Coordinating the interruption of people in human-computer interaction[END_REF], the interruption point at which interruptions arrive [START_REF] Cutrell | Notification, Disruption and Memory: Effects of Messaging Interruptions on Memory and Performance[END_REF] [START_REF] Iqbal | Disruption and Recovery of Computing Tasks: Field Study, Analysis and Directions[END_REF], the similarity between ongoing and interrupting tasks [START_REF] Gillie | What makes Interruptions Disruptive? A study of length, Similarity and Complexity[END_REF], and the interruption modality [START_REF] Latorella | Effects of Modality on Interrupted Flight Deck Performance: Implications for Data Link[END_REF]. We expand this to focus on the factors that influence people's decision process at turning interruptions into actual disruption. A factor strongly correlated to how people react to interruptions is the level of goal commitment, that is, the importance of the task and the belief that the goal can be accomplished [START_REF] Locke | Building a Practically Useful Theory of Goal Setting and Task Motivation: A 35-year Odyssey[END_REF] [START_REF] Mccrickard | Attuning Notification Design to User Goals and Attention Costs[END_REF]. The importance of the goal to the individual, will affect subsequent reaction to an interruption.

We explore the hypothesis that people's reactions to interruptions and perceived disruption are principally affected by goal-oriented strategies users adopt to evaluate incoming interruptions in order to accomplish their goals. We argue that incoming interruptions are evaluated with respect to the ongoing processes goals and priorities. Therefore, it is possible for people to be influenced by the level of commitment to a task [START_REF] Iqbal | Disruption and Recovery of Computing Tasks: Field Study, Analysis and Directions[END_REF][7] [START_REF] Van Bergen | Task interruption[END_REF]. If a task is almost completed, people can opt for finishing the task before accepting an interruption and switch to the interrupting task as task priorities are constantly changing.

Disruption Mediation Framework

Disruption Mediation Framework (DMF) introduces a multilayer software model that separates important constituents of disruption and aids in creating a computer-based disruption manager. Earlier interruption models [START_REF] Latorella | Effects of Modality on Interrupted Flight Deck Performance: Implications for Data Link[END_REF][32] [START_REF] Altmann | Managing attention by preparing to forget[END_REF][20] [START_REF] Lieberman | Out of Context: Computer Systems that adapt to, and learn from, context[END_REF] have not supported continuously monitoring the user and dynamically adapting timing and interrupting modality, among other parameters. Indeed, current interruption models focus on the user, disregarding the context from interrupting applications. Based on our work and existing research we take the approach that goal concepts and task context serve as important factors in predicting disruption. The DMF is influenced by existing literature, and is especially an extension of our explorations in three areas: tracking user activity, identifying ways of tracking ways to interrupt people, and understanding the factors that influence the interruption decision process (task completion level, task type, task complexity, task switching strategies, etc.)

[3][4][12][22][35].
The DMF identifies a decision process that evaluates incoming interruptions with respect to the current state of the ongoing activity or situation. It determines the priorities of each interruption to decide when to execute the task associated with the interruption. Previous work investigating interrupting modalities has shown that the decision process is affected by people's individual differences, such as prior experiences, motivations, and psychological factors [START_REF] Altmann | Managing attention by preparing to forget[END_REF][15] [START_REF] Horvitz | Learning and reasoning about interruption[END_REF]. The DMF also identifies goal priorities, goal commitment, and goal relevancy as other factors affecting the interruption-decision process.

The DMF introduces goals and relevance (to these goals) as a main factor for mediating interruptions. The interruption process is closely related to attention (according to the information-processing model). Attention also determines which goal concepts are relevant. These concepts then can also determine which tasks are important or have higher priority. For this reason the DMF uses concepts to provide a cognitive representation and offer insight into the user's attention. The DMF also includes, user activity, and task context as other important factors that allow examining incoming interruptions in order to evaluate their potential for disruption (see Fig. 1). Our approach is meant to be usable and easily implemented for any interactive digital system.

Relationships Between Goals and Concepts

SuwatanaPongched [START_REF] Suwatanapongched | A More Complex Model of Relevancy in Interruptions[END_REF] classified interruptions into three categories: 1) Interruptions relevant to the primary task that assist completion of the primary task. 2) Interruptions irrelevant but related to the primary task, although not contributing to completion of the primary task. 3) Interruptions irrelevant and unrelated to the primary task. These interruption categories are useful; however, people often perform more than one task at a time in order to accomplish their goals as several tasks can be grouped as being part of a single goal. The above interruption classification schema can be extended to include the relationship to the user's goals. That is, interruptions irrelevant to the primary task can be related to the user's goal, thus contributing to completion of one of the user's goals. Some of these goals are unique in that the user might be willing to sacrifice a certain amount of primary task attention in order to achieve them. Literature agrees that irrelevant, unrelated interruptions can be harmful to the primary task, and that they elicit frustration and anxiety [START_REF] Cutrell | Notification, Disruption and Memory: Effects of Messaging Interruptions on Memory and Performance[END_REF][13][15] [START_REF] Mcfarlane | Coordinating the interruption of people in human-computer interaction[END_REF]. Our work has also shown that people can benefit from interruptions if they are relevant to the ongoing task or the user's goals [35].

Current approaches for reasoning about the user's goals have focused on sensing user actions directed at achieving a domain-specific goal [START_REF] Gievska | Examining the Qualitative Gains of Mediating Interruptions during HCI[END_REF]. This approach is dependent on previous examination of the desired domain and is limited to known domains or the domain itself. Instead, the DMF uses concepts such as topic of documents, and communication derived from the user environment as a way to reason and think about the underlying user's goals. Tools such as, WordNet1 and ConceptNet2 can support context-oriented inferences over real-world texts [START_REF] Singh | Open Mind Common Sense: Knowledge acquisition from the general public[END_REF], by supporting semantic similarity computations and by performing query expansion from a given concept. These semantic knowledgebase systems make reasoning about goals through concepts a practical approach.

Activity Monitoring

The DMF focuses on supporting people's goals as a mean to reduce disruption. The degree to which interruption mediated interfaces support the user goals is a key factor that determines their success. The success is also determined by inferences generated from interpreted knowledge; we call virtual sensors [START_REF] Lieberman | Out of Context: Computer Systems that adapt to, and learn from, context[END_REF]. Thus, The goal is to identify "sensors" that generate sufficient knowledge independent from domain-specific sources. These domain-independent sources provide the basis for mediating disruption when no other data sources are available. Low-level factors, such as micro tasks and HCI interactions (mouse and keyboard behavior, windowing activity, etc.) can be used for mediating interruptions, as they provide a fail-soft system response when no domain-specific data is available.

Tasks

The benefit from accepting interruptions should be balanced with respect to the ongoing task. The challenge consists of balancing interruptions at a task level while supporting the user's goals. Our approach extends and makes use of some constructs defined by previous research in the area of interruptions. This research indicates that disruption is related to several task factors: 1) Task complexity and similarity of the primary or interrupting task [START_REF] Gillie | What makes Interruptions Disruptive? A study of length, Similarity and Complexity[END_REF][22], 2) Interruption relevancy to the primary task, task stage when interrupted [START_REF] Czerwinski | Instant Messaging: Effects of Relevance and Time[END_REF], 3) Interruption coordination method [START_REF] Mcfarlane | Coordinating the interruption of people in human-computer interaction[END_REF], 4) Modalities of the primary task and interruption [START_REF] Arroyo | Interruptions as Multimodal Outputs: Which are the Less Disruptive?[END_REF] [START_REF] Latorella | Effects of Modality on Interrupted Flight Deck Performance: Implications for Data Link[END_REF]. In our framework task context includes Number of Tasks, Task level, Task Type, Task Complexity, Task Difficulty, and Task priority.

Disruption Manager

We developed a test-bed to evaluate dynamic interruption systems. The test-bed allows the examination of the relation between ongoing behaviors, task actions, goals and interruptions. The application is implemented as a three-layer architecture system (see Fig. 2). A low-level layer includes implicit low granularity information such as keystrokes and mouse movement activity. An intermediate layer includes the activities and information to which some of the low granularity data can be extracted and summarized, such as reading, switching tasks, paying attention, etc. And a top layer or knowledge layer includes the information or concepts related to the user goals. The disruption manager monitors the user state (current activity), concepts surrounding the user's goals: history of recently accessed documents, web pages and search queries, the interrupting message relevance to these concepts, and concept priority. The manager then identifies messages that should be allowed to interrupt the user or delayed to an appropriate time within task execution. The disruption manager uses monitoring modules to track the user state, concepts surrounding the user's goals and interrupting message concepts. The system has one module for each context (goals, tasks, and activities) in which the interruption content is examined and a decision module that mediates interruptions on multiple services based on the evaluations of the context modules. The mediator uses several auxiliary modules for interfacing with an Instant Messaging client to both read in the Instant Message (IM) content and manipulate the timing and presentation of the IM. A context module that uses natural language on the goal-level layer; experience, interest, and reading for the intermediate layer; and mousetracking, task-stage, and task-exit for the low-level layer. These architectural elements in disruption manager are described in the following section.

Context Modules

Each Context Module is responsible for evaluating a particular aspect of the interruption, the system, and the user. These modules are derived from aspects of the disruption model. The modules convey their evaluations to the Decision Module as appropriateness of showing the IM at a given moment. The manager's top level monitoring layer uses Google Desktop 3 and ConceptNet [37] engines as services running on the user's computer. Google Desktop keeps an up-to-date index of files and documents and their contents. ConceptNet is a commonsense knowledgebase with facts from the Open Mind Commonsense corpus [START_REF] Singh | Open Mind Common Sense: Knowledge acquisition from the general public[END_REF]. Its concise semantic network contains 200,000 assertions and supports practical textual-reasoning tasks over real-world documents.

Natural Language Module. (NL) implements the part of the disruption module concerning the relatedness of the content of the IM to documents the user is working with. It uses natural language processing and commonsense reasoning to develop an understanding of the interruption and documents, and attempts to compare the interruption to each document. These comparisons are aggregated into a relevance score, indicating the relevance of the IM to all documents examined. A separate score focuses on relevance to the current document. The NL Module has three major components. The first is responsible for obtaining the text of an IM (this is done through the IM Interface). The second component is responsible for locating files of interest and obtaining the contents of those files. The manager application queries the Google Desktop Engine for recently accessed documents, files of interest (PDF, DOC, PPT, etc), emails, IMs and web pages and parses them using ConceptNet and a natural language processing engine. Files of interest are the files open on the user's computer, as well as recently viewed documents and webpages. The system also obtains a list of open files using VBScript and the Microsoft PsTools 4 library. The system then uses Google Desktop [39] to locate and read those files and to search for web pages viewed in the past hour in the web cache, and to find and read the documents in the user's My Recent Documents folder. The third component uses document-level functions in ConceptNet (text normalization, commonsense-informed part-of speech tagging, semantic recognition, chunking, surface parsing, thematic-role extraction, and pronominal resolution) to extract the verb-subject-object-object frames from recently accessed documents. The entire contents of both the IM and all of the retrieved documents are individually fed into the MontyLingua 5 natural language processing suite. The MontyLingua suite provides both lexical parsing of text and commonsense reasoning through the OpenMind 6 commonsense database The NL Module extracts from MontyLingua interpretation key words and concepts in the texts, uses a thesaurus to find possible synonyms for those words and concepts, and then counts the number of times the important words, concepts, and synonyms from the IM appear in the other documents. The NL module then extracts all the concepts in a document, assigns them saliency weights based on lightweight syntactic cues, and computes their weighted contextual intersection.

Concept connections in ConceptNet's semantic network allow the contextual neighborhood around a concept to be found by spreading activation radiating outward from a source concept node. The more frequently the number of important words or concepts appears, the more relevant the content of the IM is likely to be. The output of the NL module is the average number of times a key word or synonym in the IM appears per sentence in all the searched documents. This module allows the manager to summarize text of active documents, identify the documents gist topics, evaluate notifications, capture and classify incoming messages, detect if actions are required from the user, keep track of topics related to ongoing and past goals, and determine if incoming interruptions should be presented to the user.

Mouse-tracking Module (MT).

This MT module is composed of three components and represents the portion of the disruption module concerning the user's familiarity with and their depth of involvement in their task. This component records the user's mouse movements and reasons about the user by comparing his or her mouse movements to a database of many people's mouse movements. The Mousetracking Module serves to determine the experience of a user with a website, the interest level of a user in a website, and whether the user is reading or scanning a website. This module is based on experimental data monitoring mouse activity while browsing websites [START_REF] Chen | What Can a Mouse Cursor Tell Us More? Correlation of Eye/mouse Movements on Web Browsing[END_REF]. The manager's low-level proxy-based layer installed on the user's computer monitors and categorizes mouse movement activity into low granularity behaviors (scrolling, menu, text input, clicking) and user states (reading, deciding, scanning, and waiting). The manager uses a naïve Bayes classifier and wrapper based feature selection because it is computationally inexpensive, which is important when running on a web-browser due to its limited resources. The mouse tracking module outputs data representing the percent experience, the percent interest, and the percent likelihood the user was closely reading a webpage. The Decision Module uses these heuristics to estimate the depth of user involvement with their current task, with the idea being that the more deeply involved a user is with their current task, the more costly it is to interrupt the user.

Task Stage Module (TS).

The TS module is responsible for determining if a user is at the beginning, in the middle, at the end, or between tasks. It does this by looking for discontinuities or changes in keyboard and mouse usage and windowing behavior. This is used to determine the user cognitive load based on HCI interactions. The TS looks for significant changes to the number of keystrokes per minute, mouse time per minute, or windowing behavior indicating that the user is changing tasks, or subtasks, as at those moments interruptions are more likely to be less disruptive [START_REF] Iqbal | Disruption and Recovery of Computing Tasks: Field Study, Analysis and Directions[END_REF].

Existing Tasks Module (ET).

The ET module attempts to gain an understanding of persisting tasks the user may have, even though they are not currently working on them, and corresponds to the part of the disruption module which determines if interruptions relate to other tasks the user has but may not be currently working on. The ET module returns the percentage to which an interruption is appropriate for past tasks. This information is a heuristic for how often the user cares about what the interruption is about, and thus also on how significant tasks related to the interruption are to the user.

Decision Logic Module (DL).

The DL module is the central component of the disruption manager. Whenever an IM arrives, DL determines the appropriateness of that message. It polls all Context Modules for their evaluations of the IM, and decides how to proceed. Once the disruption manager decides an interruption should be presented, it delays the interruption until an appropriate time in order not to disrupt the ongoing micro-task or activity. If the interruption is relevant to the user's goal, the manager gives priority to this interruption, and presents it as soon as possible; while minimizing disruption on the ongoing task. Delaying standards are lowered over time to guarantee the message will be displayed within 2 minutes. In addition, the manager's decision rules are based on findings showing that interruptions related to topics the user has worked on in the past, have the potential to be significant to the user's goals and should be allowed [START_REF] Bailey | Iqbal Understanding Changes in Mental Workload During Execution of Goal-directed Tasks and Its Application for Interruption Management[END_REF][22] [START_REF] Mccrickard | Attuning Notification Design to User Goals and Attention Costs[END_REF]. Thus, the manager limits the number of unrelated interruptions in order to reduce perceived disruption. The manager also limits interruptions whenever this ratio increases and allows interruptions related to prioritized topics whenever confidence values are above predefined thresholds. The decision module for the disruption manager is implemented as add-on to Trillian™; a stand-alone chat client that supports AIM, ICQ, MSN, Yahoo Messenger, and IRC. This allows the disruption manager to be easily deployed and integrated into current systems without requiring the user to lose existing contacts, learn a new interface. The IM client provides unique customization functionality, such as, contact message history, and an advanced automation system to trigger events based on anything that happens in the client. This allows the disruption manager to "catch" incoming interruptions and control them (see Fig. 3).

Evaluation

An experiment evaluated Disruption Manager's effectiveness in mediating interruptions compared to existing interfaces. The experiment elicits disruption by presenting interruptions while the user is engaged on a task. These interruptions present themselves as notifications that carry a message associated with them, requiring the user to switch to another task. In addition, the message associated with each interruption can either be related or unrelated to the user goals. The experiment evaluated Disruption Manager with respect to productivity and user satisfaction. Productivity refers to objective performance metrics such as overall goal completion, the time taken to finish an activity, task or goal. While user satisfaction refers to subjective metrics designed to evaluate the perceived user satisfaction for given tasks and overall goals. The main dependent variables were performance and perceived disruption. Several other variables were used to confirm the task was performed properly. These variables included task time, number of notifications attended to, time spent on each email, and STAI 7 score.

Hypothesis

1. People using Disruption Manager will have higher performance than people in the No-Manager condition. 2. People using Disruption Manager will be more efficient in their task than people in the No-Manager condition.

3. People using Disruption Manager will report less perceived disruption than people in the No-Manager condition.

Method

The effectiveness of Disruption Manager was assessed using a between subjects experimental design. Subjects were randomly assigned to one of two conditions: 1) Communications mediated by Disruption Manager or 2) Communications presented normally as the IM application receives them (No-Manager). In the Disruption Manager condition, the manager controls email notifications presented based on whether the email is related to the ongoing activity and several factors. Fig. 3 shows the filtering stages that each interruption must go through before being delivered to the user. The manager allows people to complete the task without unnecessary distractions. That is, related IMs are presented (almost) right away so that the subjects can benefit from the IM. On the other hand, irrelevant IMs are delayed until a subtask is finished the. The manager's behavior can be summarized with the following rules:

• Relevant IM are presented after small changes in activity, such as quick task switches, or after finished finding an item, updating values, text entry, etc. • Irrelevant IM are presented after subjects finish gathering data for one customer, or finish sending email. User Task Scenario. The scenario consisted of customer service and order processing activity for an e-commerce site. We simulated a typical small business environment where customer service representatives take email orders from several customers and process each order individually trying to satisfy the customer's demands and complete a sale. This represents the most commonly reported task for 77 million workers who used computers at work in 2003 according to a Survey by the USA Bureau of Labor and Statistics [START_REF]Bureau of Labor Statistics Current Population Survey for Computer and Internet Use at Work[END_REF]. Subjects were told that customer service representatives obtain a commission based on their sales and instructed subjects to play the role of a customer representative. Adding this role guaranteed that subjects would perform the task to the best of their abilities and encouraged subjects to obtain a bigger profit. The customer service scenario serves as a good proxy for representing high-level goals. Rather than specifying detailed task-based goals, the scenario presents an abstract goal requiring subjects to create their own definition of the goal based on the task constrains. The abstract task goal is to satisfy the customer demands based on their requirements. This type of settings addresses the situation where a system might not be able to identify the exact representation of the user's goals, and relies instead on the specific concepts surrounding the user's goals. Subjects were instructed that the experimental setup evaluated an automated system that filtered customer e-mails and an Instant Message (IM) system for price updates, and requests from fellow employees. Centering the experiment on the email manager system focused subjects' attention o the email content, rather than on their own reactions to IM. This warranted that people would react to interruptions, as they would normally do.

User Task description. The tasks presented in the scenario are similar to those used in a customer service task done at a large supply company in the purchasing department. Small to medium e-commerce sites still require human intervention and process customer orders in a similar way. The main task is focused on processing email orders; the task includes different types of customers, and the importance of satisfying customer's requests. The tasks are controlled by two systems for improving productivity: 1) An automated system that assigns different type of customers to different employees through the day, as to maintain a balanced workload. The system classifies and sorts customer emails depending on the type of service requested and the customer's demands (high accuracy, fast response, high volume, and low-volume clients). 2) An instant messaging system that allows its employees to share pricing information with one another. Sharing information benefits the company and participants receive a bonus based on the company performance.

User Task details. Each order included a short email script where customers described which items they wanted to buy and why. The short email script included enough information to convey the expectations and motivations from real clients and hint the condition type. Each email script was designed so that it would remind subjects how they should process the email and to reduce the task completion time. Subjects worked with an email client that sorted customer emails and placed them in separate folders. Each of these folders had to be completed before moving to the next one. The task required subjects to scan their email folders, decide which type of customers they might be working with and what type of service to provide these customers. The subjects then found the listed price for the items requested by each customer, and arranged the products so that the customer was able to buy as many items as possible while accommodating their preferred products; all within the customer's budget. Subjects were encouraged to use their own intuition and taste in order get them involved in the task. Two out of four emails had artificially introduced budget errors (similarly to a customer error). Errors required subjects to drop one or two items from the order, correct the item price, and recalculate the order quantities based on the customer budget restrictions. Emails with no errors received prices updates so that the update could be reflected on the email price. The errors ensured subjects devoted enough attention to the task and to keep the task from becoming monotonous. The number of items per email order varied from 3-5 items to keep the scenario more realistic and short enough not to overwhelm subjects, and to provide enough time for interruptions in the middle of the task.

Pilot Studies.

Several pilot studies showed that we had indeed created a cognitively effortful scenario and that subjects should perform a minimum of two email orders before becoming familiar with the task. As a result of these pilots several changes were introduced: 1) the length of each condition was reduced to 10-15 minutes long.

2) The number of emails to be processed by condition was reduced from an earlier experiment with 6 emails to 4 emails so that subjects wouldn't be overwhelmed with too many orders needing processing.

3) The number of interruptions was reduced from 5 interruptions per email to 3 in order to allow subjects to successfully complete the order without excessive interruptions. These pilot explorations also helped choose the quality and urgency tasks to focus on in the formal experiment.

Protocol

Each mediator condition was presented in three stages, an introduction stage, a quality stage, and an urgent stage. The introductory stage served to fully familiarize subjects with the experimental task. This within-subjects condition explored how the task is performed when the task is highly prioritized. The quality and urgent conditions were selected because the exhibited similar traits in pilot experiments. 40 subjects were randomly assigned to two conditions: Disruption Manager, and No manager. Subjects were first presented with the interface and a walkthrough of the task based on a script previously rehearsed by the experimenter. In order to obtain a consistent response to interruptions, the walkthrough included an exemplification of potential interruptions and how they should be deal with them A practice session allowed participants to become familiar with the interface, familiar with the content, and familiar with the interrupting messages. The practice session also allowed subjects to identify the benefits from attending to interruptions. The practice session lasted until subjects completed all questions and were satisfied with their answers. On a second practice run, timed sections were introduced in order to introduce this feature and allow subjects to experiment with different navigations techniques.

Results

The hypothesis regarding performance was confirmed. Mediating interruptions yielded higher performance than without mediation supported by planned comparisons indicating a significant difference on performance based on the manager type F(1, 37) = 473.92, p<.001. The disruption manager conditions showed 26% and 32.5% performance increase over no mediation for the quality and urgent tasks respectively. Additionally to the increases in individual performance, Disruption Manager subjects were more able to share pricing information by replying to IMs, therefore improving overall goal completion or company's profits. This was confirmed by the hypothesis stating that subjects under the Disruption Manager condition would be more efficient in their task. The ratio of IMs responded to was about 5 times higher for the disruption manager condition. This indicated that the manager did much better at presenting interrupting messages (relevant information) at the right time. Participants on the manager condition responded to 58% and 51% of the IMs received for quality and urgent tasks. Whereas, they only responded to 12% and 8% of the messages on the No-manager condition, see Fig. 4. Perceived Disruption. There was no main effect of manager type in perceived disruption F(1,37) = .089 p=.7, nor were there any significant contrasts between Quality and Urgent tasks. Thus, our third hypothesis that subjects would perceive less disruption was not confirmed. Both manager categories demonstrated a similar moderate (0.5) disruptive effect across all task categories: 0.5 -0.6 with no manager and 0.5 -0.5 with disruption manager. While more data might show a main effect here, the shear point that automated disruption management did not increase perceived disruption and that Subjects were not aware of the system being different is clearly exciting and present here.

Discussion

We have presented a Disruption Management Framework and Disruption Manager System that reduces disruption by taking into account the user goals tasks, and activities. Our results show a general computer interface capable of evaluating incoming interruptions in relation to their benefits to the user's goals and the disruption to the ongoing task, improves performance and overall productivity. Our evaluation shows a 26% and 32.5 % performance increase for task prioritized by quality and urgency. The manager also increased goal and task efficiency by presenting relevant information at the right time and allowing participants to respond to 5 times as many interrupting messages. Furthermore, participants showed no perceived difference in disruption for IM's delayed by our intelligent system. Delaying IM's by variable amounts of time did not interfere with the user goals and did not increase perceived disruption. Disruption manager can in fact reduce unnecessary disruptions without reducing a person perceived control. This work represents a new style of interface, one that can recognize and mediate important communication effectively. The approach improves a user's ability to work fast or accurately by attending to what the computer knows will be more relevant.

Conclusions and Future Work

In an increasingly complex information world, mediating information is critical and new tools must be developed to gather information about attention from low-level data. However, we must consider that as new sensors are being developed, their implementation is time consuming and user acceptance increases slowly. Therefore, domain-independent sources of information are still usable as means to provide failsafe disruption mediation. In this paper we used several tools developed to investigate attention and user interest from domain in dependent low-level metrics. These tools demonstrate that a disruption manager is able to rely on Goals, Tasks, Implicit Metrics, Activity, Mediator Filters, Sensors and virtual sensors. These tools are bound to improve; everyday new technologies and approaches provide the much-needed insight into people's computing activities.

More and more software acquires an ability to decide when it is appropriate to accept or project information to and/or from particular people, particular places, and particular projects. This will have profound influence on the social dynamics between people, the ability for people to accomplish their work, their homework, their social responsibilities, and even their own goals. In addition to mediating interruptions based on goal-related information, it is important to understand how people manage interruptions and what makes interruptions be perceived as disruptive. Future work should investigate the factors that influence people's decision process at turning interruptions into actual disruptions. For instance, interruptions relevant to user's goals might not be perceived as disruptive, even if unrelated to the ongoing task. In these situations people might be more likely to switch to the interrupting task in order to accomplish their goals, or not. Further information is needed regarding the user task priorities and their influence on perceived disruption: How do changes in task priority affect people's reactions to interruptions? What are the people's strategies and reactions for dealing with interruptions?

Fig. 1 .

 1 Fig. 1. The Disruption Management framework (DMF) identifies goal concepts, user activity, and task context: the three main components to mediate and filter disruption from implicit metrics.

Fig. 2 .

 2 Fig. 2. Disruption manager three-layer architecture: A low level layer includes implicit low granularity information (top). An intermediate layer includes the user activities and information to which some of the low granularity data can be extracted and summarized (middle-left). And knowledge layer includes the information or concepts related to the user goals (right).

Fig. 3 .

 3 Fig. 3. Disruption manager's Decision Logic Module: a layered filtering process determining the stages that each interruption must go through before being delivered to the user.

Fig. 4 .

 4 Fig. 4. Instant Message Response Ratio: Subjects responded to instant messages 5 times more when using Disruption Manager while dealing with quality and urgent goals.

  • Allow Instant Message notifications if relevant to current email /customer request (active email, document, or webpage). o Relevant presented almost immediately. o Wait until finished task or task switch. • Delay Instant Message notifications if relevant or moderately relevant to current email /customer request (active email, document, or webpage) o Wait until a task break. o Wait for a task switch only • Delay Instant Message notifications if not relevant to current email /customer request (active email, document, or webpage).

o Wait until email sent or task completion.

Princeton University "About WordNet." http://wordnet.princeton.edu

ConceptNet http://csc.media.mit.edu/conceptnet

Google Desktop: http://desktop.google.com/

PsTools http://www.microsoft.com/technet/sysinternals/utilities/pstools.mspx

Montylingua: http://web.media.mit.edu/~hugo/montylingua/

OpenMind: http://www.openmind.org

State-Trait Anxiety Inventory for Adults


Elba Del C Valderrama
email: elba.valderrama@acm.org

Jarmo Kauko
email: jarmo.kauko@nokia.com

Jonna Häkkilä
email: jonna.hakkila@nokia.com

Albrecht Schmidt
email: albrecht.schmidt@acm.org

In Class Adoption of Multimedia Mobile Phones by Gender -Results from a Field Study

Keywords: Mobile phones, children, technology adoption, rural schools, developing countries, novelty effect, learning, boys, girls

In this paper we share our findings from a field study conducted in Panama, focusing on adoption of mobile phones in classroom settings. Our initial findings reveal that during the initial phase of use, boys adopt mobile phone usage faster and explore more functionality; while girls take more time to familiarize themselves with the phones. Girls seem to maintain a better focus on the learning activities using the mobile phones across all tasks. When the task implies an active role then boys also showed high concentration. The videos recorded by the children as part of the learning activities showed a remarkable difference in roles between girls and boys. These findings suggest that it is important to consider the different adoption and exploration strategies of girls and boys with new technologies when designing tools for mobile learning.

Introduction

There is a great potential of using mobile phones for learning in both developed and developing countries. For most people living in developing countries, mobile phones are the only computing technology they know and have access to. This makes mobile phones a potential alternative for computer supported learning. Additionally, it is known that children are very fast in taking on new technologies, such as mobile phones. Nonetheless, using them for learning in formal classroom settings is still not common and little is known how to do this in a successful way. Most of the approaches found in the literature have focused on the development of dedicated learning games or applications to cover one specific learning subject or content. Understanding children's practices of learning to use new technology could support the creation of better learning applications and teaching practices.

We conducted an explorative field study to learn how pupils in developing regions take up the multimedia features of mobile phones in classroom settings to support their learning activities. Our study took place in Panama, a small country in Latin America with a population of 3.5 million [START_REF]Contraloría General de la República de Panamá[END_REF], and where approximately 40% of the population lives in poverty [START_REF]UN-Panama: Erradicar la Pobreza Extrema y el Hambre[END_REF]. The experiment was run in two schools in rural areas.

Although mobile phone assisted learning is an emerging field, there exists some earlier research in the area. The interaction between children and multimedia phones has been investigated in developed regions e.g. in [START_REF] Puikkonen | Playing, Performing, Reporting -A Case Study of Mobile Minimovies Composed by Teenage Girl[END_REF][START_REF] Jarkievich | In the Hands of Children: Exploring the Use of Mobile Phone Functionality in Casual Play Settings[END_REF], where Jarkevich et al. [START_REF] Jarkievich | In the Hands of Children: Exploring the Use of Mobile Phone Functionality in Casual Play Settings[END_REF] studied the adoption of mobile phones by small children in the kindergarten, and Puikkonen et al. [START_REF] Puikkonen | Playing, Performing, Reporting -A Case Study of Mobile Minimovies Composed by Teenage Girl[END_REF] analyzed short videos created by girls with their mobile phones. However, our research focuses on the use of multimedia phones in developing countries. Here, pioneering work has been done for example by Kumar et al. [START_REF] Kumar | An Exploratory Study of Unsupervised Mobile Learning in Rural India[END_REF], who studied the utilization of mobile phones in unsupervised learning environments in rural India; and Kam et al. [START_REF] Kam | Designing Digital Games for Rural Children: A Study of Traditional Village Games in India[END_REF], who explore the adoption of local game practices for learning games with mobile phones in schools, also in India. Hollow and Masperi [START_REF] Masperi | An evaluation of the use of ICT within primary education in Malawi[END_REF] explored the use of video learning tutorials for handheld devices in primary schools in Malawi. In contrast to this work, our research looks at the opportunities that arise from using general multimedia mobile phones as generic learning tools -much like paper and pencil. Qualitative findings of how teachers adopt the use of multimedia for their own teaching style; and needs and strengths of their pupils in a similar setting as used in the research presented in this paper have been reported in [START_REF] Valderrama Bahamóndez | Utilizing Capabilities for Mobile phones[END_REF].

Method

The starting point for our research reported in the paper was based on the results from a set of interviews, discussion and surveys to teachers and children realized previously in Panama [START_REF] Bahamóndez | A Survey to Assess the Potential of Mobile Phones as a Learning Platform for Panama[END_REF][START_REF] Valderrama Bahamóndez | Utilizing Capabilities for Mobile phones[END_REF]. The surveys and interviews were conducted to get an overview of the use of technology in schools, the acceptance of using mobile phones for learning, how teachers and children imagine mobile phones could be used for learning, and the learning needs that could be supported by mobile phones in Panama. The main findings revealed that teachers and pupils have a high access to mobile phones [START_REF] Bahamóndez | A Survey to Assess the Potential of Mobile Phones as a Learning Platform for Panama[END_REF], and they would welcome support especially for Math, Language arts and Nature [START_REF] Bahamóndez | A Survey to Assess the Potential of Mobile Phones as a Learning Platform for Panama[END_REF][START_REF] Valderrama Bahamóndez | Utilizing Capabilities for Mobile phones[END_REF]. It was desired that any learning application should include multimedia features. Teachers were open to use mobile phones in their classes, and they considered the school the most suitable place for using a mobile phones for learning [START_REF] Bahamóndez | A Survey to Assess the Potential of Mobile Phones as a Learning Platform for Panama[END_REF][START_REF] Valderrama Bahamóndez | Utilizing Capabilities for Mobile phones[END_REF]. Based on these findings we decided to conduct an explorative field study to understand how children and teachers adopt the use of the mobile phones in class and how phones could support learning in class in the context of rural areas in developing countries.

Setup of the Field of Study

The field study was realized in two schools in Cocle, a province located in the west of Panama City. The first study was conducted in a rural multigrade school in the village El Retiro. The participants attended 4 th , 5 th and 6 th grade; pupils from the 4 th and the 5 th grade worked together in the same classroom under the supervision of the same teacher (multigrade). We have described the learning activities and general findings from the field study realized in this school in [START_REF] Valderrama Bahamóndez | Utilizing Capabilities for Mobile phones[END_REF]. The second school was Maria de Tirones, an urban school located in the village of Rio Hato. There we worked with 2 classes of the 5 th grade. Although only one of the schools was classified as rural, most of the pupils in both schools came from families with low or very low income. In total, 78 children participated in the study, 36 from the first school and 42 from the second one. Additionally, we worked with 6 teachers, including the two respective grade-teachers and the English teacher. The gender distribution was 41 girls and 37 boys, and the mean age was 10.5 years. As the learning activities using the mobile phones were planned by teachers to target the specific grade and learning goals, some activities were analyzed from a subset of all participants. The study ran for two working-weeks in each school, between September and October 2010.

As we want to explore the use of multimedia in class and to avoid any bias if children used their own phones, we provided each child and teacher with a Nokia 5530 Music Xpress equipped with an SD card of 4 GB. With this model we were also able to explore touch screen interaction, as touch-screen phones are currently the trend in the market. In both schools we gave a brief introduction to the teachers showing the multimedia features and applications of the mobile phones. Teachers were free to decide which multimedia features and applications they used during their classes. The learning tasks were designed exclusively by teachers and respected the planned lessons scheduled previously by them for the school term.

Only three of the children had never used a mobile phone before, but none of them had prior experience with touch-screen interaction on mobile phones. Mobile phone features and applications were introduced to children when needed for a learning task. After the first week all the children knew well how to use the phone. The mobile phones were used only during the school; teachers delivered them to children when starting the class and collected them once the class was finished. On average, children used the phones between 2 and 4 hours daily. In addition to the instructed tasks designed by teachers, children had a chance to explore the mobile phones freely. The mobile phones were used only as portable computers, thus the ability to do phone calls was not available during the field study. To monitor the usage of the mobile phones, we installed a logger application in all the phones. The logger run on the background and recorded the applications that had been opened by the pupils. The logger application created a daily text file with the phone IMEI, timestamp and name of the application launched. In addition, the logger took a screenshot of the active application every 20 seconds. The first author of this paper was present during the field study doing observations and giving support in case of any technical problems.

Adoption of the Use of Multimedia Phones during the Learning Activities

One of the goals of this research was to explore gender differences in using multimedia features of the mobile phone in class. From our observations made during the field study, apparently, boys seemed to be more distracted than their female peers while working with an assignment on the phone. For example, they explored other applications during an instructed task. Also, our observations suggest that boys are faster to adopt the mobile phones than girls. As initial point, we analyzed two learning tasks: watching a video, and filming a video. The choice of these two tasks was based on the opportunity of analyzing different roles of the children: one as active role (filming video) and the other as passive role (observing a video).

Watching videos. Overall, there were six learning tasks involved watching videos, and the videos varied in terms of topic and duration. We restricted our population to two 5 th grades groups who observed the same Math video (introduction of the multiplication of fractions), to ensure comparability. The video has duration of 5.7 minutes and was watched by 21 girls and 18 boys. In our first analysis, see fig. 1 (left), we found a gender difference in how many times the pupils observed the same video. In general, boys watched the video 1.2 times less than their female peers. To confirm this difference statistically, we conducted 2x2 between-subjects ANOVA to analyze the net time used for watching the video. The analysis revealed a significant difference between the groups 1 and 2 (F 1 , 35 = 6.65, p < 0.05). However, the difference between genders was only marginally significant (F 1 , 35 = 3.77, p = 0.060). As illustrated by fig. 1 (left), group 2 observed the video less times than group 1, while maintaining the trend between girls and boys. illustrates that the number of distractions by boys (39%) is approximately double in comparison to their female peers (19%).This also explains the shorter time used by boys for watching the video. We notice also that in group 1 girls show no distractions, and the number of distractions by boys is considerably lower in group 1.

An explanation of the variation between the two groups showed in both graphs is that in the first group teacher used the video to introduce the concept (multiplication of fractions), while in the second group the topic was already introduced and the video was used to reinforce the learning. Another explanation is the learning effect. The group 1 observed the video during the first day of the field study and group 2 during the fourth day. This means that pupils in group 2 were already familiar with the mobile device. For both groups, however, this was the first time they were introduced to playing videos using the phone.

Filming videos. Although there were around 7 tasks involving filming video, most of them were realized in a team -every child had to film a video in one learning activity only. We analyzed the children's behavior during this task. In this activity, pupils had to go out to the school garden to film and describe a part of a flower. Children were free to choose the flower(s). This task was performed by children from the 5 th grade of the first school, including 8 girls and 4 boys. The task was conducted during the third day of the trial, but it was the first task involving video recording. In general, we noticed two major gender differences: (1) All the girls worked in pairs -one filming while other presenting and vice versa; in contrast all the boys worked alone and filmed their own videos; (2) boys had no technical problems when filming, while two girls had difficulties with the task.

Fig. 2. Screenshot of the video recorded by the children for Nature Science class

Boys maintained focused in the flower as the main subject (fig. 2 right), while girls were more concerned about acting in front of the camera (fig. 2: left and center). All the girls cut off the flowers and held it in the hand in front of the camera; in contrast all boys completed the tasks without cutting it. As the boys created their own videos, they did not appear on the film in person. Six girls presented in front of the camera (full view), while two only partially (hand and lower body). The zoom option was neither taught nor explored by the participants.

Learning/adoption during explorative use

In addition to instructed learning activities, children were allowed to explore other functions of the mobile phone. Teachers reserved specific time slots (between 10 and 15 minutes) for free exploration, but unscheduled explorative usage also occurred during and between the instructed learning activities. We analyzed the explorative use of the phones mainly from log files containing active application history data. Findings from the quantitative log data were then contrasted by qualitative observations.

Log File Analysis

We did not have any a priori hypotheses, as the original focus of the study was in the learning activities. Due to lack of control in the field trial method, the results are affected by unknown random and potentially confounding factors. Therefore, the following results should be interpreted merely as hypotheses for future experiments unless strongly supported by qualitative findings. To avoid bias in selecting metrics and comparisons, the log files were analyzed by a researcher who was not involved in the field trial.

Log files contained a time-stamped history of active applications. We analyzed the usage during the first 4 days, and excluded 3 users who participated less than 4 days. First, the applications were categorized in system applications (e.g., home screen, application menu), learning applications (e.g., Camera, Paint Pad) and explored applications. A subset of the last category was further classified as game applications. Second, a set of metrics were extracted from the log files. We analyzed the results using mixed-models ANOVA with gender and school as between-subjects factors and day as a within-subjects factor.

Time used for learning applications varied between the schools and days, but there were no systematic differences. Overall, the mean time used for learning applications was 60.8% (sd = 21.3%). For explored applications, there were significant main effects for gender (F 1,53 = 5.98, p < 0.05) and school (F 1,53 = 6.86, p < 0.05). In the first school, the mean exploration time by boys was 8.5% (sd = 13.4%) and by girls 3.6% (sd = 3.7%). In the second school, the corresponding results were 14.8 % (sd = 15.1%) for boys and 8.9% (sd = 8.9%) for girls. The remaining time was used in system applications. Idle time was excluded from the analysis using a threshold of 1 hour without an application switch.

By analyzing the explorative usage further, we found a significant gender difference for game applications (F 1,53 = 5.99, p < 0.05). There were also significant schoolday and gender-school-day interaction effects. A post-hoc analysis with Bonferroni correction revealed a significant day effect for the second school (F 3,132 = 10.91, uncorrected p < 0.000005). This reveals that time used for games increased day by day, as illustrated by Fig. 4. Overall, the time used for games was more significant for boys (µ = 4.1%, sd = 9.6%) than for girls (µ = 0.3%, sd = 0.9%). Similarly to these findings, during the field study we observed that girls played games notably less than their male peers. One girl commented to us "I find this game [car racing] very boring and difficult", while a boy stated "I like this game [car racing] … is easy but I have to be concentrated". The most played game was the Global Race-Raging Thunder ® where the player drives a car in a racing competition. Most of the boys played alone, but four boys discover that turning on Bluetooth allowed them to play against each other.

In order to analyze the scope of explored applications, we calculated the amount of new applications discovered daily by each user. System and learning applications were excluded from this analysis. We found significant main effects for gender F 1,53 = 12.28, p < 0.001 () and day (F 3,159 = 5.28, p < 0.005). The mean amount of discovered applications was higher for boys (µ = 3.5, sd = 3.0) than for girls (µ = 2.6, sd = 2.2). The discovery rate decreased moderately with time. Cumulative amounts of discovered applications are illustrated in Fig. 5 (left).

We also analyzed exploration activity by calculating the frequency of switching the active application per hour. This metric includes learning and system applications, such as the application menu. We found a significant main effect on activity by day (F 3,159 = 5.28, p < 0.005). Further analysis shows that there is an increasing trend in the activity during the first 3 days. Interestingly, the activity seems to decrease after the third day, as illustrated in Fig. 5 (right). Decreasing activity and slightly decreasing discovery rate may indicate that users found the most interesting applications during the first three days.

Discussion and Conclusion

Mobile phones have a great potential as learning tools. In the learning process children need to learn to use the basic functions and a variety of applications of the mobile phone to utilize the tool. Children are fast to adopt these skills, especially if they find the device interesting. Games and media applications can have an important role in maintaining interest and introducing new functions. On the other hand, such applications can distract the actual learning process.

A design challenge for future mobile learning devices is to balance pragmatic learning activities and free exploration. Our results indicate that there is a notable gender difference in how much children weight these activities. Boys are more active in free exploration and learning new applications, games in particular. Therefore, boys discover the basic functions of the phone faster. Girls are typically more focused on the learning task, but may be accidentally interrupted by problems in operating the device. One possible reason discouraging girls from exploration is that the existing games are more targeted towards boys.

The initial surveys and interviews [START_REF] Bahamóndez | A Survey to Assess the Potential of Mobile Phones as a Learning Platform for Panama[END_REF][START_REF] Valderrama Bahamóndez | Utilizing Capabilities for Mobile phones[END_REF], as well as the post-interviews after the field study indicate that teachers in Panama would welcome the use of technology in classroom. In particular, they welcome the use of mobile phones, as they are more commonly available and widely used than computers in Panama. Teachers with no prior experience of these technologies, react more skeptically to the use of technology for learning.

Because of the explorative and uncontrolled nature of our study, where each teacher designed their own learning activities, we received a variety of results and insights. As the learning tasks varied in content and form from grade to grade, the results are difficult to compare directly. The focus of this study was to capture the initial adoption of the mobile phones, a new technology for the children, thus the paper is limited to findings in this area. Nonetheless, we believe that our findings contribute in better understanding how children in unprivileged areas adopt the use of multimedia phones for learning with no preinstalled learning applications. Our future work will focus on longer term usage trends and gender differences in both in and out classroom settings. Also, we plan to conduct a controlled experiment to evaluate how using mobile phones actually affects the learning performance.

Finally, as touch-screen phones are a growing trend in the market, it is expected that in the near future the technology will become a commodity in low-end devices and reachable to people living in developing regions. This will enable the use of touch-screen technologies and widen the spectrum of potential mobile applications also for learning purposes.

Fig. 1 .

 1 Fig. 1. Number of repeated observations of the Math video by gender and by class group (left), and the percentage of interruptions in the video watching task (right). Group 1 and 2 represent the pupils of 5 th A and 5 th B respectively. Together refers to both groups.Fig. 1 (right) shows the number of distractions while watching the Math video by gender. We define distractions as the occasions when the pupils pause/stop playing the video and open another application on the mobile phone instead. Fig. 1 (right)illustrates that the number of distractions by boys (39%) is approximately double in comparison to their female peers (19%).This also explains the shorter time used by boys for watching the video. We notice also that in group 1 girls show no distractions, and the number of distractions by boys is considerably lower in group 1.An explanation of the variation between the two groups showed in both graphs is that in the first group teacher used the video to introduce the concept (multiplication of fractions), while in the second group the topic was already introduced and the video was used to reinforce the learning. Another explanation is the learning effect. The group 1 observed the video during the first day of the field study and group 2 during the fourth day. This means that pupils in group 2 were already familiar with the mobile device. For both groups, however, this was the first time they were introduced to playing videos using the phone.

Fig. 3 :

 3 Fig. 3: Mean percentage time used for game applications.

Fig. 5 .

 5 Fig. 5. Cumulative amount of discovered applications (left), and application switch frequency (right).


Celeste Lyn Paul
email: cpaul2@umbc.edu

Anita Komlodi
email: komlodi@umbc.edu

Wayne Lutters
email: lutters@umbc.edu

Again?!! The Emotional Experience of Social Notification Interruptions

Keywords: Emotion, interruption, One-Word-Response, methodology, notification, user experience

This paper describes a post-hoc analysis of the relationship between the socialness of an interruptive notification and the emotional tone of the words used to describe the experience through a One-Word-Response (OWR). Out of the 89 responses analyzed, 73% of participants used emotional words to describe their notification experiences. There was a significant relationship between the emotional tone of a OWR response and the socialness of an interruptive notification experience and participants were 3.2 more likely to describe social interruptive notifications with positive words than negative words.

Introduction

Notifications are intended to be a useful service provided by an information technology to help users maintain awareness of events and information while they focus on other tasks. In the case of desktop-based notifications, this information can be delivered in the form of a message and displayed on a screen in a variety of ways -either by itself or in combination with an auditory or haptic cue-in order to gain the user's attention. These services can notify the user about new emails, keep track of calendar events, update chat availability status, and inform of new incoming messages from friends. Typically these notifications interrupt the user while engaged in other tasks, for example word processing or image editing.

Different types of notifications have different types of interruptive effects on the user. Interruptions could be beneficial by delivering anticipated information or they could have a negative effect by creating a distraction. Distractions or other negative experiences could affect the user performance and satisfaction of the user's current activity or of the notification system itself. Creating an effective notification system requires design trade-offs that balance user awareness with user distraction.

Social communication services actively connect people together through email, chat, and social networking. These services are constantly updated -delivering new information to subscribers all the time. As participation in social communication services increases, the amount of new information a user must be aware of increases. As a result, the number of notifications increases and so do the interruptions from those notifications.

Previous work informs us when during an activity is best to interrupt users [START_REF] Czerwinski | A Diary Study of Task Switching and Interruptions[END_REF][START_REF] Miyata | Psychological Issues in Support of Multiple Activities[END_REF], how to best design software to manage this [START_REF] Cadiz | Designing and Deploying an Information Awareness Interface[END_REF][START_REF] Mankoff | Heuristic Evaluation of Ambient Displays[END_REF][START_REF] Matthews | A Toolkit for Managing User Attention in Peripheral Displays[END_REF], and how to interrupt users in specialized domains [START_REF] Avrahami | IM Waiting: Timing and Responsiveness in Semi-Synchronous Communication[END_REF][START_REF] Vastenburg | Considerate home notification systems: a field study of acceptability of notifications in the home[END_REF]. The literature does not help us understand for what reasons users should be interrupted. Understanding "why" depends highly on understanding the notification and the context and there is little research in this area. We set out to investigate these contextual factors through an exploratory survey. The study asked participants to report a recent interruption experience and their reaction to it. Among other questions they were asked to describe the experience in one word.

Emotion is an important aspect of the user experience and influences how users understand, interpret, experience, and interact with technology [START_REF] Boehner | How Emotion is Made and Measured[END_REF][START_REF] Forlizzi | Understanding Experience in Interactive Systems[END_REF]. However, emotion can be difficult to study because it is context dependent. There is no single method that is best for studying emotion and the best method is often determined by the scope of the study and type of data desired [START_REF] Isomursu | Experimental evaluation of five methods for collecting emotions in field settings with mobile applications[END_REF][START_REF] Lopatovska | Researching Emotion: Challenges and Solutions[END_REF]. There are a range of methods available for studying emotion that include: using pictogram surveys such as the Selfassessment Manikin (SAM) [START_REF] Lang | Behavioral treatment and bio-behavioral assessment: computer applications[END_REF] and Emocards [START_REF] Desmet | Designing products with added emotional value; development and application of an approach for research through design[END_REF], multimedia tools such as Experience Clip [START_REF] Tahti | A Proposal of collecting Emotions and Experiences[END_REF] and 3E [START_REF] Isomursu | Experience clip: method for user participation and evaluation of mobile concepts[END_REF] that allow participants to create media to express themselves, and physiological measurements such as EKG, heart rate, blood pressure, respiratory rate, and galvanic skin response [START_REF] Lopatovska | Researching Emotion: Challenges and Solutions[END_REF]. The One-Word-Response technique described in this study (see Methodology) is most similar to the participant selfassessment methods used to study emotion.

Methodology

The purpose of the study described in this paper was to explore the interruptive notification experience with a focus on understanding the context of the experience and to help identify areas for further study. An interruptive notification is a notification that displays in a way that actively draws the user's attention. Emotion was discovered as an emergent factor during analysis which inspired further post-hoc analysis. This paper presents the results of the post-hoc analysis of the relationship between the emotional tone of words elicited from a One-Word-Response question and the socialness of the interruptive notifications collected from the main study.

The main study collected 139 responses regarding a participant's recent notification interruption experience using Amazon Mechanical Turk (AMT). AMT was used as a way to gain access to a more representative population sample and an alternative to the common approach of using college students. Participants were recruited and paid through the AMT system. Participation in the study was limited to participants in the United States to help control for English language skills, and was limited to participants who had at least a 95% Human Intelligence Task (HIT) approval rating (the percentage of HITs correctly completed) in order to control for participants who are known to not follow instructions and provide low-quality responses.

The study instrument was a web-based form that asked participants to describe a recent interruptive notification experience through a combination of open-ended (text) and closed-ended (selection or scale) questions about the details of their experiences:

1 The main study was conducted in two parts. The All-Notifications part asked participants to respond based on any recent interruptive notification experience. The Social-Notifications part asked participants to respond based only a recent social-only interruptive notification experience. The study instruments were the same for both parts except that the instructions for the Social-Notifications part specified socialonly interruptive notification experiences. Each study part was conducted over three rounds of AMT studies until 20 responses that followed instructions were collected. Separating the studies into multiple rounds is best for AMT studies because newer studies requiring fewer responses have better response rates than older studies requiring many responses. Table 1 describes how the study rounds were conducted.

One-Word-Response One-Word-Response (OWR) is a word association technique that is a short, direct question that requests the participant to respond with a single word. Word association is an elicitation technique that aims to get an immediate reaction rather than a thought-out response. OWR differs from other survey question types in that it is a simple question with a simple response and requires no deliberation to respond to. This question type is common, but we formalize the technique in this study. Although the study instructions asked participants to provide a "description", to our surprise, many of them gave emotional words as responses. We report the analysis of these emotional responses in this paper. Word association techniques can be susceptible to priming [START_REF] Hines | Automatic Semantic Priming: Effect of Category Exemplar Level and Word Association Level[END_REF], a memory effect due to a previous influence or exposure, and so the responses must be considered within the context of other influences in the study. The OWR in this study was worded as: "Using one word, how would you describe the notification?" Data Quality Responses were reviewed to ensure the collected data was about a recent notification experience and not an experience similar to but not the same as interacting a notification. Each response was evaluated based on two rules: First, if the notification was part of the main task and required a response before the user could continue this main task (For example, a browser security alert requiring the user to accept a cookie before continuing); and second, if the response was a web-based pop-up (For example an advertisement). Responses that met a rule were discarded since participants did not follow instructions. Of the 139 responses collected, 17 responses met one of these rules and were discarded for a total of 122 valid responses collected.

Data Coding

Coding rules for notification socialness were developed ad-hoc as part of the study design while OWR emotion coding rules were developed post-hoc in response to the need for additional analysis.

Notification Socialness. Participants were asked to describe a recent notification experience including what they were doing at the time of the notification and details of the notification. These details contained information about the application or service that sent the notification and what the notification was about, e.g. "An incoming message from a friend on Facebook" and "A notification that I needed to update my anti-virus software". From these descriptions, responses were coded as either social or non-social. If the notification came from a social application or service it was coded as a social notification. If the notification was not a social application or service it was coded as non-social notification.

OWR Emotion. OWR responses were coded based on an emotional dictionary that defined the positive or negative tone of the emotion [START_REF] Hein | Feeling Words List[END_REF]. OWRs were coded either positive for words with a positive emotional tone, negative for words with a negative emotional tone, or descriptive for words that had no emotional tone and simply described the experience. Words that had an unclear emotional tone were further investigated by evaluating the context of the open-ended responses. For example, two cases of "reminder" were clearly descriptive, while one case of "remindful" was clearly a positive word when the context of the open-ended responses considered.

Even though the OWR question asked participants for a "description" of their experiences, most responses contained emotion. Of the 122 participant responses, 89 words (73%) had a positive or negative emotional tone with 58 positive words (24 unique) and 31 negative words (13 unique). "Annoying" was the most popular negative word (n=15, 48% of all negative words). There was no single popular positive word, with the top five positive words as "informative" (n=9, 15% of all positive words), "useful" (n=7, 12%), "helpful" (n=6, 10%), "important" (n=6, 10%), and "exciting" (n=4, 7%). Table 2 lists the positive and negative OWR responses. The prevalence of emotion suggests that the interruptive notification experience had a significant emotional effect on participants such that their reactions to the interruptive notifications were emotional. Perhaps participants found it easier to draw on an emotional word to describe their experience, which supports design literature that includes emotion as part of the interaction experience.

A Chi-square Test of Independence showed a significant relationship between the OWR emotional tone (positive/negative) and socialness of the interruptive notification (social/non-social) (6.376, df=1, p=.012). A Phi coefficient showed a significant weak relationship between emotional tone and socialness (r=.268, p=.012). A theta odds ratio showed that social interruptive notification experiences were 3.2 times more likely to be described with a positive OWR word than a non-social interruptive notification. Table 3 provides a summary of statistical analysis.

Social interruptive notifications.

Notifications from Facebook were the most frequently reported source of social interruptive notifications (n=31). These included interruptive notifications from all features of Facebook including chat, new mail messages, and other notices. Email (Gmail, Inbox.com, Outlook,

• Useful (7) • Helpful (6) • Important (6) • Exciting (4) • Cool (3) • Convenient (2) • Effective (2) • Efficient (2) • Interesting (2) • Pleasant (2) • Discreet (1) • Easy (1) • Fast (1) • Fun (1) • Good (1) • Great (1) • Happy (1) • Humorous (1) • Necessary (1) • Polite (1) • Remindful (1) • Simple (1) • Thankful (1) • Annoying (15) • Unimportant (3) • Boring (2) • Irritating (2) • Again?!! (1) • Bad timing (1) • Bothered (1) • Distracting (1) • Forgettable (1) • Pestering (1) • Time waste (1) • Untimely (1) • Useless (1)
Thunderbird, Yahoo!, but does not include Facebook messages) was the second most reported source of social interruptive notifications (n=18), followed by chat (AOL Instant Messenger and MSN Messenger, not including Facebook chat, n=5). More positive words used to describe social interruptive notifications than negative words.

Of the 45 social interruptive notification experiences, 35 were described using a positive emotional word compared to 10 that were described using a negative emotional word. Many participants expressed the social benefits of social interruptive notifications, as one participant stated, "I like knowing when someone quotes me [on a website]" (OWR "informative"). Another participant described a social obligation, "I didn't want to make my friend wait; [I responded immediately because] it was polite" (OWR "humorous"). These results suggest that social interruptive notifications are likely to be a positive experience.

Non-social interruptive notifications. Software and security updates were the most common non-social interruptive notifications. Operating system updates from Windows (versions XP, Vista, and 7) and Mac OS X as the most frequently reported source of non-social interruptive notifications (n=18). Security software (Avast, AVG, Immunet, Kaspersky, McAfee, Microsoft Security, Norton) were a close second (n=15), followed by various software-related update services (Adobe n=7, HP n=4, Java n=4). However, there was no difference in the ratio of positive or negative words used to describe non-social interruptive notifications. Of the 44 non-social interruptive notification experiences described, 23 were described with positive words and 21 were described with negative words.

A further look into the experience of non-social interruptive notifications revealed mixed feelings about receiving notifications about software updates and security services. Some participants did not mind non-social notifications because, "It is always good to know your virus protection is working," (OWR "great"). Other participants expressed dislike because, "They interfere with what I was currently doing," (OWR "irritating"). Other factors may influence why one interruptive notification can be a better experience than another. For example, frequency of nonsocial interruptive notifications may be an important contextual factor. As one participant explained, "It's important to know that everything is working okay, but maybe not everyday," (OWR "boring"). Another participant stated, "The less often I get [notifications], the more likely I am to listen to them," (OWR "annoying"). This suggests two things about non-social interruptive notifications. First, that non-social interruptive notifications may be an emotional experience but not an overwhelmingly positive or negative one. Second, that non-social interruptive notifications may have additional factors that influence the overall experience; that is to say, context matters more for understanding the non-social interruptive notification experience.

Study Limitations

The purpose of the main study described in the Methodology was to collect information about recent experiences of interruptive notifications. The nature of our methodology may have resulted in participants to recall rather than to report their experiences. A study design with a different methodology could better support reporting and result in different OWR responses or emotional content. As previously stated in the Methodology, emotion was not a planned study factor. Due to our main study methodology and the post-hoc nature of our analysis, we were unable to capture a baseline of participants' emotional state before and after the interruptive notification experience. This type of comparison is common in emotional design research.

Finally, while the OWR and other study quests were descriptive in the way they were presented to participants, there was possible evidence of word priming. Some of the qualities participants were asked to rate in the main study appeared in the OWR responses as emotional words (e.g. useful and interesting). However, priming is not always s a negative effect and could be used as a methodological strategy. For example, in order to control context of responses, participants could be primed in a way to influence the scope of how they respond.

Conclusions

We found a significant relationship (Chi-square 6.376, df=1, p=.012) between OWR emotional tone (positive/negative) and the socialness of an interruptive notification (social/non-social). Although the overall strength of the relationship was weak (Phi r=.268, p=.012), a positive word was more likely (theta odds ratio, 3.2) to be used to describe a social interruptive notification experience than a negative word. Social interruptive notifications stand out as particularly interesting results since many users' computing experiences are shifting from a work-related to more a social environment. Our results are important for two reasons. First, our results identify emotion as an important factor in the interruptive notification experience. Second, our results reinforces existing literature on emotion as an important factor in interaction design.

While socialness was planned as a factor to further investigate, the results presented in this paper has also lead us to include emotion as a factor to examine in any future work. We plan to continue investigating the contextual factors of the interruptive notification experience. A future field study will follow a small group of users who know and interact with each other over social communication tools. The primary research method will be a software-supported diary study with secondary methods to log user and system behavior. The diary study is unique in that the software will help negotiate common challenges found in diary study methodology such as consistency in event sampling, consistency in participant data entry, and consistency in participation over time.

Table 1 :

 1 Description of study rounds for data collection

		Round 1	Round 2	Round 3	Total
	All-Notifications	20	20	20	60
	Social-Notifications	32	20	27	79
			Total collected responses	139
			Total valid responses	122
		Total OWR emotional tone responses

Table 2 :

 2 Positive and negative OWR responses with frequency

		Positive Words	Negative Words
	•	Informative (9)

Table 3 :

 3 Statistical analysis of OWR emotion x notification socialness

	Notification		OWR Emotion		Chi-square	Phi
	Socialness	Positive Negative	Total	Value (df)	Sig.	Value	Sig.
	Social	35	10	45
	Non-social	23	21	44	6.376 (1)	.012	0.27	.012
	Total	58	31	89