Sharon Lynn Chu

Francis Quek
email: quek@vt.edu

Information Holodeck: Thinking in Technology Ecologies

Keywords: Ecology, technology, devices, displays, thinking, sensemaking, objectification, embodied interaction, design

Information can be persistently represented on a multitude of devices beyond a single screen and session. This paper explores how technological display and device ecosystems (DDEs) may support human thinking, learning and sensemaking. We propose a theoretical foundation that extends Vygotsky's sign mediation triangle to include digital information. Through a process we call objectification, perceivable objects, e.g. interface objects, tangible technologies, can be associated with signs to support thinking. We present a qualitative study of learning in a testbed DDE with 12 graduate students. We developed a method that traces digital objects within 'thinking episodes' to help us evaluate how technology configurations support objectification. Our findings relate two storylines of how DDE technologies may afford objectification. Our work advances a method informed by psychological theory to examine device ecologies for their potential for learning, and articulates affordances for the design of technology that can help to support higher thought.

Introduction

In the TV Sci-Fi series, Star Trek Voyager, there is an Emergency Medical Hologram Mk 1 (EMH) doctor played by Robert Picardo. Although the EMH was designed to simulate a personality, at the beginning of the series he/it was just a program restricted to the Sick Bay (which had 'Holo-emitters' to support his materialization), and could take physical form for periods no longer than necessary to see an emergency patient. But because Voyager was stranded far from home without medical support, the technical staff 'stabilized his matrix', so he could remain persistently corporeal. Over time, they extended the technology coverage to enable him to materialize in larger portions of the spaceship, and acquired a 'mobile holo-emitter' that enabled him to be material continuously. He became 'real'. This technology tale is relevant to our deployment and use of technology because just like the doctor, information with which we interact is at the same time digital bits and materialized representations on screens and devices. Just as an ecosystem of technological enablements support the materialization of the doctor, a multitude of devices are increasingly able to support persistent representations of information materiali-zations beyond the restrictions of a single screen and a single session. The question is how may our information become 'real' and why this may be important for learning.

We have a good understanding of how technology augments our human physical skills (e.g., digging), and our cognitive skills (e.g., memory) to a certain extent. It has proven more difficult for us to understand how technology can augment the higher human thought. Investigations on information, communication, artifact and media ecologies, have addressed mostly philosophical, behavioral, design and technical perspectives of these new environments. This paper explores how these technological display and device ecosystems (DDEs) in the physical world may function to support human thinking, learning and sensemaking.

First, we review and synthesize the literature on technology ecologies, and then present a model to make sense of the process of thinking in DDEs derived from the theories of the Russian psychologist, Lev Vygotsky. We describe a study that uses our theoretical framework to investigate how students think in the DDEs that they form, and conclude the paper by discussing technology affordances and their implications for the design of DDEs for learning.

A Review of Technology Ecosystems

A range of research has hitherto applied the metaphor of a biological ecosystem to human activities with technology for illustrative purposes and to stimulate intellectual discussions. However, this body of work does not always form a coherent whole, and it is a challenging undertaking to present a comprehensive synthesized account. We reviewed many technology ecology notions in the literature, but in the interest of space, we shall simply list them and expand only on the most relevant ones here.

The overall message underlying the different positions in the literature is that artifacts, devices, systems and products cannot be studied in isolation but can only be truly understood when seen in the broader perspective of the universe they inhabit. Depending on the position taken, the universe can consist of one's physical context, other artifacts used, or one's practices and culture using technology. We classify the perspectives into three categories: philosophical positions, empirical study results and technical frameworks.

Among the theoretical or philosophical positions taken on technology ecology concepts, one can find the 'media ecology' by McLuhan [START_REF] Strate | Studying Media As Media: McLuhan and the Media Ecology Approach[END_REF], Altheide's [START_REF] Altheide | An Ecology of Communication: Toward a Mapping of the Effective Environment[END_REF] 'ecology of communication' or 'communicative ecology', the 'information ecology' by Nardi & O'Day [START_REF] Nardi | Information Ecologies: Using Technology with Heart[END_REF], Tungare et al.'s [START_REF] Tungare | Personal Information Ecosystems and Implications for Design[END_REF] 'personal information ecosystem', Krippendorf's [START_REF] Krippendorff | The Semantic Turn: A New Foundation for Design[END_REF] 'ecology of artifacts', and Rick's [START_REF] Rick | Towards a classroom ecology of devices: Interfaces for collaborative scripts[END_REF] proposition of a 'classroom ecology of devices'. Gibson [START_REF] Gibson | The ecological approach to visual perception[END_REF], Suchman [START_REF] Suchman | Human-Machine Reconfigurations: Plans and Situated Actions[END_REF] and Norman [START_REF] Norman | The Psychology of Everyday Things[END_REF] have also used the ecological metaphor.

Research of technology ecologies that present an empirical study of some sort include Huang, Mynatt & Trimble's [START_REF] Huang | Displays in the Wild: Understanding the Dynamics and Evolution of a Display Ecology[END_REF] 'display ecology', Enquist, Tollmar & Corry's [START_REF] Enquist | Interaction Ecologies[END_REF] 'interaction ecology', Dearman & Pierce's [START_REF] Dearman | It's on my other Comp-uter![END_REF] 'computing with multiple devices', Forlizzi's [START_REF] Forlizzi | The Product Ecology: Understandng Social Product Use and Supporting Design Culture[END_REF] 'product ecology', Jung et al.'s [START_REF] Jung | Toward a Framework for Ecologies of Artifacts: How are Digital Artifacts Interconnected within a Personal LIfe?[END_REF] 'personal ecology of interactive artifacts', and Coughlan et al.'s [START_REF] Coughlan | The Conceptual Framing, Design and Evaluation of Device Ecologies for Collaborative Activities[END_REF] 'device ecology'. From the management sciences, Bailey & Barley [START_REF] Bailey | Teaching-Learning Ecologies: Mapping the Environment to Structure Through Action[END_REF] present an extensive 'shadowing' study of 'teaching-learning ecologies', tracking knowledge as it moves through six engineering firms. Their focus however was on people and not technology. Some of the technical frameworks that have been proposed to implement technological ecologies are Loke & Ling's [START_REF] Loke | Analyzing Observable Behaviours of Device Ecology Workflows[END_REF] use of petri nets to represent the state of devices, the 'task migration framework' by Pyla et al. [START_REF] Pyla | Continuous User Interfaces for Seamless Task Migration[END_REF], and Pierce & Nichols' [START_REF] Pierce | An Infrastructure for Extending Applications' User Experiences Across Multiple Personal Devices[END_REF] framework based on instant messaging to enable multi-device user experiences.

Among the different conceptions of ecologies that we reviewed, only Coughlan et al. [START_REF] Coughlan | The Conceptual Framing, Design and Evaluation of Device Ecologies for Collaborative Activities[END_REF] and Jung et al. [START_REF] Jung | Toward a Framework for Ecologies of Artifacts: How are Digital Artifacts Interconnected within a Personal LIfe?[END_REF] presented formal empirical studies about technology use in a learning context. Rick [START_REF] Rick | Towards a classroom ecology of devices: Interfaces for collaborative scripts[END_REF] points out the importance of a classroom ecology, but does not provide any supporting study. Coughlan et al.'s investigation informs the design of ecologies by studying transitions in foci across devices (a tabletop computer with a mirrored projection, laptops, a telephone) in three short controlled activities, carried out in a "technologically-enhanced indoor space". Communication across devices was provided by a Central Management System, and instant messaging. The focus of their study was on how device ecologies can support collaboration. Study results presented a set of "seams" that represent disconnects in a device ecology that can affect users' behaviors. Their study however gave little indication of how one can understand whether or how learning has occurred within the context they constructed.

Jung et al. [START_REF] Jung | Toward a Framework for Ecologies of Artifacts: How are Digital Artifacts Interconnected within a Personal LIfe?[END_REF] studied one's network of personal artifacts through the lens of 'factors' and 'layers' within a 'personal ecology of interactive artifacts', described as a "set of all physical artifacts with some level of interactivity enabled by digital technology that a person owns, has access to, and uses". They make use of two methods called the Personal Inventory, based on a simplified version of the Repertory Grid Technique, and the Ecology Map, which consists of sketching using sticky notes to probe about a person's device ecology. Their exploratory study with ten graduate students found that perceived attributes of an artifact can be classified into two categories, designed properties (physical, functional, informational, interactive aspects) and subjective values (experiential, emotional, social). They further specify the different types of relations that artifacts in a personal ecology can have, based on: purpose of use, context of use, or subjective meaning. Their study results, although very helpful to understand the nature and types of technological ecologies, again do not consider the process of learning.

We conceive of a technology ecology where devices function, not as individual gadgets, but in ecosystems to deliver an experience. As such, we define a DDE as a mesh of interacting displays and devices that enable the manifestation and manipulation of digital information to deliver a cohesive learning experience [START_REF] Yew | The Physicality of Technological Devices in Education: Building a digital experience for learning[END_REF].

A Model of Thinking in DDEs

In the context of our definition of a DDE above, we present a high-level model of how thinking can be understood with regards to interactions with digital information through physical technological devices. Figure 1 Externalization of TOs as MOs allows the user's perceptual and spatial abilities to participate in the thinking process. This idea is related to that of 'distributed cognition' [START_REF] Hollan | Distributed Cognition: Toward a New Foundation for HCI Research[END_REF], which states that human knowledge and cognitive processes are offloaded into the environment as external representations. Our contribution is that we suggest the mechanism by which cognition is distributed into the environment through the DDE.

In the next section, we make use of Vygotsky's sign mediation theory to describe a mechanism by which the different processes in our high-level model take place. Other perhaps more commonly known theories that have been derived from or are closely associated with Vygotsky's theory include, for instance, activity theory and distributed cognition.

The Sign Mediation Triangle and Materialized Thought

We turn to Vygotsky's theories to understand thinking in DDEs because he proposes a way by which things in the environment may be brought into the very process of thinking. According to sign mediation theory, language is conceived of as a psychological tool by which both cultural (interpersonal) and psychological (intrapersonal) thought are 'mediated' [START_REF] Vygotsky | Mind in society: The development of higher psychological processes[END_REF][START_REF] Vygotsky | Thought and Language[END_REF]. Signs are selfgenerated linguistic stimuli [START_REF] Vygotsky | Mind in society: The development of higher psychological processes[END_REF] that extend the operation of human cognition beyond the confines of the strictly biological system. Figure 2 illustrates Vygotsky's sign mediation theory. For example, a student in algebra may be introduced to the summation concept, the TO: ! + ! + ! …. She understands and is able to perform the operation. However, if she had to think of details of the concept each time she applies it, the limits of her memory, attention, and mental processing would make further advancement untenable. Thus, she encodes this concept as a mental 'sign' -the concept of 'summation'. She is able to think of the operation simply as ∑, and to employ this in further learning (e.g.

! ! -1 ! !!!
). As the sign becomes 'internalized' it becomes in essence the object in her thinking. She can 'unpack' the sign as needed to attend to the details.

In Vygotsky's model, signs may take the form of both internal or external symbols (a stick between a child's legs becomes his horse, and a block represents an idea [START_REF] Vygotsky | Mind in society: The development of higher psychological processes[END_REF]), or as an abstract entity grounded in language. Externally instantiated signs are referred to as 'material carriers' (MC) of thought [START_REF] Mcneill | Gesture and Thought[END_REF] (see Figure 2). Any perceiva- ble object (spatial location, gestures, objects or even sounds) in the environment can opportunistically and temporally be appropriated for use as MCs to assist thinking by bringing spatial ability and perception into play. In theory, the MC can be anything that may or may not resemble the mental object. In our example, the MC for convolution can be a specific hand gesture or a written * symbol.

In the early 1900s, Vygotsky obviously never encountered the magic of computation and modern display technology. The number of MCs one can entertain at any one time is limited by the meanings one can assign and recall for amorphous objects and space. We advance a theoretical framework, in Figure 3, that extends the model of the sign and MC, to include the 'magic' of digitality. The top of Figure 3 from thought object (TO) to mental sign and material carrier (MC) replicates Vygotsky's mediation theory. In the information world, a TO can be encoded in a digital object (DO). The DO can at times even extend the TO, such as with information about how the convolution formula was derived. The mental sign, as well, thus becomes associated with the DO. Through DDE technologies, the DO is expressed as a MO, e.g. as a website displayed in a browser on a mobile device. If one mentally appropriates this MO in the process of thinking, in essence shedding 'intentional regard' [START_REF] Dourish | Intentionality[END_REF] to the object, the MO becomes synonymous with the MC, i.e. the binding process of objectification.

Our model expands the power of MOs to support thinking through MCs in two ways. First, MOs are iconic and provide mnemonic reference in ways that arbitrary objects and space cannot. This potentially expands the number of MCs one can employ over longer periods of time. Second, digital media that are encodings of one's thought objects (and hence associated with one's mediating signs) can serve as an external long-term detailed representation of a piece of knowledge, thus extending the depth of thinking one can handle. Our model describes a thinking process that is different from simply opening a document on a screen to refresh one's memory about an idea because then the document simply becomes something one queries for information rather than wields in the process of thinking. With multiple users, the MO, being the perceivable component of the model, becomes (in co-located situations) or may be made (in remote scenarios) the shared object among all users (Figure 4). We envision users taking advantage of the

4

The Study

We conducted a qualitative study of the manifestation of the thinking process in DDEs. Adopting the hermeneutic perspective, the inquiry was to seek "understanding rather than explanation" and to uncover systems of meaning from the participant's vantage point [START_REF] Patterson | Collecting and Analyzing Qualitative Data: Hermeneutic Principles, Methods, and Case Examples[END_REF]. We follow the methodology specified by Patterson & Williams [START_REF] Patterson | Collecting and Analyzing Qualitative Data: Hermeneutic Principles, Methods, and Case Examples[END_REF] who advance that a hermeneutic approach begins with establishing a point-ofview (our 'objectification framework'), from which an 'organizing system' is derived to meaningfully organize, interpret and present data [START_REF] Tesch | Qualitative Research: Analysis Types and Software Tools[END_REF].

Study Methodology

The study enabled a set of students to experience a DDE for the purpose of doing a knowledge discovery assignment over a two-month period, and capturing their experience through self-reports and interviews. The participants were 12 computer science students in a graduate class. The assignment, to be completed in teams of three, was to research and write a report about the emergent field of 'Physical Computing'.

The students were each given a 'testbed suite' of devices comprising an iPod Touch, an iPad, and a 27" iMac to use as their own throughout a semester. The iMac, embedded in a custom casing that allowed it to be laid on the table horizontally or at an angle, was endowed with a touch overlay to enable touch interaction (see Figure 5). The rationale for the dissemination of devices of various form factors was that we wanted to provide the students with an experience of a heterogeneous DDE. Additionally, to provide a basic information architecture that crosses devices in the ecology, we installed the free file sharing service Dropbox, the notetaking service, Evernote, with a paid subscription, and the paid PDF reader GoodReader 1 on the iPad. The first two services are based on the cloud, and GoodReader can be paired with Dropbox to allow data synchronization and transfer. We provided the students with a tutorial session on how to use the three services/applications prior to the study.

The class was held in our research center, which contains several large display screens spread out in different meeting rooms, and a large interactive vision-based 1 www.dropbox.com, www.evernote.com http://www.goodreader.net/goodreader.html

Figure 5 Suite of devices available to study participants

touch screen prototype [START_REF] Verdie | Mirrortrack: tracking with reflection -comparison with top-down approach[END_REF]. All students had constant access to the building. They were asked to use technology as much as possible while doing the assignment, including the device suite given to them, the large display screens around the center, and any other devices that they own such as laptops and other desktops. They could also use the devices freely for any other purposes.

Data Collection

We conducted semi-structured interviews and surveys to capture the students' experiences with technology before they were given our suite of devices (pre-experience), and after they have used them for the semester (post-experience). Each interview lasted for about an hour and was audio recorded. The purpose of doing a pre-and post-interview was not to perform an experimental comparison, but rather, to be cognizant of the initial conditions under which our participants joined the study. The pre-experience interviews and surveys collected data about the students' behaviors and 'strategies of use' of the devices, as well as their attitudes toward and perceptions of the devices and processes. More specifically, the survey, which was completed in the presence of the interviewer, asked about the list of devices used and owned, the duration and frequency of use, familiarity with devices, purpose of use, perceived usability of devices (measured on a 7-point likert scale by adapting the IBM usability questionnaire [START_REF] Lewis | IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use[END_REF] with dimensions like ease-of-use, comfort, efficiency, satisfaction), and data sharing methods. Qualitative comments were also encouraged during the completion of the survey. The interview addressed similar themes as the survey, and added questions about the role of devices in their idea generation and paper writing processes, and device interactions in their existing ecologies.

The post-experience interviews asked about the process of assignment completion; impact of the devices on practice; problems with devices, information sharing, and the writing process; context/situations of devices use; influencing factors of information use; personal sensemaking; work distribution and team coordination; use of file/data sharing services; longer term device use; general assessment of the ecology; desired changes; meeting contexts; idea generation process; and, information presentation.

Data Analysis

All pre-and post-interviews were fully transcribed with timestamps. Survey data were imported into a spreadsheet for analysis. To separate relevant from irrelevant verbiage in the transcripts, two coders did a first round of analysis of all the transcripts to identify 'thinking episodes'. Following Barker's ([30] in [START_REF] Bailey | Teaching-Learning Ecologies: Mapping the Environment to Structure Through Action[END_REF]) concept of "behavioral episodes", we define a thinking episode as a "coherent run of [thinking] in which the constituent [processes] have a constant direction, a purpose". Some examples include 'filtering out important keywords', 'searching for information', and 'annotating papers'. We acknowledge that thinking is a process with no explicit end point or necessary resulting outcome. However, the method of tracking thinking episodes allowed us to identify possible situations when the students engaged in thinking, and was thus presented with opportunities for learning. It is not our interest in this paper to evaluate the uptake of knowledge or retention per se. Moreover, in this paper, we understand the commonly used term of sensemaking as being a necessary part of thinking.

In order to make sense of our data on the students' experiences of their ecologies, we engaged in what we call a 'DO (Digital Object)-path analysis'. The steps involved in our analysis, illustrated in Figure 6 (Left) together with an example analysis (Right), were as follows: each thinking episode was analyzed to distinguish the different components of TOs, DOs, MOs, and technologies used. We noticed that certain objects had a high frequency of occurrence in our collated list. To focus our analysis thus, we decided to trace the use of three of the common DOs: GoogleDocs documents, Evernote notes, and PDF documents. We shall call these our three focus objects for our study. All thinking episodes where any of the three DOs were manifested as MOs were copied to a separate spreadsheet for analysis. We then analyzed the different technology configurations in which the DO in question was used in each of the relevant thinking episodes.

By tracing the model components in context of use, we determined whether there were indications that objectification had occurred in any form for the particular task at hand. Two separate coders identified objectification indications by making a judgment as to whether the MO (e.g. Evernote note displayed on a tablet screen) brought spatial ability and perception into the process of thinking. Subsequently, a feature extraction was done, whereby we identified the characteristics of the technologies, or technology configurations, that seemed to have supported or hindered the objectification process. The characteristics were further categorized along uncovered themes.

Findings

To frame our findings, we will first describe the starting context of our participants from results of the pre-interviews/surveys, with an emphasis on processes of thinking with devices. This also conveys an understanding of the work strategies of the participants in the testbed DDE of the study, since practices, as we anticipated, did not change significantly over the two months. We then describe case scenarios from our DO-path analysis, before expounding on derived technology affordances for DDEs.

Initial Experience of DDEs

It was evident that some of the students already had a rather extensive ecology of devices prior to the study. The most common devices that they possessed were laptops, desktops and smartphones. Fewer owned tablets, music players and large displays (Figure 7). Among those who owned them, laptops and tablets were used daily. Desktops were mostly used only two to three times per week. All, apart from two, indicated that they used their smartphones everyday. The use of music players and large displays was more sporadic. Laptops and smartphones were mostly ranked first in familiarity, followed by tablets and music players. Students were generally least familiar with desktops and large displays. Their use of the devices can be classified into three types: instantaneous use, e.g. checking email, calendar, news, social networks (tablet, smartphone); extended use, e.g. reading papers, video streaming (laptop, tablet, large display, music player); and long-term use, e.g. backup of work, managing Internet (desktop, music player).

All except one had experience with Dropbox prior to the study, with an average duration of use of a year. Only three of the students had used Evernote, with an average of two months of use. All of the others however made use of other notetaking applications such as GoogleDocs and TextMate. Among those who already had a tablet, only one of them made use of GoodReader. The other students used other PDF readers/annotators such as Adobe Reader and QuickOffice. In terms of usability, the laptop (M = 2.57) was ranked the highest, followed by the tablet (M = 2.41), the desktop (M = 2.39), the smartphone (M = 1.92), the music player (M = 1.88), and finally the large display (M = 1.85).

In the existing DDEs of the students, the thinking process for assignments mostly followed the standard loops of foraging and sensemaking [START_REF] Card | Piroli Sensemaking Processes of Intelligence Analysts and Possible Leverage Points as Identified Through Cognitive Task Analysis[END_REF], with the prominent use of the laptop and desktop throughout. Actions in which a thinking process was evident on these devices included for instance, "just open[ing] up Microsoft Word and start typing in ideas", taking notes in a text file on Dropbox "when I'm surfing and I find something interesting", "categorizing my papers through…folders", and "start[ing] to kind of do a treelike structure from the cited references".

Those who had a tablet reported making use of it only occasionally for purposes of work. We could not identify many thinking episodes in their accounts of use of tablets. Among the few that we found such as for annotating readings, reading information in the browser or notetaking in the native 'Notes' application, the annotations were only "like an intermediary step to a final annotation", the website was "just [to The use of non-digital materials, such as sticky notes, notebooks and pieces of paper, was evident mostly in the 'foraging' part of the process. Paper materials appeared to have been used only as quick, temporary MCs and rarely had their content transferred to digital in the same exact form. One participant, for instance, recounted that she would write down 'idea fragments' on paper, and then "when I finally get my idea, I would put it on TextEdit", after which she would throw away the paper pieces. Or for reading papers, another participant "tend to print them out", "take some notes on it. And then turn it into some thrash". A number of the participants also reported relying only on memorization, "I remember some keywords of it, so I can get it when I want to", without the use of any devices.

Objectification as Manifested in Testbed DDEs

Recall that our analysis traces three kinds of 'focus objects' (PDFs, Evernote notes, & GoogleDocs) used by our participants as DOs. Our DO-path analysis of the postinterviews brought to light different ways in which the objectification process occurred in the DDEs that the students experienced. We follow one of our three 'focus' objects as it was involved in two example storylines extracted from thinking episodes.

Storyline 1: Identifying relevant sources for the assignment. One team employed a strategy by which they brainstormed a set of keywords and organized these on a whiteboard as a concept map, after attempting to use the vision-based interactive touch screen and failing. From this, they selected a subset of keywords to explore, and used these for literature searches. One of our participants related the following process, which we map to our model:

1. He searched for sources using the keywords and left these in tabs in his laptop browser. These were not saved to disk nor named. The browser tabs served as a 'bag of finds' whose relevance was yet to be determined ("I didn't save them on my computer. They were opened in my browser actually, because I wanted to filter what I have and see if I need to take these or not.").

2. He brought his bag of finds on his laptop to the next meeting and presented them to the team, and selected some to take ("I just renamed the paper to keep track of which is which. If I saw robotics … so this is the robotics paper. If there is a lot of robotics papers, I will say robotics 1, 2, 3, and or I would tie it to the name of the author."). It is at this point that the participant associates particular PDFs with signs. Relating this to Figure 3, the 'concept of robotics paper as relevant to physical computing' was the thought object, the sign was the name of the file, and the manifest object was either the displayed PDF document or the named file icon in the folder.

3. He would copy the PDFs to his iPad for further reading, and refined the naming of the PDF according to his understanding of the paper ("So after you read the paper … OK … this paper is about this. A way I used to do is to just rename the paper to keep track of which is which.").

In this storyline, we observe the formation of signs and the use of manifest objects as MCs to help the participant make sense of how each paper (representing a specific concept) fits in 'Physical Computing'. This strategy was seen in two other participants. One had put her 'bag of papers' on her iPad and had them open in GoodReader tabs to share and discuss with her teammates ("… papers that were open on it … since we were talking about what we researched. I was able to say I found this paper, it talks about this and this"). She was using the iPad on which the PDF was materialized as an MO to support her discussion and thinking (the team did not actually read the paper on the iPad screen, the participant merely used it as an objectified sign to reference the information the PDF contained). A third participant employed an almost identical strategy with a slight difference in that he did an initial filtering of his 'bags of finds' on his laptop browser tabs, and winnowed these to 15 -20 papers that he saved and named in his Dropbox. He copied these to his iPad for further reading and annotation. There was also evidence of his employment of filenames as meaningful signs to give him mnemonic access to the PDFs, and his use of the displayed PDFs as MCs in the process of reasoning about 'Physical Computing'.

Storyline 2: Creating the assignment report. One of the participants organized his paper building from an outline written in a GoogleDocs document. This outline served as the conceptual frame on which he worked: "I have the outline and each point I know which PDF is linked to what, so I would read the PDF and while I am reading, I get an idea, and I start writing on the paper." He read the paper on his iPad and wrote in GoogleDocs on his laptop simultaneously. He mentally associated each outline item in his GoogleDocs outline with several PDFs.

Here, we see that he has a two-level conceptualization of the PDF paper he is reading. The idea of an outline item is a thought object. Both the mnemonic name of the PDF and the concomitant name of the outline point serve as signs for the TO. The TO has two different simultaneous MOs. The outline item in the opened GoogleDocs document is a MO with which he associates the TO, but it is also clear that he "knows which PDF [it] is linked" to. A second MO is thus the display of the PDF on his iPad. Further, since the PDF summoned to the iPad is open to reading and inquiry, the contents of the PDF themselves become another level of TOs, signs, and MOs. At each level, objectification may occur to allow the use of the MO as an MC for reasoning. It is precisely this nature of signs that are able to hold entire concepts for thinking, and be unpacked into its contents for inquiry at a finer degree of abstraction that makes the theory a potent vehicle to understand thinking in DDEs.

This same two-level strategy was employed by another participant who used PDFs in Dropbox folders to support conceptualization. The mnemonic names of the PDFs served as the signs and the preview display of the PDFs (quick look feature in the Macbook) in the Dropbox folder served as the MO or MC when he referred to it while writing. The participant wished he could add more information to the sign at the level of the whole PDF, stating "Originally we discussed annotating them (the PDFs), but we found that it was kind of difficult to actually do it because we just wanted to have a summary of each paper but Dropbox on the iPad would not let us create a text file [of the same name] next to the papers." This shows that the participant was thinking at the level of the PDF as a whole, and wanted to associate meta-information at the level of PDF within the folder structure.

Another three participants exhibited the same two-level conceptual structure using yet other methods to provide overarching structure of how the sign associated with each PDF relates to the paper as a whole. Two of them employed a citation manager, e.g. Endnote, Mendeley, to maintain citations and to add an annotation at the level of the whole citation for each PDF. An entire PDF thus could be used as a contained unit of thinking for paper writings. Our last participant recorded to use this strategy employed Evernote for her higher-level organization. She had an Evernote note for each PDF document, describing its contents at the level of its place in the whole 'Physical Computing' paper. Evernote allowed her to attach the PDF directly to the note, enabling the same two-level sign-MC strategy.

Technology Affordances of DDEs

To recap, our theoretical framework used for analysis is that digital objects may be materialized through DDEs into manifest objects, and that these MOs may be appropriated through a process of objectification to serve as material carriers that support thinking. We extracted 50 thinking episodes that related to GoogleDocs documents, Evernote notes, and PDF documents. Our DO-path analysis traced the three 'focus' MOs in terms of thinking processes, TOs, DOs, MOs, technologies used, relevant quotes relating to affordances and experience, and the occurrence of an MO-MC binding (coded as hindrance or support).

The second part of the DO-path analysis, of which a sample is shown in Figure 8, coded the quotes extracted in the first part of the analysis for objectification indicators and 'support' (in the moment or/and in the long-term) or 'lacking' characteristics of the technology configurations used. Indicators included for example referencing materials as thoughts, reports of pointing to objects as ideas, and use of space as an organizing structure. Figure 9 shows the distribution of thinking episodes in which the MO was able to be coded as either having a support or hindrance role in relation to the students' thinking processes. The synthesis of the technology affordances that we found supported, were lacking, or hindered objectification to occur is presented in Figure 10. The features were derived from quote-supported storylines like Storyline 1 and 2 above. Storyline 1, for example, shows evidence of the creation of mental signs as part of the reasoning workflow, and how objectification may have occurred by which MOs, The persistence of the 'bags of finds' in the laptop browser allowed the participants to think of these finds as 'stuff I found' without necessarily forcing them to a premature commitment to meaning or interpretation. The mobility of the iPads supported opportunistic action so that the bags of finds and the named PDFs could be used to support thinking. The transparency of action between searching, renaming, and reading allowed for "a complete experience". In Storyline 2, all the participants regarded the entire PDF as the DO related to a particular TO at the level of the organization of the entire paper being written. The participants all employed a strategy of encapsulating this level of conceptualization of the paper with a very short description that served as the MO. This allowed the participant to bring perception into the process of thinking by using the MO to serve as an MC. The extent at which this objectification process occurred however was highly dependent on the transparency of interaction to annotate/associate meta-data to the high-level description of the TO, and the persistence of the display of the MO. The immediacy of being able to quickly drill down to the content of the PDF document as well supported the unwrapping of successive layers of abstraction and encapsulation.

Discussion

We organize the discussion of our work under three themes: the objectification framework as a method for the study of technology ecologies; the experiences of students in DDEs; and technology affordances to aid in the design of DDEs.

An Objectification Framework for Thinking in DDEs

Digital information is 'magical'. We can duplicate, copy, summon to visual presentation, search, etc. almost at will. Yet, despite this malleability of pixels on screen, we often revert to physical, material things to help us make sense of information because they more readily help us to objectify thought. Starting from Vygotsky's sign mediation theory, we proposed that thought objects can be related to digital objects that can be materialized in DDEs as manifest objects. Through objectification, the student appropriates the MO to serve as a material carrier, thereby bringing the MO into the loop of thinking. Our DO-path analysis, as presented in the two example storylines, showed the promise of our objectification framework, and affordances in the DDE that appears to support objectification. We successfully traced the components of our model in different situations in our data transcripts to elucidate the thinking process.

Figure 10 Technology affordances for objectification

Other similar 'path analysis' methods have been proposed before for the study of technology use, notably in works that argue for the importance of context in HCI. For example, in 'activity theory' (AT), the unit of analysis is human activity, mediated by the use of artifacts or tools. One traces the subject, object, actions and operations in an analysis approach using AT [START_REF] Nardi | Studying context: A comparison of activity theory, situated action models and distributed cognition[END_REF][START_REF] Kaptelinin | The Activity Checklist: A Tool for Representing the "Space" of Context[END_REF]. In a similar vein, situated action models identify the "activity of persons-acting in setting" as the focus of study. Using the approach, one traces the relations between individual and environment at a "very fine-grained level" as the human engages in dynamic uses of artifacts [START_REF] Nardi | Studying context: A comparison of activity theory, situated action models and distributed cognition[END_REF]. Distributed cognition (Dcog) looks at the "cognition of a system in terms of its function". It requires one to trace "representation [states] inside and outside the head" as the central unit of analysis [START_REF] Halverson | Activity Theory and Distributed Cogntion: Or What Does CSCW Need to DO with Theories?[END_REF]. Our method of tracing DOs does not exclude artifacts (devices and displays), individuals (students), or their activities (thinking processes). Adopting a different focus, it provides a way instead to make sense of the flow of information, literally 'food for thought', from the environment to the mind of the individual (or vice-versa).

Experiencing the DDEs

With currently available technologies, students already build ecologies with their personal devices in daily life. As highlighted by Jung et al. [START_REF] Jung | Toward a Framework for Ecologies of Artifacts: How are Digital Artifacts Interconnected within a Personal LIfe?[END_REF], "every newly designed interactive artifact will inevitably become a part of someone's ecology". The question is however whether it will be an effective part of his/her ecology. We found many situations when the student was faced with 'gaps' in his/her DDE, which hampered the smooth flow of his/her thinking. He/she then had to find other technologies that could bridge the 'gaps'. In this sense, the DDEs that one constructs today are 'adhoc ecologies': the different components do not function as a synergistic whole by design, but places the burden of synchronization and file control on the user [START_REF] Tungare | Personal Information Ecosystems and Implications for Design[END_REF].

Furthermore, we found that the use of the individual devices was mostly characterized by what we call the 'portal thinking' effect. We posit that many current interfaces and devices are designed based on the Cartesian view that external things and thinking processes are mutually exclusive. The result may be that the screen then acts only as a conduit through which information is summoned. With computational advancements (e.g. larger screens, mobile systems) we may have multiple 'portals', but they remain data straws to draw information to view. It is unfortunate thus that the potential of interface, visual and physical components of devices to be MCs for thinking often fails to be harnessed, and instead, technology is used as mere input, output or 'projection' devices.

Designing DDEs for A Learning Experience

We believe that there is a lack of designed support to optimize technology's role of augmenting humans' higher thought. From our findings, we proposed technology affordances that appears to help the process of objectification. These can be classified into seven groups, although they are all interrelated at certain levels. Below, we discuss the central theme of each group, while providing considerations for system design features that can potentially embody each theme. It must be noted however that our proposition of technology affordances in DDEs for learning does not imply that one can design such a complete ecosystem for learning. Variations in individual learning styles and the diversity of subject matter to be learned would preclude such rigidly constructed ecosystems. Drawing insight from the inveterate paper ecology that has supported learning since Gutenberg's printing press, we suggest that learning ecologies, be they physical or digital, have to remain ad-hoc but with designed support. The paper ecology, that includes writing implements, tables, bookshelves and libraries, books, and the myriad paper clips, staplers, rulers, and folders, has evolved over more than half a millennium and provides a vast set of components that are flexible enough for the individual to appropriate them to construct his/her environment for specific learning experiences. The digital ecology as well, needs to evolve technologies that possess affordances but yet are malleable enough to support learning.

Affordance 1: Expectation of interaction (includes transparency, shareability): It is key that components of DDEs are able to not only interoperate in some way, but also provide an expectation of interaction to users. Our writing implements for example can be expected to write with few impediments on paper. Of course, the paper ecology is constrained by materials and physical laws (e.g., we do not have to worry about book 2.0 falling through the surface of table 3.1 because of incompatibilities) while all interactions in DDEs have to be designed and implemented. Also, the cultural longevity of paper has built expectations and constraints (e.g., pencils do not work on leather portfolios) into the user community that digital technologies cannot always rely on. In our study for instance, the students reported that they decided to use the whiteboard for brainstorming particularly because they knew that they would be able to take a picture of it with their iPad cameras later on. Conversely, a clear example of failure of this aspect in our study is one instance where the students spent one entire meeting only to set up shared Dropbox folders and Evernote notebooks. Work in middleware and system interoperability, as well as direct interaction methods, are important to enable this affordance of 'expectation of interaction' to take place.

Affordance 2: Immediacy (includes persistence, reference, situatedness, accessibility, multitasking, opportunistic action): Immediacy concerns features that allow the user to display, manipulate, and use DOs across devices without going through one or more indirect actions. For example, if a user has a physical paper that she wants a friend to read, she drops it on the table in front of him. However, if she had the document on her laptop, she may have to put it into Dropbox and tell him where to get it before they can discuss its contents. The lack of immediacy in this scenario hinders the opportunistic use of the document as a focus of discussion or thought. Immediacy is closely associated with transparency. Transparent interoperability across platforms supports immediate action as do consistency of interaction techniques (consistent ways to move and manipulate MOs across platforms is critical to support their use as MCs). In our study, participants used persistence to allow information to stay immediate and more easily participate in their thinking: on the laptop, they aligned their Word document and the PDF papers side by side. Others used their iPad as a 'persistent' secondary display for the PDFs.

Affordance 3: Concreteness (includes dedicated use, spatiality, tangibility): Components that possess the characteristic of 'concreteness' in a form appears to better support objectification. In our study we saw that the whiteboard that affords the use of space and the iPad that opens applications using the full screen real estate, for example, aid information to become what Heidegger [START_REF] Heidegger | Being and Time[END_REF] calls a 'thing', something tangible for the student to grasp in her thinking process. A technological environment that requires the user to hold thoughts in mind while looking for the appropriate device to record them offers little support for objectification. Work on embodied interaction, physicality, and tangible user interfaces that address how to enable the user to easily assign thoughts to concrete materials contribute to this affordance.

Affordance 4: Iconicity (includes structure, mnemonicity, atomization, customization): Objects to which we have assigned meaning become more easily internalized than neutral objects. We have seen the use of file renaming and organization of folder structure as instances when iconicity has enabled MOs to be used as MCs of thought. The few works in HCI that have looked at familiarity of interfaces are potent informants of this affordance. Atomization is a related feature that supports the association of DOs to TOs. A TO is typically an atomic idea at some level of abstraction, in the same way that a 'unit of analysis' specifies an entity that is a coherent whole at a certain level. Take the idea of convolution in our earlier example. A Wikipedia page on convolution would be an apropos DO for the concept, but a whole book on signal processing would not (even if it contains a section on convolution). Features to support atomization (e.g., bookmark individual components of larger text documents) have been investigated for example in hypermedia research.

Affordance 5: Common ground (includes simultaneous focus and control):

To be able to focus on thinking in groupwork, the technology should provide support for students to easily create shared MCs. A common MO may not necessarily lead to the same mental signs for two different people. This is essentially the question of intersubjectivity [START_REF] Gillespie | Intersubjectivity: Toward a dialogical analysis[END_REF]. Physical things inherently allows for several users to have simultaneous focus and control. A page on the table can be seen by everyone around the table; several users can write on the whiteboard at the same time. In digital technologies however, the students always needed a separate 'situating channel' (e.g. speech, instant messaging, comment lines) to set common ground together with the 'information channel', where work is carried out. This factor also appeared to have prompted the more intensive use of GoogleDocs, which has a simultaneous editing capability, over Evernote for notetaking. Work in computer-supported collaborative would be relevant to further inform the design of this affordance into technologies.

Conclusion

This paper contributes significantly to the area of technology ecologies, focusing on how they may help us to think and to learn. We identified a gap in the literature of the plethora of technology ecology/ecosystem concepts previously proposed in terms of investigating learning. Second, we extend Vygotsky's sign mediation theory to the digital world by proposing how thought objects may relate to digital objects, and their manifestation through technology ecologies. Third, we propose a way through the concept of objectification and method of DO-path analysis to examine device ecolo-gies for their potential for learning. And fourth, we articulate affordances that can inform either design guidelines or evaluation metrics for the development of device and display ecologies that allow us to think effectively.

One limitation of our study may be that it was conducted with computer science students who may have had a different perspective on technology. Researchers from other domains like history may require additional support to piece together fragmentary information. Such extension can be a rich domain for future research. Although we do not claim to propose a holistic explanatory theory of cognition, we do believe that the objectification framework manages to uncover basic thinking processes irrespective of the student's domain expertise. Knowledge discovery is foundational to any form of learning. In a future study, we plan to employ our model to make sense of students' in-situ speech on top of their self-reported experiences of learning using technology. In the face of changing perspectives, HCI researchers should begin investigating and designing technologies that help us think, or in our Star Trek tale, that make the doctor 'real'.

Figure 2 Figure 1

 21 Figure 2 Vygotsky's sign mediation theory

Figure 3 Figure 4

 34 Figure 3 Extension to the sign triangle

Figure 6 .

 6 Figure 6. Left. Analysis procedures; Right. Example analysis

Figure 7

 7 Figure 7 Distribution of devices owned at start of study

Figure 8 Figure 9

 89 Figure 8 Objectification indicators and Technology affordances from DO-path analysis

Acknowledgements

This project has been supported by NSF grant No. IIS 1059389.