Richard Beckwith
email: richard.beckwith@intel.com

Jo Boaler
email: joboaler@stanford.edu

Hilda Borko
email: hilda.borko@colorado.edu

Janet Bowers
email: jbowers@math.sdsu.edu

Matt Brown
email: mwbrown@uic.edu

David Carraher
email: david_carraher@terc.edu

Cathy Casserly
email: casserly@earthlink.net

Milton Chen
email: miltchen@stanford.edu

John Cherniavsky

Paul Cobb
email: paul.cobb@vanderbilt.edu

Daniel Cogan-Drew
email: daniel.cogan@tufts.edu

David Crismond
email: davidc@cc.gatech.edu

Ken Dauber
email: kdauber@stanford.edu

Chris Dede
email: chris_dede@harvard.edu

Sharon Derry
email: sharond@wcer.wisc.edu

John Doherty
email: doherty@fxpal.com

David Dwyer
email: dwyer@apple.com

Wolfgang Effelsberg

Michael Eisenberg

Fred Erickson
email: ferickson@gseis.ucla.edu

Margaret Farren
email: margaret.farren@dcu.ie

Jamie Fenton

Barry Fishman
email: fishman@umich.edu

Janice Fournillier

John Frederiksen
email: frederik@u.washington.edu

Alvaro Galvis
email: alvaro@concord.org

Shelley Goldman
email: sgoldman@stanford.edu

Ricki Goldman

Louis Gomez
email: gomez@nwu.edu

John Graham
email: johng@graham.com

Jim Gray
email: jim.gray@sri.com

Jim Greeno
email: jgreeno@stanford.edu

Pam Grossman
email: pamg@stanford.edu

Kenji Hakuta
email: hakuta@stanford.edu

Rogers Hall
email: rogers.hall@vanderbilt.edu

Thomas Hatch
email: hatch@carnegiefoundation.org

Randy Hinrichs
email: randyh@microsoft.com

Cindy Hmelo
email: chmelo@rci.rutgers.edu

Chris Hoadley

Eric Hoffert

Sherry Hsi
email: sherryh@exploratorium.edu

Toru Iiyoshi
email: iiyoshi@carnegiefoundation.org

Aditya Johri
email: ajohri@stanford.edu

Eammon Kelly
email: akelly1@gmu.edu

George Mason

Don Kimber
email: kimber@pal.xerox.com

Tim Koschmann
email: tkoschmann@siumed.edu

Maggie Lampert
email: mlampert@umich.edu

Rachel Lotan
email: rlotan@stanford.edu

Kay Mcclain
email: kay.mcclain@vanderbilt.edu

Brian Macwhinney

Michael Mills
email: mmills@stanford.edu

Vic Nalwa

Ricardo Nemirovsky
email: ricardo_nemirovsky@terc.edu

Tom Prudhomme

Lou Pugliese
email: oncourselpugliese@oncourse.org

Jeremy Roschelle
email: jeremy.roschelle@sri.com

Joe Rosen

Nora Sabelli
email: nora.sabelli@sri.com

Janet Schofield

Vivian Schoner
email: vschoner@lt3.uwaterloo.ca

Dan Schwartz
email: daniel.schwartz@stanford.edu

Miriam Sherin
email: msherin@northwestern.edu

Greg Shrader
email: gshrader@northwestern.edu

Brian Smith
email: bsmith@media.mit.edu

Mike Smith
email: msmith@hewlett.org

John Spaihts

Reed Stevens

Jim Stigler
email: stigler@psych.ucla.edu

Sue Talley
email: sue.talley@pepperdine.edu

Sarah Walter

Raul Zaritsky

Introduction

This report documents a workshop on the use of digital video in the study of learning and teaching, held at Stanford University's Wallenberg Hall on November 25 and 26, 2002. The workshop was funded by the National Science Foundation through a Center for Innovative Learning Technologies (CILT) seed grant. CILT was founded in October 1997 with the aid of a 5-year grant from the NSF, with the goal of stimulating research and development in technologies to support learning in science, mathematics, engineering, and technology in grades K-14. In addition to the workshop itself, the seed grant from CILT funded the development of a community website that serves as a repository of resources and a discussion forum for a broad range of issues around the use of digital video in inquiry.

The premise of the project is that digital video is an increasingly important medium for recording and analyzing interactions in classroom and other educational contexts, but that there is relatively little community support for researchers and practitioners who make use of digital video for this purpose. At the root of this problem, ironically, is the widespread availability of inexpensive video recording and editing technologies. The barriers to entry for digital video are now lower than ever, but guidance about how to proceed once in is sparse and scattered. As a result, we are seeing a great deal of effort being expended to produce a growing number of independent toolsets and approaches. The purpose of the workshop was to gather a number of the most active practitioners in this area to uncover the common fruits of these diverse experiences, and to identify areas in which collaboration and consolidation are possible.

Workshop and website purpose, approach and structure

This project had two complementary goals: to gather together a group of researchers and practitioners to assess the state of the use of digital video in the study of learning and teaching; and to form the basis of an ongoing community focused on digital video as data for discovery, rather than simply a vehicle for presenting records of activity. The organizers invited a diverse group of 84 experts from a variety of fields and institutional backgrounds (see Appendix A for a list of invitees and attendees). Ultimately 65 people attended the workshop, including 42 from universities, 11 from industry, 7 from nonprofit organizations, 4 from foundations, and one representative from the NSF. Several previous NSF-funded workshops have dealt with the technical and research issues affiliated with digital video records for learning and teaching research, and for teacher education [START_REF] Lampert | New technologies for the study of teaching: Report of the NSF Workshop held June 9-11[END_REF][START_REF] Macwhinney | Multimodal studies of instructional discourse (Report from the Classroom Discourse Workshop[END_REF].

The workshop had several distinct components, designed to accomplish the project's dual goals. Through keynotes, short 5-minute presentations, and panels, attendees shared with each other their individual work and their assessments of the state of the field. Through demonstrations and posters, those attendees who have developed tools for video analysis were able to share their work with others in interactive engagements, providing the opportunity for productive one-on-one conversations and perhaps future collaborations. Finally, a series of breakout sessions on thematic topics produced collective snapshots of the state of the field and recommendations for future directions.

The breakout sessions were chosen by attendees from a longer list prepared ahead of time by the workshop organizers.

In addition to the workshop itself, the project produced a community website (http://cilt.stanford.edu) that is intended to serve as a longer-term repository and discussion space for researchers interested in digital video inquiry in the learning and social sciences, including both attendees at the workshop and others. The website was developed at SCIL and includes functionality for uploading documents and references to resources on the Web, discussing resources and general topics, and maintaining a membership list. The website includes a large number of resources uploaded by community members, and promises to be a useful node in the community network.

Workshop talks and breakouts

3.a Keynote addresses

Norman Winarsky, SRI International's Vice President, Ventures and Strategic Business Development spoke about new developments in video technology from Sarnoff Labs (http://www.sarnoff.com/) that allow video records to be manipulated at the level of objects, rather than just as streams of pixels. He pointed out that the MPEG-4 video standard includes the notion of object-based manipulation, which has however posed great difficulties in terms of speed and processing requirements. New advances in chip technology, however, such as the Acadia chip, make it possible to do real-time manipulation of objects in video records. Winarsky demonstrated this capability through a video scene in which the moving image of a man was seamlessly removed in real-time, leaving only the background. This technology has great potential for educational and research applications -for example, distracting elements could be removed from a scene at one point in an analysis, and then reintroduced later to test whether their presence was meaningful.

Ken Hay described his ongoing work at the University of Georgia in which a variety of cameras, audio recorders, and computing devices produce multiple streams of data regarding a single event. This approach ensures that the event is fully documented, but poses difficult issues in data management and integration. Hay described some aspects of a response to these issues, such as real-time coding of 'nodes' of activity within an event. The amount of data that this approach generates has led Hay's group to consideration of a concept with very broad applicability: the amount of 'friction' involved in creating and, most importantly, storing, retrieving, and using video records. Friction refers, in this case, to the amount of time and, secondarily, expense that is required to make a video record useful. If friction is high (for example, by requiring a great deal of time transferring tapes to digital formats) then researchers will tend to make relatively less video and to make less use of it in their work. If friction is reduced, then we can expect digital video to be more heavily used. Hay urged that friction be quantified as much as possible, and that attention be directed within the community to reducing the amount of friction associated with digital video inquiry.

3.b 'Firehose' talks

Following the keynote addresses, participants heard a series of very brief (approximately 5 minute) talks in which presenters shared their research and tool development efforts. This format has been used successfully in other CILT workshops, and provides a means for providing participants with a quick survey of developments in the field, which can then be followed up on later if desired.

Sue Talley described online video cases being used for preservice teacher education at Pepperdine University. She observed that new teachers are more interested in viewing 'authentic' teaching situations than in seeing 'best practices,' which, she contends, are hard to demonstrate in a naturally occurring teaching situation. This produces some tension in the pedagogical purpose of videos between accuracy and integration into the curriculum. In fact, some student teachers found classroom video to be a way of breaking down the isolation of the classroom by giving them a view into a variety of professional situations that would otherwise be difficult to observe.

Along similar lines, Miriam Sherin of Northwestern University described how she has used group viewing of classroom video to stimulate professional vision among mathematics teachers. She found that 'video clubs' produced a greater degree of understanding and excitement than was found for people working alone. The presence of other teachers fostered more accurate and richer perceptions of the teaching situation.

Reed Stevens of the University of Washington described software for producing what he terms 'video traces' -that is, base video clips with a layer of verbal and visual annotation. These video traces have a wide variety of potential uses, one of which is also the development of professional vision in a field of inquiry (see [START_REF] Goodwin | Professional vision[END_REF]. Video traces provide records of expert viewing of scenes, and also provide a means of capturing novice perceptions so that they can be subjected to expert evaluation and feedback.

Probably the most elaborated examples of professional vision on display at the workshop were those described by Thomas Hatch of the Carnegie Foundation for the Advancement of Teaching. The Carnegie Foundation, through the vehicle of the Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) and the Carnegie Knowledge Media Laboratory, provides exemplary teachers with resources and technical support to fully document an extensive aspect of their teaching, such as a course. CASTL provides fellowship support to allow teachers to devote themselves for a period of one or two years to the documentation and improvement of their teaching. Video is used as one vehicle for documenting professional practice.

Frederick Erickson of UCLA posed a different but related problem -how to use extensive video of an exemplary teaching practice (in his case, an investigation of the physics of matter, energy, and motion in a classroom of kindergarten and first grade students at the UCLA Lab School) to inform diverse audiences without producing a single narrative. Rather than producing a single perspective on this teaching practice, Eriksen's goal is to enable users (preservice teachers, experienced teachers, or researchers) to explore the video and other documentary material in ways that reflect their own interests. A primary difficulty is one of professional vision -how to deal with the fact that not all viewers are capable of making sense of the complex pedagogy being represented, and hence need support in order to get value out of the viewing.

Along somewhat similar lines, though not in the area of teacher training, John Graham of Broadware Technologies demonstrated a system, based on open-source Web technologies, to allow global users access to source video and image data produced by research projects anywhere in the world (http://www.telascience.org/). The system allows users to share their annotations and comments on data with other users, making it possible to store and retrieve multiple perspectives on the same underlying data.

Milton Chen of Stanford University also demonstrated a new technology for video distribution (http://graphics.stanford.edu/~miltchen/VirtualAuditorium/), in his case a system for enabling videoconferencing with up to 25 distributed users, all with PCs with simple USB cameras and relatively low bandwidth (100 Kbps). The system holds great potential for increasing interaction in distance learning, by enabling instructors and students to see and hear each other whether or not they are located in the same physical space.

Other participants highlighted the use of video technology to enhance classroom instruction directly. Arnetha Ball and Jo Boaler of Stanford University reported on the results of a PT3 grant to support the use of technology, including video technology, handheld devices, and computing devices in classrooms. Their work draws explicitly on learning theory to integrate technology into project-based instruction. Dr. Boaler also reported on a series of video cases in the area of mathematics education, for use in teacher training and in education research.

Several talks highlighted the use of video for research purposes. Ricki Goldman described a Web-based tool called 'Orion' designed to support 'quisative' ethnographic research into teaching situations -that is, research that explicitly acknowledges the multiplicity of points of view in complex social situations, and attempts to display those perspectives as data rather than attempting to come to a universal perspective. In this approach, videographers include others beside the researcher, and videos can be frankly persuasive texts.

Rogers Hall of Vanderbilt University argued, more generally, that the process of moving from a concrete video to a general proposition with scientific status should be transparent and reversible, in order to preserve the status of video as evidence. He suggested that providing 'viewers' with access to the original video and the means to author their own perspectives might be a useful way to maintain focus on the process of fact and argument construction out of video evidence.

David Carraher of TERC (Technical Education Research Center) described the concept of 'video papers' combining text and hyperlinked video into a Web-based tool for rapid presentation of research results. The Video Paper project is both an attempt to produce a usable tool, and an investigation of how multimedia technologies will change the form and content of scholarly research over the next decade, as [START_REF] Pea | New media communication forums for improving education research and practice[END_REF] highlighted in a National Academy of Education chapter on the future of educational research dissemination.

Mike Mills of Stanford University demonstrated DIVER (for Digital Interactive Video Exploration and Reflection), a video analysis and collaboration tool developed with the aid of NSF MRI and SGER support. DIVER is intended to allow researchers and practitioners to author their own video-based commentary on video sources, without the need for advanced video editing skills (see [START_REF] Pea | The DIVER™ Project: Interactive Digital Video Repurposing[END_REF]. DIVER includes the ability to upload video clips and annotations to the Web, where others can view analyses and add comments.

Brigid Barron of Stanford University showed how video analysis could yield significant insights into the nature of collaborative learning processes. Video records of student project work and interaction with teachers makes it possible to trace variations in learning to differences in the nature of interaction in small groups. Video capture makes it possible to compare differences in group processes across a relatively large number of groups by enabling subsequent coding of interactions. Interactional competencies -such as skill in managing group attention, use of conversational techniques to move the group towards consensus, and so on -are significant mediators of learning, even holding domain knowledge constant.

Finally, Brian MacWhinney described the NSF-funded 'TalkBank' project, a highly successful effort to produce a corpus of audio and video materials for researchers of human language over a variety of disciplines and subdisciplines. TalkBank includes, in addition to language records, a number of tools designed to aid analysis, including transcription and annotation tools. TalkBank also provides, by virtue of its success and the range of materials it holds, a useful source of experience regarding legal and ethical issues around video record access and use.

3.c Breakout sessions

On the first full day of the workshop, Roy Pea presented a list of possible topics for breakout groups that reflected themes expressed in the published work of participants (see Select Bibliography), in prior CILT workshops, or in industry developments and, more generally, in the published literature of the field. The purpose of the groups was to gather a set of researchers and practitioners around a topic in order to generate a collective understanding of the state of the field and of potential future directions and their relative priorities. Potential breakout group topics included:

1.

Requirements for tools to use in digital video coding, annotation, transcription and publishing. A key issue is to assess whether the variety of tools obscures a core set of commonly-needed functionality that should drive tool development and consolidation in the future. A related issue is whether a standard for data interchange, possible XML-based, could reduce the fragmentation of data produced by the variety of incompatible tools.

2.

Video data access conditions, including issues of publishing rights, security, and human subjects requirements.

3.

Use models for digital video for informal learning. This includes the use of video in developing 'professional vision' -that is, the characteristic way of looking at an event that professionals in a discipline develop.

4.

Use models for digital video in teacher education. An important issue in this area is to understand the existing infrastructure of teacher education programs, and to be aware of the value added of digital video over text in case studies and reports.

5.

Video as evidence. If digital video is to become widely used and accepted as evidence in the learning sciences, it is necessary to develop practices for argumentation and statistical use of video. Otherwise, common arguments remain that the selective capture of video is anecdotal and used primarily for buttressing the beliefs of an investigator rather than for systematically testing scientific conjectures. 6.

Computational video as an instrument for building theory. How can sizeable databases of digital video of learning and teaching provide a new instrument for theory building in education and learning? 7.

Best practices for high-quality digital video and audio capture in classrooms.

Digital video records vary greatly in quality, and researchers and practitioners could benefit from a compilation of the community's experience with producing usable records. 8.

Designing a digital video collaboratory. A collaboratory is a vehicle for gathering together and making available otherwise disparate resources. How would such a collaboratory for digital video inquiry in the learning sciences be structured, and what practical steps can we take towards instituting such a community resource? 9.

Digital video reading/writing relationships. The complexity of the process of producing and using digital raises questions of authorship and point-of-view. If a video is captured, edited, and annotated by different people, who is the author? What is the difference between viewing a video and reading a text? 10. Digital video cases and assessment. What is the process by which people learn from digital video materials? Given the amount of time and resources devoted to producing video material for educational purposes, there is relatively little research about how video works in learning, and what conditions facilitate or hinder the effectiveness of video.

From this list, participants coalesced around a set of four breakout topics. The breakout groups met for several hours, and reported back to the full group of attendees the following morning.

I. Development of Professional Vision -Roy Pea (Reporter)

This breakout group focused on questions of professional vision and learning with digital video. The group developed an inventory of important research questions that should be addressed in this area and sketched a community project to forward understanding of the educational and professional development role of video records. The group focused on the development of professional vision among teachers, and discussed the challenges of characterizing and studying teacher expertise.

As an initial matter, the group faced the problem of the complexity of teaching as an activity. The teacher development literature is unsettled on the question of precisely what constitutes 'expertise' in teaching, on how to distinguish between domain-specific teaching expertise and expertise in general, and on how to assess such forms of expertise. K-12 teaching standards have been established by several organizations. INTASC (Interstate New Teacher Assessment and Support Consortium), a program of CCSSO (the Council of Chief State School Officers) in which 35 states participate, has thus far released standards developed for teaching Arts, Foreign Language, Mathematics, Science, and students with disabilities), which they intend to be used as a resource for states to use to develop their own standards for new teachers. NBPTS (National Board for Professional Teaching Standards) has established a large set of standards for the voluntary recognition of accomplished teachers "at the highest level of teaching in the different disciplines," and at different grade levels (http://www.nbpts.org/standards/stds.cfm#stdsfaq), with over 24,000 total National Board Certified Teachers. Although the content standards for both INTASC and NBPTS have been developed to align with the K-12 student standards that the subject matter professional organizations have established, so as to characterize what teachers should know and be able to do to effectively teach to the standards, they are not well aligned on what counts as expertise, and are in any case not focused specifically on the question of professional vision. Moreover, the concept of professional vision in some noneducational fields (such as auto mechanics) seems different from its use in teaching, in that even expert teachers may see different elements in a teaching situation as important. Digital video may help to capture this kind of variation, but it is not clear how to use video to inculcate a skill that is itself not well understood. Finally, it was remarked that it seems odd in such an unsettled sphere to have a group of non-teachers discussing how teachers might develop professional vision.

Mindful of these issues but still desiring to discover what coherence there is in this area, the group took up the question of how to understand the learning uses that people make of digital video records. In the context of professional vision, the relevant outcome variable has to do with changes in professional practices achieved through teacher learning with the video materials. Such achievements may be defined in terms of changes in information processing behavior -do teachers see their classrooms differently after having received video-based professional development, for example, with different categories of phenomena observed, and a more nuanced sense of the conditionalized knowledge to deploy when such-and-such recognizable learning and teaching situation arises? It may also be defined in terms of more distal measures, such as teaching behavior -do teachers teach differently? For example, it may be that teachers adopt a broader range of strategies for dealing with learning issues if they view video in which those strategies are used in a classroom. We could also measure learning outcomes -do students learn differently or better with a teacher who has received support in developing professional vision? It was also noted that metacognitive and strategic knowledge may be needed for teachers to learn from a video representation of practice in ways that can improve their own practice.

In order to be more specific about these effects, the group developed a list of kinds of uses of digital video that made clear the diverse variety of ways in which digital video can be used for promoting teacher learning. These included uses of video to: (1) illustrate different developmental levels of student thinking or teacher thinking; (2) highlight uncommon but important practices; (3) show a broad variety of exemplars of a particular teaching strategy or a student misconception; (4) provoke reflective conversations about a troubled teaching moment in a safe mentoring environment; (5) to model advanced levels of thinking in teaching; (6) enable highly-focused attention to specific aspects of a teaching phenomenon; (7) provide a 'common ground' experience for a cohort teacher group; (8) distinguish contrasting cases (such as exemplary use of a teaching strategy versus a 'near miss' in its effective use); (9) provide visions for what is possible; (10) compress lots of experience that it would take a long time to have in the world; (11) support role-playing in instructional decision-making (what would you do now?); (12) use for grounding predictions for what will next happen in the classroom, and then for discussing variations of predictions from what then happens when the video continues to play; (13) help teachers build categories of important phenomena that you want them to remember and learn to see in their own practice; and (14) enable leaps in time scales (as in longitudinal video that could illustrate changes in student or teacher thinking over months and years).

The group also considered the question of whether video material should be 'authentic', or whether simulations could be used instead of actual classroom footage. The latter approach offers the benefits of circumventing the problem of student privacy and making it easier to generate 'pure' examples, but it risks producing scenes that are discounted by practicing teachers. Even in terms of authentic video of classroom teaching, the group noted, there are considerable differences across teacher education and professional development programs in what kinds of practices videoproducers aim to capture and then highlight in the editing process for creating video case studies. On what principles are these decisions made, and how can the field move from art to science in the design of effective video case study materials, and in learning about the consequences of the instructional uses of such video cases under different conditions and with teachers of different background and skill levels? Might it be possible to develop criteria for recognizing "the most powerful teaching moments" in teaching practice from which other teachers could learn most effectively?

The recurrent issue was also raised of the tradeoffs involved in pre-structuring video case studies in terms of phenomena highlighted, lessons to be learned, and using scaffolding to suggest application of the phenomena to one's own practices, versus a more discovery-oriented approach in which teachers take more of the responsibility of identifying the values of the videorecords of teaching practices for their own teaching through their own reflections and discussions.

Finally, the group developed a list of recommendations for future research and development, including the following:

1. The use of video cases by teachers is not well understood, and should be researched directly. If we understood better how teachers learn from video cases, it would be easier to determine how video cases should be structured, and where there are unmet needs for case material. 2. Research is required into the broader effects of changes in professional vision. We assume that seeing the world differently produces changed behavior, but the connection is not well established empirically. 3. The research community would benefit from a common database of interesting teaching and learning events. Such a database could serve as a common reference point for studies that use these videos as an elicitation device for studying professional vision among teachers.

II. Requirements for tools -Eric Baumgartner (Reporter)

This group focused on addressing requirements for video analysis tools, most importantly the need for interoperability and easy data transfer among tools. Rather than requiring researchers to adopt one or another comprehensive tools, researchers should be able to use different tools for different tasks. This implies, however, that tools should be interoperable at the level of data transfer, so that researchers could use one tool with confidence that their results would be portable to other tools.

Much of the interaction of researchers with video records is directed toward codingthat is, towards associating some kind of structured metadata with the video or some element of the video. Currently, there exists no standard way of representing coding schemes in a way that makes them comparable, or even of expressing codes and their relation to video records in a way that is compatible across tools and coding schemes. The group recommended that the community adopt XML as a data format for describing coding schemes and metadata. XML has the advantage of being agnostic with respect to programming language, display method, etc., and of being readable by humans. This would require, as a preliminary effort, the production of a standard XML-based approach to describing coding schemes.

A challenge for such an effort, raised by Nora Sabelli of SRI International, is that the learning science community does not have a generally agreed upon theory of teaching and learning, and hence it is not possible to link coding schemes to a common definition of the entities and phenomena of interest. This complicates the problem of developing a standard representation of coding categories, since each researcher or community of researchers defines the phenomena of the field, as well as the dimensions of variation. On the other hand, the use of video data makes it possible to produce and publish exemplars of coding categories, allowing members of a research community to communicate around concrete examples of behavior, rather than textual descriptions.

On the level of the video tool itself, it would be extremely valuable to automate the 'chunking' of video into codable units, such as scenes, interactions, conversations, etc. There is some progress being made along the lines by commercial products such as Virage-where the motivation is indexing and retrieval for news video, more than for research analysis-and in other research work on image and video recognition systems.

The group reflected on the plethora of video tools used by one or more of the workshop participants, including: Case Creator (Bowers), CLAN (MacWhinney), DIVER (Pea, Mills), iMovie (Frederiksen), ITMD (Integrated Temporal Multimedia Data: Hay, Hickey), LessonLab (Talley), Orion (Goldman), Teachscape (Pea, Shrader), Transana (Derry), VAST (Video Analysis Support Tool: Sherin), VideoPaper Builder (Carraher, Nemirovsky), VideoTraces (Stevens), Access Grid (Downey), Nudist/nVivo, and atlas.ti. More generally, the group developed through a survey of existing tools a list of functionalities supported by one or more tools, as follows:

1 The group produced a set of follow-up recommendations:

1. It would be productive for the research community to group tool-related resources, using the categories of functionality identified above, in order to make more apparent what tools are available for what purposes-and to consider what a more coherent approach to providing a video "workbench" for researchers and educators might be. 2. A workshop should be held to identify a common set of core codes for a specific research community, in order to develop a model for systematizing and expressing a coding scheme for the community. This would involve asking researchers to code video in advance and using common video to facilitate consensus building. 3. The workshop should draw on the expertise of an expert in translating metadata into XML, in order to guide the transition from coding schema to sharable data.

III. Access to digital video records -Dan Hickey (Reporter)

The basic issues addressed by this group were the privacy and access issues posed by digital video records of learning and teaching. Unlike textual representations of research subjects, video records potentially include massive amounts of information about children and teachers, some of it potentially embarrassing and in need of careful reviews to protect the privacy of subjects and maintain confidentiality of data when appropriate.

These issues are encountered most obviously in the context of protecting the rights of human research subjects in human subject reviews, where research involving video records often faces heightened scrutiny by Institutional Review Boards (IRBs). It is apparent that informed consent for uses of videorecords of learning and teaching can mean very different things to IRBs and their members, as different personal values come to play in judgements concerning potential risks and benefits from research participation. Many IRBs do not consider uses of research videorecords of learning and teaching problematic in their repurposing for teacher education, or uses for illustrating learning phenomena. But some instititions consider video data to be 'secondary' and require destruction of videorecords after research studies transcribe and code such data, which makes impossible any data sharing or revisitation of original source data for reinterpretation. Other IRBs require reconsideration of video data use for each and every repurposing, in that all individuals involved in the recordings must approve each new specific use of the videoclips.

The key issue in human subjects review compliance is disclosing foreseeable risks. The Human Subjects Panels of IRBs typically have oversight responsibility for the review of all University projects that involve human subjects in non-medical research to ensure that the rights and welfare of the subjects are adequately protected with informed consent review. Panel reviews commonly involve approval of a clearly worded consent form which assures that the subject (or responsible parent) is fully informed of the risks inherent in participation and of the benefits which might be reasonably expected. As the basic age for participation in research is 18 years, parental permission is requested for their child's participation in educational research.

There is currently very little precedent for identifying the range of those risks. One alternative is to identify the worst case scenario, to disclose that possibility, and to minimize the likelihood of such risks of such harms. For example, a child may become a standard example for the wrong answer to a question. Or if a specific school is identifiable in a video, it may be possible for a criminal to encounter or seek out a video on the Web and use the video for the purposes of identifying and kidnapping a child.

Correlatively, there is also insufficient common practice and understanding concerning the benefits that might be reasonably expected from uses of videorecords of learning and teaching for promoting advances in scientific understanding of education and enhanced educational practices.

The universality of Web access also leads to the question of security. One way to mitigate the risks to subjects of being included in a video is to limit access to known users, and to ensure that the video cannot circulate beyond that group (for example, by using streaming technology rather than file downloads to make the video available).

The TalkBank site, which is funded by the NSF and which maintains a large collection of audio and video data, addresses this problem by requiring users of its collection to agree to an extensive code of ethics. This code obligates anyone who uses TalkBank data to avoid criticism of individuals depicted in the materials. Anyone who violates the code of ethics is subject to community censure, although not to other sanctions. This code is an attempt to balance the privacy and other interests of research subjects with the scientific needs of researchers, and represents the judgment that personal criticism has sufficient potential for social harm that it should be banned.

Another approach that has been considered is the use of alternatives to videorecordings of situated behaviors as they occur naturally in real classroom settings, such as dramatizations and labeling of performances-as-acted that will allow illustrations of teachers and students engaged in both desirable and undesirable practices. In such circumstances, media releases rather than human subjects releases are used, since the point is not human subject research but media capture as in a play or movie that can then be shown to others. A central problem here is that the performed versions of learning and teaching may not sufficiently resemble the real thing so as to serve the educational and reflective learning purposes that are intended. Research is needed to examine this question. Furthermore, teacher educators who have worked extensively with video case studies of teaching practice commonly emphasize how crucial it is for the teachers learning from these cases that they are filmed in real classrooms, and not enacted behaviors.

Another possibility is the use of digital masks of identity, such as blurring of faces and transformation of voices to prevent identification of individuals. Workshop participants did not know of any systematic approaches to such a method that would make it commonly useful. Research would also be necessary to identify the range of research questions that could best be addressed with such masking, and which could not. For example, discourse and interactional analysis frequently relies upon information about gaze direction, joint attention, and such information in the human displays would not be available in videorecords utilizing digital masking.

The group recommended that the participants at the workshop contribute examples of approved human subject reviews to the CILT website, so that community members can see what kinds of provisions have passed muster with local IRBs.

IV. Use models for digital video in teacher education -Art Recesso (Reporter)

The key issue for this group was whether there is a common model for how digital video can be used in teacher education. The huge variation in schools of education and in school districts makes it very difficult to determine a universal use model, and hence difficult also to devise a content or service model that would be appropriate for the various contexts of use. This is particularly the case since preservice training and professional development among teachers is tied to licensure and regulatory systems that impose varying requirements on teacher education programs.

A more promising approach than producing a single body of material is to call attention to the very large volume of video that is available for repurposing, depending upon local needs. In that regard, it would be useful to survey existing tools for their suitability for repurposing existing video for use in teacher education programs. Even looking at the tools represented at this CILT workshop, there is a variety of possibilities that could be of use, depending on the needs of a program. Once repurposed (for example, by creating a set of clips and annotations using the DIVER tool) the resulting package could be available to other teacher education programs to serve as a resource and inspiration.

The group raised two questions for the future that arise from this discussion: 1. How can incentive structures be aligned to encourage sharing of video materials and repurposing of them, both within schools of education and among teachers? 2. How can an editorial function be encouraged, so that there is some guidance for teachers and teacher educators about what video is available?

Next Steps

The energies of the group assembled suggest that it would be very productive to work to define multi-institutional collaboratives that tackle complementary facets of the research and technical problems affiliated with advancing the field of using digital video for inquiry in learning and teaching. After the workshop, a new partnership was developed that, along these lines, included Stanford University (Pea), Carnegie-Mellon University (MacWhinney), and U. Georgia (Hay), who submitted an NSF-Information Technology Research (ITR) proposal in February 2003 for a Digital Video Collaboratory that was defined to be responsive to the requirements and needs specified at the CILT workshop by the research community. Other parties are discussing the potential of NSF-ROLE program submissions and Teacher Professional Continuum program grants.

We also believe that it will be of continuing value to the field for its diverse disciplinary contributors to work together in advancing uses of the CILT communityoriented website (http://cilt.stanford.edu) that has been developed for the diverse disciplines (education, psychology, anthropology, linguistics, computer science, computer vision, artificial intelligence, media studies) that use and contribute to digital video inquiry in learning and education to share their profiles, projects, bibliography, URLs, and threaded discussions. This site will be hosted on an ongoing basis by the Stanford Center for Innovations in Learning (SCIL), and promoted at AERA, EARLI, SIGGRAPH, ACM Multimedia, and other professional society meetings and publications.

There were noteworthy concerns about the need to work on a framework for defining more clearly in a way that would benefit IRB reviews and the ethical conduct of work in this emerging field the risks and benefits of repurposing of research videorecords of learning and teaching for teacher education and for multi-investigator research studies after the fact of their collection.

Finally, it would be a significant development for the learning sciences field if the recommendation of the Tools Group led to action in the community-of collaborative groups convening in the enterprise of identifying a common set of core codes for specific learning and teaching research topics, so as to develop XML models for systematizing and expressing coding schemes for specific learning sciences and teacher education research communities. Such an enterprise would involve researchers coding videos in advance and using common video to facilitate consensus building.

Appendix B -Invitation Letter

Date: Fri, 17 May 2002 Dear Friends and Colleagues, For the past four years, I have been directing a distributed Center for Innovative Learning Technologies (CILT, see http://www.cilt.org), funded by the National Science Foundation, with my colleagues John Bransford, Marcia Linn, Barbara Means and Bob Tinker. As part of our work, we have held annual workshops on vital themes in the learning sciences and technologies, such as visualization and modeling, ubiquitous lowcost computing, technology and assessment, and tools for learning communities. One aim for these workshops has been to share new developments, learning, and foster collaborative teaming in order to catalyze advances in K-14 learning technologies R&D.

In this final year, I am pleased to invite you to the CILT workshop that I am organizing as a small group, around issues concerning digital video inquiry in learning sciences research and teacher education. We consider your work on one or another of the many facets of this topic (or its technical underpinnings) to be central to our group that will work together on the issue of how we can share what we are learning and move the field forward in significant ways, as described below. This topic is one about which I have considerable personal research interests for many years, and this effort is made timely by the state of digital video and web-based technologies, research and educational needs, and an emerging community orientation to collaboration and collective advancement of the field.

Our plans grew out of explorations for the final year's work of CILT's Community Tools theme efforts at an April 2002 AERA "pre-conference." The group that began that planning effort is represented among the invitees; we are very hopeful that you can join us for the following reasons-characterized in a bit more detail in the enclosed CILT workshop proposal:

-The need is great for establishing a distributed community of researchers and developers that can "augment our collective intelligence" about how to most productively advance the theories, methods, and tools we use for capturing, annotating and analyzing, and sharing digital videorecords of learning and teaching activities.

-The work of not only the invitees but the field more broadly could benefit considerably by leveraging the insights that various research and industry groups are developing on these issues, if only we could work to develop a community-oriented website to capture and share resources, expertise, problems and solutions, as we are discovering or developing them.

What do you need to do at this point? We need you to SEND US DATES during September, October and November 2002 when you CANNOT currently make it for this event.

The current plan is to have folks arrive in time for a dinner launch of our discussions and informal sharing of our tools and platforms for doing this kind of work, and a full day afterwards of work together, with an agenda that will include brief presentations, workgroups, and reporting out and group discussion on priorities that could inform subsequent collaborative proposal writing for NSF and other agencies or foundations on the issues we consider central for moving the field and our respective research efforts forward.

I will be hosting this CILT workshop in a newly renovated building in the Stanford Quad-"Wallenberg Hall"-in which our new Stanford Center for Innovations in Learning (SCIL) is located. We will have available for our use during these meetings in our building a broad range of quite novel collaborative groupware tools and large display technologies. We believe you'd find this a compelling and valuable event, and we welcome your involvement.

[It is important to note that this invitational workshop will work as CILT workshops have throughout these past years: We will host the administrative, facilities, and food expenses affiliated with planning and running the event, but will not be able to fund the travel and hotel expenses affiliated with your participation. Our funding simply does not allow it. However, I should note that we have available a $500 honorarium affiliated with your contributions to the community-oriented website content development (not simply coming to the meeting!) -and will be in contact subsequently about the simple terms of engagement under which we would send you this check. We believe you will find this a useful and interesting thing to do -and that you will benefit from everyone else's work as well -and that it may provide an additional incentive for you to join us.)

Please send your availability ASAP (through "block-out dates" that are absolutely NOT possible) to: Debby Angus <debby@stanford.edu>. We will aim to get back to you in the next couple of weeks to resolve the meeting time so that our plans become firm. birds-of-a-feather or other thematic gatherings that can be defined "in real time" at the workshop. In particular, we recognize that there are multiple groups that have developed tools and frameworks for video capture, annotation, analysis, and reflection, and many other groups that have collected extensive video-recordings of learning and teachingthere should be ample opportunities to become acquainted with one another and to explore productive partnerships and collaborations. We also see the potential for coordination and leveraging of ongoing streams of technology or research work about which many participants may not be aware. **If we receive your video by Tuesday, November 19 th , in either cassette or digital form, we will make do the conversion work to make it readily accessible during the CILT workshop for use with any one of a number of computers running DIVER. Because of encoding time restraints and disk limitations we need to restrict your total video to 3 minutes or less (can be multiple clips or a single clip).

Desired Video Formats: NTSC video only, in any of the following formats: Cassettes (either mini-DV, Hi8, or VHS); or QuickTime and AVI files that are already encoded (on standard removable media such as a CD, Zip disk, etc.). It will be important for tapemedia for you to pre-cue your tapes to the start of the short segment you'd like digitized and specify the time duration for the clip. Also, while all submitted materials will be returned at the workshop, please do not send unique originals of video-records.

Instructions: you can ship to Joe Rosen, Attention: CILT.DVI Project, Stanford University, Stanford Center for Innovations in Learning, Wallenberg Hall, 450 Serra Mall, Building 160, Stanford CA 94305-2055 (Cell Phone: 415-505-0853). For questions, contact: Joe Rosen joro@Stanford.EDU.

*COMMUNITY-ORIENTED WEBSITE*

Since the workshop was conceived in Spring 2002, we have worked at Stanford to develop the framework for a community-oriented website that we wish to develop further before the workshop, and make broadly available to the research community after our workshop.

In my view, this effort is made timely by the state of digital video and web-based technologies, research and educational needs, and an emerging community orientation to collaboration and collective advancement of the field.

But it is exceptionally hard to track new developments in the pertinent and nonoverlapping subfields that can contribute to the highest quality research and development work in digital video inquiry for learning and education. We are devoted to making the effort -with your help -to build community knowledge in an ongoing manner to overcome this challenge, and to hosting and maintaining the website.

*REQUEST FOR YOUR ACTION*

We have built the first release of the CILT DVI website, and you need to go there on the web and register (your password will show up within 24 hours):

The URL is http://cilt.stanford.edu What you will need to do is very simple, and it will be of benefit to all of us at the workshop (and in the future, to those outside the group participating in late November):

(1) Establish your Member Profile in the Member Directory:

On the home page, once you are registered, click on My Account on the left side of the home page and, after selecting Edit My Member Page and Information, fill out your profile, webpage, and interests data fields.

(2) Post select items -especially YOUR OWN -in the Library:

We have already populated the community Library with over 200 items, including scientific papers (.pdf, .doc or URLs), technical reports, product website URLs, research lab URLs, and so on. This is simply a start, and we are sure your own work and the references and sites you considerable most important are not yet wellrepresented. It is very simple to add items.

To see if an item is already in the Library, you can search at the top of the page for the Library (http://cilt.stanford.edu/library.php). All items have also been "tagged" with one or more as exemplars of a set of working categories that we defined in an earlier planning workshop, and category definitions are described at (http://cilt.stanford.edu/category_about.php). Items will often have multiple categories, and we encourage you to use them.

To add an item, simply click the Post a Document link on any library page. You can either Browse your hard-disk to upload a file or Post from the Web by typing in a URL for a site or file location. Cut/Paste an Abstract if you have one, and click on the appropriate categories for the Document Information. There is another button to click if are the Author of the document. Select Post Document and you are done -> any registered website participant will now be able to search, find, and download the Library document, or go to the web link you have provided! NOTE: We expect to award an honorarium of $500 to the ten most prolific new contributors to the workshop website by the time that we convene on November 25 th . (Your graduate students and staff can help.)

*TECHNOLOGY DEMOS*

One of the most effective mechanisms for CILT workshops has been for researchers to illustrate their work during demonstration sessions, and our schedule allows for several different multi-hour sessions when you can show your work in parallel with others.

While we are not able to have plenary talks from all participants in the interest of discussion and breakout group activities, we encourage you to demonstrate your work.

We will have available a large number of tables with chairs next to power switches and Ethernet connections where you can set up your demo on Sunday night, November 24 th , from 6-10 pm. LET US KNOW RIGHT AWAY IF YOU PLAN TO SET UP A DEMO AND WHETHER YOU WILL ALSO BE BRINGING A POSTER AS A BACK-DROP FOR THE DEMO (Email to Debby Angus debby@stanford.edu).

We know that some individuals/collaborative groups will show tools for digital video analysis or case studies as well as content, while others will show specific digital video collections, for example on a DVD. Please bring the computer that you normally use to demo your software or video-records.

*NEXT STEPS*

• Please register as a member at the CILT.STANFORD.EDU website.

• Provide new Library items -including your own work -to this communityoriented website.

• Select 3 mins of video to send for digital video tool exploration at the workshop.

• Notify Debby Angus of your technical needs for your demos and whether you will also bring a poster for display next to your demo.

• Stay tuned for later emails on parking,

We will also be in touch on other practical details for the workshop in the weeks ahead. We are very much looking forward to meeting with you and developing common ground and exploring the future for digital video inquiry in learning and education.

Sincerely yours, Roy Pea

I

  am really looking forward to this event. I hope you can join us! Sincerely yours, Roy Pea Professor of Education and the Learning Sciences (http://www.stanford.edu/~roypea) Director, Stanford Center for Innovations in Learning and Director, Center for Innovative Learning Technologies

  ***HANDS-ON WORK with YOUR DIGITAL VIDEO*** With NSF funding for tool development in support of a Digital Video Collaboratory, our team at Stanford has developed what we call DIVER (Digital Interactive Video Exploration and Reflection), an environment for data capture, analysis, and web-sharing of digital video.As one element of this workshop, we are inviting you to send us pre-selected video of learning or teaching that you have a strong interest in exploring or showing in digital form during the workshop.

*PURPOSES*

Like the many other previous workshops that have been sponsored by the NSF-funded Center for Innovative Learning Technologies (http://cilt.org), our aim is to establish a distributed community of researchers and developers that can "augment our collective intelligence," in this case, about how to most productively advance the theories, methods, and tools we use for capturing, annotating and analyzing, and sharing digital videorecords of learning and teaching activities.

• One of our activities will be building up a collective understanding of the state-ofthe-art and emerging priorities to synthesize in a subsequent report to the National Science Foundation.

• A second will be working to launch a community-oriented website on digital video inquiry in learning and education (about which more below).

• A third will be exploring hands-on demonstrations of many software and hardware tools and collections of digital video data.

• A fourth will be joining in discussions of potential new collaborations toward the establishment of "digital video collaboratories" for learning and teaching.

*GROUP EXPERTISE and WORKSHOP SPIRIT*

The group we have assembled has an exceptional range and depth of experience in using digital video for research in student learning and teacher learning. Many of you will be bringing along technology systems you have developed to support your work, and/or video case studies, profiles, or video data analyses that you would like to share. We are especially keen to enable new collaborations to emerge from the workshop, and have arranged for much of the time to be devoted to demonstrations, informal discussions, and


Roy Pea

Michael Mills

Joseph Rosen

Kenneth Dauber

Wolfgang Effelsberg

Eric Hoffert Versatility

The Diver Project: Interactive Digital Video Repurposing

Keywords:

The Digital Interactive Video Exploration and Reflection (Diver) system lets users create virtual pathways through existing video content using a virtual camera and an annotation window for commentary. Users can post their Dives to the WebDiver server system to generate active collaboration, further repurposing, and discussion. W ith the inexorable growth of low-cost consumer video electronics and broadband networking, communicationcentric, symmetric video-point-of-view sharing will eventually supplant broadcast-centric and asymmetric approaches to video content and repurposing. In sheer volume, the use of existing video to share perspectives will likely far outstrip video repurposing for commercial purposes and for different device platforms.

Thus, automatic, real-time repurposing of content designed for one technical environment configuration (a device or network, for example) to fit other technical environments is but one, albeit important, facet of the content repurposing problem. Another fundamental issue in content repurposing is user-driven content reuse, prompted by users' desire to reinterpret content and to communicate and share their interpretations.

The Digital Interactive Video Exploration and Reflection (Diver) system provides facilities for dynamic time/space cropping and annotation of archival video to support digital video repurposing. Diver lets users create point-of-view video tours of original content in a way that supports sharing, collaboration, and knowledge building. Diver works like a video camera: a user points, pans, tilts, and zooms with the virtual viewfinder to create an infinite number of perspectives on a video record. Although other digital video editing and effects tools offer similar repurposing features (Adobe Premiere's zoom transition effects and cropping and Adobe After Effects' side-byside group clip compositing, for example), none offer a comparable intuitive and easy-to-use interface, and few can infinitely repurpose audio and video without forcing the user to wait through a rendering process.

Diver overview

Much of our work on Diver involves recording human activities such as teachers and students in a classroom or research group meetings using a conventional or panoramic digital video camera. Such scenarios involve a broad range of complex human interactions of interest to researchers in the learning and social sciences, such as meeting behaviors or instructional discourse.

A video-analysis phase follows the videorecording phase. [START_REF] Jordan | Interaction Analysis: Foundations and Practice[END_REF] Using a mouse, users can zoom, pan, and tilt a virtual camera on an overview of the source video. The virtual camera dynamically crops still image clips, or records multiframe video pathways through the panoramic video to create a Diver worksheet. A Dive consists of a set of reorderable panels (inspired by VideoNoter), 2 each containing a key frame or thumbnail representing a clip, and a text field that can contain an accompanying annotation. Source video can support an unlimited number of Dives.

After creating a Dive, a user can upload it to WebDiver, a Web site for interactive browsing, searching, and display of video clips and collaborative commentary on Dives. Diver automatically packages the Dive for WebDiver as an XML document with associated media files. The Dive's Web representation includes the key frames, annotations, and video clips. Colleagues can thus share Dives over the Internet.

Figure 1 gives a schematic representation of the recording, Diving, and Web-sharing phases. pervasive, they often serve as Diver's source material. However, it's difficult to use such video records in social science research. The videographer's choice of what to include (and what not to include) limits the usefulness of videos recorded with conventional digital cameras not only for the original researcher, who can't recover events relevant to the analysis occurring off camera, but also for other researchers who might be interested in different aspects of the filmed event.

By supporting panoramic video, Diver avoids this problem. Panoramic video has been used in teleconferencing [START_REF] Kimber | FlyAbout: Spatially Indexed Panoramic Video[END_REF] and in surveillance, or as one component in an immersive simulation of an event, such as a football game or concert. [START_REF] Pintaric | Immersive Panoramic Video[END_REF] We argue that panoramic video is of particular interest in all settings where the video material's purpose and application isn't precisely defined or known at recording time.

Video recording system

Image resolution is a major technical challenge in recording panoramic video. For a user to zoom into an area of the panoramic video and still see sufficient detail, the camera must capture the video at the highest possible resolution.

In principle, a single-lens system can record panoramic video: a spherical mirror, somewhat similar to a fisheye lens, collects the light from 360 degrees and redirects it onto a camera's charge-coupled device (CCD) chip. The system can then store, compress, and edit the camera's digital video stream like any other video stream. Such systems are relatively inexpensive and readily available. Examples include the BeHere camera (http://www.behere.com) and the camera used in Ricoh's Portable Meeting Recorder. [START_REF] Lee | Portable Meeting Recorder[END_REF] However, single-lens systems produce low-resolution images: the number of pixels on the CCD chip, designed for a small angle, must now suffice for surround vision-that is, a much wider angle. Consequently, a user zooming in on an area in the video will see a low-quality image.

Several recent multilens systems-FullView (http://fullview.com), Fuji's FlyCam, [START_REF] Sun | Panoramic Video Capturing and Compressed Domain Virtual Camera Control[END_REF] and Microsoft's RingCam, [START_REF] Cutler | Distributed Meetings: A Meeting Capture and Broadcasting System[END_REF] for example-address this problem. They capture scenes using multiple lenses and CCDs with slightly overlapping fields of vision and stitch the frames together to construct panoramic images. However, these systems introduce a new challenge: stitching and dewarping multiple frames in the computer, perhaps in real time. Multilens systems running at full resolution generate a very high bit rate, often too high to be transferred over a single peripheral component interconnect (PCI) bus.

Diver uses a FullView camera system with five mirrors and five cameras, and this system feeds the camera streams to a PC where FullView software stitches and dewarps the video streams for live preview (at a reduced frame rate and/or reduced resolution, and always without sound). The FullView system uses three parallel PCI buses on the PC processing the streams to achieve such a high data transfer rate.

Because of these limitations, we've developed a tape-based recording system capable of recording at the full resolution and full frame rate of NTSC-DV (digital video), which is desirable for repurposing panoramic video (that is, 720 × 480 × 5 cameras, at ~30 fps). A system for capturing multichannel sound with a microphone array generates corresponding audio.

To ensure compatibility with 4:3 aspect-ratio video and panoramic video content at a variety of aspect ratios, Diver supports a general approach to video analysis, collaboration, and interaction, allowing for a diversity of aspect ratios and resolutions for recorded video material. This provides a flexible approach for recording, analyzing, and sharing a broad range of video material independently of the specifics of the captured video material.

User interface

The Diver software consists of the Diver desktop application and the WebDiver server system. The desktop application is a native Windows application coded in C++ that builds on Apple's QuickTime tools and uses the QuickTime video format and QuickTime browser plug-in to display and control the video.

The design rationale for Diver's user interface is rooted in cognitive science and human-computer interaction research. Our goal for the tool is to augment the fundamental and social activity of human visual interpretation and communication about what's interesting or relevant to a given purpose-namely, looking at a complex scene, noticing events, and commenting on the focus of attention.

The Diver interface consists of three distinct regions, each corresponding to an element in the look-notice-comment cycle, as Figure 2 illustrates.

The overview window (look region) provides access to the original source material and provides standard video controllers.

The magnified viewing window (notice region) shows the part of the original scene that the user selects by dragging and resizing a rectangle overlay (the virtual camera) across the overview window, panning over and zooming into regions of interest.

This overview/magnified viewing interface aims to satisfy the well-known dual-awareness principle of human-computer user interface design: it lets the user zoom in on details of an information source while maintaining orientation and context. [START_REF]Readings in Information Visualization: Using Vision to Think[END_REF][START_REF] Mills | A Magnifier Tool for Video Data[END_REF] Other researchers have explored related navigational interfaces in experimental [START_REF] Teodosio | Panoramic Overviews for Navigating Real-World Scenes[END_REF][START_REF] Prihavec | User Interface for Video Observation over the Internet[END_REF][START_REF] Rui | Viewing Meetings Captured by an Omni-Directional Camera[END_REF] and commercial applications of panoramic video technology (FullView and BeHere, for example).

As a user drags the virtual camera rectangle across the video source overview, the viewing window gives a dynamically updated and magnified view of the circumscribed region. Two buttons make the viewing selection a flexible authoring tool. Mark takes a temporally tagged snapshot of the viewing window contents and automatically creates a new panel in the screen's third region: the Dive worksheet (comment region). In addition to a video thumbnail representing a marked frame or clip, a worksheet panel contains a text field in which users can describe the selection's noteworthy characteristics.

Pressing the record button creates a new panel. At the same time, moving the selection rectangle across the overview window creates a recording of the spatiotemporal selection shown in the viewing window. Thus the user can record a dynamic path through the original source video. Pressing the record button again ends the recording. The user can annotate the recorded pathway clip by entering text in the Dive panel.

In creating a unique point-of-view virtual tour of the original video source, the Diver user creates a Dive. The Dive can serve as a storytelling device for playing back both the marked locations and the dynamically cropped pathway recordings through the original video source and the records' annotations.

Panels contain the data elements constituting a Dive (see Figure 2): ❚ static and dynamic time and space video markers-a static marker points to a single frame in the video (temporal coordinate) and the location of the virtual viewfinder in that frame (spatial coordinates); dynamic markers result from recording the virtual camera's time/space path as the user guides it through the source video.

❚ thumbnail representations of time and space video markers-thumbnail-sized images copied from the source video at the marked time and space.

❚ annotation text-the commentary associated with the marker (that is, the text the user typed into the panel). ❚ time/space cropped video clips-thumbnails in a panel that also symbolize portions of the source video. Double-clicking a thumbnail (or dragging and dropping it back into the viewing window) repositions the source video at the corresponding time/space coordinates.

IEEE MultiMedia

Full-resolution panoramic video recorder

Diver's user interface supports panoramic video. As Figure 3 demonstrates, the user gets a 360-degree horizontal view of the unfolding scene while the video plays as a "peeled back" cylinder with a wide-aspect ratio.

To accommodate even higher resolution video capture than native FullView live preview and direct-to-disk video can achieve, we developed an alternative full-resolution video tape-based recorder. Our general-purpose Diver recorder system captures the FullView camera array output at full resolution (720 × 480 × 5) and full frame rate (~30 fps) to five DV tape recorders (VTRs). The Diver recorder system later transfers select video tape segments to a PC, where our software reassembles in postprocessing the multiple recordings into seamless full-resolution panoramic movies encoded in QuickTime and ready for Diving.

Virtual path movies

The Diver file format supports the efficient creation of Dives. As a user drags the virtual camera viewfinder across the source overview window, Diver records time (the video time) and space (a viewing region's position within the video's border) references, as Figure 4 shows. The virtual viewfinder in Figure 4 is rectangular, but it can also be circular or any selection shape. Marking parallel video regions of interest using multiple concurrent pathways with the appropriate user interface is also feasible.

What's recorded when the user presses the record button is not a new video; rather, it's the digital video collaboratory (.dvc) file information needed to replay the dynamically cropped pathway that the virtual camera traced through the original video frame sequence.

When a user plays back a .dvc file, Diver resets the current video time and space to the previ-ously saved coordinates. Before displaying each frame of video in the virtual camera window, Diver scales the video's borders (in real time, onthe-fly) to match only the viewing area coordinates associated with the same frame (video time) in the recorded virtual camera path.

Using the .dvc list recording scheme instead of rendering new video clips has two significant advantages: The structure and extensibility of the XMLbased .dvc file format lets users reformat Dives into many display and interaction environments. For example, on a lightweight device supporting only text and image display, a user could extract and show just the text annotations and video image thumbnails.

❚

Infinite panoramic audio mixing

The Diver panoramic recording system captures up to 10 independent audio channels, each mapped to an associated field of view captured by the FullView camera. (Near-term enhancements will incorporate 16 or more independent audio channels into the panoramic recording array.) Diver embeds these multichannel sound recordings as navigable QuickTime audio tracks in the postprocessed panoramic movies. When playing back the recordings, Diver can steer panoramic sound so only the tracks associated with the virtual viewfinder's current focus become audible. Thus, Diver users can easily repurpose audio recordings and dynamically create infinite virtual audio mixes.

WebDiver server system

Users can upload their Dives to a WebDiver server where others can share them, promoting dialogue and collaboration around video clips. Transforming content for the Web occurs in a series of client-and server-side automated tasks invisible to the user. The user chooses the Export to WebDiver menu option in the desktop Diver application, initiating text, image, and video-clip transformation processes.

The Diver system formats and transfers annotation text, time, and space markers and other metadata associated with each Dive (file names, file sizes, and so on) to the WebDiver server in an XML format. At the server, PHP scripts and a MySQL database store and retrieve Dive content for delivery to client browsers as dynamically generated and JavaScript-enhanced HTML pages. Diver transforms thumbnail images from video bitmaps to JPEG files.

Because QuickTime plug-ins don't support real-time playback of time/space cropped movies, accommodating dynamic playback of such movies in Web pages across multiple platforms and browsers would require a custom-coded plug-in for each of the most popular operating systems and browsers. As a possible cross-platform alternative, we experimented with a Diver Java applet architecture, but found that QuickTime for Java in its current implementation isn't robust enough to support time/space cropped movies. Rather than write our own custom plug-ins, we rendered all Dive video into standalone QuickTime video clips. Users can play the videos directly from the WebDiver Web site using standard QuickTime players.

During the video clip export process, Diver uses the .dvc file to trace the virtual camera's path through the time/space cropped video. It rebuilds the QuickTime clips frame by frame by copying time/space regions of the source video into the new standalone files. Diver also converts the original video source file to QuickTime in a compressed format of smaller geometry. These conversions constitute a technical video repurposing step in Diver.

Figure 5 shows the XML schema for a Dive. When the desktop Diver finishes the transformation, the application automatically opens the user's Web browser and dynamically generates and displays a new Web page containing a file transfer and HTML form frameset. Submitting the form completes the Dive transmission (represented by the XML file containing Dive metadata, image thumbnails, and automatically generated video files) from the desktop to the Web server.

When a user later opens a Dive on the WebDiver site, shown in Figure 6, Diver reconstitutes the content as HTML and JavaScript, dynamically generating the Web pages from the media files and XML data. WebDiver pages in the browser look similar to desktop Dives and contain the same data and media elements.

In on Dives by adding comments to the panels in the manner of a threaded discussion. WebDiver users can also perform keyword and metadata searches across all annotations.

Diver usage scenarios and implications

Multimedia records are an increasingly important type of data for researchers in many fields because they capture the complexity of "mind in context"-human interactions and behaviors in situations such as the classrooms and the home. Researchers today lack tools for sharing or discussing these data readily with fellow scholars and practitioners. Diver seeks to bridge this gap, enabling collaborative analysis of learning and teaching video records in a distributed community of researchers and practitioners.

For example, social scientists can select and annotate video clips of human behavior from the workplace, home, or learning settings, and share their analyses with distant colleagues. They can investigate areas in which video has proven useful, such as patterns of student participation in instructional discourse, uses of diagrams, small group collaboration, and teaching. [START_REF] Barron | When Smart Groups Fail[END_REF][START_REF] Frederiksen | Video Portfolio Assessment: Creating a Framework for Viewing the Functions of Teaching[END_REF][START_REF] Lampert | Teaching, Multimedia and Mathematics: Investigations of Real Practice[END_REF][START_REF]The Power of Video[END_REF] Used diagnostically, Diver can tease out differences in how novices and experts notice social, behavioral, and physical phenomena in video. Researchers can use Dive panels as structured templates for assessing trainees in various fields; they can direct users to particular Diver clips and ask them to answer specific questions. Local or remote experts can provide critiques as threaded comments in WebDiver.

For training purposes, experts in a domain could construct Diver videos from Dives created to serve as think-aloud reflective commentaries, offering "guided noticing" help for novices.

More generally, Diver is relevant to any human activity domain with dynamic or static media in which community engagement in analysis and education using those media might bear fruit. For example, virtual classroom visits are increasingly used in teacher education for professional development (Teachscape.com, Lessonlab.com, and Teachfirst.com, for example).

To ensure that Diver reflects user needs, and to introduce Diver into relevant user communities, we recently held a design kitchen in the learning and social sciences departments at Stanford University. Faculty and graduate students used Diver to examine their own video data while the Diver team observed their use of the tool and recorded their reactions. The initial user reaction was positive, with several research teams already incorporating Diver into their work with digital video data-for example, in studies of collaborative learning in mathematics, group discussions of text comprehension, uses of diagrams in physics education, and teacher preparation.

We're working on a pilot project with several California universities in which K-12 teachers use Diver to document and reflect on their teaching practices in terms of specific quality rubrics for planning, instruction, assessment, and reflection.

Outlook

Although our current work focuses on video records in learning research and educational practices, Diver can aid collaborative analysis of a broad array of visual data records, including simulations, 2D and 3D animations, and static works of art, photography, and text. In addition to the social and behavioral sciences, substantive application areas include medical visualization, astronomic data or cosmological models, military satellite intelligence, and ethnology and animal behavior.

Diver-style user-centered video repurposing might also prove compelling for popular media with commercial application involving sports events, movies, television shows, and video gaming.

Future technical development includes possible enhancements to the interface to support simultaneous display of multiple Dives on the same source content, a more fluid two-way relation between desktop Diver and WebDiver, and 59 January-March 2004 solutions to the current limitations on displaying and authoring time/space cropped videos in a browser context. These developments support the tool's fundamentally collaborative, communication-oriented nature. MM

Figure 1

 1 Figure 1. Overview of the Diver video exploration and reflection system.

Figure 2 ❚

 2 Figure 2. Diver user interface. The overview window (bottom left) shows the full video source. The magnified viewing window (upper left) shows a selected image from the scene. The annotation window, a.k.a. Dive worksheet (right) lets users comment on the frames or path movies they create.

Figure 3

 3 Figure 3. Panoramic video as input for Diver. The user gets a 360-degree view of the scene around the camera's location.

Figure 4 .

 4 Figure 4. Graphical representation of the .dvc file format.

Figure 5

 5 Figure 5. Dive XML schema. A <panel> data block is written for each panel in the Dive worksheet.

Figure 6 .

 6 Figure 6. A Dive from the WebDiver site, as it appears in a browser window.

  addition to the Dive's author, other WebDiver users can explore the full source video used to create it. Moreover, users can collaborate

	<dive id= owner=>
	<name></name>
	<annotation></annotation>
	<video type= id=>
	<video_file bytes=></video_file>
	</video>
	<panel id= order=>
	<annotation></annotation>
	<video type=>
	<time_in></time_in>
	<time_out></time_out>
	<source_video></source_video>
	<video_file bytes=></video_file>
	<thumbnail_file
	bytes=></thumbnail_file>
	</video>
	</panel>
	</dive>
	IEEE MultiMedia
	58

Acknowledgments

Diver, WebDiver, Dive, and Guided Noticing are trademarks of Stanford University for our software and affiliated services, with patents pending. This work was supported by the US National Science Foundation (grants 0216334 and 0234456) and the Hewlett Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of its funders.

We used a QuickTime for Windows-based panoramic video postprocessing application, developed from camera calibration tables provided by the manufacturer (FullView.com), and with the programming assistance of Dirk Farin, a visiting scholar from the University of Mannheim, Germany. Bob Smith, SCIL's director of technology services, made significant design contributions to the Diver panoramic work as did Dan Nelson, who contributed to the panoramic sound recording process. Thanks also to Ulrich Neumann and the team at the University of Southern California's Integrated Media Systems Center for sharing their expertise and experiences with panoramic video recording.

This new journal will emphasize the algorithmic, mathematical, statistical and computational methods that are central in bioinformatics and computational biology including…