Pierre Dillenbourg
email: pierre.dillenbourg@epfl.ch

Fabrice Hong
email: fabrice.hong@epfl.ch

The mechanics of CSCL macro scripts

Keywords:

Macro scripts structure collaborative learning and foster the emergence of knowledge-productive interactions such as argumentation, explanations and mutual regulation. We propose a pedagogical model for the designing of scripts and illustrate this model using three scripts. In brief, a script disturbs the natural convergence of a team and in doing so increases the intensity of interaction required between team members for the completion of their collaborative task. The nature of the perturbation determines the types of interactions that are necessary for overcoming it: for instance, if a script provides students with conflicting evidence, more argumentation is required before students can reach an agreement. Tools for authoring scripts manipulate abstract representations of the script components and the mechanisms that relate components to one another. These mechanisms are encompassed in the transformation of data structures (social structure, resources structure and products structure) between script phases. We describe how this pedagogical design model is translated into computational structures in three illustrated scripts.

Introduction

When two students work together, one often focuses on low-level aspects of the task while the other spontaneously pays more attention to metacognitive aspects [START_REF] Miyake | Constructive interaction and the iterative process of understanding[END_REF][START_REF] O'malley | Understanding explanation. Cognitive science research report[END_REF]. As collaboration proceeds, these roles may shift, with the regulator becoming the regulated and vice-versa. Under certain conditions, this mutual regulation process is internalized by the students, giving rise to the development of self-regulation skills [START_REF] Blaye | Collaboration as a facilitator of planning and problem solving on a computer based task[END_REF]. This phenomenon has been observed by psychologists and can be converted into a pedagogical method by explicitly assigning each of the two roles to student peers. In the reciprocal tutoring method, for instance, one student reads a paragraph and the second asks comprehension-monitoring questions before roles are switched for the next paragraph [START_REF] Palincsar | Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities[END_REF]. This turn-taking mechanism could further be translated into a physical artifact. Imagine a table in the form of a see-saw commonly found in children's playgrounds (Fig. 1). When the student who is at the lower end of the see-saw is able to manipulate the objects on the table, the second student is automatically at the higher end and has a more global view. This table mechanism translates the pedagogical design into a physical event: if one student is carrying out the task, the other is automatically forced to adopt a regulatory role.

This example is expandable to other pedagogical methods. For instance, socio-cognitive conflict can be triggered by placing students at two orthogonal viewpoints of a pyramid [START_REF] Doise | The social development of the intellect[END_REF]. These psychological studies lead to pedagogical methods which trigger epistemic conflicts among team members. This mechanism can also be adapted to the table example: The solid axis between chairs could be formed in such a way as to force students to adopt perpendicular viewpoints.

The pedagogical methods addressed in the following sections rely on mechanisms that induce specific interactions in the same way that the mechanics of the two table examples would influence interactions. While pedagogical mechanisms are usually described by verbal principles, they can be translated into physical mechanisms (as in the table examples above) and of course into computational mechanisms. The purpose of the present paper is to describe these rather simple computational mechanisms.

Why scripting?

Collaborative learning is not always effective; its effects depend on the richness and intensity of interactions engaged in by group members during collaboration [START_REF] Dillenbourg | The evolution of research on collaborative learning[END_REF]. Learning outcomes are related to the emergence of elaborated explanations, the negotiation of meanings, the quality of argumentation structures and the mutual regulation of cognitive processes. The holy quest of CSCL is to establish environments that directly or indirectly favor the emergence of rich interactions. This is commonly referred to as 'design for conversations' [START_REF] Roschelle | Designing for conversations[END_REF] or 'designing interactions' [START_REF] Jermann | From mirroring to guiding: A review of the state of art technology for supporting collaborative learning[END_REF][START_REF] Dillenbourg | Basics of computer-supported collaborative learning[END_REF]. Many non-exclusive approaches exist by means of which a CSCL environment can directly or indirectly shape group interactions, namely:

1. By designing a communication tool, for instance semi-structured interfaces, that proposes predefined speech acts in the form of buttons or sentence openers [START_REF] Baker | Flexibly structuring the interaction in a CSCL environment[END_REF][START_REF] Veerman | Software for problem solving through collaborative argumentation[END_REF][START_REF] Soller | Supporting Social Interaction in an Intelligent Collaborative Learning System[END_REF]); 2. By shaping (graphical) representations of the task and the objects to be manipulated by students [START_REF] Roschelle | Designing for conversations[END_REF][START_REF] Suthers | Representational bias as guidance for learning interactions: A research agenda[END_REF];

Fig. 1 A pedagogical method translated into a physical mechanism 3. By forming groups in a specific way [START_REF] Wasson | Identifying coordination agents for collaborative telelearning[END_REF][START_REF] Hoppe | Can analytic models support learning in groups?[END_REF][START_REF] Inaba | How can we form effective collaborative learning groups?[END_REF][START_REF] Muehlenbrock | Learning group formation based on learner profile and context[END_REF]Wessner and Pfister 2001) 4. By providing team members with a representation of their interactions in order to foster regulation at the group level [START_REF] Dillenbourg | The socio-cognitive functions of community mirrors[END_REF][START_REF] Jermann | Group mirrors to support interaction regulation in collaborative problem solving[END_REF]) 5. By providing feedback on the quality of group interactions [START_REF] Mcmanus | Monitoring computer-based problem solving[END_REF][START_REF] Inaba | Development of the intelligent discussion support system for collaborative learning[END_REF][START_REF] Ayala | A collaborative learning environment based on intelligent agents[END_REF][START_REF] Barros | Analysing student interaction processes in order to improve collaboration. The DEGREE approach[END_REF][START_REF] Constantino-Gonzalez | A coached collaborative learning environment for Entity-Relationship modeling[END_REF]. See also various similar approaches as reviewed by Jermann et al. (2001) 6. By scripting the collaboration process using specific phases, roles and activities.

The term 'script' has been used to refer to two approaches that share the same goal but are nonetheless different:

-Micro-scripts are dialogue models, mostly argumentation models, which are embedded in the environment (Approach 1 above) and which students are expected to adopt and progressively internalize. For instance, a micro-script may prompt a student to respond to the argument of a fellow student with a counter-argument [START_REF] Weinberger | Fostering computer supported collaborative learning with cooperation scripts and scaffolds[END_REF]). -Macro-scripts are pedagogical models, i.e. they model a sequence of activities (a combination of Approaches 3 and 6 above) to be performed by groups. For instance, argumentation can be triggered by collecting students' opinions and pairing students with conflicting opinions.

The terms micro/macro refer to the granularity of the prescribed actions: a four-second turn of dialogue in micro-scripts versus a 4-h task in macro-scripts. A more important difference between micro and macro scripts is the fact that, in contrast to pedagogical models (macro scripts), dialogue models (micro scripts) are expected to be internalized. The present paper focuses on macro-scripts.

Examples of CSCL macro-scripts

Three examples of macro scripts developed by the present authors will be presented in the following. While the first two scripts (ArgueGraph and ConceptGrid) have been multiply tested and are integrated in a script authoring environment (http://manyscripts.epfl.ch), the third script (WiSim) has not yet been tested in a real context.

ArgueGraph

The aim of ArgueGraph [START_REF] Jermann | Dialectics for collective activities: An approach to virtual campus design[END_REF] is to trigger argumentation between peers. This is achieved by collecting opinions and forming pairs of students with opposite opinions. An experiment showed that conflicts led pairs to produce justifications that were not proposed by any peer in the individual phase [START_REF] Jermann | Elaborating new arguments through a CSCL scenario[END_REF].

Phase Level Activity 1

Individual Each student responds to an on-line multiple choice questionnaire. These questions do not have right or wrong answers, but rather reflect different viewpoints. These viewpoints are to be addressed by the teacher in Phase 4. For each answer, the student is expected to write a few lines justifying his or her choice. Class This debriefing session aims to reformulate the elements mentioned by the students using the correct terminology, to structure them and to integrate them into a theoretical framework. The teacher synthesizes the arguments provided by the students (Fig. 2, right), asks them to provide further clarification, rephrase their justifications, compare them and so forth. 5

Individual Each student chooses one of the questions discussed and writes a summary of all arguments collected by the system in Phases 1 and 3, structured according to the framework developed in Phase 4.

Similar approaches have been developed by pairing students with different hypotheses in an inquiry-based learning environment (Gijlers and de Jong 2005) (Fig. 2).

ConceptGrid

ConceptGrid is a sub-class of the JIGSAW scripts [START_REF] Aronson | The jigsaw classroom[END_REF]. Team members acquire complementary knowledge by reading different papers. The concept grid can only be constructed when each member explains the concepts about which he/she has individually read (Fig. 3).

Phase Level Activity 1 Group

The group distributes the roles among its members. Each role is associated with a few papers to read. 2

Individual Each student reads the papers associated with his or her role. 3 Group The group distributes the concepts to be defined among its members. 4

Individual Each student enters a 5-10-line definition of the concepts (s)he has been assigned. 5

Group The group constructs a concept grid, i.e. concepts are ordered on a map in such a way that two neighbouring concepts can be explained in just a few sentences (Fig. 3). 6

Class This debriefing session aims to reformulate the definitions and relations provided by the students, to structure them and to integrate them into a theoretical framework.

Fig. 2 ArgueGraph: Left the opinions map in Phase 2; Right the tool collects all answers for debriefing (Phase 4)

WiSim

WiSim addresses two pitfalls of enquiry-based learning (De Jong and Van Joolingen 1998): the lack of explicit negotiation of parameters (Phase 3) and the difficulty of interpreting the representations of results (Phase 6). Script software is run on mobile phones and exploits the fact that mobile devices are often exchanged among teenagers. WiSim is designed for co-presence situations: co-located groups of students interact face-to-face and use their mobile phone as tiny computers to support their activities.

Phase Level Activity 1 Class The teacher conducts an introductory lecture providing background knowledge on the phenomenon to be simulated. 2 Group Students physically form groups in the class. One group member creates the group in the database via his/her phone application and the other members join by entering the name of the group to which they belong on their phone. 3 Group Team members negotiate the values of the parameters which are to be entered by each member. 4

Individual Each team member sends his or her parameter values to the simulation using his/her mobile phone (Fig. 4). 5

Individual

Script concepts

Despite the first "C" in CSCL, the scripts presented above are not restricted to computerbased activities, but integrate a variety of mediated or face-to-face activities across different spaces. The scripts are predominantly used in co-present settings and aim to enrich traditional university teaching through collaborative learning activities (Fig. 4). Despite the second "C" in CSCL, these scripts are not purely 'collaborative', but include both individual activities such as reading a paper or writing a summary, and collective (class-wide) activities such as debriefing sessions. We do not see any reason to develop pedagogical methods which exclusively rely on group activities. Individual reflection is required in order to transform experience into learning and class-wide activities are especially valuable when it comes to structuring the informal knowledge that emerged in previous phases. Figure 5 depicts the distribution of activities across these three social planes for the script instances presented above.

In other words, the scripts presented are neither purely computerized, nor purely collaborative. We termed this two-fold extension 'integrated scripts'; the scripts integrate several activities distributed across multiple places and different social planes (individual, collaborative, collective) within a single workflow. This broadened script concept places the teacher back at the centre of activity, not for lecturing, but rather for "orchestrating" [START_REF] Randolph | Images of management in a learner-centered classroom[END_REF][START_REF] Dillenbourg | Basics of computer-supported collaborative learning[END_REF] script activities. Hence, our script environment includes features that enable the teacher to follow student activities in real time and, if necessary, modify the script, for instance by changing groups or deadlines [START_REF] Dillenbourg | Flexibility in macro CSCl scripts[END_REF].

Authoring scripts

The similarities among the script examples presented so far and the scripts developed by our colleagues (see Acknowledgments section) raise the expectations that a large number of scripts could be produced simply by combining basic script components. This led a group of European scholars to describe scripts using a set of components and operators [START_REF] Kobbe | Specifying computer-supported collaboration scripts[END_REF]. Components describe the sequence of activities in an abstract way, as in any authoring language or educational meta-data. A number of script authoring tools have been developed over the last few years and have encountered growing interest, e.g. CeLs [START_REF] Ronen | Structuring, sharing and reusing asynchronous collaborative pedagogy[END_REF]) and LAMS (http://www.lamsinternational.com). The Argue-Graph and ConceptGrid scripts are embedded in an authoring environment that enables the designer to edit the contents of the script and to tune certain parameters such as group size (in ConceptGrid). This environment (manyscripts.epfl.ch) differs from that of CeLs or LAMS: it has a low level of abstraction, only enabling the editing of scripts that are predefined, but it implements the scripting mechanisms addressed in this paper. ManyScripts handles three levels of abstraction [START_REF] Dillenbourg | Designing integrative scripts[END_REF], presented here in descending order:

-Scripts. ManyScripts stores scripts such as ArgueGraph or ConceptGrid independently of content. -Script instances. The designer may select a script and provide it with specific content (questions in ArgueGraph, roles, concepts and readings in ConceptGrid), producing an instance, such as "ConceptGrid-on-Knowledge-Management". -Script sessions. To run a script instance with a specific class of students, the teacher creates a script session. This session holds data such as students' names and the timing of each phase. Initial session data are supplemented by runtime data such as, in the case of the ConceptGrid, the composition of the groups, the definitions produced by the students and the position of the concepts on the grids.

A possible higher level of abstraction, script families, discriminates classes of scripts which use the same pattern: ConceptGrid uses the Jigsaw pattern i.e. it distributes knowledge among group members; ArgueGraph relies on a conflict-raising pattern; the Reciprocal Tutoring script [START_REF] Palincsar | Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities[END_REF] relies on a mutual regulation pattern. This level of abstraction is not represented in ManyScripts.

The efficiency of a script is not intrinsic to the script definition, but also depends on the specific instance and the session. For example, the potential learning outcomes of ArgueGraph not only depend on the quality of the scenario per se, but also upon the quality of the questions (do they lead students to express divergent or convergent answers?) as well as on the extent to which the students engaged in a specific session have different ab initio opinions on the topics addressed by the questions.

This paper does not aim to present script authoring tools per se, but to address the scientific challenge underlying the development of such an environment. One frequently addressed challenge is the elaboration of ontologies describing the learning activities that are to be edited and sequenced by authors. We explore a second challenge, namely, how to capture the dynamics of a script, i.e. the mechanics referred to in the introduction. The mechanisms that favor the emergence of productive interactions lie in the relationship between activities and are located in transformations of the underlying data structures. While addressing data structures may sound like a back-office technological detail, we actually consider the key differences between a CSCL script and a genuine lesson plan to be embedded in the underlying workflow and data structures. In addressing the challenge of capturing script mechanics, we use a model that articulates these data mechanisms with the raison d'être of a script, i.e. scaffolding targeted types of interaction.

SWISH mechanisms

The SWISH model relates emerging group interactions to the way in which the task or its resources are distributed across team members. It may appear counter-intuitive to formalize task distribution given that collaborative learning is often defined as the process of constructing and maintaining a shared understanding of the task [START_REF] Roschelle | The construction of shared knowledge in collaborative problem solving[END_REF]. However, the effects of group learning are not dependent on the construction of a shared understanding per se, but rather on the effort exerted by group members to develop a shared understanding despite their differences [START_REF] Schwartz | The emergence of abstract dyad representations in dyad problem solving[END_REF]. Macro scripts engineer these differences: a script tunes the effort which is necessary for students to reach a shared understanding. Effort refers to the frequency and quality of the previously mentioned types of interaction: explanations, argumentation, and mutual regulation. The methods adopted in tuning the collaborative effort vary across the three presented scripts:

-ArgueGraph increases collaborative effort by selecting conflicting pairs and hence augmenting the argumentation which is necessary to reach an agreement. -ConceptGrid increases collaborative effort by fragmenting knowledge (readings) and hence augmenting the need for explanation among team members. -WiSim increases collaborative effort by distributing simulation inputs across different phones and hence requiring students to negotiate values and coordinate their experimental design.

For the tuning of collaborative effort, each of the scripts creates some kind of difference or 'split' among the students involved:

-ArgueGraph relies on the differences of opinion that naturally exist among students in order to form pairs. -ConceptGrid creates artificial differences in the knowledge among peers by providing them with different resources to read. -WiSim creates role differences by allocating different responsibilities in setting up parameters.

In these three examples, script designs involve disturbing a 'natural' collaborative system in such a way that the interactions necessary to maintain/restore collaboration produce the desired learning outcomes. We refer to this design principle as "Split Where Interaction Should Happen" (SWISH). It can be summarized in three axioms:

1. Learning results from the interactions in which students have to engage in order to compensate for the 'split' introduced by the script, i.e. constructing a shared solution despite distributed resources, knowledge, etc. 2. The nature of the 'split' thus determines the nature of interactions. Interactions are mechanisms for overcoming task splits. 3. Task splits can therefore--in reverse engineering--be designed to trigger the very interactions that the designer wants to foster: Split Where Interaction Should Happen.

Increasing collaborative effort too much may of course also damage collaboration [START_REF] Dillenbourg | Over-scripting CSCL: The risks of blending collaborative learning with instructional design[END_REF]. As is often the case in educational design, a trade-off is required: while psycholinguists describe the mechanisms by which natural conversation tends to minimize collaborative effort [START_REF] Clark | Referring as a collaborative process[END_REF], collaboration breaks down and effects such as the free rider effect emerge if understanding one another becomes too difficult. The aim of educational design is to achieve the best compromise between ease and workload, a trade-off that we term 'optimal collaborative effort' [START_REF] Dillenbourg | Sharing solutions: persistence and grounding in multi-modal collaborative problem solving[END_REF]. The same trade-off is also targeted in discovery learning methods, as expressed in the term 'guided discovery'.

Data structures

SWISH is not a computational but rather a pedagogical design model, i.e. a way for designers to think about scripts. The essence of SWISH is not embedded in the description of a sequence of activities but encrypted in the data structures that underlie the script execution. To develop a computational account of SWISH, it is thus necessary to relate its principles to data structures and operators. We describe data structures at an abstract level, i.e. independent of their implementation. At this level, the mechanics of scripts can be described using simple tables or matrices and with a limited set of operators.

Social structure

Macro script activities occur across three social planes (see "Script concepts" section): the whole class, a student group and the individual student. Both the class and the set of individuals remain static throughout the script, whereas groups are dynamic structures that may evolve with time, e.g. when forming groups dynamically on the basis of individual behavior or on the basis of rotation. In many scripts, each group member is required to play a specific role. These four basic elements (students, groups, class and roles) define the social structure of any script. This simple structure enables the definition of phase transitions as matrix transformations based on generic operators. Some scripts do not differentiate roles within teams. This situation can be simply represented by inserting the same role into each row of the matrix. Roles and groups do not always define a strict partition of a class. The social structure does not presuppose that a student belongs to only one cell or that one cell contains a single student (a matrix may have two students per role or two roles per student). Such cases require, however, special attention when it comes to implementation. Tolerating 'imperfect' social structures is necessary in order to guarantee flexibility of CSCL scripts. This includes, for example, coping with student number 17 when groups of four are to be formed in a class of 17 students, continuing with the script when one team member drops out of the class, etc. Two ways of providing flexibility have been implemented in the ConceptGrid environment: the "joker" and "spy" roles [START_REF] Dillenbourg | Flexibility in macro CSCl scripts[END_REF]. The 'joker' is an extraneous group member who is allowed to play any role within the group (see Armin in Group3, Table 1). The 'spy' role allows members of the incomplete Group2 (Table 1) to borrow products produced by those members of another group who play the role that is missing in Group2.

In some scripts, the social structure changes over time. Some JIGSAW scripts include phases in which students with role x in their respective team meet with those playing the same role in other teams. These two types of groups (groups comprising members with different roles and groups comprising members with identical roles) are respectively represented on the left and right hand sides of Table 2. Such a rotation was embedded in the UniverSanté script [START_REF] Berger | Teaching community health by exploiting international socio-cultural and economical differences[END_REF], which combined co-present and distance argumentation phases. The script was used by medical students from Switzerland, Lebanon and Tunisia for a course on public health. In co-present meetings, students from the same country studied different diseases (Table 3, left) while, in distance meetings, students from different countries studied the same disease (Table 3, right).

Roles and resources

Roles are key components of script mechanics. The notion of "role" is a synthetic way to differentiate the contributions expected from team members and to communicate these expectations to the students. For instance, one team member may be asked to adopt the role of team coordinator in a project-based script or to play 'Piaget' in a pedagogical debate. A role may simply be a label from which students can infer what they are supposed to do. Designating roles is one way of defining the 'didactic contract' [START_REF] Brousseau | Théorie des situations didactiques[END_REF]. We encounter two types of roles in CSCL scripts, 'induced' roles and 'natural' roles. Induced roles are roles proposed by the script with a low or high degree of coercion. Natural roles refer to the exploitation of differences that exist between students before entering the script.

Roles can be induced by restricting access to specific resources, documents or tools, or by defining specific responsibilities related to student authentication in the system.

-Documents. For instance, in the ConceptGrid, a student adopts the role of Jean Piaget by reading three papers written by Piaget; papers that the other team members will (probably) not read. While the documents associated with a particular role are accessible to other roles, students tend to focus on those resources for which they have a defined responsibility (in our experience, other students never volunteered to read all the papers). -Tools. In the RSC script [START_REF] Betbeder | Symba: A framework to support collective activities in an educational context[END_REF], roles (which in this case are defined by the students themselves) are related to the use of specific tools at different phases of their project (Table 4). -Responsibilities. In WiSim, each team member--identified by his mobile phone--is responsible for entering values for certain simulation parameters. Having access to specific functions of an application is similar--just different in granularity--to having access to different software tools as discussed in the previous point. Reciprocal Tutoring [START_REF] Palincsar | Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities[END_REF]) is a script according to which students are alternatively assigned the role of 'reader' and 'questioner'; while the former reads a paragraph, the latter poses predefined questions.

Other scripts rely on "natural" differences between students, i.e. differences that are not generated by the script. Natural differences occur in two situations:

-Natural differences are known a priori when the class consists of different audiences, i.e. students with different backgrounds. For instance, the script developed by [START_REF] Rummel | Can people learn computer-mediated collaboration by following a script[END_REF] forms pairs comprising students of medicine and psychology, i.e. student roles naturally resulting from their respective background knowledge. In the UniverSanté script, medical students came from different countries (Switzerland, Lebanon, Tunisia and Cameroon), which was highly relevant given that they were required to discuss public health issues and that health problems as well as solutions (e.g. prevention) vary across countries. Many "natural" differences can be exploited on the basis of geographical differences (climate, etc.), social differences, urban versus country contexts, etc. -When natural differences exist within a single audience, e.g. 'natural' differences of opinions among students of the same class, these can be captured by software and utilized in the same way as roles. This is the case in the ArgueGraph script in which the recording of student opinions in Phase 1 is used for the subsequent forming of groups. [START_REF] Gijlers | Confronting ideas in collaborative scientific discovery learning[END_REF] use the different hypotheses produced by students as 'natural' differences in an inquiry-based learning environment.

We do not propose a classification of resources since this has been carried out elsewhere and has led to various standards [START_REF] Friesen | Interoperability and learning objects: An overview of E-learning standardization[END_REF]. We would like to emphasize that script mechanics rely on connections between people and resources and that these connections are often expressed using the concept of roles.

Products

Individuals, groups and classes produce objects at all phases of a script. These products range from simple data to complex multimedia objects. All efforts to generalize the description of products should rely on existing standards for metadata. This is easier to apply to large than to small objects. Larger objects (long texts, software pieces, etc.) are usually stored in an explicit way. Smaller objects comprise data such as "Olga selected Answer 2 for Question 5" in ArgueGraph or "Group 5 placed Concept 2 in cell [3 4]" in ConceptGrid. Within a clean modular architecture, these data remain 'internal' to the software tools (a chat tool, a whiteboard, a questionnaire etc.) that are included in the script as black boxes. This high modularity facilitates script authoring but inhibits scripting mechanisms which require

project SPSS Transana Piaget X X Coordinator X X Vygostky X Statistician X X Weinberger X Dialogue analyst X X
the reuse or processing of data that are internal to other components (for instance, using questionnaire data for the purpose of group formation as in ArgueGraph).

Resources and products can be distinguished by the fact that resources are predefined, i.e. they exist before the script is run, while products result from the unfolding script. However, this distinction is not very robust since the product of a given phase may become a resource for activities in a later phase.

The specificity of CSCL scripts is that all products are associated with a social entity, i.e. an individual, a group or a class. The ownership of products makes it possible to aggregate/ dissociate products according to social plane.

Operators

A script is a workflow, i.e. a time-based sequence of operators that transform data structures, the social structure (student, groups, classes, roles) and object structures (resources and products). Each operator produces the phase N+i matrices from the phase N matrices. Different types of operators exist:

-[Products Matrix → Social Matrix] The group formation operator in ArgueGraph uses the products matrix from Phase 1 to produce the social matrix of Phase 3. -[Social Matrix → Resources Matrix] In ArgueGraph, simple aggregation operators generate pairs in Phase 3 using the answers provided by members in Phase 1 and provide the teacher (Phase 4) with all answers to a specific question. In ConceptGrid, aggregation is used to provide the teacher with the cross-grid view required for conducting the debriefing session (Phase 5). -[Products Matrix → Products Matrix] Aggregation is used in WiSim to turn individual parameter values provided by each member of the group into the parameter set required for running the simulation. Conversely, WiSim uses a distribution operator in Phase 6 to provide group members with fragments of the simulation results, different views of the same results or results produced by different models. -[Social Matrix → Social Matrix] Some Jigsaw scripts and the UniverSanté script [START_REF] Berger | Teaching community health by exploiting international socio-cultural and economical differences[END_REF]) apply a 90 degree rotation operator, i.e. forming groups using the rows or columns of the role X group matrix as illustrated in Table 3.

Script handicap

As expressed by the SWISH principles, the distribution of resources and roles among team members determines the effort to be exerted by students in order to compensate for this distribution, i.e. the intensity of the interactions required for completion of the joint activities.

A script artificially increases the effort necessary to achieve a joint product, as depicted in Fig. 6, where Δ 1 represents the effort that pairs would have to naturally exert to reach a consensus, Δ 2 represents the increased effort pairs would have to exert due to the script design and Δ 3 represents the degree of divergence remaining upon task completion. In a script such as ArgueGraph, the value of Δ 1 mainly depends on the script session, i.e. the natural heterogeneity of the class, while the difference between Δ 1 and Δ 2 is an intrinsic feature of the script. This difference corresponds to script effects, not in terms of learning outcome but rather in terms of the need for interaction. Metaphorically speaking, the deeper the valley, the more effort the team has to exert in order to reach the summit on the right hand side. As mentioned earlier, if the valley is too deep, climbing may become intractable.

Let us use a simple metric for Δ 1 , Δ 2 and Δ 3 , a disagreement rate ranging from 0 to 100%, where 0% refers to unrealistic perfect agreement. The value of Δ 2 corresponds to what we refer to as 'optimal collaborative effort'. The value of Δ 3 decreases if the joint activity in phase 3 forces team members to form a joint understanding, but there is of course no activity that can force them to reach a perfectly shared understanding.

Since scripts span different social planes, Δ can be computed in different ways: the degree of disagreement between the individuals within a group Δ IG , the degree of disagreement between the individuals within a class Δ IC and the degree of disagreement between the groups in a class Δ GC . While collaborative scripts and this paper focus on Δ IG , Δ IC and Δ GC also have to be considered in integrated scripts given their relationship to the richness of class-wide activities, in particular to the debriefing sessions. In the following sub-sections, we use these parameters to describe how SWISH mechanisms are instantiated in the ArgueGraph and ConceptGrid examples.

SWISH mechanisms in ArgueGraph

The initial degree of disagreement (Δ 1 IC ) in ArgueGraph can be estimated by comparing the answers to the individual questionnaire (phase 1). Let us consider a class of six students, each of whom answers a single question with one of six possible answers. Disagreement is 0% if all students provide the same answer and 100% if each student provides a different answer. With an actual class of 20 students and questions with four answers, the value of Δ 1 IC will of course somewhere between these extreme values. We simply compute Δ 1 IC as follow: for each answer to each question, we compute the percentage of other student who gave a different answer and then compute the average percentage.

We recently ran an ArgueGraph session with 14 doctoral and postdoctoral fellows at EPFL. The questionnaire included ten questions, each of which had between two and four possible answers. The value Δ 1 IC was 47%. The rate of disagreement ranged from 78% (for a critical question with four possible answers) to 0% (for a question that was too obvious or too politically correct and had only two answers). It is important to design questions that lead students to express divergent opinions in Phase 1; in our experience, questions with a politically correct flavour fail to produce sufficient divergence. We also examined whether individuals systematically had a different opinion from the rest of the class: This personal rate of disagreement was surprisingly stable, ranging from 48 to 58%. Once groups have been formed by the script, we may compute the rate of disagreement within each pair (Phase 2) and average them: in the previously mentioned ArgueGraph session, the value of Δ 2 IG was 59%. Our pair matching algorithm is actually more subtle than such simple percentage calculations expressed as Δ 1 and Δ 2 . For instance, on a scale "never," "rarely," "sometimes," "often" and "always," the answers "often" and "sometimes" should be treated as being closer to each other than "often" and "never." Therefore answers in ArgueGraph are differentially weighted, but we nonetheless use this disagreement rate as a simple appraisal of the collaborative effort that illustrates the SWISH principles and is reusable in many contexts.

At the end of their argumentation (Phase 3), pairs are required to provide a single answer to each question. The computable value of Δ 3 IG is therefore 0%, although this obviously does not reflect the actual level of disagreement: pairs may 'accept' a common answer without truly agreeing [START_REF] Baker | Negotiation in collaborative problem-solving dialogues[END_REF], because they are polite or tired, or they may have the "illusion of a shared understanding" [START_REF] Ross | The false consensus phenomenon: An attributional bias in selfperception and social perception processes[END_REF][START_REF] Cherubini | Grounding is not shared understanding: Distinguishing grounding at an utterance and knowledge level[END_REF]. It is even possible, although it can not be verified, that the engagement often observed in the debriefing session (Phase 5) results from the frustration felt by some students in being forced to make choices that do not reflect their opinion in a satisfactory way. For the sake of interest, we may examine whether groups converge more in Phase 3 than individuals in Phase 1. We compute Δ 3 GC in the same way as Δ 1 IC but by considering group answers instead of individual answers: the value of Δ 3 GC is 51%. Not only do answers provided by students in a questionnaire not perfectly match their actual opinions, but in addition, their divergence of opinions does not accurately predict students' engagement in argumentation. The intensity of argumentation in teams (Phase 3) does not exclusively depend on Δ 2 IG , but also on the personality of the students (some intensively defend their original individual answer) and on status differences (the class included, for instance, fresh PhD students as well as more experienced scholars). Similarly, the intensity of argumentation in debriefing (Phase 5) not only depends on Δ 3 GC but also on other factors including, for example, the timing of the whole sequence [START_REF] Dillenbourg | Designing integrative scripts[END_REF]. In other words, we do not consider the rate of disagreement to be a cognitive measure, but rather a behavioural indicator of script mechanics: SWISH mechanisms can be implemented as operators that transform script data structures (social matrix, products matrix and resources) between phases in order to increase Δ IG .

SWISH mechanisms in ConceptGrid

We can apply the same type of analysis to the ConceptGrid. We ran four sessions of the ConceptGrid in a Master's course on Computer-Supported Cooperative Work. Each session involved between eight and ten groups of three students working on sets of nine concepts. Grid dimensions were 4×4 with the exception of the third session, in which the grid was 3×3.

An estimate value of Δ IG can be computed when concepts have been individually defined by students. The products matrix (Table 5) stores who defined which concept. In Table 5, results are aggregated across groups. A 1 is inserted in cell i,j if the concept i has been defined by role j . The number in cell i,j corresponds to the number of times the concept i has been defined by role j . Table 5 shows that the distribution of concepts across roles varies across teams. Some concepts (e.g. concepts 2 and 5 in grid 1) were defined by the same role in all teams. Most concepts are rather clearly associated with a role. Nonetheless, some concepts were defined by several roles (e.g. concepts 8 and 9 in grid 4). The role specificity of concept i in grid g can be computed as the standard deviation of column i in grid g . If we average role specificity across all concepts in each grid, we observe that this value constantly decreased throughout the fours session of ConceptGrid in the given Master's course (Fig. 7). The log files from these experiments do not allow us to discriminate the main author of a concept from the final author (somebody who might simply have edited the definition); hence this loss of specificity could also indicate a growing trend to edit the definitions produced by peers. In a paper questionnaire, 47% of students said they sometimes edited the definition produced by their partner. Nonetheless, if concept-role specificity could be computed accurately, it would serve as a good estimate of Δ 4 IG prior to students' construction of the grid (Phase 4).

The key phase in ConceptGrid is the phase in which groups construct the grid, or more precisely, in which two concepts defined by different roles are juxtaposed on the grid. Team members have to explain these concepts to one another in order to write a short text describing the relationship between the two concepts. Of course, if one student reads about concepts C 1 , C 2 and C 3 , it is natural that (s)he juxtaposes these concepts on a grid. In the first The number of groups involved in each session was 10, 9, 8 and 8, respectively session, for instance, the concepts "intellectual capital" and "knowledge management"--both of which had been defined by the same role in all groups--were juxtaposed on the grid by nine of the ten groups. Extra effort is required whenever students juxtapose concepts that have been defined by different authors.

In Fig. 8, the grids have been annotated in such a way as to allow identification of the role players who defined each concept. Let us refer to the relationships between concepts defined by different authors as "cross-author". In the grid on the left, six of a total of ten relationships defined by the team were cross-author relationships, and 7 of a total of 12 in the grid on the right. The value of Δ 5 IG can be simply estimated by the number of crossauthor relationships: the higher this number, the greater the explanation effort required to construct the grid.

The value of Δ 5 IG does not depend on the script per se, but rather on the specific script instance. In Session 2, for example, most teams (eight of nine) juxtaposed "group memory" and "transactive memory" despite the fact that these two concepts were in each case defined by different team members. These two concepts thus have high affordance of social interactions. The estimation of Δ 5 IG could be refined by not only counting the number of cross-author relationships, but also by weighting each relationship with the concept-role specificity: if role x has to explain concept i to role y , the explanation will have to be more elaborate if concept i is highly specific to role x than if it is also sometimes defined by role y .

Of interest in the debriefing phase (Phase 6), is that many different relationships have been defined by different teams. This is captured by Δ 6 GC , i.e. the diversity of the grids constructed by the groups, as illustrated in Fig. 9.

As was the case for the ArgueGraph, the SWISH mechanisms that drive ConceptGrid are embedded in simple operators that relate three data structures: the social matrix (which student has which role), the individual products matrix (concept i is defined by role x /team t ) and the group products matrix (concept i is juxtaposed with concept y ). The product of these Fig. 8 Juxtaposing two concepts requires elaborate explanation when the concepts have been defined by two different students Fig. 9 Concept relationship matrices expressed by each group. A dot appears in cell ij when the concept i has been juxtaposed to concept j . The upper left matrix is empty because this group did not clearly understand the task in constructing the first grid matrices allows the computation of data such as the concept-role specificity and other above-mentioned variables.

Conclusions

SWISH is not a cognitive model: it does not explain why students learn through argumentation, explanation or mutual regulation. The cognitive effects of inducing such interactions have been established by other researchers. SWISH is rather a pedagogical design model, i.e. a set of principles that designers may apply to help scaffold specific classes of interaction. As is true of most pedagogical principles, these principles are not deterministic: the actual induction of target interactions depends on numerous parameters (e.g. the personality of both students and teachers), besides the script itself. Run-time adjustments are required: teachers must permanently regulate scripts. While constructivism has often been confused with "teacherlessness," our socio-constructivist scripts assign the teacher the central role of orchestrating multi-level activities.

The specificity of these principles lies in the fact that their operationalization relies on mechanisms for the transformation of data structures (social matrix, resources matrix and products matrix) that are handled by the different activities of the script. This set of mechanisms, which we refer to as the "mechanics" of scripts, and the SWISH model constitute the core difference between a genuine lesson plan and a CSCL script. Unfortunately, these mechanisms also represent an obstacle in the abstraction process that is required for developing authoring tools.

Fig. 3

 3 Fig. 3 Students are required to construct a grid in which they explain the relationship between juxtaposed concepts

Fig. 5

 5 Fig. 5 Distribution of script activities across three social planes (numbers refer to phases)

Fig. 6

 6 Fig.6Natural divergence (Δ 1 ) is increased by the script (Δ 2 ) and is to be compensated for by the team (Δ 3 )

Fig. 7

 7 Fig. 7 Evolution of concept-role specificity across the four ConceptGrid sessions

  students, i.e. pairing of students who provided conflicting responses in Phase 1. 3 Group Pairs answer the same questionnaire as in Phase 1. The environment provides them with the answers and justifications provided by each peer in Phase 1. Pairs must select a single answer. 4

2 Class

Based on the responses in Phase 1, the system produces a map of opinions (see Fig.

2

/ left). The teacher discusses this map with students and forms pairs in such a way as to maximize distance between

Table 1

 1 Social structure of CSCL scripts

	Class	Group 1	Group 2	Group 3	….	Group i
	Role 1	Pierre	Michel	Jacques
	Role 2	Jean	Emma	Lena
	Role 3	Sophie	<SPY>	Tamara
	<Joker>			Armin

Table 3

 3 Social matrix rotation in the UniverSanté script

	Phase i	Switz.	Lebanon	Tunisia	Phase j	Cancer	Diabetics	Alcohol
	Cancer	Amy	Andrea	Alberto	Switz.	Amy	Bernie	Marie
	Diabetics	Bernie	Brigitte	Bruno	Lebanon	Andrea	Brigitte	Michelle
	Alcohol	Marie	Michelle	Marco	Tunisia	Alberto	Bruno	Marco

Table 2

 2 Social matrix rotation in a Jigsaw script

	Class	Group H1	Group H2	Group H3	Class	Group E1	Group E2	Group E3
	Role 1	Amy	Andrea	Alberto	Role 0	Amy	Bernie	Marie
	Role 2	Bernie	Brigitta	Bruno	Role 0	Andrea	Brigitta	Michela
	Role 3	Marie	Michela	Marco	Role 0	Alberto	Bruno	Marco

Table 4

 4 The roles X resources matrix

	Roles/	Paper	Paper	Paper	Paper	Roles/tools Email	MS
	documents	1	2	3	4	forum

Table 5

 5 Ownership of definition according to role for each of the 4 sessions

	GRID	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept
	1	1	2	3	4	5	6	7	8	9
	Role 2 1	0	1	0	1	6	6	6	0
	Role 1 0	0	8	9	9	1	2	1	0
	Role 3 9	10	1	1	0	2	1	2	1 0
	GRID	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept
	2	1	2	3	4	5	6	7	8	9
	Role 2 1	1	2	3	1	2	7	7	3
	Role 1 8	8	0	0	0	7	1	1	6
	Role 3 0	0	7	6	7	0	0	0	0
	GRID	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept
	2	1	2	3	4	5	6	7	8	9
	Role 1 0	7	0	6	5	1	0	0	3
	Role 2 6	1	5	0	2	5	0	1	1
	Role 3 1	0	2	2	1	1	8	7	4
	GRID	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept	Concept
	2	1	2	3	4	5	6	7	8	9
	Role 2 0	7	0	6	3	1	3	4	3
	Role 1 5	1	1	1	3	5	2	2	1
	Role 3 2	0	6	1	0	1	1	2	4

Computer-Supported Collaborative Learning

P.Dillenbourg, F. Hong

Acknowledgments This work was partly supported by the Swiss Center for Innovation in Learning (SCIL, University of St. Gallen) which is funded by the Gerbert Rüf foundation. Special thanks to SCIL members Taiga Brahm and Sabine Seufert. The paper was also supported by Kaleidoscope, a European Network of Excellence, in particular members of the MOSIL and COSSCILE groups, including Frank Fischer, Lars Kobbe, Armin Weinberger, Andreas Harrer, Nils Malzahn, Paivi Hakkinen, Raia Hamalainen, Sanna Jarvela, Ulrich Hoppe and Pierre Tchounikine. We also benefited from the contributions of our team members Shuja Parvez and Patrick Jermann and our former colleague Daniel Schneider.


Carolyn P Rosé

Gregory Dyke

Nancy Law

Kris Lund

Dan Suthers

Christopher Teplovs

Ulrike Cress

C Penstein

; Provocateur

K Becu-Robinault

W Chen

M Chiu

S Goggins

L Hill

C Hmelo-Silver

I Howley

H Jeong

B De Leng

J Kimmerle

C K Looi

Y Matzuzawa

E Mayfield

R Medina

Y Mor

Y Niihara

J Oshima

R Oshima

P Papadopoulos

K Sawyer

H Shirouzu

Y Song

G Stahl

J W Strijobs

S Trausan- Matu

C Wang

A Wise

H Wee

Leveraging Researcher Multivocality for Insights on Collaborative Learning -White Paper

come   L'archive ouverte pluridisciplinaire

Introduction and motivation

This workshop targets researchers in the Learning Sciences, Computer Supported Collaborative Learning (CSCL) and Technology Enhanced Learning (TEL) communities who are interested in how human interaction leads to learning. Such researchers come from many different disciplines (psychology, linguistics, cognitive science, computer science, didactics, etc.) and thus employ diverse methods in pursuing their specific research goals as well as hold diverse theoretical assumptions in relation to these goals. Many of us are interested in the richness that an interdisciplinary approach to studying learning in human interaction can provide, but in order to profit from this, we must find a systematic way of leveraging our diversity to further our understanding in spite of potential incommensurable differences that may occur across traditions. Recognizing this diversity as a necessary multivocality has led our group to reflect upon ways in which such multivocality can be productive for the communities involved. In particular our objective is to make progress towards better understanding the role of human interaction in learning, an understanding that should transcend disciplinary boundaries. This workshop proposes a systematic method for promoting fresh dialogue between the relevant research traditions with the objective of making new claims about learning. This workshop proposal continues the trajectory of a series of prior workshops. At ICLS 2008 ("A Common Framework for CSCL Interaction Analysis"), we explored dimensions along which analytic efforts can be characterized, and attempted to identify a common framework that would enable comparison of analyses and building shared analytic tools. Confronted with the multivocality that makes such unification difficult, we shifted our focus at CSCL 2009 ("Common Objects for Productive Multivocality in Analysis") to identifying the basis for dialogue between different traditions. One major conclusion was that multiple analyses of shared data sets provide a promising basis for discussion, these data sets constituting "boundary objects" (rather than "common objects") that make discourse possible. At the Alpine Rendez-vous 2009 ("Pinpointing Pivotal Moments in Collaboration"), we followed up on this conclusion by having researchers from different theoretical and methodological traditions analyze the same data sets. The analyses were focused on the identification of "pivotal moments" in collaboration. Different conceptions of pivotal moments were identified, but in all cases they provided good starting points for further analysis of how learning arises from interaction. At ICLS 2010 ("Productive Multivocality in the Analysis of Collaborative Learning"), we expanded the corpora on which researchers from different theoretical and methodological traditions performed their analyses and we proposed an initial structure for a book focused on the multiple analyses of shared data, arising from our different gatherings. The objective of this new workshop proposal "Leveraging Researcher Multivocality for Insights on Collaborative Learning" is twofold. First, we will discuss how the multiple analyses carried out on each paradigmatic corpus we chose from previous workshops contributed to specific new insights on collaborative learning. Secondly, we will build dialogue between complementary researcher views that can be introduced into the book.

Workshop description

The workshop was structured as a working meeting towards forming a consensus view of what the story will be in the book, which will be the final product of our workshop series. As in our earlier workshops, the structure of this workshop was organized around datasets. The book will include multiple analyses of 5 different datasets: Hajime's fractions dataset, Wen Li's Group Scribbles Electricity dataset, Carolyn's 9th Grade Biology dataset, Nobuko's Knowledge Forum dataset, and Keith's Chemistry dataset. The workshop gave more time to all but the Chemistry dataset since the majority of researchers working on the Chemistry portion of the book were not able to participate in the ARV. However, this dataset was given some consideration in all but the initial session so that its message would figure into the integrated story that will be presented in the book. A draft version of the book was available to all workshop participants ahead of time in order to facilitate rapid progress during the workshop itself.

Wednesday Session 1 08:30-12:30 Within-Dataset Multivocality Discussions

The goal of the initial session of the workshop was to welcome the participants, to introduce the book in its draft form, and then to divide into two parallel sessions, each focusing on two datasets. In those sessions, there was time to discuss the dataset as a whole and each draft analysis. Discussion focused on issues raised in the analysis, especially places where discrepancies between analyses came up in the drafts or presentations, or where participants who were not authors raised questions or objections. Not surprisingly, because our participants represent a broad spectrum of methodological and theoretical perspectives, there were challenges raised, especially relating to how analyses were set up, since many assumptions are made in this process that relate back to that range of perspectives. For example, in Parallel session 2, questions related to the setup of the Goggins social network analysis in terms of parameter settings were raised based on the Stahl ethnographic style analysis. Questions were also raised relating to the selection of the datasets themselves -especially with regard to whether it is beneficial to highlight what can be learned from what went wrong in the data, or whether it is more beneficial as a contribution to the CSCL community to focus on places where knowledge building was more ideal. The decision was eventually to place value both on the ideal examples of knowledge building as well as ones where things go wrong (as a reality check for the community, especially for young researchers with idealistic views of how to change the world with technology, and also as an illustration of how a multivocal approach to iterative, data-driven design and development is valuable as well as a multi-vocal analysis of data from more mature systems and interventions

Wednesday Session 2 16:30-20:15 Multivocality within and across datasets

In Session 2, the discussions from session 1 were summarized and reported back to the whole group since each parallel session was attended by only half of the workshop participants. In addition to reporting back, this session gave opportunity for the whole group to begin its consensus building process. What emerged from this session was deeper questions about the nature of multivocality itself. One realization was that despite the controversies that did come up, we found that we were more on the same page than we anticipated, even across apparently very different theoretical and methodological camps. We questioned whether we should have reached further out to include researchers more different from ourselves, or whether the experience of the workshop series had brought about a mind meld in such a way that the differences we started with had been lessened over time. Questions were raised about whether very different perspectives were really as incommensurate as we initially believed. We began to see that some of us choose methodologies for pragmatic reasons rather than deeply philosophical ones, and in that case, the mind meld is more natural. Session 3 was a whole group working session divided into two parts. In the first part, we set aside one table for each of the 5 datasets, where the discussant for that dataset sat. All other participants divided up into traveling cohorts who visited each table in a round-robin/speed-dating style. In each of the rounds, the traveling cohort group at each table provided feedback based on the discussion from the day before about the current analyses under discussion for that dataset. These notes were collected and integrated by the discussant and then reported back to the whole group. These sets of notes will be used by those discussants in their writing of the discussion chapters for the book. These summaries also fed into the next phase of the group work, which was an affinity diagramming activity designed to identify issues that were not adequately discussed so far for the book as a whole. What came out of this discussion is that we need some high level, cross-cutting chapters that discuss the concept of pivotal moments and multivocality, a best practices methodology chapter for researchers just getting in to a multivocal style approach, and a lessons learned chapter that recaps what we take away from the experience of this workshop series. We also chose to delete two planned chapters that seemed less important than the chapters that emerged from the affinity diagramming discussion. One of those was a tools chapter that has been "rebirthed" in the form of a chapter about representations for data analysis and how these are embodied in analysis technology. As a wrap-up, the editors of the book worked out a plan with deadlines for moving forward to the final preparation of the book, which we plan to submit to a publisher in Fall of 2011.

3.Emerging Research Questions

Summarizing what came out of the discussions we realized several things during this workshop, which eventually became themes presented in the Symposium we presented at CSCL 2011. One was that through multivocal analysis, some things that we expect to look different based on our initial understandings can turn out to be much more similar than we had thought, while on the other hand, things that we assume are similar based on a high level understanding of operationalizations can turn out to be quite different in important ways when we examine them up close in the same dataset. In both cases, a multi-vocal analysis is valuable in that it challenges researchers to reconsider their assumptions, to sharpen their operationalizations, and to catch mistakes. After years of working together through our workshop series, and noticing how natural and easy it is now to communicate about our variety of analyses, we began to wonder if we learned to be too polite and accepting, or if perhaps our analyses seemed more compatible because we weren't digging deeply enough into the details. Beyond the issues discussed above, some additional questions that came out include the following:

What is the role of statistical methods in multivocal analysis? It's obvious what role it plays in quantitative approaches -but within our repertoire of datasets, we included one where complex statistical techniques were applied to a dataset that quantitative researchers would have considered too small for such techniques. In the spirit of multivocality, should we accommodate such an approach? If we do, are we relaxing our commitment to rigor? Do we then open up our community to a lower standard in terms of accumulation of ratified knowledge? Also, at the heart of qualitative research approaches is the idea that it takes human judgment and contextual understanding to identify those interesting interactions worthy of an up close investigation. Can we use statistical techniques to identify those instances that are unusual? Are these techniques really capable of identifying the ones that don't fit the statistical distribution for the right reasons or in a theoretically interesting way? Are those instances that are important for theory building at least a subset of those instances that can be identified this way?

Questions related to falsifiability also came up with respect to the idea of a pivotal moment. In the fractions dataset, different analysts agreed on some pivotal moments and disagreed on others. What does this mean about the nature of the construct? Should all analysts agree? Is the multivocality then for the purpose of triangulation? If discrepancies are tolerated, does that mean we are not using multivocality for triangulation? Does it mean that the construct of pivotality cannot be falsified? Is it then just a tool for facilitating discussion rather than an indicator of something that has external validity?

Grand Challenges

Building a comprehensive framework for exchanging research data and analyses from different research teams in order to deepen the discourse, coming to a convergent interpretation and identifying further research questions.

What problems of the European education system are addressed, and what are the long term benefits for society?

In TEL and CSCL different research groups focus on different aspects of collaborative learning. Some do small case studies, others have larger samples that allow for longitudinal studies or quasi-experimental designs. The teams focus on different teaching methods (scripting; co-operative learning; knowledge building), gather different kinds of data for their research (text data; video data, log files, performance tests) and apply different methods (interaction analysis; pre-post test designs; multi-level methods).

In order to make use of this variety of research data across the different research groups these relevant data sets should be shared and made accessible. The data sets and related analyses could serve as boundary objects and stimulate fruitful discussion across the different research approaches. This would not just show the multivocality in CSCL research, but could also serve as a means for converging evidence about the potentials and effectiveness of TEL and CSCL. This allows not just an overview about the effectiveness of CSCL in teaching and learning for researchers and the scientific community, but also for stakeholders and practitioners.

Furthermore, sharing of datasets and analyses would ensure that results be easier to validate and replicate, facilitating peer-review and leading to more generalisable results to be shared with stakeholders and practitioners.

What are the main activities to address this Grand Challenge Problem?

Development of a technical infrastructure for supporting open data.

Development of a framework for data sharing. This framework consists of recommendations for technical formats, ethic standards and metadata. It describes the needed preparation of data and the documentation of analyses and results.

Development of a framework of how to exchange results (both the analyses as replicably performed and their interpretation) and "lessons learned" among researchers, practitioners and stakeholders.

Build a supportive structure for a dialogical interpretation of the data in order to make the community and stakeholders aware what results converge among the different data sets and different interpretations and in order to identify open questions.

Implementation and formative evaluation of this infrastructure.

What is the timeframe for the Grand Challenge Problem?

About 3 years are needed in order to develop and implement the infrastructure and achieve a critical mass of relevant data. Existing infrastructures such as PSLC Datashop1 [START_REF] Koedinger | A Data Repository for the EDM community: The PSLC DataShop[END_REF], MULCE2 [START_REF] Reffay | Sharing corpora and tools to improve interaction analysis[END_REF], CAViCoLa [START_REF] Harrer | Towards a Flexible Model for Computer-based Analysis and Visualization of Collaborative Learning Activities[END_REF], and Tatiana3 [START_REF] Dyke | Tatiana: an environment to support the CSCL analysis process[END_REF] might potentially be improved upon, adapted or adopted to lessen this timeframe.

What are measurable progress and success indicators?

• Development of an infrastructure (not just a database, but also communication opportunities, meetings, events etc.).

• Number of shared data sets and different kinds of analyses on each of them.

• Involvement of representative groups

• Quality of the stimulated discourse.

How can funding be attracted?

Some core EU-research teams which also integrate research teams from North America and Asia should be funded (by a Network of Excellence or an Integrated Project) which develop the infrastructure, share own data and take the responsibility to make the infrastructure sustainable.

Many funding agencies (e.g. NSF) are increasingly requesting projects to submit a data management plan which describes how data will be persistently warehoused. A platform and infrastructure for sharing could synergistically be funded with such a goal.

One of the challenges -experienced by all existing projects -lies in infrastructure sustainability beyond the funding period.

Researchers and Communities

In short, our Grand Challenge is building a comprehensive framework for exchanging research data and analyses from different research teams in order to deepen the discourse, coming to a convergent interpretation and identifying further research questions in the spirit of multivocality. Multivocality requires a multi-disciplinary community to make it work. First, we need the different orientations brought by different fields to challenge one another. We need psychologists who study the connection between discussion behaviour and cognitive processes to challenge ethnographic researchers who focus on group cognition, and vice versa. We need computer scientists who build computational models of discourse to allow interaction analysts to dig into their data so that they can grapple together with the tensions between overly simplistic generalizations represented by statistical distributions and overly complex contextualized representations that arise from a qualitative approach. We need a community where multivocal collaborations are happening in order to ensure that any infrastructure we build will have the proper affordances for facilitating and not hindering that process.

From a practical standpoint, in order to move forward in a sustainable way we also need the involvement of multiple fields. We need computer scientists to build and maintain the infrastructure for storing and analysing data, but they can't do it effectively without a close partnership with the researchers who will use those tools. Some of the technology we need is still under development, such as the technology to pre-process interaction data in order to prepare it for visualization, sequence analysis, etc. While strides towards developing and improving this technology have been made in the CSCL community as well as in the language technologies community, more work is needed to make this technology more That effort itself is interdisciplinary, involving experts in linguistics, sociology, philosophy, and machine learning. We need experts in visualization to help us design representations that people can interpret. But we need interaction analysts involved in that process to ensure that the impressions conveyed by the visualizations have face validity.

In order to make this vision a reality, we also need some non-research staff with the skills to make things happen in a professional way, including people with marketing expertise, people who have experience developing financial models that work at an international level, people who know how to build and maintain databases that run properly and keep data safe, and people who can offer technical support.

  ).

	09:15-09:25	Hajime Fractions Corpus Presentation	Bio Corpus Presentation and
		(Hajime)	Analysis(Carolyn)
	09:25-09:35	Analyst 1 (Stephan)	Analyst 1 (Ulrike & Joachim)
	09:35-09:45	Analyst 2 (Ming)	Analyst 2 (Gerry)
	09:45-09:55	Fractions Meta-Discussant (Kris)	Analyst 3 (Sean)
	09:55-10:05		Bio Meta-Discussant (Cindy)
	10:05-10:20	Coffee Break	Coffee Break
	10:20-10:30	Group Scribbles Corpus Presentation	Knowledge Forum Corpus Presentation
		(Dan)	(Nobuko)
	10:30-10:40	Analyst 1 (Heisawn)	Analyst 1 (Chris)
	10:40-10:50	Analyst 2 (Richard)	Analyst 2 (Nancy)
	10:50-11:00	Analyst 3 (Kris)	Analyst 3 (Ming)
	11:00-11:10	Group Scribbles Meta-Discussant	Knowledge Forum Discussant (Nobuko)
		(Dan)
	11:10-11:25	Compare notions of pivotal moments	Compare notions of pivotal moments
		(Kris)	(Carolyn)
	11:25-11:40	Discuss roles of representations/	Discuss roles of representations/
		visualizations (Dan)	visualizations (Chris)
	11:40-12:15	Discussion of insights into	Discussion of insights into Multivocality
		Multivocality and Grand Challenges/	and Grand Challenges/ prepare afternoon
		prepare afternoon presentation (Kris)	presentation (Carolyn)
	12:15-16:30 Lunch and Free Time
	08:30-08:45 Opening Remarks and Introductions (Carolyn, Ulrike, and Greg)
	08:45-09:15 Overview of Book (Carolyn, Kris, Dan)
	09:15-12:15 Parallel Corpus Break Outs
	Time	Parallel Session 1	Parallel Session 2

Thursday Session 1 08:30-12:30 Methodological Insights and Book Theme

	16:30-16:45 Hajime Fractions Report (Kris)
	16:45-17:00 Productive multivocality discussion (Kris)
	17:00-17:15 Group Scribbles Report (Dan)
	17:15-17:30 productive multivocality discussion (Dan)
	17:30-17:45 Bio Dataset Report (Cindy)
	17:45-18:00 productive multivocality discussion (Cindy)
	18:00-18:15 coffee break
	18:15-18:30 Knowledge Forum Report (Nobuko)
	18:30-18:45 productive multivocality discussion (Nobuko)
	18:45-19:00 Chemistry Dataset Report (Carolyn)
	19:00-19:15 productive multivocality discussion (Carolyn)
	19:15-19:45 Full group discussion and summary of multivocality lessons (Kris)
	19:45-19:55 Discussion about Grand Challenges (Ulrike)
	19:55-20:15 Planning for day 2 (Kris)

13:30-15:00 Final Book Planning and Commitments

	08:30-08:40 Opening remarks on today's work (Carolyn)
	08:40-09:30 Feedback to discussants, "Speed dating" style (one discussant per table)
	09:30-09:35 Discussants prepare response
	09:35-10:00 Discussants report back (Carolyn, Cindy, Nobuko, Dan, Kris)
	10:00-10:15 Coffee break
	10:20-10:40 Affinity Diagramming Part 1: Participants write thematic comments on sticky notes
	10:40-11:20 Affinity Diagramming Part 2: Participants put up notes round robin style
	11:20-11:30 Affinity Diagramming Part 3: Identification of (Nancy, Chris, and Carolyn)
	11:30-12:00 Group Discussion (Carolyn)
	12:00-12:30 Wrapup discussion of themes and STELLAR Grand Challenges (Carolyn)
	12:30-13:30 Lunch
	Thursday Session 2

13:30-15:00 Closing Discussion (Dan)

http://pslcdatashop.org/

http://mulce.org/

http://code.google.com/p/tatiana/