Jan-Willem Strijbos
email: jwstrijbos@fsw.leidenuniv.nl

Gerry Stahl

Running head: Coding procedure for small group chat Methodological issues in developing a multi-dimensional coding procedure for small group chat communication

Keywords: content analysis, methodology, reliability, threading, coding

In CSCL research, collaboration through chat has primarily been studied in dyadic settings. This article discusses three issues that emerged during the development of a multidimensional coding procedure for small group chat communication: a) the unit of analysis and unit fragmentation, b) the reconstruction of the response structure and c) determining reliability without overestimation. Threading, i.e. connections between analysis units, proved essential to handle unit fragmentation, to reconstruct the response structure and for reliability of coding. In addition, a risk for reliability overestimation was illustrated. Implications for analysis methodology in CSCL are discussed.

Coding of communication processes (content analysis) to determine effects of computersupported collaborative learning (CSCL) has become a common research practice [START_REF] Barron | When smart groups fail[END_REF][START_REF] Webb | The development of students' helping behaviour and learning in peer-directed small groups[END_REF][START_REF] Fischer | Knowledge convergence in computer-supported collaborative learning: The role of external representation tools[END_REF]. In the past decade, research on CSCL has opened new theoretical, technical and pedagogical avenues of research.

Comparatively less attention has, however, been directed to methodological issues associated with coding [START_REF] Strijbos | What we know about CSCL: And implementing it in higher education[END_REF].

Early attempts to analyse communication in computer-supported environments focused on counting messages to determine students' participation and on mean number of words as an indicator for the quality of messages. Later, methods like 'thread-length' analysis and 'social network analysis' expanded this surface-level repertoire. Now the CSCL research community agrees that surface methods can provide a useful initial orientation, but believes that more detailed analysis is needed to understand the underlying mechanisms of group interaction.

Content analysis is widely applied in collaborative learning research (see [START_REF] Barron | When smart groups fail[END_REF][START_REF] Gunawardena | Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing[END_REF][START_REF] Strijbos | Content analysis : What are they talking about ?[END_REF][START_REF] Schellens | Collaborative learning in asynchronous discussion groups: What about the impact on cognitive processing?[END_REF][START_REF] Weinberger | A framework to analyze argumentative knowledge construction in computer-supported collaborative learning[END_REF]. Communication is segmented into analysis units (utterances), coded and their frequencies used for comparisons and/ or statistical testing. Increasingly, collaborative learning studies are moving to a mixed-method strategy [START_REF] Barron | When smart groups fail[END_REF][START_REF] Hmelo-Silver | Analyzing collaborative knowledge construction: Multiple methods for integrated understanding[END_REF][START_REF] Strijbos | The effect of roles on computer-supported collaborative learning[END_REF] and new techniques are being combined with known ones, such as multilevel modelling of content analysis data [START_REF] Chiu | Rudeness and status effects during group problem solving: Do they bias evaluations and reduce the likelihood of correct solutions[END_REF].

At present, however, the number of studies reporting on the specifics of an analysis method in detail is limited. With respect to content analysis this is highlighted by how many citations still reference [START_REF] Chi | Quantifying qualitative analysis of verbal data: A practical guide[END_REF], whose article was until recently the most cited article regarding the methodological issues involved. Within the CSCL community an academic discourse is gradually developing on issues such as analysis scheme construction, comparability and re-use [START_REF] De Wever | Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review[END_REF], unit of analysis [START_REF] Strijbos | Content analysis : What are they talking about ?[END_REF] and specific processes like argumentative knowledge construction [START_REF] Weinberger | A framework to analyze argumentative knowledge construction in computer-supported collaborative learning[END_REF]) -but many issues remain.

Background

This article reports on an attempt to use coding under circumstances that may be typical in CSCL research, but where coding has not generally been applied. The theory behind our research focuses on group processes and the meaning making that takes place in them. It is elaborated in Stahl (2006a) and [START_REF] Stahl | Computer-supported collaborative learning: An historical perspective[END_REF]. The theory there recommends ethnomethodologically-informed conversation analysis as the most appropriate analysis methodology, but we wanted to try to apply a coding approach as well.

Coding is most frequently used to compare research groups under controlled experimental conditions with well-defined dependent variables; we wanted to use coding to help us explore initial data where we did not yet have explicit hypotheses. Coding is often used in cases of face-to-face talk (e.g., in a classroom) or between communicating dyads; we were interested in online text-based synchronous interaction within small groups of three to five students.

Educational and psychological research using coding generally takes utterances or actions of individuals as the unit of analysis; we wanted to focus on the small group as the unit of agency and identify group processes. In undertaking our inquiry into the use of coding under these circumstances, we strove for both reliability and validity. In this article, we take a close look at reliability and address issues of validity in our discussion.

Our test site, the VMT project (http://mathforum.org/vmt/), is developing an online service for students to engage in math discourse at a distance. This project takes a design-based research approach [START_REF] Stahl | Virtual Math Teams project: An overview of VMT[END_REF]. It started very simply with a well-known technology (AOL IM) and the established Math Forum Problem of the Week (PoW) service. The PoW service targets students in grades three through twelve. It provides 'creative, non-routine challenges' for volunteers around the globe. The service is divided into four separate branches: algebra, geometry, pre-algebra and math fundamentals. The reported work with the coding scheme was conducted at the end of the first year of the five-year research project (for examples of the problems see http://mathforum.org/pow/vmt/allproblems0304.html).

We wanted to understand what was happening in the chats along a number of dimensions.

We wanted insights that would help us to develop the environment and the pedagogical approach. In particular, we were interested in how the students communicated, interacted and collaborated. We were also interested in how they engaged in math problem solving as a group. So we drew upon coding schemes from the research literature that addressed these dimensions while developing the VMT coding scheme.

VMT coding scheme

Multi-dimensional coding schemes are not a novelty in CSCL research, but they are often not explicitly defined. [START_REF] Henri | Computer conferencing and content analysis[END_REF] distinguishes five dimensions: participation, social, interactive, cognitive, and metacognitive. [START_REF] Fischer | Fostering collaborative knowledge construction with visualization tools[END_REF] define two dimensions: the 'content' and 'function' of utterances (speech acts). Finally, [START_REF] Weinberger | A framework to analyze argumentative knowledge construction in computer-supported collaborative learning[END_REF] use four dimensions: participation, epistemic, argument, and social. These studies assign a single code to an utterance, or they code multiple dimensions that differ in the unitisation grain size (i.e., message, theme, utterance, sentence, etc.).

The first step in the development of the coding scheme was to determine the unit of analysis; its granularity can affect accuracy of coding [START_REF] Strijbos | Content analysis : What are they talking about ?[END_REF]. We decided to use the chat line as the unit of analysis mainly because it is defined by the user. It allowed us to avoid segmentation issues based on our (researcher) view. We empirically saw that the chat users tended to only 'do' one thing in a given chat line. Exceptions requiring a separate segmentation procedure were rare and too insubstantial to affect coding. We decided to code the entire log, including automatic system entries. In contrast to other multi-dimensional coding schemes unitisation is the same for all dimensions: a chat line receives either a code or no code in each dimension-this allows for combinations of dimensions and expands the analytical scope.

We decided to separate communicative and problem-solving processes and conceptualised these as independent dimensions. Our initial scheme consisted of the conversational thread (who replies to whom), the conversation dimension (based on [START_REF] Beers | Computer support for knowledge construction in collaborative learning environments[END_REF][START_REF] Fischer | Fostering collaborative knowledge construction with visualization tools[END_REF][START_REF] Hmelo-Silver | Analyzing collaborative knowledge construction: Multiple methods for integrated understanding[END_REF], the social dimension (based on [START_REF] Renninger | Mentor-participant exchange in the Ask Dr. Math service: Design and implementation considerations[END_REF][START_REF] Strijbos | The effect of functional roles on perceived group efficiency: Using multilevel modeling and content analysis to investigate computer-supported collaboration in small groups[END_REF], the problem-solving dimension (based on [START_REF] Jonassen | Communication patterns in computer mediated and face-to-face group problem solving[END_REF][START_REF] Polya | How to solve it[END_REF], the math-move dimension (based on [START_REF] Sfard | Analyzing tools: Perspectives on the role of designed artifacts in mathematics learning: Special issue[END_REF] and the support dimension (system entries and moderator utterances).

Then we spent the summer trying to apply these codes to ten chats that we had logged in Spring 2004. Naturally, we wanted our coding to be reliable, so we checked on our inter-rater reliability as we went along. Problems in capturing what was taking place of interest in the chats and in reaching reliability led us to gradually evolve our categories. As the dimensions became more complicated with sub-categories, it became clear that some of them should be split into new dimensions. We ended with the categories in Appendix A, and the additions during calibration trials have been italicised (the math move and support dimension are not discussed in the remainder of this article and therefore not shown).

It turned out that it was important to conduct the coding of the different dimensions in a certain order, and to agree on the coding of one dimension before moving on to consider others. In particular, determining the threading of chat in small groups is fundamental to understanding the interaction. For the participants, confusion about the threading of responses by other participants can be a significant task and source of problems [START_REF] Fuks | R-U-Typing-2-Me? Evolving a chat tool to increase understanding in learning activities[END_REF][START_REF] O'neill | Text chat in action[END_REF]. For researchers, the determination of conversational threading is the first step necessary for analysis [START_REF] Cakir | Thread-based analysis of patterns of collaborative interaction in chat[END_REF].

Agreement on the threading by the coders establishes a basic interpretation of the interaction.

Then, individual utterances can be assigned to codes in a reliable way. In addition, we were interested in the math problem solving. So we also determined the threading of math argumentation, which sometimes diverged from the conversational threading, often by referring further back to previous statements of math resources that were now being made relevant. Determining the problem-solving threading required an understanding of the math being done by the students, and often involved bringing math expertise into the coding process.

In this article, we focus on three issues that emerged in our attempt to apply a coding scheme in preliminary stages of CSCL research:

(a)

We tried to use the natural unit of the chat posting as our unit for coding. This rarely led to problems with multiple contents being incorporated in a single posting, but rather with a single expressive act being spread over multiple postings.

(b) The reconstruction of the chat's response structure was an important step in analysing a chat. We developed a conversation thread and a problem-solving thread to represent the response structure.

(c)

The goal of acceptable reliability drove the evolution of the coding scheme. The calculation of reliability itself had to be adjusted to avoid over-estimation for sparsely coded dimensions.

Unit fragmentation and response structure reconstruction

We started with the calibration of the conversation dimension and combined this with threading in a single analysis step, but quickly discovered that 'threading' actually consisted of two issues namely 'unit fragmentation' and 'reconstruction of the response structure'. Unit fragmentation refers to fragmented utterances by a single author spanning multiple chat lines.

These fragments make sense only if considered together as a single utterance. Usually, one of these fragments is assigned a conversational code revealing the conversational action of the whole statement, and the remaining fragments are tied to the special fragment by using 'setup' and 'extension' codes. This reduces double coding. Table 1 provides an example of both codes: line 155 is an extension to 154 and together they are a 'request' and line 156 is a setup to line 158 forming a 'regulation'.

CSCL research on chat technology previously focused on dyadic interaction (e.g., research on argumentation; Andriessen, Baker, & Suthers, 2003), which poses few difficulties to determine who responds to whom. In contrast, the VMT's small group chat transcripts revealed that the chain of utterances was problematic. A discussion forum uses a threaded format that automatically inserts a response to a message as a subordinate object in a tree structure, and in a similar vein, a prefix is added to the subject header of an e-mail reply.

Current chat technology has no such indicators identifying the chain of utterances. Moreover, while there is no confusion about the intended recipient in a dyadic setting (the other actor), students in small groups often communicate simultaneously, making it easy to loose track of to whom they should respond. Coding small group conversation in a chat required the reconstruction of the response structure as shown in Table 1. Delay between utterances proved to be important. For example, lines 157 and 158 fully overlap (no delay) and the delay between lines 166 and 167 of 16 seconds reveals that the short utterance of 167 is more likely to be connected to 166 than 164. Our reasoning is that it takes only a few seconds to type and submit this utterance, and if line 167 was intended as a response to line 164 this utterance would have appeared before or simultaneous with line 166.

Connecting utterances to handle unit fragmentation and to reconstruct the response structure is performed simultaneously, and referred to as 'threading'. The threading is performed separately from the conversational coding, including assignment of extension and setup, because not all spanned utterance connections concern fragmentation. There is one infrequent exception of a spanned utterance in the shape of three fragments coded as 'explain/critique' + 'elaborate' + 'extension', but this emphasises that coding of extend and setup should be performed separately. In other words, threading only reconstructs connections between the user-defined chat lines that form a) a fragment of a spanned utterance or b) a response to a previous utterance, but the nature of the chat line is decided during coding and not during threading. It also highlights that a coder should be familiar with the codes to ensure that s/he knows which lines should be considered for threading because the conversational code depends on whether or not a thread is assigned.

Calibration trials for the problem solving dimension revealed a similar need for the reconstruction of a problem-solving thread -to follow the co-construction of ideas and flow of problem-solving acts (e.g., proposing a strategy or performing a solution step) -prior to the coding of problem solving.

Calibration trials showed that threading is of utmost importance for the analysis of chatbased small group problem solving and should be assigned prior to the (conversational)

coding. In the next section we will discuss the reliability for threading and coding of three dimensions in detail, as their calculation presented additional methodological issues -more specifically the risk for reliability overestimation. In line with [START_REF] Strijbos | Content analysis : What are they talking about ?[END_REF] we address reliability stability by presenting two trials, each covering about 10% of the data.

Reliability of threading, coding and reliability over-estimation

Reliability of threading

Threading is already a deep interpretation of the data and therefore a reliability statistic should be determined. The calculation of 'threading reconstruction' reliability proved complicated, because coders can assign a thread indicator to a chat line or not, assign an indicator to the same chat line or to a different chat line. As a result, only a proportion agreement can be computed. We used three coders (first author and two research assistants)

and computed two indices for all possible dyads:

• for the assignment of a thread or not by both coders (% thread);

• for the assignment of the same thread whenever both assigned a thread (% same). the field of content analysis [START_REF] Neuendorf | The content analysis guidebook[END_REF][START_REF] Riffe | Analyzing media messages: Using quantitative content analysis in research[END_REF]. Given the various perspectives in the literature, a range of .70 to .80 for proportion agreement can serve as the criterion value. Combined results for the conversational thread reveal that, on average, both coders assign a thread in 80.7% of all cases. Overall, 72.2% of the thread assignments are the same. These combined results show that the reliability of conversational threading is actually quite stable and fits the .70 to .80 range.

The results of both reliability trials reveal for the problem-solving thread that, on average, in 87% of all the instances both coders assigned a thread. Of all threading assignments by either coder 91.5% are the same. These results show that the reliability of problem-solving threading exceeds the .70 to .80 range. It should be noted that the problem-solving thread is often the same as the conversation thread, so the reliability indices are automatically higher.

The R2 selection also contained fewer problem-solving utterances than R1, so the problemsolving thread is more similar to the conversational thread and thus reliability higher. Since the reliability of problem-solving threading depends on the number of utterances that actually contain problem-solving content, it will fluctuate between transcripts. Therefore, the first trial should be regarded as a satisfactory lower bound: 77.1% for thread assignment and 89.9% for same thread assignment.

Reliability of three coding dimensions and reliability overestimation

Given the impact of the conversational and problem-solving threads during the calibration sessions, codes were added or changed, definitions adjusted, prototypical examples added, and rules to handle exceptions established. Nine calibration trials were conducted prior to the reliability trials. We used three coders (first author and two research assistants) and adopted a stratified coding approach for each reliability trial: the coders first individually assigned the conversation threads, followed by a discussion to construct an agreed upon conversational thread, after which each coder independently coded the conversational and social dimension.

Next, coders first individually assigned the problem-solving thread before a discussion was held to construct an agreed upon problem-solving thread, followed by assigning the problemsolving codes. Between both reliability trials, minor changes were made in the wording of a definition or adjusting a rule. Mastery of the coding procedure is laborious. Per dimension, it takes about twenty hours of training and discussion with an experienced coder.

In contrast to our initial conceptualisation of the dimensions as being independent we have been thus far unable to avoid ties between some of the conversational codes and the problemsolving dimension. Coding qualitatively different processes, social versus problem-solving, using the same data corpus was problematic -foremost regarding 'elaborate', 'explain' and 'critique' categories. The implications of ties for the validity of the coding scheme should be determined, but this is beyond the scope of the current article.

Calculating the reliability for the conversation, social, and problem-solving dimensions proved to be less straightforward than expected. Each chat line receives a conversation code and can have either one or no code for any other dimension, but not all chat lines are eligible to receive a particular code. The social and problem-solving dimensions only apply to a portion of all of the chat lines, and the pool of valid units will fluctuate between different pairs of coders. When not all units are eligible to receive a code we should decide how we handle units coded by only one coder and the units not coded both coders in the reliability Perspectives in the literature on a criterion value for kappa differ, but in our opinion these criteria-intermediate, strict and lenient-apply best: below .45 'poor', .45 to .59 'fair', .60 to .74 'good', and .75 and above 'excellent' [START_REF] De Wever | Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review[END_REF][START_REF] Landis | The measurement of observer agreement for categorical data[END_REF][START_REF] Neuendorf | The content analysis guidebook[END_REF]. We apply the same criteria to alpha. Table 3 shows for reliability because it does not correct for chance agreement, and we report this solely for comparison. Kappa is computed because this is the most widely used statistic that corrects for agreement by chance. However, recent publications revealed that kappa behaves strange, i.e.

the kappa for two coders with a radically different distribution of frequencies over categories will be higher than coders with a similar distribution [START_REF] Artstein | Kappa 3 = alpha (or beta)[END_REF][START_REF] Krippendorff | Reliability in content analysis: Some common misconceptions and recommendations[END_REF]. Alpha does not suffer from this statistical artefact, so it should be preferred. We retain kappa for comparison because alpha is not widely used in CSCL or educational research. We will first discuss the pair-wise comparisons for the social and problem-solving dimension.

When only those units coded by both coders are included in the computation -κ 1 and α 1the reliability is consistently higher than proportion agreement, which is expected because κ 1 and α 1 do not treat all units coded by only one coder as disagreement. It should be noted that alpha affords to 'include' missing values in the data matrix, however, units coded by only one coder are ignored in the final computation. So, although it seems that more units are included there is computationally no difference with the case where these units are excluded (Table 3 shows the number of units that 'appear' to be used for the computation for α 1 but they are in reality the same as for κ 1 ).

When the missing values for units that were coded by only one coder are categorised 'no code' and this 'extra' category is included in the computation -κ 2 and α 2 -reliability drops.

This is stronger for the social dimension as compared to the problem-solving dimension, and is caused by the number of missing values; more missing values lead to a stronger downward correction when these are treated as disagreement. Alpha and kappa have similar values, but differ slightly (caused by the different distribution of frequencies over categories).

When the missing values and all units that were not coded by both coders are included and categorised as 'no code' -% A , κ A and α A -proportion agreement is consistently higher, α A is higher than α 2 for the social and problem-solving dimension but is lower than α 1 for the social dimension and equal to α 1 for the problem-solving dimension. The same pattern is visible for the three kappa indices.

Since proportion agreement does not correct for chance agreement and kappa suffers from a statistical artefact, alpha is preferred. Excluding missing values in the computation neglects a source of disagreement and inflates reliability, so α 1 is not adequate. Including all units that were not coded by both coders appears appealing and consistent but treats those units that are conceptually not eligible to receive a code as agreement. So, α A also inflates reliability and is not adequate. Including only those units coded by either coder, categorising missing values as 'no code', is the most strict computation. Thus, α 2 should be preferred although this statistic is a slight underestimation of the possible 'eligible' units -because it ignores the ambiguous units that both coders considered but did not code -but this is favoured given the substantial overestimation if missing values are excluded or all non-coded units are included.

The pair-wise comparisons provide insight into the performance of particular coders, but if more than two coders are available this should be preferred. We had three coders and alpha is suited to compute reliability for more than two coders (although Fleiss kappa can also correct for multiple coders it applies only to nominal data, alpha can also be used for ordinal, interval and ratio data). Again, α 2 is preferred over α 1 and α A for the case of three coders, and appears the best approximation for the reliability for the social and problem-solving dimension.

Considering the reliability statistics for three coders, alpha for the conversation dimension can be considered 'good' for both trails, .653 for R1 and .689 for R2. The alpha for the social dimension can be considered 'fair' for both trials, .462 for R1 and .480 for R2. The alpha for the problem solving dimension is 'poor' for R1 (.370) and 'fair' for R2 (.523).

Discussion

CSCL research using chat technology has focused primarily on dyads. The VMT project investigates chat-based small-group problem solving. During the development of a multidimensional coding scheme to analyse interactions in these groups, three new issues emerged that have strong implications for content analysis methodology and practice in general and chat communication in particular.

The first methodological issue concerns unit fragmentation. We chose the chat line as the unit of analysis because this is defined by the user, but frequently an utterance spanned across several chat lines makes sense only when considered as a whole. Consequently, connections between these units were required prior to coding, and two codes were added to the conversation dimension to mark these fragments (setup and extension).

The second issue concerns the need to reconstruct the response structure. Whereas in a dyadic chat the intended recipient is always the other partner, it is not easy to determine this in a small group. Similarly to fragmentation, the connection between chat lines forming a chain of responses needs to be reconstructed prior to coding of the conversation dimension.

Furthermore, the delay between chat line postings proved to be relevant to determining the response structure. Also, a coder must be familiar with the conversational codes. Assignment of both types of connections is performed simultaneously and termed 'threading' and a deep interpretation of what is going on in the chat. Aggregating all coding divergence would result in very low reliabilities, so agreement on threading prior to coding is necessary.

The third methodological issue concerns reliability calculation. We conducted two trials and computed the reliability for both types of threading. Reliability for the conversation and problem-solving threading could only be expressed as a proportion agreement, but this proved to be sufficiently reliable. Calculation of reliability for the social and problem-solving dimension was problematic: not all chat lines are valid analysis units for these dimensions and can lead to overestimation of their reliability. The extent of overestimation was shown by non-coded units are included. If available the use of more than two coder is preferred, and the valid pool of units should be reported (see for example Hurme & Järvelä, 2005, p. 6).

We included proportion agreement and Cohen's kappa for comparison, although both statistics are problematic. Overall, coding reliability -Krippendorff's alpha for three codersranged 'poor' to 'good' in the first trial and 'fair' to 'good' in the second trail. Nevertheless, reliability is only one aspect of a coding scheme-addressing the extent to which the coding can be reproduced-and it should not be mistaken for validity. We conclude with some reflections on validity.Once we had reliable coding of ten chat logs, we looked for statistical patterns. It turned out that the chats almost fell into two sets depending upon whether the students had seen the math problems in advance of their chats or not. However, there were two anomalous chats that fell into the wrong sets. The use of codes brought this anomaly to our attention, but could not explain it. Using conversation analysis, we could see a difference in interaction patterns that we termed expository versus exploratory [START_REF] Mercer | Is "exploratory talk" productive talk?[END_REF][START_REF] Zemel | Analyzing the organization of collaborative math problem-solving in online chats using statistics and conversation analysis[END_REF]. Subsequently, we found that students working in our chat environment developed methods of interacting that were not adequately captured-let alone explained-by codes adopted from the work of researchers investigating other media or from a priori theories of interaction. For instance, we determined that 'math proposal adjacency pairs' often play a distinctive driving role in our math chats [START_REF] Stahl | Sustaining group cognition in a math chat environment[END_REF].

Ethnomethodologically-informed design-based research needs to grasp the methods that participants creatively invent in response to innovative learning situations and technologies; they cannot simply reduce everything to instances of categories of actions generalised from past studies.

Also, we are particularly interested in group cognition (Stahl, 2006a) that takes place at the group unit of analysis, while coding schemes generally focus on the individual. For instance, we look at problem solving by the group as a whole (Stahl, 2006d). Our coding scheme tried to capture group phenomena like proposal bid-and-uptake or interaction question-and-answer by coding these as sequences of individual contributions (e.g., offer followed by response). The format of chat logs and the traditions of coding practice misled us to fragment group interactions into individual contributions. We now want to look at paired interactions and longer sequences as atomic elements of chats.

As the VMT environment evolved and incorporated a shared whiteboard, graphical referencing, math symbols and other functionality, even our multi-dimensional coding of utterances could not capture the increasingly complex and innovative interactions (Stahl, 2006e). To understand the unique behaviors as students adapt to the new environmentcustom technology, pedagogical guidance, open-ended math worlds-we need to look closely at the design of unique group interactions, and not simply code them with pre-existing categories, no matter how multi-dimensional and reliable. While general codes can be applied to many of these phenomena, they do not capture what is new, as required for design-based research. Reducing the chat to a sequence of codes that are general enough to be applied reliably, can eliminate the content and details that are of particular interest [START_REF] Stahl | Rediscovering CSCL[END_REF]. This is a paradox of reliable and valid coding efforts in exploratory CSCL research.

196

.699 % = percentage agreement, κ = Cohen's kappa, α = Krippendorff's alpha, κ 1 = kappa with missing excluded, α 1 = alpha with missing excluded, κ 2 = kappa with missing as disagreement, α 2 = alpha with missing as disagreement, analysis units in italics, % A , κ A , and α A = percentage, kappa and alpha when all units are included.

  ************************************ Insert Table 1 about here ************************************

  problem-solving thread. ************************************ Insert Table 2 about here ************************************ A threshold for the proportion agreement reliability of segmentation does not exist in CSCL research[START_REF] De Wever | Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review[END_REF][START_REF] Rourke | Methodological issues in the content analysis of computer conference transcripts[END_REF], nor in

  only units coded by both coders (exclude units with missing values) b) categorise missing values as 'no code' and include this category; c) categorise missing values and non-coded units as 'no code' and include this category. For possibilities a) and c) we calculated three reliabilities indices as suggested by De Wever et al. (2006): proportion agreement (%), Cohen's kappa (κ) and Krippendorffs alpha (α) (the latter two correct for chance agreement) for each dimension and pair of coders.Option b) was only computed for kappa and alpha. To determine whether the reliability is sufficient the .70 to .80 range is mostly used as criterion for proportion agreement.

  the reliability results for the conversation, social and problem-solving dimension. ************************************ Insert Table 3 about here ************************************ Although proportion agreement is still often used, it is insufficient to serve as an indicator

  calculating reliability for the case where a) only units coded by both coders are included (missing values are excluded), b) missing values are categorised as 'no code' and included in the computation, and c) missing values and non-coded units are categorised as 'no code' and included in the computation. We computed and compared three reliability indices and concluded that excluding missing values and including all non-coded units lead to overestimation. Including missing values as a 'no code' category is the most strict computation and a slight underestimation of the reliability. In our opinion a slight underestimation should be favoured given a substantial overestimation if units with missing values are excluded or all

Table 2

 2 presents the results for both reliability trials for each pair of coders. The first trial (R1) consisted of 500 chat lines and the second trial (R2) consisted of 449 chat lines. The top of table 2 presents the results for the conversational thread and the bottom the results for the

Table 1

 1 Threading reconstruction (derived from reliability trial R1)

	Line Name Utterance	Time	Delay T1	T2	T3 TA
	154 AME How about you fir	7:28:03 0:15
	155 AME Do you agree	7:28:35 0:32 154		154 154
	156 AME nvm	7:28:50 0:15
	157 MCP I used cos(22.5) instead of .924.	7:28:55 0:05 151 153 153 153
		Got 4.2498ish
	158 AME lets go on	7:28:55 0:00 156 156 156 156
	159 AME Its close enough	7:29:16 0:21 157 157 157 157
	160 AME How about 4.25?	7:29:22 0:06		157 157
	161 MCP I guess use 4.6^ -4.25^ to get	7:29:53 0:31 160 160	160
		BV^2
	162 AME ya	7:30:03 0:10 161 161 161 161
	163 MCP Then 16 * that, again	7:30:05 0:02	161 161 161
	164 AME I got 1.76 or so	7:31:03 0:58		161
	165 MCP yes	7:31:09 0:06 164 164 164 164
	166 AME So the perimeter should be 28.16	7:31:28 0:19	164 164 164
	167 FIR	ye!	7:31:44 0:16 166 164 166 166
	168 FIR	*YES!	7:31:51 0:07 167 167 167 167
	T1 = Thread coder 1, T2 = Thread coder 2, T3 = Thread coder 3, TA = Agreed after
	discussion.

Table 2

 2 The proportion agreement indices for the conversational and problem-solving thread by coder

	pair and reliability trial
			Conversational thread
		R1		R2
	Pair	% thread	% same	% thread	% same
	1 -2	.832	.731	.835	.712
	1 -3	.778	.727	.824	.749
	2 -3	.750	.687	.832	.730
			Problem-solving thread
		R1		R2
	Pair	% thread	% same	% thread	% same
	1 -2	.756	.928	.942	.983
	1 -3	.805	.879	.909	.967
	2 -3	.753	.890	.880	.935

Table 3 .

 3 Proportion agreement, kappa and alpha by coder for the conversational, social and problem-solving dimension

	Conversation dimension

Table 3 .

 3 Proportion agreement, kappa and alpha by coder for the conversational, social and problem-solving dimension (continued)

									Problem-solving dimension
						R1								R2
			Missing	Missing	Missing and no-code		Missing	Missing as	Missing and no-code
			excluded	'no code'	units included (U = 500)		excluded	'no code'	units included (U = 449)
	Pair	%	κ 1	α 1	κ 2	α 2	% A	κ A	α A	%	κ 1	α 1	κ 2	α 2	% A	κ A	α A
	1 -2	.469	.631	.628	.382	.385	.821	.622	.613	.657	.674	.666	.588	.576	.864	.766	.762
		178	127	178	178	178				178	158	178	178	178
	1 -3	.351	.564	.543	.229	.242	.782	.514	.504	.553	.649	.662	.484	.464	.804	.675	.665
		172	97	172	172	172				195	147	195	195	195
	2 -3	.439	.542	.520	.339	.340	.834	.618	.608	.556	.576	.654	.485	.469	.815	.688	.667
		148	106	148	148	148				190	146	190	190	190
	3 coders			.563		.370			.576			.650		.523
				181		181						196

225182

Acknowledgements

The authors express thanks to Murat Cakir, Nan Zhou, Ramon Toledo and Johann Sarmiento for their hard work in developing, testing and using the coding system in the reliability trials. The reported research was funded by NSF grants 0325447 and 0333493.

Appendix A. VMT coding scheme example (italic signals addition during calibration)


Michiel B Oortwijn
email: oortwijn@fsw.leidenuniv.nl

Monique Boekaerts

Paul Vedder

Jan-Willem Strijbos

Helping behaviour during cooperative learning

Keywords: Helping behaviour, Mathematical ability, Ethnicity, Cooperative learning

Is helping behaviour (i.e., solicited help and peer tutoring) during cooperative learning (CL) related to subsequent learning gains? And can teachers influence pupils' helping behaviour? One hundred one 5th grade pupils from multiethnic schools, 10-12 years old, participated in the study. Forty two pupils (31 immigrant) worked in an experimental condition, characterized by the stimulation of solicited high quality help and 59 (24 immigrant) worked in a control condition. It was found that learning gains were predicted positively by pupils' unsolicited helping behaviour (i.e., peer tutoring) and negatively by solicited help. Furthermore, teachers were able to affect pupils' low quality solicited help only. Lastly, immigrant pupils used less helping behaviour than local pupils, irrespective of CL setting.

Introduction

Often, during direct teaching in elementary schools there is little room for pupils to ask their classmates for help [START_REF] Johnson | Learning together and alone: Cooperative, competitive, and individualistic learning[END_REF]. Pupils often are inclined to ask the teacher for help, since they view the teachers as more able to facilitate learning as compared to their peers [START_REF] Newman | Students' perceptions of the teacher and classmates in relation to reported help seeking in math class[END_REF]. This is unfortunate since educational research has demonstrated that interactions between peers can augment their learning gains (e.g., [START_REF] Chinn | The structure of discourse in collaborative learning[END_REF][START_REF] Gillies | The effects of cooperative learning on pupils with learning difficulties in the lower elementary school[END_REF][START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF]. Thus, there is increased interest in the mechanisms that bring about effective peer interactions, that is, the constituents of peer interactions that are related to higher learning gains, the context factors that affect peer interactions, and the relation of peer interactions with learning gains (e.g., [START_REF] Fuchs | Effects of peer-assisted learning strategies in reading with and without training in elaborated help giving[END_REF][START_REF] Gillies | The effects of communication training on teachers' and students' verbal behaviours during cooperative learning[END_REF].

One of the most consistent findings in the literature is the positive effect of high quality verbal helping behaviour on learning gains [START_REF] Gillies | Children's cooperative behavior and interactions in trained and untrained work groups in regular classrooms[END_REF][START_REF] King | Structuring peer interaction to promote high-level cognitive processing[END_REF][START_REF] Topping | Trends in peer learning[END_REF][START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF][START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF]. Following [START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF], high quality verbal helping behaviour is defined here as those utterances of peers that ask for explanations, give explanations, or apply them on the task at hand. In the present study we investigated how high quality helping behaviour during cooperative learning (CL) affects pupils' subsequent learning gains. Our study differs from that of [START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF] in that we investigated both solicited and unsolicited helping behaviour and additionally considered three factors that might mediate the relationship between high quality verbal helping behaviour and learning gains, namely the role of the teacher and the pupil background characteristics, such as ethnicity and prior knowledge.

High quality helping behaviour

Researchers of peer interactions and learning tend to use different concepts and distinctions when exploring peer interactions. For instance, Nelson-Le Gall and Clor-Scheib (1985) distinguished executive help seeking (i.e., asking for an answer) from instrumental help seeking (i.e., asking for an explanation). [START_REF] Vedder | Cooperative learning: A study on processes and effects of cooperation between primary school children[END_REF] proposed that for instrumental help to be effective, the help receiver must understand the help given, have an opportunity to apply it, and actually apply it. Webb and her colleagues [START_REF] Webb | Constructive activity and learning in collaborative small groups[END_REF] integrated the aforementioned elements of solicited helping behaviour into an elaborate coding scheme that distinguished between high and low quality verbal helping behaviour in asking, giving, and applying help.

The effects of both solicited help giving and help receiving have been extensively studied. Studies have consistently reported that the help giver benefits from providing high quality solicited help (e.g., [START_REF] King | Structuring peer interaction to promote high-level cognitive processing[END_REF][START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF]. [START_REF] King | Structuring peer interaction to promote high-level cognitive processing[END_REF] asserts that this is because it stimulates the help giver to elucidate and reorganize knowledge and to recognize mistakes. In a group setting, a learner who is explaining a concept to another learner has to 'tune' the help to the cognitive level of the receiver. Through the reorganization of the knowledge needed for the attuning, the learner who gives the explanation comes to understand the concept more thoroughly. Clearly, solicited high quality helping behaviour may also benefit the help receiver. [START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF] emphasize that high quality help is only useful to the receiver when it is sufficiently elaborated, correct, on time, and links up to the need for help. However, the most accurate predictor of learning gains is whether or not the help receiver applies the help that is given. Not all help is asked for. Sometimes a pupil assumes the role of tutor, guiding the problemsolving process of another pupil, the tutee, by asking problem-solving questions or giving assignments that are aimed at solving the problem. We refer to this type of unsolicited helping behaviour here as peer tutoring. [START_REF] Topping | Trends in peer learning[END_REF] showed in a review study that peer tutoring can increase the learning gains of both the tutor and the tutee. Most studies on peer tutoring have focused on cross-age peer tutoring and generally have provided support for a positive relation between peer tutoring and learning gains [START_REF] Topping | Cross-age peer tutoring of science in the primary school: Influence on scientific language and thinking[END_REF]. [START_REF] Van Keer | Effects of explicit reading strategies instruction and peer tutoring on second and fifth graders' reading comprehension and self-efficacy perceptions[END_REF] suggested that the positive effect on reading performance was larger for crossage peer tutoring than for same-age peer tutoring. [START_REF] Topping | Cross-age peer tutoring of science in the primary school: Influence on scientific language and thinking[END_REF] found that tutoring not only boosted the learning gains of the tutees, but also of the tutors. In their study, the cognitive ability of the tutors was roughly of the same level as that of the tutees. They concluded that same-age peer tutoring might be just as effective as cross-age peer tutoring for students' learning gains. However, they also suggested that peer tutoring is most effective if the participating students are free to be a tutor or a tutee, depending on the nature of the problem. This is in keeping with a study by [START_REF] Robinson | Peer and cross-age tutoring in math: Outcomes and their design implications[END_REF] who argued that cross-age tutoring makes it almost impossible to establish reciprocal tutoring and therefore is less effective than reciprocal same-age peer tutoring.

Peer interactions during CL and learning gains 1.2.1. The teacher's role

Teachers play an important role in CL. What they are doing and not doing affects the quality of pupils' problem-solving process considerably. Teachers who promote complex cognitive communication between pupils boost the quality of peer interactions and performance [START_REF] Chinn | The structure of discourse in collaborative learning[END_REF][START_REF] Gillies | The effects of cooperative learning on pupils with learning difficulties in the lower elementary school[END_REF]. However, teachers seem to have difficulties to teach pupils to use high quality helping behaviour [START_REF] Gillies | Structuring co-operative learning experiences in primary school[END_REF][START_REF] Meloth | The role of the teacher in promoting cognitive processing during collaborative learning[END_REF]. A possible reason is a high task load for the teacher resulting from the requirements for CL [START_REF] Turner | Creating contexts for involvement in mathematics[END_REF]. Specifically, teachers need to plan learning activities geared at the acquisition of content knowledge and new domain skills as well as learning activities that help students to cooperate effectively. Additionally, teachers need to guide the CL skills of both individual students and groups as a whole. Effectively and efficiently satisfying these combined requirements takes effort and time on the part of the teacher, which means that promoting effective CL is a long-term project (Webb, Nemer, & Ing, 2006). Gillies andAshman (1997, 2000) demonstrated that when teachers do successfully stimulate high quality helping behaviour, pupils' communicative skills and performance are boosted: they found that pupils had higher learning gains, and their interactions were characterized by more high quality solicited helping behaviour and peer tutoring. Other studies have confirmed the positive effect of the stimulation of pupils' elaborated helping behaviour on their use of high quality helping behaviour [START_REF] Fuchs | Effects of peer-assisted learning strategies in reading with and without training in elaborated help giving[END_REF] and peer tutoring [START_REF] Nixon | Emergent writing: The impact of structured peer interaction[END_REF].

Student characteristics

Several studies have shown that students' characteristics, such as ethnicity and prior knowledge, influence their helping behaviour and learning gains in a CL setting.

Ethnicity. In the Netherlands, there are three major discernable ethnic groups with respect to their performance at school: (a) Moroccan, Turkish, and Antillean youth, (b) Surinamese and other ethnic youth groups (e.g., Asian, former Yugoslavia), and (c) Dutch youth. [START_REF] Tesser | Rapportage Minderheden 2001[END_REF] have shown that especially the performance of the Moroccan, Turkish, and Antillean groups falls behind. Research has revealed that the linguistic setback of these groups is a possible explanation of their low academic performance [START_REF] Tesser | Rapportage Minderheden 2001[END_REF]Vedder & Horenczyk, 2006). Since these ethnic groups together form the majority of immigrant youth in the Netherlands, it may come as no surprise that the academic performance of pupils in multicultural schools falls below the national mean [START_REF] Bosker | Een herijking van de gewichtenregeling [A recalibration of the financial arrangements for students with an educational setback[END_REF]. However, a study by [START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF] revealed that the implementation of CL can decrease the educational setback of immigrant pupils. They found that teachers who promoted pupils' use of solicited high quality helping behaviour boosted the solicited high quality helping behaviour of immigrant pupils and their mathematical setback was reduced, as compared to national pupils (see also [START_REF] Calderón | Effects of bilingual cooperative integrated reading and composition on students making the transition from Spanish to English reading[END_REF].

Prior knowledge. Several studies have shown that students with high prior knowledge are more able to stay focused on the group task and to plan and evaluate their actions [START_REF] Hmelo | Effects of high and low prior knowledge on construction of a joint problem space[END_REF][START_REF] O'donnell | Interactive effects of prior knowledge and material format on cooperative teaching[END_REF]. [START_REF] Puustinen | Help-seeking behavior in a problem-solving situation: Development of self-regulation[END_REF] argued that pupils with low prior knowledge are less able to self-regulate their learning, which heightens their need for structured group work and guidance by the teacher. Evidently, immigrant students in the Netherlands fall into this category of pupils.

Aims -Hypotheses

In this study we investigated how high quality verbal helping behaviour is related to learning gains in a CL setting. We were interested in how the interaction between ethnicity and prior knowledge with the teacher's role affects pupils' helping behaviour (both solicited and tutoring) and their subsequent learning. To assess the effect of solicited high quality helping behaviour on learning gains we constructed a CL curriculum that borrowed rules for solicited high quality helping behaviour from [START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF] and [START_REF] Webb | Constructive activity and learning in collaborative small groups[END_REF]. We manipulated the role of the teacher: one group of teachers was required to stimulate pupils' solicited high quality helping behaviour (experimental condition), while the other group was required not to do this (control condition). Pupils' prior knowledge in mathematics was defined as "mathematical ability". Pupils were enrolled in ethnically diverse classrooms.

Our hypotheses were the following: Both tutoring behaviour and high quality solicited verbal helping behaviour will be positively related to posttest mathematical performance (Hypothesis 1). National pupils will have an advantage over immigrant ones as regards the frequency with which they display tutoring behaviour and high quality solicited verbal helping behaviour because of the more limited linguistic proficiency of immigrant pupils (Hypothesis 2). Pupils in the experimental condition will use more high quality solicited verbal helping behaviour than in the control condition (Hypothesis 3). Following [START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF] and [START_REF] Gillies | The effects of cooperative learning on pupils with learning difficulties in the lower elementary school[END_REF], however, we expected that there will also be interactions of condition, ethnicity, and mathematical ability. Specifically, we predicted that immigrant pupils and pupils with low mathematical ability will display higher learning gains and more solicited high quality helping behaviour in the experimental condition. Also, national pupils with low mathematical ability will display more solicited high quality helping behaviour in the experimental condition as compared to immigrant pupils with low mathematical ability (Hypothesis 4).

Method

1. Sample

The total sample comprised 48 groups of 3 to 4 pupils each (N = 166), from 10 classes of 5th grade. Each teacher and his/her classroom were randomly assigned to the experimental or control condition. Ten teachers (nine Dutch, one immigrant; one male, nine female) participated in this study (M = 41 years, SD = 8.6). Four of them were experienced teachers (i.e., teaching more than 10 years, two in the experimental and two in the control condition). Two CL lessons were video-recorded in order to gain a representative picture of the interactions of the pupils. Since we were only interested in the peer interactions, not in the teacher-pupil interactions, we selected only those recordings in which the peer interactions were not interrupted by the teacher. A subsample of 27 groups (n = 101) qualified for further analysis, consisting of 53 boys and 48 girls (M = 135.2 months, SD = 6.4). The mean length of the two video recording episodes was 1372.4 seconds (SD = 142.4) and did not differ between conditions.

----------------------------------------Insert Table 1 about here - ---------------------------------------There were 12 groups in the control condition (n = 42) and 15 groups in the experimental condition (n = 59). The groups were narrow-heterogeneous in terms of mathematical ability (high-middle or low-middle) and were formed by the teacher and the researcher. As regards ethnicity, pupils were considered national when at least one parent was of Dutch origin and immigrant when both parents were of foreign origin. There were 11 national and 31 immigrant pupils in the control condition. In the experimental condition there were 35 national pupils and 24 were immigrant pupils (see Table 1). The majority of the immigrant pupils were of Moroccan or Turkish descent: 67% in the experimental condition and 87% in the control condition. Mathematical ability and linguistic proficiency were used as covariates in the analyses.

Instruments

As a manipulation check a teacher checklist on CL implementation and videotaped teacherpupil interactions was used. Pupil performance was assessed with a mathematical pre-and posttest, a linguistic proficiency test. Helping behaviour was assessed with a coding scheme for verbal peer interactions.

Teacher checklist on CL implementation

Teachers rated on a 4-point Likert-scale (1 = 'very often' and 4 = 'very little') the extent to which they had implemented a number of CL rules. A principal component analysis with varimax rotation revealed a three-factor solution. The solution explained 71 % of the variance. All factor loadings were higher than .50. The first factor (18 items, Cronbach's α = .97) comprised statements about general CL rules (e.g., ''I teach the children not to interrupt each other''). The second factor (5 items, Cronbach's α = .81) referred to the rules for giving help and receiving help (e.g., ''I teach the children to keep asking when someone poses an unclear question''). The third factor (4 items, Cronbach's α = .84) regarded the feedback on the CL process (e.g., ''At the end of each lesson I discuss with each group what is going well and what should be improved''). Each teacher completed the checklist at the end of every other mathematical lesson, starting at the first lesson, amassing five checklists in total.

Videotaped teacher-pupil interactions

All teachers were videotaped during two, randomly selected, lessons. The teachers were not told in advance which CL lessons we would videotape. All recordings were rated by two independent scorers, one of whom was double blind to the experimental manipulation. The coders filled in a coding scheme of 14 items. A principal component analysis with varimax rotation was applied; 62 % of the variance was explained. All factor loadings were .50 or higher. The first factor (6 items, Cronbach's α = .71) concerned whole-class reflection on the group work (e.g., ''Did the teacher reflect on group performance in the prior lesson?''). The second factor (8 items, Cronbach's α = .86) regarded the teacher's activities during the group work (e.g., "Did the teacher encourage group members to ask each other questions?"). The items were rated on 3-point Likert-scale (1 = 'little' and 3 = 'often'). The inter-coder reliability (calculated over two recordings, approximately ten percent of the total number) was satisfactory: for Factor 1 kappa = .73 and for Factor 2 kappa = .62.

Mathematical ability

Scores from a curriculum-independent mathematical test by the Central Institute for Test Research (CITO; [START_REF] Janssen | Rekenen-Wiskunde 3. Handleiding[END_REF] were used to assess the baseline mathematical performance of all pupils. The teachers scored the test for all pupils. Previous research has shown that CITO has a good internal consistency, Cronbach's α = .94 [START_REF] Evers | COTAN Documentatie van tests en testresearch in Nederland [COTAN Documentation on tests and test research in The Netherlands[END_REF].

Mathematical posttest

The mathematical posttest consisted of multiple choice items that assessed general knowledge of area, scale, fractions, percentage, and circle diagrams. A reliability analysis of the data obtained in this study revealed that the internal consistency was satisfactory, Cronbach's α = .75. A previous study demonstrated that the mathematical posttest significantly correlated with CITO, r = .77, p < .001 [START_REF] Oortwijn | Is coöperatief rekenen op multiculturele basisscholen effectiever dan klassikaal leren? [Is collaborative learning during math at multiethnic elementary schools more effective than direct teaching?[END_REF].

Linguistic proficiency

This test was taken from the National Testing Service, used to assess pupils' learning progress in elementary schools [START_REF] Janssen | Rekenen-Wiskunde 3. Handleiding[END_REF]. The scores of the two dimensions of the test, namely vocabulary and reading comprehension, were averaged in our study into the new variable linguistic proficiency. This variable was used to determine whether pupils' language proficiency affected their helping behaviour. Thus, it was used as relevant background characteristic of the pupils.

-

----------------------------------- Insert Table 2 about here -----------------------------------

Coding of verbal peer interactions

The coding scheme of [START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF] on solicited verbal help was adapted to suit the needs of this study. The resulting coding scheme was made up of four categories (see Table 2). [START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF] discussed the relationship of both help giving and help receiving with learning gains and investigated the relationship of help receiving with learning gains. We investigated both relationships.

Category 1, need for help, was composed of two subcategories: (a) request for an answer (low quality questions); it comprised request for information, e.g., ''What is the answer to this one?'', and general request for help, e.g., ''I don't get it'' (see also [START_REF] Webb | Help seeking in cooperative learning groups[END_REF] for a discussion of help that is not indicated by a question). (b) request for an explanation (high quality questions).

Category 2 targeted the level of verbally provided help by the help giver. It was composed of two subcategories: (a) low quality help, comprising unclear help, undesired help, and (numerical) outcome only, and (b) high quality help (explanation with a (part of a) problem-solving step).

Category 3, constructive activity on the current problem by the help receiver, was composed of two subcategories: (a) low quality constructive activity, that is, no reaction and acknowledgement of the help received and copying the provided (numerical) outcome, and (b) high quality constructive activity, that is, working out part of a problem-solving step and working out one problem-solving step.

We excluded the category constructive activity on the next problem from our study. Since authentic, open-ended mathematical tasks were used, none of them were truly independent from each other. Therefore, no satisfactory distinction could be made between the current and the next problem. However, we added another category (Category 4) that assessed the frequency of peer tutoring by counting the number of tutor actions. Tutor actions were defined as unsolicited utterances aimed at stimulating a group member to give a problem-solving oriented response by asking a question, giving an assignment, or giving help. In the experimental condition, we only manipulated solicited help, not peer tutoring, and had no pre-set idea of which pupils should be tutors frequently and which pupils would be frequent tutees. A pupil action was scored as tutoring when (a) a tutor asked a problem-solving oriented question, or gave a problem-solving oriented assignment, or explanation, and (b) the tutee gave a problem-solving oriented response -see also the example in Table 2. We marked for each tutor action which pupil took on the role of tutor and which pupil(s) assumed the role of tutee (see Appendix for an example).

The inter-coder reliability was calculated on six recordings (approximately 10% of the total sample) between two observers (the first author and a second coder, unfamiliar with the study). For Category 1 the agreement between the two coders was 83%, and kappa was .73. For Category 2, the agreement was 76%, and kappa was .60. The agreement was 72% for Category 3, and kappa was also .60. For Category 4 (unsolicited help), the agreement was 88% and kappa was .76. In a number of cases (5% of all utterances) one or both of the coders found an utterance to be impossible to be coded. These utterances were removed from the dataset for both coders (pair wise deletion). The second coder, who was blind to the manipulation, individually scored all the videotaped peer interactions.

Procedure

CL training

During a two-hour workshop the first author explained the essentials of effective CL to the teachers and instructed them how to implement it in the classroom. Subsequently, the teachers trained their pupils in two lessons how to effectively work in groups. In the first lesson general social CL rules were discussed and practiced. These rules required pupils to check whether: ''everyone cooperates'', ''everyone listens to each other'', ''everyone shares their knowledge and opinions'', and ''everyone agrees''. In the second lesson more specific CL rules were discussed and practiced. Adapted from [START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF], these rules were about high quality helping behaviour. Regarding receiving help, pupils were instructed to (a) ask precise questions, (b) continue asking in case of ambiguities, (c) think before asking a question, and (d) ask for help on time. With respect to giving help, pupils were instructed to (a) fine-tune the level of help to the need for help that is being requested, (b) give a clear and precise answer, (c) let the help receiver apply the help that is given, (d) continue to ask if the question for help is unclear, and (e) give help when needed.

CL mathematical curriculum

The CL mathematical curriculum consisted of nine one-hour lessons followed by a mathematical exam. The teachers in the control condition were required not to intervene in pupils' interactions. They were instructed only to interact with the pupils to tell them to talk less loudly (and not disturbing other groups), to listen to each other, or to stop making fun of each other. Teachers in the experimental condition were instructed to stimulate pupils' solicited high quality helping behaviour, as specified in the second lesson of the CL training, and to promote their use of general CL rules of the first lesson as much as possible. Lesson-to-lesson protocols were used to help the teachers implement CL in their condition (experimental and control condition).

The mathematical assignments used in this CL curriculum were authentic mathematical assignments. These are mathematical tasks with a strong narrative structure and which are embedded in contexts familiar to the children, such as calculating the area of classrooms in their school. They dealt with area, scale, fractions, percentage, and circle diagrams. All mathematical assignments were adjusted for CL purposes using authentic mathematical assignments from the regular mathematical curriculum. Pupils worked on two assignments per lesson.

During two randomly selected lessons -one somewhere at the beginning (Lesson 1-5) and one near the end of the CL curriculum (Lesson 7-9) -video recordings were made of the peer interactions in both the experimental and the control condition. After the CL mathematical curriculum all pupils individually completed a mathematical exam.

Results

We start with the manipulation check. Regarding the teacher checklist, we found that teachers in the experimental condition reported instructing pupils more in the use of helping behaviour throughout the CL curriculum, t(21) = -3.37, p < .005, than the teachers in the control condition, with a large effect size, Cohen's d = 1.48. Remarkably, no differences were found on general CL rules and on extent of feedback on the CL process.

Regarding the videotaped teacher-pupils interactions we found that teachers in the experimental condition elaborated more on the group work at the start and the end of the lesson than teachers in the control condition, t(16) = -1.78, p < .05, which equates to a moderate effect size, Cohen's d = .58. No differences were found for the factor CL activities during group work.

Hypothesis 1 4.1.1. Analyses at the individual level

Hierarchical regression analyses of the performance of the individual pupils were employed to investigate whether peer tutoring and high quality solicited helping behaviour predicted subsequent mathematical performance. The predictor variables were ethnicity, condition, mathematical ability, requests for explanations, number of tutor actions provided by the tutor, number of tutor actions received by the tutee, high quality help, and high quality constructive activity. Posttest mathematical performance was the criterion variable (see Table 3). The analysis showed that mathematical ability was the main predictor of posttest mathematical performance. Of the other variables entered in Step 2 mathematical ability and requests for explanations were significant predictors. Requests for explanations were negatively associated with posttest mathematical scores. Remarkably, inclusion of the variables Condition and Ethnicity in the equation did not significantly changed the explained variance.

-----------------------------------Insert Table 3 about here ----------------------------------In a similar hierarchical regression analysis, in which the Number of Tutor Actions Provided by the Tutor and Number of Tutor Actions Received by the Tutee instead of Requests for Explanations were included, only a positive relation between Number of Tutor Actions Provided by the Tutor and posttest mathematical performance was found. After Step 2, mathematical ability and the number of tutor actions provided by the tutor were positively predicting posttest mathematical performance (see Table 4).

------------------------------------Insert Table 4 about here - ----------------------------------

Analyses at the group level

We performed analyses at the group level in an attempt to corroborate the findings we found at the individual level. A number of researchers have suggested that more valid conclusions on learning in a social setting can be drawn when one takes into account multiple analytical perspectives (e.g., [START_REF] Rogoff | Observing sociocultural activity on three planes: Participatory appropriation, guided participation, and apprenticeship[END_REF]. Due to the small sample size, the relationship of helping behaviour with posttest mathematical performance could not be evaluated with a multi-level approach. Inspired by earlier studies using a similar approach [START_REF] Gillies | The effects of cooperative learning on pupils with learning difficulties in the lower elementary school[END_REF][START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF], we conducted analyses at the group level by aggregating individual scores for each team.

The variable Group Level Tutoring Behaviour was created as the mean number of tutor actions provided by the tutors in each group. The variable had a severe skewness and kurtosis. A logarithmic transformation reduced the kurtosis and skewness to a value < 1, which is acceptable (DeCarlo, 1997). Partial correlations were calculated for requests for explanations and group level tutoring behaviour, corrected for mathematical ability. We found a negative trend for requests for explanations (M = 1.65, SD = 1.34), r = -.36, p < .08 (two-tailed). No relationship was found between group level tutoring behaviour and posttest mathematical performance. However, when we selected only those groups with a high number of tutor actions (i.e., at least one tutor action per group member, n = 10), we did find a positive trend, provided we corrected for mathematical ability (M = .57, SD = .44), r = .63, p < .07 (two-tailed).

Hypothesis 2

The linguistic proficiency of the immigrant pupils was lower than that of the national pupils, t(88) = 3.46, p = .001. We explored whether the lower linguistic proficiency of the immigrant pupils was related to the use of tutor actions. We expected that if national pupils provided more tutor actions than immigrant pupils, this would be related to a more limited linguistic proficiency of immigrant pupils. An independent samples t-test revealed a significant effect, t(21) = 2.89, p < .01. A Mann-Whitney test corroborated this finding, Z(21) = -2.42, p < .02. National pupils who provided tutor actions had a higher linguistic proficiency than immigrant tutors.

Hypothesis 3 and 4

We carried out a 2(condition) x 2(ethnicity) MANCOVA to analyze the relationship of condition with high quality solicited helping behaviour. Mathematical ability was the covariate. Additionally, we carried out a regression analysis to investigate the effect of the interaction Condition x Mathematical ability and the interaction Ethnicity x Mathematical ability on high quality solicited helping behaviour.

Analyses at the individual level

In the above MANCOVA the dependent variables were the seven categories of the coding scheme (i.e., request for an answer, request for an explanation, low and high quality of provided help, low and high quality constructive activity, and tutor actions).

- -----------------------------------Insert Figure 1 about here ---------------------------------------No main effects were found for condition and ethnicity, although there was a trend for condition. We found a 2-way interaction effect for the relation of condition and ethnicity with the level of verbally provided help, Wilks' Lambda = .89, F(2, 95) = 5.76, p < .005, η² = .11. National pupils in the control condition provided more low quality help than immigrant pupils, F(1, 38) = 4.94, p < .04, η² = .12 (see Figure 1). In the experimental condition no such difference was found. Regarding low and high quality constructive activity and tutoring no effects were found.

We conducted a regression analysis to test the impact of the Mathematical ability x Condition and Mathematical ability x Ethnicity interactions (predictor variables) on the categories of the coding scheme (criterion variables). To compensate for effects of multiple testing, we used a stricter alpha (α = .01). No effects were found on the subcategories of Need for Help. In the subcategories of Level of Verbally Provided Help, the interaction Ethnicity x Mathematical ability explained 17% of the variance of high quality help, β = -.42, t(43) = -2.79, p < .01 (see Table 5). Immigrant pupils with low mathematical ability provided more high quality help than national pupils with a low one. Immigrant pupils with medium to high mathematical ability provided less high quality help than national pupils with a medium to high one. Regarding the subcategories of Constructive Activity on the Current Problem the interaction of ethnicity with mathematical ability explained 19% of the variance in low quality constructive activity, β = -.36, t(67) = -3.08, p < .004. Immigrant pupils with low mathematical ability showed more low quality constructive activity than the respective national pupils. In contrast, immigrant pupils with medium to high mathematical ability showed more low quality constructive activity than the respective national pupils. We found no relations between the aforementioned criterion variables and tutoring (Category 4).

----------------------------------Insert Table 5 about here ----------------------------------

Analyses at the group level

We investigated whether the interaction effect of ethnicity and condition with low quality help that we found at the individual level was maintained at the group level. We recoded ethnicity into a group-level variable: we considered a group national if there was one or no immigrant pupil present and we regarded a group as immigrant when there was one or no national pupil in the group. The groups with an even distribution of immigrant and national pupils (n = 4) were left out of the group level analyses. We conducted an ANCOVA analysis in order to examine whether we could support the two-way interaction effect of condition and ethnicity with low quality help, corrected for mathematical ability, which we found at the individual level. The group level analysis corroborated the 2-way interaction effect we found at the individual level, F(1, 18) = 6.63, p < .02, η² = .27. National groups provided more low quality help in the control condition than immigrant groups. We could not confirm the relation between mathematical ability and ethnicity with high quality help and low quality constructive activity which we found in the analyses at the individual level.

Summary of the findings

The frequency of requests for explanations was negatively related to posttest mathematical performance and the number of tutor actions provided by the tutor was positively related to posttest mathematical performance, both at the individual and the group level. These findings partly support Hypothesis 1. In addition, we found that national pupils provided more tutor actions than immigrant pupils. This was associated most notably with a lower linguistic proficiency of immigrant tutors, as Hypothesis 2 predicted. Hypothesis 3, however, that predicted a condition effect on high quality helping behaviour was not verified. Instead there was an interaction of condition with ethnicity and mathematical ability as Hypothesis 4 predicted. National pupils in the control condition provided more low quality help than immigrant pupils. This finding was corroborated at the group level.

Analyses at the individual level further showed that, regardless of condition, immigrant pupils with low mathematical ability provided more high quality help and used more low quality constructive activity than their national counterparts. In contrast, immigrant pupils with medium to high mathematical ability provided less high quality help and used less low quality constructive activity than their national counterparts.

Discussion

The finding that the number of tutor actions provided by the tutor was positively related to subsequent mathematical performance of the tutor is in line with other studies that also pointed out the significance of peer tutoring in CL [START_REF] Duran | Styles and sequences of cooperative interaction in fixed and reciprocal peer tutoring[END_REF][START_REF] Gillies | Children's cooperative behavior and interactions in trained and untrained work groups in regular classrooms[END_REF], 2000;[START_REF] Topping | Trends in peer learning[END_REF]. [START_REF] Topping | Trends in peer learning[END_REF] suggested that for peer tutoring to be effective for both the tutor and the tutee, it has to be stimulated by the teacher. However, our instructions to the teachers specifically targeted the solicited helping behaviours and not peer tutoring. On the other hand, the study of [START_REF] Duran | Styles and sequences of cooperative interaction in fixed and reciprocal peer tutoring[END_REF] suggests that peer tutoring is most successful when the tutor and tutee interact on an equal or reciprocal basis. A stronger relation between peer tutoring and posttest mathematical performance might have been found if the teachers had been instructed to stimulate reciprocal tutoring.

This study also demonstrated that pupils do not have to be the best in mathematics to be successful tutors. There is mounting evidence that peer tutoring is most successful when the tutor and tutee cognitively challenge each other, meaning that their cognitive abilities are roughly the same [START_REF] Topping | Trends in peer learning[END_REF][START_REF] Topping | Cross-age peer tutoring of science in the primary school: Influence on scientific language and thinking[END_REF]. The fact that national pupils assumed the role of tutor more often might have to do with their higher linguistic proficiency. It might also be related to their willingness to assume the tutor role.

We found no relationship of provided high quality help and high quality constructive activity with subsequent mathematical performance. A reason could be a transfer problem. In the present study, and different from the [START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF] study, the mathematical posttest contained problems that were meant to be different from the type of assignments that the pupils completed during the CL lessons. [START_REF] Webb | The development of students' helping behavior and learning in peer-directed small groups[END_REF] used a program-dependent mathematical test, whereas we used a mathematical test that was more general and program-independent. A second reason might be pupils' lack of CL experience. [START_REF] Limón | On the cognitive conflict as an instructional strategy for conceptual change: A critical appraisal[END_REF] suggested that minimal prior CL knowledge hampers productive participation in CL. Our study showed that pupils had minimal experience with and knowledge of CL. It is possible that the pupils adopted the solicited low quality helping behaviour when solving a mathematical assignment because they were more comfortable with it. Other studies also reported that teachers are less successful in increasing high quality helping behaviour if pupils lack the skills for high quality peer interactions [START_REF] Prichard | Team-skills training enhances collaborative learning[END_REF]. In addition to a lack of CL experience, the mathematical tasks may have been too complex (both linguistically and mathematically) for the pupils -of whom a significant part had both a linguistic and a mathematical setback -resulting in a cognitive overload. Research by [START_REF] Pollock | Assimilating complex information[END_REF] has suggested that pupils who lack the necessary skills to solve complex tasks (i.e., linguistic and mathematical low-achievers) are prone to a cognitive overload. These pupils might be helped more with a highly structured direct teaching setting (see also [START_REF] Tesser | Rapportage Minderheden 2001[END_REF].

Regarding the teacher's role during CL we found that teachers were unable to stimulate pupils' high quality helping behaviour, although they were successful in reducing low quality helping behaviour. Three reasons are put forward here for this finding. First, the CL curriculum was not long enough. Researchers have demonstrated that teaching pupils to use high quality helping behaviour is a long-term, time consuming process [START_REF] Cohen | Restructuring the classroom: Conditions for productive small groups[END_REF]Webb et al., 2006). Nevertheless, an earlier study [START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF] did reveal that the teachers were able to stimulate pupils' solicited high quality helping behaviour on a relatively short term. These researchers implemented a CL mathematical curriculum that covered ten weeks and audiotaped the peer interactions in the eighth week. This differs from our study, in which we videotaped the peer interactions of each group twice, the first near the beginning and the second near the end of the CL curriculum. We did this in order to gain a more representative picture of the frequency of the occurrence of the (sub)categories of the coding scheme. It is plausible that we could not corroborate [START_REF] Webb | Promoting helping behavior in cooperative small groups in middle school mathematics[END_REF] results because the children were not yet familiar with the use of high quality helping behaviour during the first video recording.

Second, teachers in the experimental condition reported instructing their pupils more in the use of solicited high quality helping behaviour than in the control condition. Nevertheless, observation of videotaped teacher-pupil interactions revealed that the teachers only instructed their pupils in the use of solicited high quality helping behaviour at the beginning and end of the group work; not during the group work. Why did the teachers not give CL feedback when the pupils required it? It might be that the teachers did not accurately perceive the effectiveness of their own CL activities. Earlier studies have made similar suggestions (e.g., [START_REF] Vedder | The role of the task and reward structure in cooperative learning[END_REF]. Indeed, researchers have suggested that teachers are not well equipped to implement effective CL (e.g., [START_REF] Gillies | Structuring co-operative learning experiences in primary school[END_REF]. It could also be that, although the teachers in this study were given detailed instructions (both orally and in written form), individual differences in teaching style blurred differences between the two conditions. Previous studies have suggested that teaching style is hard to change (see [START_REF] Gill | Changing preservice teachers' epistemological beliefs about teaching and learning in mathematics: An intervention study[END_REF] for a more detailed discussion), which might well mean that changing the teaching style in order to let teachers implement more effective CL takes longer than the 11 lessons this CL curriculum consisted of.

The third reason, which is related to the second, regards the lack of experience of both the pupils and the teachers with CL. Webb et al. (2006) pointed out that pupils tend to copy teacherpupil interactions in their own interactions with fellow group members. When a lack of CL experience results in the teachers not instructing pupils properly in the use of helping skills and in giving no example of good practice, pupils are likely to model behaviour that is poor on examples of help giving. This might explain why we found no difference between the two conditions both with respect to the instruction of solicited high quality help by the teacher and the use of solicited high quality help by the pupils. The aforementioned finding suggests that further research is warranted to investigate how teacher background characteristics (e.g., experience with CL) affect the effectiveness of their teaching behaviour during CL.

Finally, we found mixed results for the role of ethnicity in helping behaviour. We could not find support for our hypothesis that immigrant pupils with low mathematical ability would use less high quality helping behaviour than the respective national pupils. However, we did find that the immigrant pupils in general incorporated less verbal helping behaviour in their communication. These findings are in line with other studies suggesting that immigrant pupils are less actively involved in group work (e.g., [START_REF] Kirchmeyer | Multicultural task groups: An account of the low contribution level of minorities[END_REF]. In our study we found that this was due most notably to a lower linguistic proficiency of immigrant pupils as compared to national pupils.

Limitations

It is conceivable that the frequency of high quality help is higher when the pupils are accustomed to CL. Thus, if we had videotaped two lessons at the end of the CL curriculum we might have found more pronounced differences in high quality helping behaviour between the experimental and the control condition. Also, the immigrant pupils were overrepresented in the control condition where in fact they formed the majority. This might have affected the results, although it is important to point out here that the immigrant pupils in the control condition did not differ from the immigrant pupils in the experimental condition with regard to their use of helping behaviour.

Conclusion

Earlier studies have shown that the stimulation of high quality helping behaviour during CL is associated with higher learning gains. Our study suggests, however, that incorporating high quality helping behaviour in CL not only strains pupils' cognitive capacities, but also places high demands on teacher behaviour. It is recommended that future studies implement training programs that take into account relevant background characteristics of both pupils and teachers (see also Webb et al., 2006). Additionally, the data revealed that, although pupils were not trained in the use of peer tutoring, there was a positive relation between peer tutoring and posttest mathematical performance. This suggests that peer tutoring requires less training than solicited high quality help and is better suited to augment mathematical performance. Moreover, peer tutoring may be more effective to reduce pupils' educational setback. L1: Ah, I get it. You multiply 3 with 2 to get the area. That's 6.

Appendix. Example of a coded interaction fragment

Utterance

------4. Tutor actions (unsolicited help) Utterance targeted at provoking a problemsolving response from a peer L1: Area is times. So, the length times the width. That's the area. So, 3 times 2 is? L2: 6. IV. Constructive activity on next problem ------* Category 2 targeted the level of helping behaviour of the help provider.

Figure 1 .

 1 Figure 1. Mean individual low quality help provided by national pupils (n = 45) and immigrant pupils (n = 56), corrected for mathematical ability in the control and the experimental condition.

Table 3

 3 Hierarchical regression analysis of pretest mathematical score, request for an explanation, condition, and ethnicity on posttest mathematical performance R² = .47 for Step 2; ∆R² = .05 (p < .03). * p < .05, ** p < .01, *** p < .001. Table 4 Hierarchical regression analysis of pretest mathematical score, tutor actions and ethnicity on posttest mathematical performance

	Variables	B	SE	ß	T	F	df	∆F
	Step 1						1, 56
	Mathematical pretest	.83	.13	.65***	6.33	40.05
	Step 2						2, 55	5.25
	Mathematical pretest	.79	.13	.62***	6.22	24.17
	Request for explanation	-.18	.08	-.23*	-2.29
	Ethnicity	.20	.36	.06	.54
	Condition	.17	.35	.05	.49
	R² = .42 for Step 1; Variables	B	SE	ß	T	F	df	∆F
	Step 1					12.58	1, 25
	Mathematical pretest	.76	.22	.58**	3.55
	Step 2					10.43	2, 24	5.84
	Mathematical pretest	.68	.20	.51**	3.39
	Tutor actions provided by	.05	.02	.37*	2.42
	the tutor
	Tutor actions received by	.05	.05	.18	1.07
	the tutee
	Ethnicity	-1.03	.66	-.29	-.95
	Condition	-.62	.65	-.17	-1.55
	R² = .34 for Step 1; R² =.47 for Step 2; ∆R² = .13 (p < .03).
	* p < .05, ** p < .01.

Table 5

 5 Regression analyses of the interaction of mathematical ability and condition, and of ethnicity and mathematical ability at the individual level on the categories of the coding scheme

	Condition x	Ethnicity x
	Mathematical ability	Mathematical ability

Pupil B …look 8.5 centimetres. You have to do it like this, like this (shows by writing on V's worksheet)

Tutoring behaviour (part 2)* Pupil V Is 8… Low quality constructive activity Pupil B No, 8.5.

Low quality help Pupil V (writes answer down) And this is 10, right?

Need for help Pupil B No, we don't have to do that one yet. Just finish this one.

-Low quality help -Organizational utterance (not used in the analyses) Pupil A Has everyone finished?

Organizational utterance (not used in the analyses) Pupil V Almost, just filling in the numbers… Organizational utterance (not used in the analyses)

* Coded as a single tutor action. Although the first tutor action is interrupted by the tutee, the utterance of the tutee is unintelligible and the tutor is not distracted by the interruption, continuing the tutor action.