Jan Willem Strijbos
email: janwillem.strijbos@ou.nl

Rob L Martens

Wim M G Jochems

Nick J Broers

THE EFFECT OF FUNCTIONAL ROLES ON GROUP EFFICIENCY: USING MULTILEVEL MODELLING AND CONTENT ANALYSIS TO INVESTIGATE COMPUTER SUPPORTED COLLABORATION IN SMALL GROUPS

The effect of functional roles on group

The effect of functional roles on group efficiency: Using multilevel modelling and content analysis to investigate computer supported collaboration in small groups.

Since the 1970s small group dynamics have been intensively studied in educational contexts.

Cooperative learning research focused initially on face to face cooperation at the elementary school level, but was gradually extended to college and higher education settings. Design of cooperative learning pedagogy focused on promoting group cohesion and group responsibility to increase promotive intra group interaction. Due to the technology push in the 1980s, resulting from rapid developments in computer mediated communication (CMC), social psychological orientations gradually lost the upper hand, giving rise to a new discipline called computer supported collaborative learning in the 1990s. Computer Supported Collaborative Learning (CSCL) is situated at the crossroads of educational psychology, social psychology, computer science and communication science. In effect, CSCL cannot yet be regarded as an established research paradigm [START_REF] Koschmann | Paradigm shifts and instructional technology: An introduction[END_REF] because theoretical debate, as well as large varieties in technological and pedagogical support of collaborative learning, still prevails. However, it has been shown that CSCL promotes meta cognitive processes [START_REF] Ryser | Effects of a computer supported intentional learning environment (CSILE) on students' self concept, self regulatory behaviour, and critical thinking ability[END_REF], that representational guidance can aid collaboration [START_REF] Suthers | The effects of representations in students' elaboration in collaborative inquiry[END_REF], that reflective interaction can be promoted with a structured dialogue interface [START_REF] Baker | Promoting reflective interactions in a computer supported collaborative learning environment[END_REF], that more elaborated problem solving is increased [START_REF] Jonassen | Communication patterns in computer mediated and face to face group problem solving[END_REF], and that high level interaction promotes higher levels of cognitive knowledge gain [START_REF] Schellens | Asynchrone discussiegroepen: Een onderzoek naar de invloed op cognitieve kennisverwerving [Asynchronous discussion groups: Investigating the influence on cognitive knowledge gain[END_REF].

Nevertheless, several researchers also identify large variations in the quality of interaction and learning outcomes [START_REF] Häkkinen | Sharing and making perspectives in web based conferencing[END_REF]Lehtinen, Hakkarainen, Lipponen, Rahikainen, & Muukkonen, 1999). On the one hand, these are caused by differences in length of studies, technology used, group size, as well as, differences in research methodology and the unit of analysis [START_REF] Lipponen | Computer supported collaborative learning: From promises to reality[END_REF]. On the other hand, the outcome of small group collaboration is mediated by the quality of group processes [START_REF] Shaw | Group dynamics: The psychology of small group behaviour[END_REF]. As the initial technological push slowly resides, small group dynamics have regained interest of the CSCL research community [START_REF] Kreijns | Identifying the pitfalls for social interaction in computer supported collaborative learning environments: A review of the research[END_REF][START_REF] Strijbos | Group based learning: Dynamic interaction in groups[END_REF][START_REF] Wood | Contingent tutoring and computer based training[END_REF]. In addition, it is gradually acknowledged that 'learning' and 'collaboration' resides on intra group interaction (Strijbos, Martens & Jochems, in press), and thus this is the primary process to be studied with respect to performance and learning benefits in CSCL settings.

The use of roles to support coordination during asynchronous CSCL

Group performance effectiveness depends, as group size increases, on the one hand on the groups' use of increased resources and alternate opinions ('process gains') and on the other hand on the handling of increased coordination and group management processes ('process losses') [START_REF] Shaw | Group dynamics: The psychology of small group behaviour[END_REF]. Conflicts regarding coordination are likely to occur in asynchronous CSCL settings, for example the group members are not present at the same time and/or place [START_REF] Benbunan Fich | Impacts of asynchronous learning networks on individual and group problem solving: A field experiment[END_REF]. In addition, asynchronous communication is 'non natural' in the sense that the immediacy of feedback, prone to face to face settings, is not present. Clearly, some support should be provided to help students overcome difficulties in group coordination.

Several processes in small group dynamics can indirectly affect coordination and the delicate balance of 'process gains' versus 'process losses'. Group responsibility is proportionally related to group performance (i.e., a greater sense of responsibility can increase group performance), whereas the effect of norms and status depends on whether these stimulate or impede group performance. Group cohesion has been shown to increase stability, satisfaction and efficient communication, as well as negative effects such as social pressure, inter and intra group aggression or conflict and polarisation [START_REF] Forsyth | Group dynamics[END_REF]. Group cohesion and responsibility are the basis of two key concepts in collaborative learning: 'positive interdependence' [START_REF] Johnson | Student student interaction: The neglected variable in education[END_REF] and 'individual accountability' [START_REF] Slavin | Cooperative learning in teams: State of the art[END_REF]. Positive interdependence refers to the degree to which the performance of a single group member depends on the performance of all other members. Individual accountability refers to the extent to which group members are held individually accountable for jobs, tasks or duties, central to group performance or efficiency.

Since roles promote group cohesion and responsibility [START_REF] Mudrack | An examination of functional role behaviour and its consequences for individuals in group settings[END_REF], they can be used to foster 'positive interdependence' and 'individual accountability' [START_REF] Brush | Embedding cooperative learning into the design of integrated learning systems: Rationale and guidelines[END_REF].

Roles can be defined as more or less stated functions/duties or responsibilities that guide individual behaviour and regulate intra group interaction [START_REF] Hare | Types of roles in small groups: A bit of history and a current perspective[END_REF]. In addition, roles stimulate members' awareness of the overall group performance and each members' contribution. "The opinions that others form about one's contribution to the group effort will likely be influenced, in part, by which roles the focal group members play." (Mudrack & Farrell, 1995, p. 559). The use of roles appears to be most relevant when a group pursues a shared goal that requires a certain level of task division, coordination and integration of individual activities.

Three main categories of roles can be distinguished: individual roles, task roles and maintenance roles, each of which is comprised of several different roles [START_REF] Mudrack | An examination of functional role behaviour and its consequences for individuals in group settings[END_REF]. However, these roles are based on a self report inventory and pertain to roles that participants can perform during collaboration. Moreover, each participant performs several roles simultaneously, thus making it difficult to implement such roles in educational contexts.

Nevertheless, these role descriptions can guide the design of roles for pedagogical purposes.

Several pedagogical approaches, developed for cooperative learning, use roles to support coordination and intra group interaction [START_REF] Johnson | Advanced cooperative learning[END_REF][START_REF] Kagan | Cooperative learning. San Juan Capistrano: Kagan Cooperative Learning[END_REF]. These roles are either based on differences in individual expertise (content based roles; cf. [START_REF] Bielaczyc | Designing social infrastructure: The challenge of building computer supported learning communities[END_REF], or individual responsibilities regarding group coordination (process based roles; cf. [START_REF] Kynigos | Perspectives in analysing classroom interaction data on collaborative computer based mathematical projects[END_REF]. It can be questioned whether content based roles are actual 'roles' or merely rigid task division. Moreover, most roles developed for cooperative learning settings comprise one single job, task or duty, mainly because they were developed for face to face collaboration in primary education. Although roles are widely regarded as an effective instructional strategy, in cooperative learning and CSCL, their effect has not been investigated systematically in both higher and primary education.

If cooperative learning pedagogies, and more specifically roles, were used in higher or distance education, they were not adapted, although students in these settings vary considerably in (prior) knowledge, experience and collaboration skills. Moreover, the collaboration assignments in higher/distance education are more complex, they take place over an extended period of time (i.e., not restricted to classroom time) and thus they require more explicit coordination than in primary/secondary education. Consequently, the previously mentioned uni dimensional roles for face to face collaboration appear inadequate to support collaboration higher/distance education, let alone asynchronous CSCL settings. Thus, explicit and detailed roles descriptions should be provided.

This article reports on a study that investigates the impact of roles that counter 'process losses' due to coordination demands. We refer to these roles as 'functional roles'. The roles are based on role descriptions in reports by [START_REF] Mudrack | An examination of functional role behaviour and its consequences for individuals in group settings[END_REF], [START_REF] Kagan | Cooperative learning. San Juan Capistrano: Kagan Cooperative Learning[END_REF] and [START_REF] Johnson | Advanced cooperative learning[END_REF]. In addition, they are adapted for an asynchronous CSCL setting in a higher/distance education context. The main research question can be summarised as: 'What is the effect of a prescribed functional roles instruction, compared to no instruction, on group performance and collaboration?'. It is expected that roles will have a positive effect on group performance (grade) and collaboration (efficiency) and that the amount of coordinative statements will decrease in favour of content focused statements. The relationship between individual characteristics and group collaboration will be investigated, as well as the suggestion by [START_REF] Mudrack | An examination of functional role behaviour and its consequences for individuals in group settings[END_REF] that individual and group perception will be more unanimous in the role condition compared to the non role condition. Self report questionnaires were used to measure students' perception of collaboration and content analysis of communication transcripts was used to investigate actual behaviour during intra group collaboration.

Analysis of non independent observations and small sample sizes

Before we proceed to the analyses and results of the self report questionnaire data, it is important to note the implications of non independent observations with respect to the analysis of intra group collaboration. This issue was only recently raised in CSCL and small group research. In research on cooperative learning frequently the ANOVA procedure has been used to investigate the impact of an instructional strategy using individual level observations (see [START_REF] Slavin | Cooperative learning: Theory, research and practice[END_REF]. This is no exception in some CSCL studies [START_REF] Reiserer | Fostering collaborative knowledge construction in desktop videoconferencing: Effects of content schemes and cooperation scripts in peer teaching settings[END_REF]. However, ANOVA appears not to be suited for this type of data. [START_REF] Stevens | Applied multivariate statistics for the social sciences[END_REF] points out that the assumption of independence, between scores of members of the same small group, is violated.

Students' perception of group performance depends on all others members' activities. Violation of independence increases as a function of the interdependence in a group, thus yielding a major increase of a Type I error. [START_REF] Stevens | Applied multivariate statistics for the social sciences[END_REF] suggest either to test with a stricter level of significance (p < .01 or even p < .001) or to use the group average. [START_REF] Bonito | The analysis of participation in small groups: Methodological and conceptual issues related to interdependence[END_REF] discusses three alternative procedures that take non independence into account, with respect to the analysis of participation in small groups: the actor partner interdependence model (APIM), the social relations model (SRM) and multilevel modelling (MLM).

Another point is that, unlike a considerable amount of studies in social psychology, CSCL is not conducted in laboratory settings. Its naturalistic context adds to its ecological validity, but simultaneously complicates analysis. Most CSCL studies suffer from a relatively small number of participants and research designs in general do not exceed 20 participants (see [START_REF] Stahl | Computer support for collaborative learning: Foundations for a CSCL community[END_REF]. Furthermore, quantitative statistical analyses are rarely used. Analysis focuses on qualitative methodologies to explore intra group interaction and the level of collaboration. MLM appears to be best suited to investigate questionnaire data that consists of self report perceptions (cf. [START_REF] Bonito | The analysis of participation in small groups: Methodological and conceptual issues related to interdependence[END_REF]. However, MLM analyses with a small sample size (less than 50) are not often reported. Therefore, the methodological and analytical considerations will be discussed in more detail in the method and results section that covers the MLM analyses.

Content analysis

Analysis of written electronic communication transcripts has gained increased attention in CSCL in the past decade [START_REF] Hara | Content analysis of online discussion in an applied educational psychology course[END_REF][START_REF] Lally | Complexity, theory and praxis: Researching collaborative learning and tutoring processes in a networked learning community[END_REF]. In general two approaches exist: the 'quantitative' and the 'qualitative' approach. In the first approach communication is coded and obtained frequencies and percentages are used in statistical comparisons. The latter deploys techniques such as phenomenography, ethnography and participant observation techniques to reveal descriptive trends [START_REF] Miles | Qualitative data analysis: An expanded sourcebook[END_REF].

Large variations with respect to the unit of analysis exist; it can be a message, paragraph, theme, a unit of meaning, illocution, utterance, statement, sentence or proposition. Common to all is that the unit is ill defined and arguments for choosing a specific unit lack (Strijbos, Martens, Prins, & Jochems, submitted). Furthermore, although it is acknowledged that reliability for a quantitative content analysis procedure is essential and many studies often report an intercoder reliability statistic -reliability is seldom addressed with respect to the unit of analysis [START_REF] Rourke | Methodological issues in the content analysis of computer conference transcripts[END_REF]. Nevertheless, examples of statistical comparison without any intercoder reliability being provided are not uncommon in CSCL research [START_REF] Pata | Framework for scaffolding the development of problem based representations by collaborative design[END_REF]. However, as [START_REF] Neuendorf | The content analysis guidebook[END_REF] states, "Without the establishment of reliability, content analyses measures are useless." (p. 141). Moreover, if the outcomes are used for statistical comparisons, quantitative content analysis requires that codes are mutually exclusive.

Hence, more rigour with respect to reliability of both 'segmentation in unit of analysis' and 'coding' is essential to warrant the accuracy of observations (Strijbos, Martens, Prins, & Jochems, submitted). Irrespective of the segmentation reliability, units should still be meaningful with respect to coding. Or in other words enable a researcher to answer the research question.

We used 'a sentence or part of a compound sentence' as the unit of analysis. A procedure to segment transcripts in these units was developed, as well as a procedure for coding. The reliability of both procedures and outcome of the analyses will be provided in the results section.

Method Participants

At the Open University of the Netherlands (OUNL) 57 students enrolled in a course on 'policy development' (PD) and 23 students in a course in the subject domain of 'local government' (LG). In total 80 students enrolled (49 male and 31 female). Their age ranged from 23 to 67 years (Mean = 34.4, SD = 9.03). Five students enrolled in both courses. Participants varied considerably in educational and professional background, which is common to higher/distance education. The course was successfully completed by 43 students, of which 33 returned both questionnaires and were included in this study.

Design of study

The study has a quasi experimental random independent groups design. The experimental manipulation involved the introduction of a prescribed role instruction in half of the groups (R groups). The instruction aimed at promoting the coordination and organisation of activities that were essential for the group project, in half of the groups. The other half of the groups was left completely self reliant regarding organisation and coordination of their activities (NR groups).

Each group initially consisted of four students and throughout the course they communicated electronically by e mail. In order to assess the effect of roles on performance, group level grades in both conditions are compared. To investigate the effect of roles on the perceived collaboration each students' perception of their team development, group process satisfaction, the task strategy, the level of intra group conflict, the quality of collaboration and the usefulness of e mail have been measured. Finally students' attitude towards collaboration and computer mediated communication was measured prior to the course and after successful completion.

Materials

Instructions. Half of the groups were instructed to use functional roles: 'project planner', 'communicator', 'editor' and 'data collector' (see appendix A), the other half received a non directive instruction (e.g. obvious, unspecific and general information regarding planning and task division) and they were instructed to rely on their intuition and/or collaboration experiences (see appendix B). Students in the R groups had to distribute the roles themselves and exerted their role for the full duration of the course (roles did not rotate). Instructions in both conditions were delivered as a short electronic text at the beginning of the course. They were also presented to students present during a face to face meeting at the start of the course.

Intake questionnaire. The intake questionnaire consisted of two sections. One section combined several scales addressing individual characteristics such as attitudes, need for closure and achievement motivation. All items were rated on a five point likert scale. These scales were all already previously tested and their reliability ranged from .78 to .86. Reliabilities that will be reported further, only apply to this study. Both attitude scales [START_REF] Clarebout | An invasion in the classroom: Influence on instructional and epistemological beliefs[END_REF] were reliable and measured at the intake and evaluation: attitude towards computer mediated communication (intake: α = .78; 8 items) and attitude towards collaborative problem solving (intake: α = .70; 7 items). A scale to assess active or passive orientation to group work (α = .63; 6 items) was constructed and tested prior to this study [START_REF] Strijbos | Vragenlijst samenwerkingsoriëntatie [Questionnaire collaboration orientation[END_REF]. Need for closure questionnaire is developed by Kruglanski (cf. De Grada & Kruglanski, 1999), translated into a Dutch version by [START_REF] Cratylus | /eed for closure (Dutch version)[END_REF], which was used in this study. Need for closure consist of five subscales: need for structure, need for predictability, decisiveness, intolerance for ambiguity and closed mindedness. The subscales 'need for structure' (α = .79; 8 items) and 'decisiveness' (α = .67; 6 items) were sufficiently reliable to be used in further analyses. Achievement motivation [START_REF] Hermans | PMT: Prestatie motivatie test handleiding [Achievement motivation questionnaire manual[END_REF] was measured using the P scale of this questionnaire (α = .86; 44 items). ICT experience was measured through several non scaled questions adapted from [START_REF] Valcke | Educational re design of courses to support large groups of university students by building upon the potential of ICT. The Journal for the integrated study of artificial intelligence[END_REF].

Finally background characteristics (such as received education/training, occupational group and branch of industry) were collected using a standard Open University of the Netherlands (OUNL) questionnaire. Out of the 80 students that enrolled in the course, 75 students (93.8%) returned the intake questionnaire. The course was successfully completed by 43 students (53.8 %), of which 33 returned both the intake and evaluation questionnaire (76.7 %). These figures indicate a high dropout rate, but this is not uncommon in a distance education context [START_REF] Martens | The use and effects of embedded support devices in independent learning[END_REF].

Evaluation questionnaire. The evaluation questionnaire consisted of forty six items, belonging to six scales, that are rated on a five point likert scale: attitude towards computer mediated communication, attitude towards collaborative problem solving, team development, group process satisfaction, intra group conflict and task strategy. In addition students were requested to answer several questions on a ten point scale (including 'perceived quality of collaboration' and 'perceived usefulness of e mail') and about twenty five open ended question or opportunities for extended feedback. Results that will be reported in this article are restricted to the six scales, which were already previously tested and reliability ranged from .76 to .92, and two questions that were rated on ten point scale: perceived quality of collaboration and perceived usefulness of e mail for collaboration. Reliabilities that will be reported further, only apply to this study. Attitude towards computer mediated communication in the evaluation had α = .84 (8 items) and attitude towards collaborative problem solving had α = .76 (7 items). Team development (α = .95; 10 items) provides information on perceived level of group cohesion, whereas group process satisfaction (α = .67; 6 items) provides the perceived satisfaction with general group functioning (both cf. [START_REF] Savicki | Gender, group composition, and task type in small task groups using computer mediated communication[END_REF]translated into Dutch).

Intra group conflict (α = .68; 7 items) provides the perceived level of conflict between group members and task strategy (α = .86; 8 items) indicates whether students perceive that their group deployed an appropriate strategy for the given task (both cf. [START_REF] Saavedra | Complex interdependence in task performing groups[END_REF] translated into Dutch).

Procedure

After course registration students were informed that the research focused on investigating the group processes of students collaborating through e mail and to determine the suitability of this format in distance education. Two weeks prior to the start of course students had to indicate whether they wanted to start with the group assignment in October 2000 or March 2001. Next, students were randomly assigned to groups and geographical distance between group members was maximised to discourage face to face meetings.

Prior to collaboration a face to face meeting was organised for all students. A separate meeting was organised for each research condition. General information and the instructions in both conditions were provided during this meeting and electronically afterwards. After the meeting all remaining contact between students was virtual. Role groups were required to inform their supervisor about the assignment of the roles in their group within two weeks. Contact with the supervisor was restricted to a single group member in the 'role' condition, whereas students in 'non role' groups were all allowed to contact the supervisor. Supervisors were instructed to answer questions that focused on the content of the assignment. Under no circumstance were they to provide support regarding coordination and group management. If a request for support was received, students in the 'role' condition were told to rely on the roles, whereas students in the 'non role' condition were told to rely on their intuition or experiences with collaboration.

Although students were instructed to use e mail, it is by no means possible nor feasible to exclude customary communication channels, such as telephone and face to face contact. If used, students were requested to send transcripts to all group members to retain transparency of communication. During collaboration the telephone was used occasionally, but most contact was by e mail. In spite of geographical distance three groups organised a face to face meeting. Five students participated in both courses and were placed in the same research condition. This did not pose difficulties in the final analyses. Some groups did not complete the course timely or were excluded from the research because only two group members remained (and thus were no longer included in the research). None of these five students finished both courses.

Results

Investigation of correlations between individual characteristics and dependent variables

Pearson correlations were computed to investigate whether the variables measured at the intake could be used as co variates. A correlation matrix was computed. No correlations were found between any of the variables measured on intake. Neither between these constructs and any dependent variables measured at the evaluation, nor between these constructs and grade were any correlations found. It was concluded that none of the variables from the intake, signifying individual characteristics, could be used as co variates in any of the further analyses.

Effect of condition on grade

Grades were administered on a group level. A Mann Whitney U test was performed to investigate the difference between the 'role' (Mean 6.6, SD = .89) and 'no role' (Mean 7.4, SD = .54) condition. A non directional test was performed. No main effect was observed for grade (Z = 1.549, df = 4).

Descriptives and correlations between dependent variables

Descriptives were computed for both conditions. A considerable spread of scores is indicated by standard deviations, occurring in both conditions.

Pearson correlations between these variables were computed for the entire sample (N = 33).

Medium to high correlations (.45 to .89, p < .01) were found between all of the variables, except for 'Attitude towards CMC' and 'Attitude towards CL'.

To avoid the problem of multiple testing (which will be addressed in more detail when the ML analyses are discussed) principal axis factoring was performed to investigate whether a possible latent variable existed. Table II shows the factor loading scores. Usefulness of e mail attributes less to the common factor than all other variables (Extraction I); therefore a second extraction was computed excluding this variable (Extraction II). The second extraction explains 79% of all common variance between the dependent variables. Based on the Extraction II, factor scores were computed.

Resulting factor can be interpreted as 'perceived group efficiency' (PGE). Standardised factor scores were computed for all variables used in Extraction II. In the subsequent analysis we will refer to this variable as PGE.

Multilevel modelling

Before discussing the outcomes of our multilevel modelling analyses a more detailed view on our dataset is required. Our sample consists of 10 groups and the number of observations in each group varies between two and four. This design is skewed, i.e. the number of observations on levels 1 (group) and 2 (individual) are not balanced (five groups with five observations each (5 x 5), 10 x 10, etc.). [START_REF] Mok | Sample size requirements for 2 level designs in educational research[END_REF] identifies three basic designs. Our design (type C in terms of Mok), is less efficient in the so called random component on both levels, however, ML analyses can be applied. Secondly, our sample size is rather small (N = 33). This has some implications for performing ML analyses, especially with respect to statistical power.

Investigating the influence of roles on perceived level of group efficiency (PGE) suggests the use of a t test or its equivalent reformulation into an ordinary least squared regression model (Ord. Least Squares OLS). However, OLS regression assumes that the residuals are independent and this assumption is obviously violated, because the scores of students in the same group will be more similar than the scores of students from different groups.

Analysis showed the intra class correlation coefficient, a measure of the dependency between scores within the same group, to be equal to .47. Failure to incorporate this interdependency among scores in a statistical model will lead to an underestimation of the standard errors of model parameters, resulting in a much larger than nominal probability of a Type I error [START_REF] Snijders | Multilevel analysis[END_REF].

Instead a multilevel model (model one) was constructed using CONDITION as a predictor of the dependent variable PGE yielding a so called random intercept model [START_REF] Snijders | Multilevel analysis[END_REF]:

PGE ij = y 00 + β 1 * CONDITION j + U 0j + e ij (1)
The score on PGE of person i in group j is the result of equation ( 1), where y 00 is a fixed intercept, β 1 is the regression coefficient of group level variable condition, CONDITION is a 0-1 indicator variable with 1 corresponding to non role group, U 0j is group level variance and e ij is individual level variance. Estimation of this model yielded the following fixed parameter values (with corresponding standard errors with in parentheses): PGE ij = .045 (.362) .027 (.502) *

CONDITION. An overview of the random parameters is provided in Table III.

The deviance reported in this table is equal to minus twice the log likelihood and can be used for a formal test of the goodness of fit of the model. By comparing this deviance value with the deviance of the model without CONDITION as predictor (the so called null or empty model), a significance test for CONDITION is provided. The effect of providing roles to group members is shown not to be significant (χ 2 = .003, df = 1, p > .05).

In general, at this point no further ML analyses would have to be performed. Unless there would be a theoretical ground to assume 'heteroscedasticity' instead of the assumption of 'homoscedasticity' underlying the fixed intercept model. To explain the implication of this assumptions we will briefly discuss model one. This model uses a fixed intercept (y 00 ). This intercept corresponds to the zero group of CONDITION. In each non role group CONDITION is given the value 1 and a constant of -0.027 is added to the fixed intercept. Thus, the fixed intercept for 'non role' groups takes a slightly lower value than for the 'role' groups. Of course the PGE score of each individual student depends on that individuals' score and the group dependent random effect (U 0j ). The model assumes that all group dependent random effects (U 0j ) values are taken from a 'normal' distribution with average zero and variance σ² U0j and that the variance of U 0j is equal for levels of CONDITION. This assumption is known as 'homoscedasticity'. Opposed to 'homoscedasticity' is the assumption of 'heteroscedasticity': the variance for group dependent random effects (U 0j ) is unequal for both levels of CONDITION.

Since roles, in theory, are likely to increase individual awareness of group efficiency, a theoretical foundation for the assumption of 'heteroscedasticity' is provided. Heteroscedasticity can be included in a ML model by allowing a random slope: the regression coefficient of CONDITION is allowed to vary in both levels (see Snijders & Bosker, 1999, p. 119):

PGE ij = y 00 + β 1j * CONDITION j + U 0j + e ij (2)
In model two the intercept, as well as the effect of CONDITION is allowed to vary for each group. Model two can be transformed into model three: IV.

The residual variance on group level has now been translated in a variance of the intercept (0.805), a variance of the regression slope (zero) and a co variance between values of U 0j and U 1j values ( 0.305). The estimation of the regression slope variance produced a value smaller than would be expected on the basis of the within group variability, and as a result the ML WIN program automatically inserts the value zero for this variance. However, in case of a limited number of observations it is not uncommon that the estimated variance between groups will be small in comparison the estimated variance within groups. This can be a consequence of the comparatively small power of the test. Thus, a closer look at the data is warranted. We looked at predictions of PGE generated for each group (R = role group, NR = non role group), based on respectively the model with random slope (RS) parameter (3) and the model without

RS parameter (1). Results are provided in Table V (for descriptives, see appendix C).

If we leave out the RS parameter, predictions of estimates based become less extreme for the 'role' groups (move closer towards zero), whereas predictions of estimates for the 'non role' groups become more extreme (move further form zero). This is caused by the underlying assumption of equal population variances in the model without random slope. Population variance of the 'role' condition is estimated as .82 for the model with random slope and as .62 for the model without random slope. Population variance of the 'non role' condition is estimated as .14 with RS and .24 without RS.

An F test for the homogeneity of variances was performed to investigate the hypothesis of equality of variances, both for the role and non role groups ANOVA, for the model without random slope (F = 2.86, df = 4, p > .10), and the model with random slope (F = 5.86, df = 4, .05 < p < .10). This difference is graphically represented in figures 1 and 2.

The results suggest to us that the assumption of homogeneity of variances leads to a distortion of a clearly discernable pattern in the data.

Content analysis

Before discussing the outcomes of the content analyses, it is must be to noted that the data consist of all contributions by all group members of the groups previously included in the MLM analyses, regardless whether they successfully finished the course or returned an evaluation questionnaire. Content analysis was performed on all e mail messages contributed by forty subjects equally distributed across research conditions (role and non role; N = 5 and n =20).

An alternative segmentation procedure that would be systematic and independent of the coding categories was developed (Strijbos, Martens, Prins, & Jochems, submitted). Although the sentence as a unit of analysis is not uncommon (e.g. [START_REF] Fahy | Patterns of interaction in a computer conference transcript[END_REF][START_REF] Hillman | A new method for analyzing patterns of interaction[END_REF], segmentation of compound sentences was added. The unit was defined as 'a sentence or part of a compound sentence that can be regarded as a meaningful sentence in itself, regardless of coding categories'. Punctuation and the word 'and' mark potential segmentation, but this is only performed if both parts before and after the marker are a 'meaningful sentence' in itself.

Intercoder reliability of two segmentation trials was .82 and .89 (proportion agreement), and was corroborated by a cross validation check on an English language dataset (.87). In addition, a coding scheme was constructed with five main categories 'task coordination (TC)', 'task content (TN)', 'task social (TS)', 'non task (NT)' and 'non codable (NOC)', and eighteen subcategories depicted in table 6. Reliability on subcategory level (Cohen's kappa) proved to be on average .60 (moderate) and on main category level .70 (substantial) (cf. [START_REF] Landis | The measurement of observer agreement for categorical data[END_REF].

Again the issue of non independence has to be taken into account. For the questionnaire data it was possible to reduce the number of dependent variables to a single factor to avoid the problem of multiple testing. Principal axis factoring of the five main categories, however, does not result in a factor that can be meaningful interpreted, therefore statistical comparisons were restricted to the number of messages, segments and the frequency for each main category on the level of the group. As ANOVA is not appropriated, the Mann Whitney U test was performed to compare the research conditions (five groups in each condition). Results are depicted in table 7.

No main effect was observed for the amount of messages send, but a significant difference was observed for the amount of segments (Z = 2.402, df = 4, p < .05). Regarding the content of the communication, a main effect was observed in favour of the role condition.

Significant more 'task coordination' (Z = 1.776, df = 4, p < .05; one sided), 'task content' (Z = 1.984, df = 4, p < .05), 'task social' (Z = 2.121, df = 4, p < .05), and 'non codable' statements (Z = 2.619, df = 4, p < .05) were made in the role condition. A one sided test was performed for 'task coordination'; it was expected that roles would decrease 'task coordination' in favour of 'task content'. Finally, a significant positive correlation was found between the amount of 'task coordination' and 'task content' statements (.73, p < .01). Kendall's tau was computed and a correlation plot revealed that most role groups (PD 1 4, LG 1) cluster in the upper right quadrant, whereas most non role groups (PD 5 7, LG 2 3) cluster in the lower left quadrant (figure 3).

Summary of results

A Mann Whitney U test revealed no main effect of roles regarding grade. Examination of Pearson correlates revealed significant high positive correlations between several variables that measured group functioning. Principal axis factoring was performed on the remaining dependent variables and one factor was extracted. The factor was interpreted as the level of 'perceived group efficiency' (PGE).

Next, multilevel analyses were performed. The intraclass correlation was regarded to be substantial enough to indicate the use of a multilevel model. Subsequent analyses revealed no difference between the 'role' and 'non role' condition regarding PGE using a fixed or random slope model. However, when the estimates of a model with random slope parameters were compared to a model without random slope parameters, a tendency was observed revealing a difference regarding the assumptions of homogeneity.

Content analysis was performed on the e mail communication that took place in the groups that were included in the multilevel analysis. All messages were divided in units of analysis and subsequently coded with one of five main categories. A Mann Whitney U test revealed more 'segments coded', 'task coordination', 'task content', 'task social' and 'non codable' statements in the role condition. Finally, a significant high positive correlation was observed between 'task coordination' and 'task content' statements.

Discussion

In this study the impact of functional roles, adapted for a computer mediated context in a distance education setting, was investigated. Such functional roles can be easily generalised to other content domains. The main research question was summarised as: 'What is the effect of a prescribed functional roles instruction, compared to no instruction, on group performance and collaboration?'.

Roles did not affect group performance in terms of a group grade. However, this might largely be due to the lack of variation (grades varied between 6 and 8.5 on a ten point scale). Some groups were given the opportunity to revise the report that they had submitted for grading, which of course decreased the variance in the final grades. Whether the group performed well or poorly, the effect of the roles is better reflected by their self report evaluation of perceived group efficiency.

The multilevel modelling (MLM) technique proved fruitful and showed that roles appear to affect the perceived level of group efficiency, i.e. to increase students' awareness of intra group interaction and collaboration. In the 'non role' condition participants appear to be less aware of these processes. The outcome of the content analysis corroborates this interpretation, as a significant difference was observed with respect to 'task social' statements. Students in the role condition contributed more statements that expressed, either a positive or negative, evaluation or attitude in general, towards the group or towards an individual group member.

Furthermore, as hypothesised -more 'task content' statements were observed in the role condition. However, the assumption that this would be due a decrease in the amount of coordinative statements was not confirmed. In fact, in the role condition the amount of coordinative statements also increased. Apparently, roles stimulated coordination and as a result 'task content' statements increased as well. Students in the role condition contributed more 'task content' and 'task coordination' statements, compared to students in the non role condition.

In this study, the MLM analyses reveal that the functional roles appear to have stimulated the 'perceived group efficiency' (PGE) and the content analyses reflect that the functional roles stimulated the amount of coordination and content focused statements through cohesion (positive interdependence) and responsibility (individual accountability). The outcomes of the MLM analysis indicates that the groups in the role condition appear to be more susceptible to intra group conflict and/or drop out. In the 'non role' condition, the lack of interdependence or responsibility appears to have less detrimental effects on intra group conflict and/or drop out.

Perhaps their self reliance provided non role groups with higher flexibility to cope with changes in the organisation and coordination of activities. Another possible explanation is that the descriptions of the functional roles were not sufficient to guide collaboration. The outcomes of the content analyses, however, clearly indicate that roles stimulated collaboration, expressed in more 'task coordination' and 'task content' statements.

We are confident to recommend the MLM technique, although it is not frequently used with small sample sizes. Nevertheless, it provides new possibilities for the analysis of non independent questionnaire data. The results, however, must be treated with some caution. This study was conducted in a high ecological valid setting, but it is imperative to investigate natural collaborating groups in an educational setting -hence the sample size is very likely to be small as it depends on the number of students that register for a course. Since many external sources, that can potentially influence outcomes, were beyond control and due to the small sample size, it can be argued that a significance level of .05 < p < .10 is justified. In addition, perceptions in the non role condition are also affected by so called free riders, i.e. group members that abstain from any effort to participate in collaboration but these members tend to rate their perception of collaboration as a very positive one. Nevertheless, the interpretation of the MLM results should be kept in perspective. This favours the interpretation of these results as a 'marginal effect' or rather a 'tendency' towards differences between the 'role' and 'no role' condition. Following the suggestions by [START_REF] Mudrack | An examination of functional role behaviour and its consequences for individuals in group settings[END_REF] the role condition can be seen as a strong situation "(…) in which most individuals will behave in similar ways (…) there are clear expectations about appropriate behaviours and adequate incentives for these behaviours exist." (p. 566 567).

Whereas the non role condition is seen to reflect a weak situation that "(…) is characterised by some ambiguity, and the definition of appropriate behaviours is more open to interpretation." (p. 567). Due to the ecological setting the results may been confounded by lack of clarity about time schedule, a lack of communication discipline or a lack of externalisation of expectations and norms regarding effort and input of group members prior to collaboration. It was confirmed that the Netherlands is a small country, as three groups organised a face to face meeting. After reviewing open ended questions in the evaluation, it was concluded that the confounding effect of these meetings on the overall collaboration could be regarded as minimal.

The reported data will be extended with a follow up study in which -apart from the use of functional roles -the need for a time schedule, communication discipline and expectations regarding input of group members are externalised prior to collaboration (currently these data are being analysed). In the near future it is planned to investigate other probable causes for PGE differences between groups in the role condition, such as role conflict and role ambiguity, and the efficiency of 'roles' which may have spontaneously emerged in non role groups through group members' previous collaboration experiences. It is clear that more systematic research regarding the use of functional roles in small groups and CSCL is needed.

Editor

Responsibility: Editing the input from all team members into a shared report Activities:

You will edit the input from all team members into a draft version of the report and distribute it among team members. They are required to respond to this draft within a timeline that you have specified (for example five days), with comments, questions, reformulations, additional information, and text formulation; You will revise each draft according to comments provided by team members. You will distribute the next version among team members with another request for comments and suggestions.

Data collector

Responsibility: Inventory of the literature database and gathering of additional information Activities:

You will make an inventory of the literature database that was provided. Based on this inventory you will indicate about which aspects sufficient or relevant knowledge or information lacks. You will distribute this inventory and analysis among team members with a request for suggestions for additional literature; Based on all comments and suggestions by team members on your inventory you will adapt the list according to their suggestions; either from the literature database or additional information sources, such as library or Internet sources; You are responsible for providing the additional information sources to your other team members, and/or distributing these sources among team members for further study in collaboration with the team member that performs the role 'project planning'. LG 2 .00 .00

LG 3 .44 .57

Table 6 Abbreviated overview of the content analysis coding categories.

  PGE ij = y 00 + y 10 CONDITION j + U 0j + U 1j CONDITION j + e ij (3) In model (3) y 00 + y 10 CO/DITIO/ j represents the fixed part and U 0j + U 1j CO/DITIO/ j + e ij the random part. Analysis of the fixed part of the model yielded the following results: PGE = .056 (.446) + .039 (.515) * CONDITION. Estimations of the random part of the model are provided in Table

Figure 1 !Figure 1 .

 11 Figure1

Figure 2 .

 2 Figure 2. Model estimates of PGE with random slope.

Figure 3 .

 3 Figure 3. Correlation of 'task coordination' and 'task content' statements per group.

Table 1

 1 Mean and standard deviations of dependent variables by experimental condition.

	Quality of collaboration	3,33 1,53	7,50 0,71	8,67 0,58	3,33 2,08	4,00 2,65
	Usability of e mail	3,67 1,53	7,50 2,12	8,33 0,58	3,00 2,65	4,33 2,08
	Team development	2,97 1,10	4,35 0,21	4,33 0,23	3,16 0,57	3,10 0,72
	Group process satisfaction	2,89 0,19	4,00 0,47	4,27 0,42	2,67 0,44	3,11 0,67
	Intra group conflict	3,05 0,29	1,93 0,30	1,57 0,14	3,05 0,21	2,62 0,54
	Task strategy	2,42 0,38	4,37 0,18	4,12 0,37	2,41 0,40	2,57 0,67
	Attitude towards CMC	3,12 1,30	3,25 1,06	3,45 0,29	3,41 0,75	3,67 0,31
	Attitude towards CL	3,05 1,07	3,92 0,10	4,05 0,43	3,09 0,82	3,05 0,58

Table 3

 3 Random variance estimates of the random intercept model. Random variance estimates of the random slope model. PGE prediction estimates by group with and without random slope parameters.

	Parameter	Estimate	Standard error
	Group level variance	.465	.285
	Individual level variance	.526	.155
	Deviance = 86.000

Table 7

 7 Mean, standard deviations and Mann Whitney rank scores for the number of messages, number of segments and the five main categories.

			Role (N = 20)		Non Role (N = 20)
		Mean	SD	Rank	Mean	SD	Rank
	Number of messages	78.20	22.30	7.2	52.40	17.47	3.8
	Number of segments	759.60	173.04	7.8	401.20	156.12	3.2
	Task coordination (TC)	63.95	16.99	7.2	37.35	20.45	3.8
	Task content (TN)	37.65	17.22	7.4	16.35	16.48	3.6
	Task social (TS)	4.40	2.73	7.5	1.95	0.48	3.5
	Non task (NT)	21.40	7.76	7.1	12.55	4.83	3.9
	Non codable (NOC)	62.55	13.73	8.0	32.10	10.33	3.0

& & & &

& & & & & & &

Code

Main Sub

Appendix A: Functional role instruction "Experience has revealed that roles can afford the work organisation and communication between team members. Each member of the team is to exert one these of four roles: project planner, communicator, editor or data collector."

Project planner

Responsibility: project planning and project progress monitoring Activities:

You are responsible for recording all activities to be performed and associated deadlines; You will supervise these to make sure that all team members comply; You will make an inventory about the groups' progress on a regular basis and you will communicate the outcome to the other team members; You will stimulate active participation of all team members to the report; You are required to set up an agenda for discussion ('Which aspects need to be discussed', 'Which aspects have priority'), make an inventory of discussion topics suggested by team members and you will compose an overview of all suggestions and decisions taken; You will initiate (and stimulate) discussion of the literature sources extracted from the database and additional information sources that your team has obtained ('Which information sources are relevant?', 'How can certain information be used in the final report?'); In case team members prefer to distribute literature sources extracted from the database or additional sources (for instance Internet), you are required in collaboration with the team member that performs the role 'data collector' to plan this distribution.

Communicator

Responsibility: communication with supervisor and progress reports Activities: Your supervisor will only contact the team member that performs this role, not the other team members. The e mail address of your supervisor is (…); You will communicate the distribution of roles in your team to your supervisor; You are responsible to make an inventory of questions and problems that team members experience during the assignment, and for communicating these to your supervisor and his/her answer to the remaining team members; You will construct an archive on the discussion of the literature, differences between perspectives, knowledge domains, and various theories that are introduced and discussed; You will construct an archive of the various versions of the report; You will initiate (and stimulate) discussion of the comments suggested by team members and changes made to the report; Every two weeks you will prepare a short progress report (half a page) that contains the most important decisions and/or developments. You will e mail this progress report to your supervisor to keep him/her informed about the progress of your team; You are responsible for submitting your teams' report to your supervisor. Below are some general guidelines on how you can proceed. It might be useful to pay attention to planning of activities and/or division of tasks.

Planning

Differences in study pace can lead to irritation, for example some students have a slower pace than others and may feel stressed by a higher pace. Also it might useful to pay attention to holidays, some students study during holidays and some do not. You might use a general planning or a planning that specifies parts of the assignment.

Task division

It might be useful to make arrangements about each team members' activities. This can either be general or specific. Is everybody going to do all tasks individually, or will the assignment be split in separate activities (one member collects data, one member writes) or will each task be divided in smaller parts between team members (one member collects data on x, one member collects data on y.)?

The authors would like to thank Mimi Crijns and Ger Arendsen for their invaluable support and assistance in gathering the data and conducting this study.

Description Example