Thanasis Daradoumis
email: adaradoumis@campus.uoc.es

Fatos Xhafa
email: fxhafa@campus.uoc.es

Joan Manuel Marquès
email: jmarquesp@campus.uoc.es

A Methodological Framework for Project-based Collaborative Learning in a Networked Environment

Keywords: Project-based collaborative learning, asynchronous virtual learning environment, conditions/methods for collaborative learning

d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

several universities in the USA, Greece and Spain, teaching a variety of courses in Mathematics and Computer Science. Since 1998 he has been working as a professor in the Department of Information Sciences at the Open University of Catalonia where he coordinates several online courses as well as the development of teaching materials appropriate for virtual learning. His research focuses on the development of methodologies and models for collaborative learning in virtual environments.

Introduction

One of the basic requirements for education today is to prepare learners for working and living in an information society in which knowledge is the most critical resource for social and economic development. Expertise and networked activities increasingly characterise the emerging nature of this work. Elaborating, managing and extending knowledge while collaborating productively with others and functioning within networks of experts will be essential for the interactive and open organisations of the future.

Besides this generic objective, in the context of distance learning, one of the fundamental issues is to provide quality teaching and learning. This fact is even more imperative today when the rapid development of information and communication technologies has initiated a shift from conventional distance learning to network-based learning. The result of these technological advancements has given rise to virtual learning environments or virtual campuses where the communicative process is crucial. In this context, the Open University of Catalonia [1] has built a large and complex organisational virtual campus that provides an innovative model for distance learning and teaching.

In this broad networked learning community infrastructure, our work seeks to investigate and facilitate learning and social interaction. In particular, we have started exploring the possibilities for new forms of learning and teaching by proposing a methodology that promotes and encourages learning and collaboration through forming smaller communities of learners working together. We investigated different methodological approaches to and scenarios for networked collaborative learning, considering several factors such as: the nature of the experience (the types of virtual collaborative learning activities), the individual and group objectives, assessment issues, tutor and student roles and commitment level, and the technology used for the implementation of the different practices.

In this paper, we describe a methodological framework that uses existing technology, namely, the Basic Support for Cooperative Work (BSCW) system [START_REF] Bentley | The world wide web as enabling technology for CSCW: the case of BSCW[END_REF], and applies an innovative scenario for developing a Project-Based Collaborative Learning (PBCL) practice that is adequately embedded in a real practically oriented educational context. In this context, we examine the conditions and methods that influence and enhance active learning through collaborative project development in shared workspaces as well as some methods for triggering collaborative processes. Our approach brings new expectations and requires changes in the attitude and reward structures for both learners and teachers, such as new roles, different pedagogic and learning methods, and technological and training support that enable learners to build up social structures, enhance learning and develop critical thinking skills.

The paper is organised as follows. Section 2 describes our research method and summarises the results of a preliminary study. In Section 3, we describe in more detail the organisational structure of our methodological framework. In particular, we present the innovative elements of the proposed approach, explain the key design issues related to planning and organisation of the learning spaces, propose a way to deal with the collaborative realisation of the project development phase, and finally discuss the project assessment issues. Section 4 presents a preliminary analysis of the experimental results and reports on the effects of this Computer-Supported Collaborative Learning (CSCL) setting on both the learners and the tutor. We especially examine one group functioning problem that arose in the progression of the groupwork and the solution proposed to tackle this problem. Furthermore, we discuss the role of the tutor in this CSCL experience, the ways the tutor supervised and guided the learning process of the students, and his awareness of the social and cognitive processes involved. Finally, Section 5 presents a global evaluation of the PBCL experience and discusses the present state of our experimental work.

Research method

The main goal of our research is to investigate how cooperation and collaboration can facilitate and encourage learning and social interaction among distance learning students in a virtual learning environment. On the one hand, traditional research on cooperative work and learning focuses on questions about how different group structures, labour division, and collective incentives can optimally improve the speed and quantity of learning [START_REF] Slavin | Cooperative Learning: Theory, Research, and Practice[END_REF]. On the other hand, emerging research on CSCL introduces the notion of knowledge management [START_REF] Greif | Everyone is talking about knowledge management[END_REF] and support for coordination in a collaborative application [START_REF] Lamarca | Taking the work out of workflow: mechanisms for document-centered collaboration[END_REF]. In fact, some recent research approaches examine the conditions under which effective collaborative knowledge building is achieved [START_REF] Baker | Designing computer-mediated epistemic interactions[END_REF]. The omission of social interaction in computer-based learning environments has been a particularly worrying problem for a long time [START_REF] Baker | The microcomputer and the curriculum. A critique[END_REF][START_REF] Cuban | Teachers and Machines[END_REF]. Nowadays, however, the situation tends to change since recent research on CSCL is exploring the ways technology can facilitate social interaction [START_REF] Benford | Broadcasting on-line social interaction as inhabited television[END_REF].

Despite the growing attention and experience related to social interaction in both research and pedagogy, we are still rather ignorant of the processes involved in it, particularly, in virtual collaborative learning and shared knowledge building in the workplace. In fact, there is a need for the development of theoretically well-grounded scenarios and tools for collaborative learning that can adequately support knowledge management and team coordination in a global virtual educational environment. Although there exist a few well-developed networked learning environments that support collaboration (e.g., [START_REF] Koenemann | Designing collaborative applications for classroom use: the LiNC project[END_REF] ), several aspects remain unclear. For example, it has not been studied sufficiently yet how different practices and networked learning environments can fit different virtual learning community needs and cultures, and support social learning and different group structures and styles.

To face this challenge, we have designed, developed, analysed and evaluated authentic collaborative pedagogical experiments within a virtual learning environment. We followed a model-based approach including initial apprehension of the problem with 'formalisation' of the application domain for understanding collaborative work and learning in a virtual environment. This initial model was based on the concept of communicative situations [START_REF] Marquès | An approach to model communicative situations for virtual cooperative learning[END_REF]. In particular, on the one hand, the work aimed to identify the different types of situations that a group of students might find when engaged in performing a learning activity. On the other hand, it determined the type of communication / interaction the group needed to be involved in so that it carried out the activity successfully. These findings set the theoretical basis of a framework for understanding the communicative aspects of collaboration.

The next step of the approach consisted of performing a further empirical study of collaborative work and learning. In particular, we conducted two successive preliminary experiments in a distance education course, called 'Information Structures'. The first experiment consisted of testing how our initial model could be used to support collaborative problem solving activities carried out by small virtual groups of students. The second one aimed at promoting a new way of studying and learning the contents of that specific computer science course. It was primarily based on performing collaborative learning activities by small, autonomous, self-regulated virtual groups without active participation of the tutor. The key design issues of this experiment concerned the conceptualisation, planning and setting up of a networked learning structure, called Virtual Study Group (VSG), aimed at supporting different collaborative learning experiences according to the learning objectives and styles a learning group specified [START_REF] Daradoumis | A methodological approach to networked collaborative learning: design and pedagogy issues[END_REF].

Both experiments were based on the BSCW system, a groupware tool that enables asynchronous as well as synchronous collaboration over the web. BSCW is a 'shared workspace' system, which supports document upload, event notification, group management and much more [START_REF]The Basic Support for Cooperative Work (BSCW) system[END_REF]. A shared workspace (or group space) is a folder that can be accessed by other people beyond its owner, typically by a group of registered users. A shared workspace can contain different kinds of information such as documents, pictures, URL links to other web pages, threaded discussions, member contact information, etc. The contents of each workspace are represented as information objects arranged in a folder hierarchy. Members can transfer (upload) information from their local machines into a BSCW workspace and set access rights to control the visibility of this information or the operations that can be performed on it by others.

An important feature that the BSCW system provides is the awareness information that allows group members to coordinate their work. The event service (activity reports) of the BSCW system provides users with information on the activities of the other group members, with respect to the objects within a shared workspace. Events are triggered whenever a member performs an action in a workspace, such as uploading a new document, downloading ('reading') an existing document, modifying a document and so on. The system records the events, and presents the recent events to each member. This particular feature proved to be very useful in our experience both for the students during the collaborative project development and for the tutor in supervising, tracking, guiding and assessing the group and individual student performance. Sections 3 and 4 illustrate how the BSCW system supports the needs of the project-based collaborative learning.

The outcomes of this study shed light on several important aspects, such as the factors that influence virtual group formation, the characteristics of effective learning teams regarding the group formation, the learning process, the group dynamics (communication, interaction, etc.), and the student and tutor roles. In addition, we gained interesting insights about the management and building of shared knowledge, the software products that supported collaboration and the principles used for individual and group assessment.

After the preliminary studies, our theoretical framework for virtual collaborative learning was modified to account for the observed outcomes and to explore new ways of collaborative work and learning. We initiated a new pedagogical experiment with a virtual class of 17 students divided into four virtual groups of four to five students. Our initial objective was to investigate a new way of working and learning the contents of the computer science course 'Software Development Techniques', based on the collaborative development of a project through the BSCW system. A further objective aimed at understanding how the best possible cooperation could be established and evolved and how successful coordination could affect the best implementation of the project in such working groups. We refer to this collaborative learning experience as Project-Based Collaborative Learning (PBCL), and we explain it in more detail in the next section.

Project-based collaborative learning: a methodological framework

In this section, we describe the methodology followed to implement the pilot experiment of PBCL in the course 'Software Development Techniques'. The key design issues are based on the conceptualisation, planning and setting up of the existing networked learning structure (VSG) mentioned in the previous section. It integrates both synchronous and asynchronous communication, though, at present, collaboration is mainly based on the latter due to the characteristics of our distance learning students.

The new experiment incorporates some important innovative elements, which differentiate the process and development of PBCL from the ones employed in the previous experiments. These distinctive elements are the following:

PBCL forms the fundamental objective of the course since it constitutes its spinal column; for this reason it is the project, which is really evaluated at the end. The goal of the project is twofold: it allows the students to exercise knowledge acquired in previous courses as well as in the current one. To that end, the project forms a significant part of the course content itself and constitutes the means on which the study, learning process, and evaluation of the students' knowledge are based.

PBCL explores new ideas and introduces new concepts in collaborative learning including:

• clearer and concrete roles attributed to the group members • a specific and more detailed methodology that focuses on how a project can be managed better and carried out successfully

• a guide, which helps students in both planning the project (task/activities distribution, timing, etc.) and choosing an adequate group structure

• a methodology of collaborative work that explains the strategies to be followed in order to achieve the best possible collaboration among the group members in a virtual environment during the project.

Planning PBCL and organising learning spaces

Groupwork in virtual environments requires special attention; thus it is a principal objective of a course including PBCL. This type of collaboration is necessary, since, when working on the project, students are often faced with situations where the group members have to make decisions that have effects on the successful continuation and completion of the project. The project to be implemented should simulate a real world problem, which would give the students an idea of how to tackle real-life complex problems as a group. Thus, students have the chance to learn and practise the following:

• methods of groupwork on a project

• decision making in collaborative work • what questions arise from working and learning in groups in a virtual environment Given this context, the organisational structure of our methodological framework is planned in three consecutive layers: initiating, forming and performing. All the three layers are set up on the BSCW networked collaborative environment.

Initiating: set up a debate on a case of virtual collaborative learning

The main goal of the first phase of our approach is to introduce the students to the new experience. To that end, a specific case is presented to the students for asynchronous discussion in a BSCW virtual workspace. The purpose of the case study is multiple. First, to prepare the students to tackle better the next phase (group formation). Group formation is considered a very important step in groupwork and learning. For this reason, students are engaged in a discussion to find out what kind of information they will need to know about their classmates, so that they form adequate groups at the next phase.

Second, to get an initial feeling and information about the way their classmates think of groupwork and learning.

Third, to become familiar with the BSCW tool and get to know the various functionalities it offers.

Finally, this discussion constitutes an initial, practical step towards the realisation of a group activity through an exchange of opinions and ideas, which can further motivate and push the students to be engaged in the real groupwork afterwards.

Forming: group formation

This phase involves three main actions: introducing, negotiating and norming. Students first introduce themselves by presenting relevant information that their classmates could use to choose possible candidate group members. Then, an interaction between possible candidates begins in order to form a group. This involves negotiation, taking into account various factors that affect collaboration, such as work pace, available amount of time, temporal coincidence, level of student knowledge/expertise, student attitude and especially the individual goals and the type of responsibility and commitment the student is willing to take. After a process of acceptance or refusal of the different intentions, negotiation will finally result in an initial proposal of group formation.

Finally, to achieve a reliable and definite learning group that could deal successfully with the project development phase, an internal discussion begins between the members of each group about the details of the working methodology and the structure of the group. The purpose of this discussion is to reach an agreement on the group norms and structure. As for the group structure, the group members choose among the following three options:

• Democratic: work distribution is performed in an equal way to all group members while everyone assumes the same types of responsibilities.

• Democratic with coordinator: the role of coordinator is assigned to a different group member in each phase of the project development; the coordinator guides and manages the work in the phase under question. The group, however, works in a democratic manner.

• Project leader: there is a member who assumes the coordinator role of the whole work during all the project development phases.

Performing: project development in virtual groups

After a group is formed and its norms are clearly specified, it is the time for performing: group members should be engaged in a collaborative work for implementing the project, using a common virtual BSCW workspace. To incite and promote collaborative interactions and make collaborative work and learning possible, we lead students through a guided process that involves achieving several learning objectives. Since the groups have to achieve specific goals, we create a context in which students have to collaborate with concrete rules and purposes. Moreover, the timing and order of carrying out these learning objectives is a very important factor for the successful completion of the project. The time period for completing the project is four months; hence careful planning of the project development phase is needed; we describe it next.

Project development phase

This phase consists of five specific learning situations related to the process of a software project development. Each learning situation is associated with a particular learning objective to be achieved by the group. This objective is very important for the definition of the learning situation, organising its structure and planning the interactions that will take place. A learning situation is clearly delimited by a variety of factors, which also determine the dynamics established within the group. These factors are the following:

• the individual goals of each group member

• the group goals to be accomplished; these goals are determined by the group as a whole in a consensual manner based on individual goals

• the characteristics and idiosyncrasies of each member

• the needs, expectations, interests, potentials and motivations of each member

• the commitment that a member establishes with the group.

Importantly, since in the norming phase (see 3.1, Group Formation) the type of commitment that each member establishes with the group is made clear, the members are able to determine more easily both the group objectives and the expectations of each member. Consequently, the learning situations in which the project is structured are adapted and refined to meet the above premises.

Below we describe each learning situation in detail by specifying both its purpose and content. This specification is crucial for designing the learning situations in the most adequate way.

Project specification/analysis

Purpose:

Study of the problem, analysis/specification of its requirements. Preliminary planning and timing.

Content: Students should discuss how to plan the work; they should study and understand the problem: what the problem asks for (analysis of the problem requirements), and specify which initial states they start from and which final goals they wish to achieve. Because a deep understanding of the problem is needed, students should work collaboratively through the whole phase. The specification document should be circulated among all the members of the group until a definite one -up to this phase -is agreed.

Project design/planning

Purpose:

Settle the problem solution and come up with the design of the application through an exploration process and by identifying possible subsystems and class hierarchies (amongst others) through an exploration process of the application. Verify and validate the obtained design.

Content:

Students should plan the problem solution by clearly indicating both the proposed methodology and the decisions made (selected from the various alternatives considered). Students' work should be done by collaborating closely between each other and should reflect the debate and the proposed alternatives. A proper task distribution should be applied to members, specifically once the hierarchies, subsystems etc., are identified.

Project implementation

Purpose: Implement the designed classes, interfaces and subsystems in a programming language Content: Students split the implementation work into smaller tasks and work individually, each programming a specific set of classes. Meanwhile they can interact any time they need some input from the others' work. Finally, they work in a group in order to integrate the different classes, interfaces and subsystems, thus obtaining the final implementation of the whole application.

Project testing

Purpose: Perform the necessary testing in order to check the proper functioning of the application.

Content:

Students should apply basic testing techniques in order to design adequate sets of tests to verify the correctness of their program.

Project report

Purpose: Elaboration of the project documentation.

Content:

Students should collaborate in order to prepare a project manual and a self-explanatory report about the project implementation.

The organisation of the shared work/learning spaces follows the structure of the project learning situations whereby the group takes care of the internal organisation of each learning situation regarding the document management and the structure of the asynchronous interactions. Thus the group space includes different subspaces each one corresponding to a learning situation. In particular, these subspaces serve as repositories for the project objects (documents, discussions, etc.) and facilitate the communication among members. There might be also other subspaces, e.g. for virtual meetings, which typically serve as auxiliary subspaces.

Assessment issues

The project assessment consists of two types of evaluation: formative and additive. Through the formative evaluation, the tutor does an active and detailed tracking of the student's activities in order to evaluate the whole learning process and the contribution to the collaborative work of each group member. For this reason, students are requested to make all their contributions explicit in the shared workspace. These contributions concern both the interactions (asynchronous discussions) and the actions (document creation and modification, and performance of other tasks such as virtual meeting planning, etc.).

The tutor also applies a qualitative evaluation to the reflection process followed by the students and to the collaborative methodology applied when working in-group. This evaluation is individual for each group member. The daily workspace activity report sent automatically by BSCW facilitates this task.

As far as the additive evaluation is concerned, the tutor performs an objective assessment of the different documents and reports produced in the different learning situations of the project development and of the final project report (group evaluation).

So, the final project assessment is the combination of the formative and additive evaluation. This assessment is further complemented by an individual qualification to a report, present in the examination centre, on which each student elaborates. This final report has two important goals: first, to obtain a student self-assessment (critical reflection of the student about the project outcome and his/her own participation in the collaborative work) and, second, the student's assessment of the group (evaluate both the work performed by the group as a whole and the functioning of the group).

Results and insights from the PBCL practice

This section presents an initial analysis of the experimental results and reports on the effects of this CSCL setting on both the learners and the tutor. We especially examine a group functioning problem that arose during the evolution of the groupwork and the solution proposed to tackle this problem. Furthermore, we discuss the role of the tutor in this CSCL experience and the ways in which the tutor supervised and guided the learning process of the students and was aware of the social and cognitive processes involved.

The dynamics established during the evolution of collaborative work

During the project-specification learning situation, some groups that had chosen the democratic option to define their group structure, had difficulties in establishing their functioning basis. Due to this, they were unable to meet the deadlines of the implementation and delivery of their product in this learning situation.

Consequently, in the following learning situations they applied the hybrid structure 'democratic structure with coordinator'. By doing so, the working dynamics established in each learning situation were characterised by democratic decision making but a coordinator was always present to guide the task distribution to the group members. He/she also managed the whole process of task implementation, and controlled the timing and the final product of the learning situation in which he/she was in charge.

In this way, every group member was in charge of the coordination of a particular learning situation, that is, of a section of the project. It was interesting to see that this group structure worked well and proved to be the appropriate structure for most of the groups in this experiment.

The tutor and student roles

• Regarding the participants' roles, we identified some desirable traits that support collaborative learning. So, on the one hand, the tutor could:

• act as a facilitator (plan and organise) and guide (counsel and show 'direction')

• motivate and offer cognitive support (make suggestions, give recommendations, clarify doubts, stimulate creativity, encourage independent thinking and its role and contribution to the group work)

• set limits and assess both the individual learner and the group as a whole.

On the other hand, the student should be:

• responsible, curious, persistent, and take the initiative

• willing to collaborate, to learn new information and new ways to learn (through collaboration) and to use technology to achieve virtual collaboration

• able firstly to organise his/her own work and help/contribute to the organisation of the group work.

Indeed, in our methodological framework, the tutor constitutes an important figure who plays a guiding, supporting and evaluating role in the learning process. Thus, during the whole process of groupwork, students received continuous support for their work and learning. More specifically, on the one hand, the tutor was attentive to the way collaborative interactions were carried through. In particular, the tutor was aware of the students' conceptual problems and doubts about the project development and thus could intervene to provide any tutorial assistance needed, monitor progress and make sure that groups were working and learning productively.

On the other hand, students also received whatever instruction and scaffolding they needed to prepare themselves for productive engagement in collaborative learning activities. This training ranged from general methodological instructions about groupwork and collaborative learning, specific guidelines, orientations and strategies about how to carry out a learning situation collaboratively to technical support regarding the tools used for implementing the experience. For example, the training showed the students how to pay attention to and benefit from contributions of the other group members, how to share and integrate the ideas of others, and how to handle disagreements constructively.

Student tracking and awareness

Student tracking and awareness influence to a great extent not only the successful completion of the project but also the whole collaborative learning process. Our experience shows that the tutor needs to apply a careful and well-studied strategy in order to relieve the overload and difficulties that this process represents. To that end, we settled on an active tracking based on several objectives, such as:

• reminding the students continuously that they are not alone 'fighting' against a project that requires thousands of code lines

• being aware of increasing difficulties as the project evolves and hence proposing ways out for the students when they get stuck due to technical difficulties

• being aware of certain undesirable situations within the group and hence proposing to the students reasonable yet human solutions to critical situations concerning group functioning

• providing group dynamic incentives so that students get more and more interested in the project

• observing individual member skills and giving adequate orientations.

It is worth mentioning here that we established two kinds of tracking, namely, individual tracking and group tracking. At an individual level, the objective was to track closely the contributions of the individual group members, while group tracking aimed to trace the group activities. In this way we were able to orientate correctly both individual and groupwork.

We have observed that the tracking process had much to do with the communication flows settled in the collaborative framework. We distinguished the following three main communication streams: 1 between the tutor and the individual members 2 between the tutor and the different groups 3 among the members of the same group.

A major difficulty with the tracking and awareness process is the rather large amount of information to be processed which might lead to a saturation of the tutor's available time.

In our framework, we found a trade-off by directing part of the communication flow to the coordinators of the groups.

Conclusion: global evaluation of the experience

At the end of the experience, students answered a questionnaire whose purpose was to collect their opinions and impressions about their learning process and outcomes during the collaborative elaboration of the project. In principle, all agreed that the project was closely represented a real situation and that they learned a lot from this practice. Moreover, they became conscious of the needs and functioning of collaborative work and learning and how to face a variety of problems that arise in a virtual group.

In the students' opinion, collaborative work can be successful if all members show goodwill and responsibility. They also believe that collaboration is necessary to deal with such large projects. In general, a positive evaluation was attributed to this experience both by the students involved and the tutor. Finally, students noted that this practice made them feel confident enough to be willing to get involved in the virtual collaborative development of a real-life project.

There are, however, some limitations and drawbacks of the framework we presented here. First, given the importance of the tutor's role, the number of group spaces in this framework should be small. This is due to the large amount of information generated within the group spaces and the overall communication of the tutor from the tracking process. Second, the group activity is permanently threatened by the overload of the group members (caused by the collaborative process), which could eventually cause some members to be unable to fulfil their commitment to the group goals and hence make it impossible for the group to complete the project.

For the time being, we are exploring new ways of PBCL based on our experience from previous experiments. In particular, we are trying to address the following questions that have been raised in our research before and need a more thorough analysis and study: How similar are the subjective learning situations experienced by the students to the beliefs of their tutor? Are the collaborative learning situations beneficial to all students or are their group members suffering from work overload, conflicts or others' problems? What specific roles does the tutor have to take in supervising and guiding the learning process of the students? What is the most adequate scaffolding process the tutor should follow to be aware of the students' weak and strong points and thus be able to intervene in order to repair deficiencies and monitor the group more effectively? It would be very interesting to investigate ways to process all the information that is available both in the global shared workspaces and in the specific group spaces and which allows for efficient student tracking and awareness. To this end, we plan to use different statistics to draw more concrete conclusions from the data we have at our disposal, based on different criteria.

•

  when and how to communicate with the other group members • what types of roles the various group members can have and what related responsibilities each member should assume • how to plan and distribute the different tasks of the project among the group members

Polytechnic University of Catalonia and since 1999 as a consulting professor at the Open University of Catalonia. Dr. Xhafa has participated in several research projects funded by the European Community. Joan Manuel Marquès graduated from the Computer Sciences Department of the Polytechnic University of Catalonia (UPC) where he is currently doing his PhD.


Henny Leemkuil
email: h.h.leemkuil@utwente.nl

Robert De Hoog

Is support really necessary within educational games?

Keywords: simulation games, learner support, modes of learning

Games can be powerful learning devices because of their interactive and multimedia capabilities, and their abilities to keep students motivated, active, deeply immersed and engaged for sustained periods of time. Yet the extent to which this translates into more effective knowledge and skill acquisition is not clear from the research reported so far. Several researchers have stressed that support tools should be added to game environments to ensure that learning takes place.

In this paper we will elaborate on this issue and will report data from experiments with a simulation game called KM Quest. In these experiments the effectiveness of several (learning) support tools was investigated. The data give indications that particular types of support (like advice) limit learning while others (feedback, just-intime information) might enhance learning. Based on these data at the end of this paper it will be discussed whether (intelligent) support within educational games is necessary.

Introduction

Games in potential are interesting educational tools because players (sometimes in cooperation with others) are actively solving challenging situated problems. In that sense they have elements in common with learning theories like constructivist learning, situated learning and collaborative learning. In these theories it is stressed that learning is an active social process in which meaning is given to experiences while solving situated realistic problems. Games can be powerful learning devices because of their interactive and multimedia capabilities, and their abilities to keep students motivated, active, deeply immersed and engaged for sustained periods of time. Yet the extent to which this translates into more effective knowledge and skill acquisition is not clear from the research reported so far. Klawe [START_REF] Klawe | When Does The Use Of Computer Games And Other Interactive Multimedia Software Help Students Learn Mathematics[END_REF] concluded that relatively small changes in game design could strongly influence the extent of the learning effectiveness of the game. Garris, Ahlers and Driskell [START_REF] Garris | Games, motivation, and learning: A research and practice model[END_REF] agree that the potential for instructional games as platforms for training is appealing but they also state that "there is little consensus on game features that support learning, the process by which games engage learners, or the types of learning outcomes that can be achieved by game play. Ultimately we run the risk of designing instructional games that neither instruct nor engage the learner (p. 442)". So, little is known about the way students learn while playing a game and about game features that support learning. In this paper we will elaborate on this issue and will report data from experiments with a simulation game called KM Quest. In these experiments the effectiveness of several (learning) support tools was investigated. Based on these data at the end of this paper it will be discussed whether (intelligent) support within educational games is necessary.

Learning within games

We assume that players use different types of information processing while playing a game and that these different modes lead to different types of learning outcomes. This distinction is based on research by Berry and Broadbent [START_REF] Berry | Interactive tasks and the implicit-explicit distinction[END_REF], Norman [START_REF] Norman | Things that make us smart. Defending human attributes in the age of the machine[END_REF] and Taatgen [START_REF] Taatgen | Learning without limits. From Problem Solving towards a Unified Theory of Learning[END_REF].

Berry and Broadbent [START_REF] Berry | Interactive tasks and the implicit-explicit distinction[END_REF] used a dynamic system, the Sugar Factory computer simulation, in which participants had to reach a certain level of sugar production by changing the number of employees. The behaviour of the simulation is based on a rule that is non-linear and contains a random component. Berry and Broadbent noticed that participants could achieve a good level of control of the system even though they remain unable to describe precisely the rules of the system in post-experimental structured questionnaires. They concluded that a task like this under certain conditions might be performed in some implicit manner. Berry and Broadbent observed a similar finding using a different task. On the basis of these results they suggested that two modes of learning could be distinguished, an implicit unselective mode (U-mode) and an explicit or selective one (that is effortful and reportable). U-mode learning is probable in situations in which there is much information in the learning environment and the key variables and their interrelationships are not salient. Especially in "rich" low transparent, interactive discovery simulations this leads to intuitive knowledge that is difficult to verbalize. Berry [START_REF] Berry | Implicit learning: reflections and prospects[END_REF] states that these two modes of learning should be seen as the two extremes on a continuum, while Swaak and De Jong [START_REF] Swaak | Measuring intuitive knowledge in science: The what-if test[END_REF] assume that these two types of learning can be seen as two parallel, at least partly independent learning systems.

Taatgen [START_REF] Taatgen | Learning without limits. From Problem Solving towards a Unified Theory of Learning[END_REF] however, has a slightly different view. In his view implicit and explicit learning are a result of two competing modes/strategies: a search mode and a reflective mode. These are two competing strategies whose evaluations depend on the expected outcomes of the strategies. The expected outcome is determined by three elements: the estimated probability of reaching the goal using a specific strategy, the expected value of the goal, and the estimated cost of reaching the goal using the strategy. These estimates change in time due to increasing knowledge and the successes and failures of applying the strategy. In most cases people will start with a search strategy because reflective reasoning has a high cost, and it is not evident that the search strategy will be unsuccessful. Especially when one has little task relevant basic knowledge, the costs of a reflective strategy will be even higher. If the search strategy is not effective, and the estimated chance of reaching the goal will get lower, people might consider changing to a reflective strategy.

When using a search strategy implicit learning is a "by-product" of normal information processing. People are actively performing a task and while doing that unintentionally learn certain things (facts, procedures, instances, examples, sequences of actions). When using a reflective mode, learning is based on information processing based on learning strategies aimed at explicitly learning, comprehending or memorizing something.

The distinction made by Taatgen resembles the one made by Norman [START_REF] Norman | Things that make us smart. Defending human attributes in the age of the machine[END_REF]. Norman describes two modes of cognition: an experiential mode and a reflective mode. "These two modes do not capture all of thought, nor are they completely independent (p.16)". The experiential mode is one of perceptual processing, it is a pattern or event driven activity. It requires some thought but the information processing is data driven and reactive. It leads to an accumulation of facts, it reactivates information that is already present in the memory system and it leads to a tuning and shaping of knowledge structures already available.

"The reflective mode is that of comparison and contrast, of thought, of decision making (p. 16)". It is slow and laborious. "Reflective thought requires the ability to store temporary results, to make inferences from stored knowledge and to follow chains of reasoning backward and forward, sometimes backtracking when a promising line of thought proves to be unfruitful……The use of external aids facilitates the reflective process by acting as external memory storage, allowing deeper chains of reasoning over longer periods of time than possible without the aids (p. 25)". According to Norman effective reflection requires some structure and organization and is greatly aided by systematic procedures and methods and the aid of other people.

This implies that the use of tools (inside or outside the environment) or the help of other people can facilitate the reflective process. Garris, Ahlers and Driskell [START_REF] Garris | Games, motivation, and learning: A research and practice model[END_REF] confirm this: "simulation games may be ineffective stand-alone training tools because people do not learn from simple exposure or experience alone to understand complex relationships …… Although our goal is to achieve self-directed, self-motivated learners, we must provide support for knowledge construction. The role of the instructor in debriefing learners is critical (if somewhat overlooked) component in the use of instructional games, as are other learner support strategies such as online help, cues/prompts, and other activities (p. 460)". A review of games research revealed that the following tools may be effective in supporting learning with games: cooperation and collaboration, debriefing and group discussions, hints and prompts, a help or advice system, additional assignments, feedback, monitoring facilities and the specification of goals.

KM Quest: A collaborative Internet based simulation game about knowledge management

In our research we used KM Quest: (http://www.kmquest.net) a collaborative Internet based simulation game about knowledge management. Several universities and institutions for higher education in the Netherlands like the University of Twente, the Radboud University in Nijmegen, the University of Utrecht, and Hogeschool Zuyd in Maastricht have used the KM Quest learning environment in courses on knowledge management. Furthermore, the Yeditepe University in Istanbul (Turkey) has used it. Goal of the game is to learn basic knowledge management concepts and actions and the steps of a systematic approach to solve knowledge management problems. Furthermore to learn to assess the KM situation of an organisation and to advise/implement appropriate interventions.

The simulation game can be played by three players who all play the same role of knowledge manager in a fictitious large product leadership organisation named Coltec for three years in the lifespan of the company (divided in 12 quarters). Collaboratively they have the task to improve the efficacy of the company's knowledge household. This is not an aim in itself but is related to objectives for the (management of the) company in general. The general goal of the simulation game is to optimise the level of a set of general organizational effectiveness variables: market share, profit, and the customer satisfaction index (that are in the top level of an underlying business simulation model that is used to simulate the behaviour of the company) by influencing the effectiveness and efficiency of knowledge management processes (knowledge gaining, development, retention, transfer and utilisation) that are in the lower level of the business model. These processes can be influenced by choosing and implementing interventions from a pool of 57 possible interventions.

In the game, players can use several resources while performing their task. They can inspect the status of business process indicators and knowledge process indicators (in three general domains) that are incorporated in the business simulation model, they can ask for Changes in the status of the business indicators will only be computed at the end of each quarter. There is no time limit to playing the simulation game. Teams set their own pace. When players think they know enough to solve the problem they indicate that they agree with the proposed interventions (by using a voting tool). After they have reached agreement the simulation game proceeds to the end of the quarter and the business simulation will calculate new values for each of the business indicators. The game ends after the players have indicated that they have implemented the last intervention(s) in the fourth quarter of the third year in the life span of the company.

To trigger activities from the players and to make sure that players are confronted with different types of knowledge management problems, at the beginning of each quarter an (unexpected) event is introduced that could affect the knowledge household of the company. Players have to decide if and how they want to react on these events. Events are generated from a pool of 50 events. Different types of events can be distinguished based on two dimensions: the locus of the event (internal or external), and the effect of the event (direct, delayed, or no effect). Effects either can be positive or negative. Players can interact with the environment and with each other by using tools and resources that are presented in an Internet environment, based on a "virtual office metaphor" (see Figure 1). Clicking on a specific element in the "office" will open a window with additional resources or tools. To support the learners in performing their task and to support learning while playing the game several features have been implemented in the environment:

A knowledge management model that describes a systematic approach to solving KM problems.

Shared worksheets related to the steps in the knowledge management model (accessible by means of the computer).

Just-in-time information (mainly in books on the bookshelf). Feedback (consisting of dynamic data from the simulation model and pre canned conceptual knowledge about knowledge management that is based on the experiences from KM experts and is coupled to certain events). Advice (only available when the value of certain indicators gets below a threshold value).

Visualisation tools (different types of visualisations to display the data from the business simulation model).

Monitoring tools (to be able to monitor their own behaviour and to reflect on it 12 quarterly reports are available on the top bookshelves). To support collaboration and communication at a distance tools are implemented like a chat box, monitoring facilities, a voting tool, shared worksheets, and embedded forums. These tools support synchronous as well as asynchronous communication between team members.

Data from experiments with the learning environment

In the KM Quest simulation game several tools/resources were embedded to support learning as stated above. Logfiles from a first experiment with this game showed that from these resources the advice functionality was used most often. Advice was available when the value of certain knowledge process indicators in the business simulation model went below a threshold value.

It appeared that in 83% of the cases where advice was available it was actually consulted. Most of the time it was the first resource players consulted when they entered a new quarter in the game. Based on the available data from that experiment however it was hard to say whether the use of advice supported learning and game play.

A literature review also did not lead to many cues about the role of advice on learning in these kinds of learning environments. There is a study by Leutner [8] that also focuses on advice in a simulation game. In this research in the advice groups, during the game the players are provided with warnings if their decisions are likely to lead to problems. Results of the experiments (with 7th grade students and with university students) showed that advice increased verbal domain knowledge, but decreased game performance. Furthermore the data indicated that system-initiated adaptive advice had short-term effects (measured directly after game play), while learner requested non-adaptive background information had long-term effects (measured by a test that was administered a week after game play).

The type of advice given in the KM Quest environment is slightly different from the type of advice that was available in the Leutner study. In that study warnings were provided. In the KM Quest environment a warning is also given, but this is accompanied with general suggestions to improve the status of the specific indicators. An example of the advice is given below:

"The value of "Efficiency of knowledge gaining in research" has dropped below a value of 4.5. This may be a reason for concern. It is possible that you overlooked this decrease of this variable because you focused to much on the event or did not include it in your measurement system. If you want to improve the value of "Efficiency of knowledge gaining in research", you could consider interventions that influence this value. Most of the time several interventions are available. In this case you could take a look at interventions of the type listed below (see Interventions handbook). Note that there maybe other interventions also.

• Interventions that aim at cooperation with partners, implementation of new projects or hiring services

• Interventions that aim at the implementation of Information Communication Technology • Interventions that aim at recruiting and hiring people".

When a player clicks on a class of interventions the chapter of the interventions handbook will be opened in which additional information about these interventions is given.

To explore the effects of the advice on learning in another experiment two versions of the simulation game were used: one with the advice and one in which this resource was removed. A pre-test, post-test design was used with an additional transfer test. It was hypothesized that students in the advice group would gain more explicit knowledge (in line with Leutners findings) and would perform better in the game because they receive hints about possible solutions. This last hypothesis is not in line with Leutners findings, but this is because the type of advice given in this experiment is somewhat different than the advice given in the experiments performed by Leutner.

Method

In this section the subjects that participated in the research will be described as well as the instruments that were used and the experimental procedure.

The students who participated in the experiment did so because the use of the simulation game was an assignment within a third year course "Knowledge management in learning organizations" at the University of Twente. All students were from the Communication Studies department and had followed a preliminary course on Knowledge management in the second year. 29 Students participated in the course (18 women and 11 men).

The game environment that was used in both conditions was slightly different than the standard version that is described above. In this case players played individually, the order of the events was not random but fixed and the same for all students, and the starting values of the indicators were lowered (several are below 5 on a ten point scale). This is done to create a situation in which events can occur that refer to internal problems related to knowledge processes. In the original game set up in the beginning of the game a lot of events are generated that have to do with external (opportunity) events because knowledge processes within the company still were running smoothly (score of 6 or higher on a ten point scale). Observations in try-outs showed that players found these kinds of events difficult to deal with when they had limited domain knowledge.

Knowledge gain is measured by comparing the score on a pre-test and a post-test. Each consisted of 26 items that contain questions to measure explicit (18 items) as well as implicit knowledge (8 items). Explicit knowledge of knowledge management concepts and principles is tested by giving students textual multiple-choice questions that refer to declarative knowledge about concepts used in the learning environment like the knowledge processes, and steps within the knowledge management model. Furthermore, questions are included that refer to relationships between indicators and interventions. Implicit knowledge items are included to measure intuitive knowledge that is difficult to articulate. These items are based on the guidelines given by Swaak and de Jong [START_REF] Swaak | Measuring intuitive knowledge in science: The what-if test[END_REF] who characterize intuitive knowledge by a "quick perception of anticipated situations". In the test a situation is given, an action is described and a set of possible post action situations is given. In each item the textual information is kept to a minimum and a picture or chart is used to present the alternatives.

A transfer task was administered after the game and the post-test were completed. The transfer task used a different type of company: a small customer relationship type of company. In the task a case description was presented with four event descriptions. For each event 4 questions were presented.

Player performance in the game was logged into a database. These data make it possible to see for each of the twelve quarters within the game, which resources people used (and in which order), which interventions they implemented and what the values of the business and knowledge process indicators were.

Procedure

In the first lecture of the course the set up of the course was described and KM Quest was introduced. In four consecutive weeks following on this lecture four two-hour sessions were planned to work with the KM Quest environment. The sessions were located in a computer room with a large set of computers. Each student had a computer. In the first session the pre-test was administered. After the game was completed the post-test was administered in the fourth session. After the students finished the post-test they received transfer task, which they had to make at home. Students were not able to get access to the game environment while they were making the transfer task.

Results

First data will be presented from the tests and about game performance. After that data will be presented about the use of advice and the other resources.

In Table 1 the test results are summarised. It shows that on the average students answered 39.6% of the questions of the pre-test correctly. For the post-test the average score was 50.3%. A paired samples t-test shows that this knowledge gain is significant (T=-4.62, df=28, p=0.00).

The data show that both groups have gained explicit knowledge. This gain is mainly in the conceptual domain. There is no significant gain in knowledge related to the knowledge management model, nor on explicit knowledge concerning relationships between indicators and interventions.

The No-advice group however, has also gained implicit knowledge, while the advice group did not show any gain in this domain. The difference between both groups on the score on the implicit knowledge items of the post-test is nearly significant (Anova: F=4.18, df=1, p=0.051).

Table 1. Average number of correct answers (and standard deviations) on the explicit and implicit knowledge items of the pretest and posttest and the score on the transfer test for the whole group and for the groups with and without advice.

Pre-expl Max=18

Pre-impl Max=8

Pre-tot Max=26

Post-expl Max=18

Post-impl Max=8

Post-tot Max=26 Transfer Max=10 Advice N=15 Looking at game performance there are hardly any differences between the experimental groups (see tables 2 and 3). Both groups have managed to improve the business and knowledge management (process) indicators significantly. Advice is only available when certain indicators get below a certain threshold value. On the average advice was available in 9.33 quarters and was consulted in 7.07 quarters. This means that in 77% of the cases where advice was available it was actually consulted. Table 4 gives an indication about the frequency of use of the other resources available in the learning environment. The Intervention handbook, the feedback and the visualization tool were the most used resources. The indicator handbook and the history books were used less frequently, and the other resources are sparsely used.

Although there are differences in the use of tools between the two experimental conditions, most of them are not significant, except for one. The no-advice group consulted the indicator handbook significantly more (Anova: F=5.22, df=1, p=0.03) than the advicegroup.

Discussion

The data indicate that the players appreciated the advice functionality, but there are also indications that the effectiveness of the advice given was low. Players in the no-advice condition score equally well on game performance, on the explicit knowledge items of the post-test and on the transfer test. The fact that in the no-advice condition there is a significant knowledge gain on test items that measure implicit knowledge while there was no gain in the advice condition even seems to indicate that advice (as it is given within the game) prevents the gain of implicit knowledge and the transfer of this type of knowledge into explicit knowledge. This could be due to the fact that players in the advice condition put in less mental effort in solving the knowledge management problems within the learning environment and rely too heavily on the suggestions given by the advice functionality. The players in the no-advice condition seem to be seeking more actively for data that could support problem solving. This is indicated by the fact that players in this group consulted the indicator handbook significantly more than the other group. A counter indicative finding could be that the advice group consulted the intervention handbook more often (although not significantly). This however, is probably caused by the direct links between the content of the advice and the content of certain parts of the intervention handbook.

There are some indications that players who are actively searching for information in the resources are learning more or performing better within the game. Frequent use of feedback or the intervention handbook improved learning in a previous experiment. This finding was not replicated in this experiment. Frequent users had higher post-test scores but did not have a larger knowledge gain. This could be due to the fact that in the reported experiment the players who used these resources often already had a "high" level of prior knowledge, while this was not the case in previous experiment. The data reported in this paper however showed that frequent use of the indicator handbook lead to higher scores on the explicit knowledge items of the post-test and on game performance measured by the level of some of the business process and knowledge process indicators included in the business simulation model.

The data are in line with findings of other experiments with the same learning environment that are reported by Purbojo and de Hoog [START_REF] Purbojo | Learning knowledge management in a collaborative game: Effects of player preparation and visualisation of variables[END_REF] and Shostak and de Hoog [START_REF] Shostak | The KM Quest business simulation model: supporting acquisition of decision-making skills for knowledge management[END_REF]. These data show that the availability of certain types of support resources like certain types of visualisations or certain types of indicator values, per se does not have an influence on learning results or game performance. These experiments however also show that a briefing session before game play is important as well as the fact that players have to cooperate. There are indications that a certain types of support (like the advice that is available) limit learning. When there is less directive support students are more actively searching for information in the resources available and while doing this gain more conceptual and implicit knowledge. Discussion with others during or after game play seems to support learning with games the most, as is also reported by Kirriemuir [START_REF] Kirriemuir | A survey of COTS games in education[END_REF]. Since relatively simple support within the game seems to make players "lazy", it can be questioned whether more advanced intelligent types of support will be effective in supporting learning within these kinds of environments. In the case of KM Quest more intelligent support could take the following form: A new version of KMQuest stores all user actions in a well-structured data base. As the program works with fixed quarters after which the values in the business model are adjusted, the moment of a quarter change also allows an analysis of the content of the data base. This analysis can lead to advice about visiting certain activities in the knowledge management model that were overlooked but are necessary for better learning. Another option is to predefine certain preferable learning trajectories and check the actual behaviour of the learners against these trajectories and give tailored advice about what to do next. A vision of the future could even be an online analysis of the content of the chat (which is also stored) to detect certain characteristic utterances that could signal learning difficulties or misconceptions that could be redressed by specific suggestions given by the system.

  additional information, and they have to choose knowledge management interventions to (try to) change the behaviour of the business simulation indicators. Most of these indicators are characterized by a decay factor. This means that the value of the indicators decreases over time when no interventions are implemented. The implementation of interventions involves costs, as well as several other activities that the players can perform. Players receive a limited budget that they can use to implement interventions and buy information.

Figure 1 .

 1 Figure 1. Virtual office interface of KM Quest.

Table 2 .

 2 Average values for some important business (process) indicators in the business simulation model at the end of the game for the whole group and for the groups with and without advice. At the bottom row the starting values. Market share, CSI = Customer satisfaction index, Profit = Total profit in 3 years in millions, JSI = Job satisfaction index, PQI = Product quality index, ATM= Average time for new product to market,

		MS*	CSI	Profit	JSI	PQI	ATM
	Advice	23.79	8.15	64.73	9.60	8.77	1.34
		(3.76)	(0.90)	(23,26)	(0.88)	(0.99)	(0.77)
	No adv.	24.43	8.33	68,58	9.50	9.04	1.14
		(3.18)	(0.61)	(19,54)	(0.76)	(0.66)	(0.55)
	Total	24.10	8.23	66,59	9.55	8.90	1.24
		(3.45)	(0.77)	(21,25)	(0.81)	(0.84)	(0.67)
	Start values	20.0	4.40	-	4.22	4.80	2.40
	* MS =

Table 3 .

 3 Average level of competence in three knowledge domains for the whole group and for the groups with and without advice. At the bottom row the starting values.

		Competence in marketing	Competence in R&D Competence in production
	Advice	8.63	8.27	6.43
		(1.22)	(1.29)	(0.79)
	No advice	8.72	8.60	6.40
		(1.00)	(1.02)	(0.71)
	Total	8.68	8.43	6.41
		(1.10)	(1.16)	(0.74)
	Start values	4.93	4.38	5.34

Table 4 .

 4 Average number of quarters in which some of the standard resources were used by the learners for the whole group and for the groups with and without advice. The resources are: feedback, the intervention handbook, the indicator handbook, history files, what and how files related to shared worksheets, and visualisation tools.

		Feedb	Intv HB Indc HB	Hist	What	How	Help	Visual
	Advice	5.73	6.13	3.13	3.67	0.80	1.00	0.47	6.73
		(4.33)	(4.36)	(1.99)	(2.19)	(0.86)	(0.76)	(0.52)	(4.93)
	No adv.	5.93	4.57	5.43	3.14	1.00	1.14	0.50	5.29
		(5.08)	(4.50)	(3.30)	(1.29)	(0.55)	(0.86)	(0.52)	(3.99)
	Total	5.83	5.38	4.24	3.41	0.90	1.07	0.48	6.03
		(4.62)	(4.42)	(2.90)	(1.80)	(0.72)	(0.80)	(0.51)	(4.48)


Dr Harvey Mellar

Dr Maria Kambouri

New technologies and adult literacy and numeracy in the UK

In this paper I shall be concerned with the issue of adult literacy and numeracy, what is often now referred

The non-UK reader may not immediately think of the UK as having significant problems in the areas of literacy and numeracy, and although it is true that complete illiteracy is rare in the UK, it is estimated that one in sixteen adults if shown the picture on the left of Figure 1 advertising a concert cannot say where the concert is being held, and one in four adults cannot calculate the change they should get out of £2 when they buy the goods displayed in the picture on the right.

Figure 1: Test items

Whilst the contexts of adult literacy and numeracy in Brazil and the UK are very different, I hope that some of the issues that I will raise about the role of new page 1 of 12 technologies in supporting adult literacy and numeracy will be nevertheless be helpful in the Brazilian context.

Lack of literacy and numeracy is claimed to be a key factor in disadvantage, high unemployment levels and social exclusion (e.g. BSA 1999). An international study of adult functional literacy carried out by the Organisation of Economic Cooperation and Development (OECD 1995 and1997), the International Adult Literacy Survey (IALS), found substantial variation across countries in literacy and numeracy levels; Scandinavian countries showed small proportions of adults operating at the lowest levels (e.g. 7% in Sweden) whilst a number of English speaking countries such as the UK, Australia, Canada, and the USA showed much higher proportions (over 20% in some cases). Translated into numbers this means that within the UK, some 7,000,000 adults are operating at low levels of literacy and numeracy. The impact on personal economic prospects and the national economy are estimated at billions of pounds per year (Bynner 1999). Yet, despite the scale of the problem relatively few of these adults at that time were taking up any relevant training -in 1998 the number taking up training was about 250,000 (DfEE 2001).

The Pisa Report (OECD 2002) gives some clues as to how the UK got into this situation in its description of the UK education system as a high quality education system, with low equity; so, at least until recently, whilst it produced some very well qualified pupils, it also failed to meet the needs of large numbers of pupils and many left school with low levels of literacy and numeracy.

In the UK a working group (DfEE 1999) on adult basic skills (literacy and numeracy) set about developing a new strategy for repairing the skills deficit, and this led to a major national remediation programme, entitled 'Skills for Life', which set out to reduce, within ten years, the numbers of people with poor basic skills to half of the 1999 levels. This strategy set out national targets, a new curriculum and systems of qualifications, planned delivery, quality assurance mechanisms, improved teacher training and inspection, and promised to deliver the benefits of new technology.

Role of new technologies

The foundation of much of this effort lay in what has come to be known as the Moser Report (DfEE 1999) and this report had forseen a vital role for new technologies (usually referred to in the UK as Information and Communications Technology -ICT) within the Skills for Life strategy, and the report made the following claims about the possible role of ICT in this strategy: page 2 of 12

• ICT is a powerful tool to raise levels of literacy and numeracy I have used the phrases new technologies and ICT in much of the discussion so far, and what I have usually been referring to was computers, and what is often called e-learning. The technologies that I have described are principally being used by institutions. However there are a number of other technologies that are likely to have an important influence in this area in the near future, and which are likely to be much more informal and personal in their use. One is the use of interactive digital TV (t-learning), and a number of companies are looking at developing approaches here, in particular building on the increasing popular use of games on interactive digital TV in order to provide literacy and numeracy activities. The hope is to reach audiences who would not have access to computers, or be interested in attending courses. Another approach being developed uses mobile technologies: mobile phones (including video-phones), and personal data assistants (m-learning). Both of these forms of technology have implications for the way we think about learning -they tend to be associated with much smaller units of learning, bite sized, on-demand learning, and are intended initially at least in order to increase the learners' self-confidence in their own ability to learn, for this is seen as the key first step in involving this particular target audience in learning. Introduction

Whilst there is some evidence of the impact of new technologies on the learning of children, much less is known about the use of new technologies with adults with skills for life needs. In addressing the needs of this group of learners we need to bear in mind that they differ from school children not only because they are older, but because they generally have bad memories of their own schooling.

Over the last seven years the Institute of Education has carried out a range of studies looking at the role of new technologies in supporting adult literacy and numeracy. In this section I'll outline the main results of one these studies. This project 2 ran from April 2000 to March 2001 and set out to investigate whether new technologies can be used effectively to provide learning to people with basic skills (literacy and numeracy) needs. This study pre-dated the implementation of the learndirect skills for life provision, and was carried out in eleven centres with a good track record of the use of computers in adult literacy and numeracy. The centres identified included Further Education colleges, community colleges, outreach centres, learning shops, a prison and a refugee organisation. Within these centres we interviewed 71 managers and tutors and 124 learners. The majority of the learners interviewed were also tested on their literacy and/or numeracy skills and 70 of these were post tested some two and a half months later.

Use of new technologies

A large number (36%) of the learners in the study had access to computers at home. This figure was only a little below the UK national average ownership figure at that time which was estimated to be around 44%. This was interesting as it had been anticipated that the level of ownership would have been significantly lower than the national average, since these learners are likely to be relatively less well off. The reason learners most often gave for not owning a computer was actually the perceived fear of theft in the areas in which they lived rather than cost.

The computing facilities for basic skills learners in some of the centres visited were excellent, but in other centres they were poor in comparison with what was made available to other learners. Learners who were new to the use of computers had little difficulty in starting to use computers, and the technology formed a barrier only for a small percentage of learners. The chief applications in use were: a range of literacy and numeracy software, office software (principally Word and PowerPoint), and the Internet (both for email and for accessing the Web). The most experienced tutors stressed the importance of close integration of the new technologies with basic skills provision. However, our study showed that the use of new technologies use in basic skills provision in most centres was at an early stage of development, and that it had not yet reached the critical threshold in terms of access, development of pedagogy, or tutor training for really effective implementation.

We found that the distinction between using new technologies to teach literacy, and introducing learners to new technologies as a new literacy was blurred in the practice of many of the tutors and learners that we spoke to. The nature of these new literacies and learners' perception of the new technologies were also undergoing change, for example there was evidence that older learners (that is, those above 24 years of age) thought of the Web chiefly as a source of information, but younger learners saw it rather as a source of entertainment.

New technologies and teaching

Whilst there were a small number of tutors who were highly experienced in the use of new technologies to teach skills for life learners, many of the tutors described themselves as relatively new to the use of technology. Even the more experienced were unaware of the range of use of new technologies in education more generally (for example in the schools, university and corporate training sectors). The people we spoke to were on the whole the enthusiasts and they indicated a great deal of resistance to the use of new technologies amongst their colleagues, particularly part-time tutors, who only had a few hours of teaching page 5 of 12 contact a week and felt that the use of new technologies would add to their workload.

One of the issues we were interested in exploring was the degree to which new technologies might enable this group of learners to work independently. Both tutors and learners expressed some support for the idea that new technologies might increase learner autonomy for certain learners, but they also saw the role of the tutor as crucial. A number of learners indicated their need for support whilst working with technology, and some even suggested that they might need more support than they needed with traditional approaches to learning. However there was a significant element -around 20% of the learners -who saw the greater autonomy in learning offered by new technologies as important for them.

Motivational impact

The use of new technologies in learning was attractive to learners, and this resulted in improved recruitment to courses. Ninety-two per-cent of learners in these centres said that they found the use of computers motivating. The learners who were being attracted through new technologies were often young men with low levels of literacy, and this was seen as a positive feature since young men are one of the hardest groups to involve in learning. However whilst new technologies were succeeding in attracting learners, there was a danger of centres promising learners more than they could deliver in terms of job opportunities.

Learning impact e learners said that computers helped them to learn, and in particular that computers helped them to concentrate and provided feedback. Figure 2 shows the percentage of those who identified specific learning goals during the first interview and the percentage of those who indicated that they had made gains towards these goals at a second interview some eight weeks later. Most felt that their literacy goals were being met, and in particular that computers enabled them to produce better presented work and to do it more quickly, however many felt that their numeracy goals were not being met and that their expectations about preparation for specific vocational areas were not being met. Many felt that they had made gains in computer skills even when this had not been one of their initial learning goals, and indeed 26% indicated that their work aspirations had altered to include a desire for a greater involvement with computers in their future work.

There was a statistically significant improvement between pre-and post-test scores in reading for learners with intermediate levels of literacy skill, but no significant improvements were found for those with very low levels of skill or for numeracy or ESOL (English for Speakers of Other Languages) though the numbers tested were small and hence less likely to show statistically significant changes. Case studies also indicated that computers could have a significant impact on the learning of some learners.

The significant improvement in reading scores is surprising in that the main activities reported by tutors were writing and preparing presentations, there was little discussion by tutors about teaching reading. There are at least two possible explanations for this difference. One possible explanation is that tutors were actually spending quite a lot of time teaching learners how to 'read the screen', but were not reporting this activity. Computer screens are considerably more complex reading environments than books, with icons, multi-layered menus, and multiple windows. Practice in reading the computer screen could well have had an impact on performance on the type of reading test item illustrated at the start of this paper where the reader has to identify a relevant piece of information amongst a number of other items. We have more recently carried out detailed observational studies of tutors teaching and have found that many tutors do indeed spend a lot of time teaching learners how to 'read the screen', but they do not necessarily describe this as teaching literacy skills. An alternative way of thinking about this might be to reflect on the idea 'omission design' (Fraser et al 1991), a principle of educational software enunciated some years ago that suggested that in designing educational software a designer needs to determine what the learner is required to do, and then take this element out of the software, so that the software can not do this for them, and as a consequence the learner is required to develop the skill.

In examining the effectiveness of computer use in skills for life provision the important pedagogic factors were found to be: assessment of learning needs, consideration of learning styles, careful integration of ICT into teaching and intensity of ICT use. Important organisational factors were found to be: availability and training of tutors, accessibility of ICT resources to basic skills learners, level of technical support, and prevailing institutional culture with respect to basic skills, ICT and learning.

Ongoing research

Three key issues were identified at the start of this paper:

• Can new technologies be used in such a way as to enable greater access to literacy and numeracy provision for adults? • In what ways can new technologies be used with this group of learners to enhance their learning? • What are the new basic literacies of the 21 st century?

Perhaps the single biggest issue for new technologies for skills for life provision is the issue of access and motivation. Computers and interactive multimedia have been seen to be a powerful incentive for learners to get involved with learning. Often learners come into provision looking for computer skills but then find that they have literacy and numeracy problems that get in their way, and so are helped to begin to focus on these. Others are motivated to stay within a course because of the computer skills element. For some learners, computers and interactive multimedia provide an alternative route into education where traditional classroom approaches have failed them in the past. The flexibility of access promised by new technologies is a major attraction. Can new technologies genuinely help to widen provision, are they in fact enabling us to address the learning divide, or is the digital divide simply replicating the learning divide? On the negative side, Gorard (2002) in an analysis of survey data on adult education came to the conclusion that: …access to ICT does not, in itself, make people anymore likely to participate in education and (re)engage with learning. Access to ICT continues to be largely patterned according to long-term pre-existing social, economic and educational factors. Thus, like educational qualifications, access to ICT is a proxy for the other, more complex, social and economic factors that pre-date it rather than as a direct contributory factor in itself. However, it is not surprising that "access to ICT … in itself" will not achieve any of these things, but what we need to is to explore ways in which new technologies can be used to improve access. In one project for Ufi we have looked at the potential of computer games for attracting learners (Kambouri, M. Schott, G. Thomas, S., Pavlou, V. and Mellar, H., 2003). In a range of ongoing projects the National Research and Development Centre has looked at the use of laptop computers in outreach community provision (National Research and Development Centre for Adult Literacy and Numeracy, 2004), the use of multimedia authoring in prison education, and the use of mobile phones in literacy provision.

Turning from the access issue to the learning issue, it is clear that the majority of skills for life learners need considerable support in their use of technologies, and that tutors also need to further develop their skills in the use of technologies in teaching. In one project we have examined the level of support that is needed to support learners working with e-learning materials (Kambouri, M., Mellar, H., page 8 of 12 Kinsella, K and Windsor, V. 2003), and in other projects we have involved teacher-researchers in classroom observation studies (Mellar, H., Kambouri, M., Sanderson, M., and Pavlou, V. 2004), and in intervention studies examining the impact of a range of forms of use of new technologies with these learners.

The third issue concerns the changing nature of literacies. Many authors have argued that digital technologies have implications for our ideas about literacy more generally and these debates have to some extent been reflected within the discussions around skills for life. So, for example, ICT is coming to be seen as a basic skill in its own right, as one UK government think tank (Post-16 E-learning Strategy Task Force 2002) writes:

Reading and writing are universally accepted as basic entry requirements for social inclusion. However, it is clear that a lack of ICT skills, including not having the skills to learn through ICT, is potentially now as great a barrier to employability and social inclusion as a lack of achievement in literacy and numeracy have been to date.

ICT user skills include those most frequently required in employment, for example, those associated with the Internet, email, word processing and other commonly used business packages. We also include an element of e-learning skills in the definition of ICT user skills; no individual who has undertaken ICT basic user skills programmes should struggle to engage with the basic principles of e-learning.

Others within the basic skills area have argued for a much wider view of technological literacy, for example humanITy (2003) writes:

Traditionally we have emphasized traditional, autonomous, discrete literacy which says: The more complex the set of skills to complete a task the fewer those who can perform them all. ICT, inclusive, collaborative literacy says: The more complex the set of skills to complete a task, the greater the number who can become involved. We need to think much more of the second and much less of the first. Carvin (2000) developing from a definition of basic literacies distinguishes a number of elements of relevant new literacies, including:

• Technological Literacy: the ability to utilize common ICT tools, including hardware, software, and Internet tools like search engines. • Information Literacy: the skills to ascertain the veracity, reliability, bias, timeliness, and context of information. • Adaptive Literacy: the willingness to learn new tools and to apply previous ICT learning to new situations.

How we define these new literacies will be extremely important in determining whether we genuinely manage to address the issue of skills for life needs, or whether we succeed in solving the last century's problems only to fail to solve those of the present century. The UK Department for Education and Skills has now designated ICT as a Skill for Life and published draft standards, and the page 9 of 12 process of developing appropriate curriculum has begun. However these standards seem to have been more carefully aligned with the demands of the workplace rather than those of learners everyday lives and these standards is likely to remain an area of controversy for some time to come.

Following

  on from the publication of this strategy the Department for Education and Skills (DfES) began a large-scale investment programme -some £1.5 billion pounds in the first three years. In November 2000 the Adult Basic Skills Strategy Unit (ABSSU) within the DfES was established with the goal of delivering the Skills for Life strategy, and in 2002 the National Research and Development Centre for Adult Literacy and Numeracy was established in order to provide a focus for the research and development work underlying this strategy.

  Figure 2: Learning goals and gains

•

  Computers and multimedia software provide attractive ways of learning

	• The Web enables access to the best materials and the most exciting
	learning opportunities
	• ICT offers a new start for adults returning to learning
	• The Internet and digital TV technology can reach into the home
	• Learners who use ICT for basic skills double the value of their study time
	acquiring two sets of skills at the same time.
	These claims about the value of new technologies for addressing basic skills
	needs have three strands. The first three claims assert that digital technologies
	provide effective and exciting educational possibilities. The next two claims are
	about using digital technologies as ways of connecting with communities of
	learners who are seen as having been excluded by traditional education
	provision. In the last claim we see a third theme, which is that new technologies
	involve a new set of skills, that new literacy skills are involved.
	Within the UK the use of ICT within the Skills for Life strategy has two major
	organizational structures. One is within adult learning centers and Further
	Education colleges, and the other is through Ufi/learndirect.
	Ufi/learndirect is a public-private partnership set up to provide on-line training
	across a number of areas, including computer skills, business skills and skills for
	life. To give an idea of the size of this undertaking, for 2002-2003 there were
	400,000 learners registered with learndirect, with over 100,000 of these taking
	Skills for Life courses. A learner's first contact with learndirect is usually through
	the telephone help line. Tutorial support is available at one of the more than
	2,000 local centers. The learning centers, which are situated in colleges,
	libraries, museums, even bars, do not belong to learndirect but are in effect
	'franchised'.
	learndirect materials are chiefly interactive electronic teaching materials
	delivered either on-line or on CD ROM, and they can be accessed at local
	centres, at home or in the workplace. The materials vary in type, there are

multiple-choice interactive exercises, exercises based around video-clips, sometimes of other learners at work, and sophisticated gaming environments 1 . The numeracy game MaxTrax is an arcade style car racing game with embedded numeracy activities. Runner is an adventure game with an urban, underground and futuristic setting typical of many commercial games, and which can be used to practice and develop literacy skills. Other approaches adopted by learndirect centres to using new technologies are outlined in a guide that the Institute of 1 Sample materials are available at http://www.learndirect.co.uk/personal/freestuff/tasters/# -last accessed 24 July 2004 page 3 of 12 Education prepared for Ufi -Using ICT to develop literacy and numeracy skills for adults: a guide

(Ufi 2001)

This research project was supported by Ufi Limited, the organisation which, together with its partners, has established an e-learning network of over 2000 learning centres and set up learndirect's national learning advice service.A summary of the research report is available at: http://www.ufi.com/press/papers/literacyguide.pdf -last accessed 24 July 2004 page 4 of 12

Bibliography

• Basic Skills Agency (1999), European Social Exclusion Project, London: Basic Skills Agency. • Brooks, G. et al (2001) Progress in Adult Literacy: Do learners learn? London: Basic Skills Agency. • Carvin, A. (2000) 'More than Just Access', EDUCAUSE Review, November/December 2000. Available at: http://www.educause.edu/pub/er/erm00/articles006/erm0063.pdf (last accessed 23 July 2004). • DfEE (1999) A fresh start: improving literacy and numeracy (The report of the working group chaired by Sir Claus Moser). London: DfEE. Available at: http://www.lifelonglearning.co.uk/mosergroup/index.htm (last accessed 24 July 2004). • DfEE (2001) Skills for Life: The national strategy for improving literacy and numeracy skills. Available at: http://www.dfes.gov.uk/readwriteplus/bank.cfm?section=211 (last accessed 24 July 2004). • DfES (2003) 21st Century Skills Realising Our Potential. Available at: http://www.dfes.gov.uk/skillsstrategy/_pdfs/whitePaper_PDFID4.pdf. (last accessed 24 July 2004). • Fraser, R., Burkhardt, H., Coupland, J., Philips, R., Pimm, D., & Ridgway, J. (1991). 'Learning activities and classroom roles with and without the microcomputer'. In O. Boyd-Barret & E. Scanlon (Eds.), Computers and learning. Wokingham: Addison Wesley & Open University • Gorard, S. (2002) Lifelong learning trajectories in Wales: results of the NIACE Adult Learners Survey 2002. Available at http://www.cf.ac.uk/socsi/ICT/niacewales.doc (last accessed 24 July 2004). • Harris, S. R. and Shelswell, N (2001) 'Building bridges across the digital divide: Supporting the development of technological fluency in Adult Basic Education learners' in Proceedings FACE Annual Conference 2001 (Ed, Mason, A) FACE, Bristol, pp. 42-53. Available at: http://case.glam.ac.uk/CASE/StaffPages/SteveHarris/SRHPubs/Building%20 Bridges%20Final%20Draft.pdf (last accessed 24 July 2004). • Horsburgh, D. and D. Simanowitz (1999). 'Basic skills and ICT: a marriage made in heaven?' Active Learning 10: 44-50. page 10 of 12 • Horsburgh, D. and D. Simanowitz (1999). 'Write and ROM -best practice for multimedia use in basic skills.' Basic Skills September: 7-9. • humanITy (2003) Briefing paper no 1 ICT development and basic skills strategy, humanITy: inclusion in the information age. Available at: http://www.humanity.org.uk/articles/pub_ictdevelopment.shtml (last accessed 24 July 2004). • Kambouri, M., Mellar, H., Kinsella, K and Windsor, V. (2003) Research into the effectiveness of learndirect materials and support system for learners with skills for life needs, Report submitted to Ufi June 2003. Available at: http://www.ufi.com/press/papers/IoE%20SFl%20Effectiveness.pdf (last accessed 24 July 2004). • Kambouri, M. Schott, G. Thomas, S., Pavlou, V. and Mellar, H. (2003) Evaluating learndirect games for learners with skills for life needs -Final Report submitted to UfI September 2003. Available at http://www.ufi.com/press/papers/UFIGamesfinalreport.pdf (last accessed 24 July 2004). • Kelly, H. (1999). 'New approaches to adult basic education.' Basic Skills, September: 7-9. • Mellar, H. and Kambouri, M. (2004) 'Learning and teaching adult basic skills with digital technology: research from the UK' in Brown, A.J. and Davis, N.E. (eds) Digital technologies, communities and education: World Yearbook 2004. RoutledgeFalmer: London. • Mellar, H., Kambouri, M., Sanderson, M., and Pavlou, V. (2004) ICT and adult literacy, numeracy and ESOL. London: NRDC. • Mellar, H., Kambouri, M., Wolf, A., Goodwin, T., Hayton, A., Koulouris, P. and Windsor, V. (2001). Research into the effectiveness of learning through ICT for people with basic skills needs June 2001. Sheffield: Ufi. Summary entitled Using ICT to develop literacy and numeracy: research summary available at: http://www.ufi.com/press/papers/literacyguide.pdf (last accessed 24 July 2004). • National Research and Development Centre for Adult Literacy and Numeracy (2004). Using laptop computers to develop basic skills: a handbook for practitioners. Available at: http://www.nrdc.org.uk/uploads/documents/doc_2838.pdf. • QCA (2004) Basic skills: Information and communication technology http://www.qca.org.uk/qualifications/types/2791.html (last accessed 24 July 2004). • The Post-16 E-learning Strategy Task Force (July 2002). Get on with IT: The Post-16 E-learning Strategy: Task Force Report. Available at: http://ferl.becta.org.uk/content_files/pages/keydocs_current/gov/GetOnwithIT _Doc.pdf (last accessed 24 July 2004). page 11 of 12 page 12 of 12 • Ufi (2001) Using ICT to develop literacy and numeracy skills for adults: a guide. Available at: http://www.ufi.com/press/papers/literacyandnumeracy.pdf (last accessed 24 July 2004).