Harvey Mellar

Maria Kambouri

Kit Logan

Sally Betts

Barbara Nance

Viv Moriarty

Emily Brewer
email: e.brewer@ioe.ac.uk

John Vorhaus

Using ICT EFFECTIVE TEACHING AND LEARNING

Preface

The Skills for Life Strategy in England has led to unprecedented investment in adult literacy, language and numeracy (LLN), major reforms of teacher education and training, and the introduction of national standards, core curricula and assessment to inform teaching and learning. We have a unique opportunity to make a step change in improving levels of adult skills. But until recently too little was known about effective teaching and learning practices, and reports from Ofsted and the Adult Learning Inspectorate repeatedly drew attention to the quality of teaching, and the need for standards to improve.

It has been a strategic priority at the National Research and Development Centre for Adult Literacy and Numeracy (NRDC) to investigate teaching and learning practices in all the subject areas and settings in Skills for Life, to report on the most promising and effective practices, and to provide teachers and trainers, along with policy-makers and researchers, with an unparalleled evidence base on which to build on the progress already made.

Our findings and recommendations are reported here, and in the four companion reports covering reading, writing, numeracy and ESOL. The five studies, which have been coordinated by NRDC Associate Director John Vorhaus, provide material for improving the quality of teaching and learning, and for informing developments in initial teacher education and continuing professional development (CPD). We are also preparing a range of practitioner guides and development materials, as a major new resource for teachers and teacher educators. They will explore and develop the examples of good and promising practice documented in these pages.

The Moser Report (Department for Education and Employment 1999) placed great emphasis on ICT, which has been classed as a 'skill for life' within the Skills for Life Strategy since 2003. It said: 'At the heart of improved quality in delivery and materials must be increased use of Information and Communication Technologies to improve basic skills.' The report went on to assert that ICT is a powerful tool to raise levels of literacy and numeracy, and that learners who use ICT for basic skills double the value of their study time, acquiring two sets of skills at the same time. This report looks into the teaching and learning of ICT, and finds evidence to support many of the claims made about learning with ICT in the Moser Report (DfEE 1999). It adds to the evidence from previous research on ICT in Skills for Life, including the DfES Survey of Needs and Impact (2003b), making important recommendations for developing teaching and learning in ways which include ICT, as new uses of literacy and numeracy and new technologies emerge.

Ursula Howard, Director, NRDC

ICT long LIVE 17/1/07 17:44 Page 6

Effective Teaching and Learning : Using ICT motivation, collaboration and purposeful action, and also highlighted the value for ICT used in this way in allowing differentiation within the classroom. Role-play was an important aspect of these activities.

The creation of learning opportunities: teaching strategies

The most effective teaching strategy was extending, where the tutor built on or added to material previously introduced, or added to a comment by a learner. There was a strong correlation between this type of teaching activity and changes in ICT skills and confidence scores.

We encouraged teaching strategies that aimed to increase the autonomy of learners so that they were able to engage in self-directed learning. There was much evidence that tutors were doing this to a greater degree than previously, There was a positive correlation between the amount of time spent by tutors managing activities with gains made in both ICT skills and confidence.

Other teaching activities that positively correlated with improvements in ICT skills were: discussing, instructing (telling learners what to do -i.e. organisational issues), listening and modelling (showing the learner how to do something using the actual technology or its representation on a SMART Board™).

One strategy was found to be negatively correlated with changes in ICT skills and confidence, and that was explaining (where the tutor offers the 'solution' and tells the learner 'how to do something'). Where the tutor actually demonstrated what was to be done ('modelling') this was much more successful.

Different types of outcomes

ICT can change the focus on the knowledge to be learnt. This was evidenced by tutors talking more about managing information, and less about learning it; more about browsing and scanning and less about reading as comprehension.

The nature of literacy tasks was also often changed in subtle ways. For example, a task previously conceptualised as writing short sentences became an exercise in relating image to text.

Going beyond the skills described in the ICT Skills for Life Curriculum and those tested in the ICT-skills assessment, tutors also sought to encourage learners to evaluate resources, and to promote a willingness to learn new skills and to transfer their skills from one context to another.

Tutors developed their teaching and skills through experimentation and reflection, before sharing their learning with peers, as part of the project's action research.

Recommendations

Development work and quality improvement

'Purposeful use' within the ICT Skills for Life Curriculum is an important concept that can help to change ICT teaching. Evidence from this project can inform the pedagogy that enables the curriculum to be effectively implemented.

The following activities contributed to improved ICT skills and confidence, and should be given greater attention by teachers and teacher trainers:

1. greater use of technology by learners 2. tutors use of the whiteboard 3. tutors managing classroom activities, extending, discussing, instructing, listening and modelling.

Teachers and teacher trainers should be made aware that the following were effective in supporting the development of ICT skills and confidence:

1. encouraging collaborative learning 2. encouraging learner autonomy, and consequently releasing teacher time to get to know their learners better and to more carefully adapt their teaching to learners' needs 3. use of a wide variety of technologies, and in particular mobile technologies to support greater flexibility 4. the use of technology to construct (usually shared) artefacts -which often allowed learners to experiment with a variety of roles, and allowed for differentiation in activities.

We recommend two forms of staff development: one for those who generate and develop models of ICT use; another for those who wish to pick up and adapt tried and tested models.

Policy

The findings of this project lend strong support to the claim in the Moser Report (DfEE, 1999) that 'Learners who use ICT for basic skills double the value of their study time, acquiring two sets of skills at the same time'. We suggest that it is a priority to support the use of ICT in the teaching of Skills for Life, as students can double the value of their study time.

To enable the best use of ICT, further development of resources and exemplar materials is needed, together with the development of appropriate forms of tutor training.

Research

Further research is needed to determine the best ways of using collaborative work with technology in provision for adult learners.

Literacies are changing as technology develops, and the relationship between ICT and other literacies continues to change and evolve -this has implications for the use of ICT in teaching Skills for Life courses, which requires continuous research.

Technologies themselves are changing. In the near future we are likely to see wider use of technologies such as interactive digital TV and mobile phones, and the further development of ubiquitous computing. Continued research will be needed into determining the most effective ways of using these technologies in adult learning.

Limitations to this research

We acknowledge the limitations of the study in terms of small sample size, although it would not have been possible to carry out a study of this kind with larger samples.

The study was fairly representative of current provision in the field. However, it was biased towards provision in further education (FE), with less emphasis on the smaller units of community-based provision.

The tests were not always sufficiently appropriate to the specific teaching interventions, which was a limitation imposed by the need to have comparable tests across a range of contexts.

2

The background to the study 2.1 The policy background

Skills for Life: Adult literacy and numeracy

In 2000, the Qualifications and Curriculum Authority (QCA) and the Department for Education and Skills (DfES) published the first national standards for adult literacy and numeracy, and the Skills for Life strategy was launched in 2001. These standards formed the basis of a number of initiatives, now established, to address levels of literacy and numeracy among the adult population. These include:

■ national qualifications in adult literacy, adult numeracy and ESOL at Entry level, Level 1 and Level 2 ■ national adult literacy and adult numeracy tests at levels 1 and 2 ■ national adult literacy and adult numeracy and ESOL core curricula.

ICT to support Skills for Life

The Skills for Life strategy emerged following the Moser Report. In its proposals for addressing adult literacy and numeracy needs in the UK the Moser Report (DfES, 1999) placed great emphasis on ICT, saying: 'At the heart of improved quality in delivery and materials must be increased use of Information and Communication Technologies (ICT) to improve basic skills.' The report went on to make the following specific claims:

■ ICT is a powerful tool to raise levels of literacy and numeracy. ■ Computers and multimedia software provide attractive ways of learning. ■ The web enables access to the best materials and the most exciting learning opportunities. ■ ICT offers a new start for adults returning to learning. ■ The internet and digital TV technology can reach into the home. ■ Learners who use ICT for basic skills double the value of their study time, acquiring two sets of skills at the same time.

In order to throw light on these claims, we have carried out a number of evaluation studies of the use of ICT (including interactive digital TV, CD-ROMs, web-based and hybrid technologies) in the teaching of adult literacy and numeracy for the Basic Skills Agency, DfES and for Ufi/learndirect (Kambouri and Kett, 2001;Mellar et al., 2001;Kambouri et al., 2002;[START_REF] Kambouri | Research into the effectiveness of learndirect materials and support system for learners with skills for life needs[END_REF]Mellar and Kambouri 2004). This research found some positive signs of the use of ICT in the teaching of adult literacy and numeracy, but suggested that there was a long way to go if the expectations of the impact of ICT on learning for this group of learners were to be met.

Much of the research in this area -both our own and that of others (for example, Hopey, 1998;Mellar et al., 2001) -was based on surveys or interviews, and while it gave an insight into some of the principal variables that may be involved, it gave little detailed account of what tutors actually do when they are using ICT in adult literacy, numeracy and language teaching. We therefore carried out detailed observational research in classrooms to examine how the use of ICT had actually impacted on the teaching and learning of adult literacy, numeracy and language, and reported on the findings in Mellar et al. (2004). The findings of this observational project then enabled the generation of a first set of hypotheses about effective strategies which formed the starting point for the present study.

ICT as a Skill for Life

By 2003 ICT was itself coming to be seen as a Skill for Life, in the White Paper, 21st Century Skills: Realising Our Potential (July 2003), ICT skills are for the first time included within the Skills for Life programme:

Until now, basic skills have referred to literacy and numeracy. In today's society, we believe it is as important that everybody can also use Information and Communications Technology (ICT), particularly in the workplace. So we shall offer basic ICT skills as a third area of adult basic skills alongside literacy and numeracy within our Skills for Life programme.

During the period of this project, the standards for adult ICT user skills were being developed by the QCA (QCA, 2005a). The standards for adult literacy, adult numeracy and ICT follow a common format and relate directly to the key skills of communication, application of number and ICT. They have also been developed to match the National Curriculum requirements for English, mathematics, ICT and the National Occupational Standards for ICT.

A pilot project looking at the teaching and learning implication of these standards was carried out (NIACE, 2005) leading to the development of a draft document, ICT Skill for Life Core Curriculum (QCA, 2006) and as a consequence a series of ICT Skills for Life Pathfinder projects began in September 2005.

However, the ICT Skill for Life curriculum was not to be funded in the same way as other Skills for Life -instead the Learning and Skills Council was called on to review its funding arrangements for ICT skills across the board. As a consequence during the period of this study many courses with quite significant ICT elements were delivered under the banner of literacy, numeracy and ESOL in order to call on the necessary funding.

Yet more recently has seen the emergence of Functional Skills, and ICT as a Functional Skill:

The QCA has begun work to develop functional skills qualifications. Functional skills are seen as fundamental skills that need to be developed and built into programmes of learning and development for young people and adults to prepare them for participation in life and work.… QCA has refocused the work originally planned for the Convergence Project (the convergence of basic and key skills qualifications into one set of qualifications) to progress this work.

In the case of ICT this means that each individual is confident and capable when using ICT systems and tools to meet a variety of needs in a range of contexts. For example they will use ICT to find, select and bring together relevant information and use ICT to develop, interpret and exchange information, for a purpose. In life and work each individual will be able to apply ICT safely to enhance their learning and the quality of their work. QCA (2005b) Alongside these developments have also been projects looking at staff development in the use of e-learning for Skills for Life tutors -notably the Learning and Skills Development Agency (LSDA) project 'Exploring e-learning'.

The Effective Practice Studies (note by Greg Brooks)

The project reported here was conducted as part of the research agenda developed by the National Research and Development Centre for Adult Literacy and Numeracy (NRDC), which is part of the Skills for Life strategy (DfEE, 2001) of the Department for Education and Skills (DfES) in England.

Four studies had the common aim of investigating effective teaching of literacy, language and numeracy to adults. The skills which the four projects covered, and the organisations which conducted them, were: The project reported here was a fifth project which covered ICT; this was conducted by the Institute of Education, University of London. This project differed from the others in that its first phase was developmental, rather than evaluative; as a consequence, only the second phase of its data-gathering resembled that of the four projects listed above, and its final sample size was smaller. Also, it had a shorter timescale, being completed in March 2005.

The other four projects all began in July 2003 and were completed in March 2006.

The motives for them were the questions posed in NRDC's Strategy, published in July 2003 (NRDC, 2003:30):

■ How can teaching, learning and assessing literacy, numeracy and ESOL be improved? ■ What factors contribute to successful learning?

Even before NRDC was set up it was apparent from reviews of the field [START_REF] Brooks | Assembling the Fragments: a review of research on adult basic skills[END_REF][START_REF] Kruidenier | Research-based Principles for Adult Basic Education Reading Instruction[END_REF] that little reliable research-based evidence existed to answer these questions, and various NRDC reviews showed that progress in amassing such evidence, though welcome where it was occurring, was slow [START_REF] Coben | Adult Numeracy: review of research and related literature[END_REF][START_REF] Barton | Adult ESOL Pedagogy: a review of research, an annotated bibliography and recommendations for future research[END_REF][START_REF] Torgerson | A systematic review and meta-analysis of randomised controlled trials evaluating interventions in adult literacy and numeracy[END_REF][START_REF] Torgerson | Adult literacy and numeracy interventions and outcomes: a review of controlled trials[END_REF][START_REF] Torgerson | A systematic review and meta-analysis of controlled trials evaluating interventions in adult literacy and numeracy[END_REF]. Four preliminary studies, on reading, writing, ESOL and ICT, were undertaken between 2002and 2004[START_REF] Besser | Adult learners' difficulties with reading: an exploratory study[END_REF][START_REF] Kelly | Teaching and Learning Writing: a review of research and practice[END_REF][START_REF] Roberts | English for Speakers of Other Languages (ESOL) -case studies of provision, learners' needs and resources[END_REF]Mellar et al., 2004, respectively). However, NRDC recognised a need to build on these to expand the research base on the practice of teaching these subjects and therefore the information available to policymakers and professionals, both teachers and trainers.

The inspiration for the design of the reading, writing, ESOL and numeracy projects, and the second year of the ICT project, was a US study of the teaching of literacy and English language to adult learners for whom English is an additional language [START_REF] Condelli | What Works Study for Adult ESL Literacy Students[END_REF].

The projects were carried out in two phases, in academic years 2003/04 and 2004/05. The targets across the two years were to recruit and gather background data on about 250 learners, assess their attainment and attitudes at two points during the year in which they were participating in the study, interview both learners and teachers, observe the strategies their teachers used, and correlate those strategies with changes in the learners' attainment and attitudes.

Features of this study

Aims

We set out to develop and evaluate effective ICT-based teaching strategies, and we were looking both for development of literacy and numeracy skills, and for development in ICT literacy skills. These strategies were collaboratively developed by the tutors involved in the work and the research team, and the involvement of tutors was a key element of the process.

We developed these teaching strategies through a series of trials using theoretically grounded ICT task designs targeted at specific learning objectives.

Though our initial aims included both literacy and numeracy skills, in practice we were relatively unsuccessful in recruiting numeracy tutors, and although we had a couple of very interesting projects during the development phase, neither could be continued into the evaluation phase for practical reasons outside our control, and so the majority of this report will be concerned with ICT and literacy and ESOL.

Specific questions

Our main questions were first to identify effective ways of using ICT to support the learning of other Skills for Life, and secondly to identify effective ways of learning ICT skills themselves.

In doing this we would be testing whether the principles we had derived from our previous work and from a consideration of the literature were useful in building effective ways of using ICT.

Side by side with these main questions we sought to gather data on a number of other specific issues:

■ the motivational impact of ICT on learners ■ how ICT was impacting on the wider learning context ■ effective support for the development of tutors in the use of ICT in Skills for Life.

Brief account of method

Design research

We have previously carried out a number of studies of the use of ICT in the teaching of adult literacy and numeracy, and from this work we concluded that there was relatively little good practice, and that research in this area needed to move forward in close coordination with developing practice. It was therefore not felt to be appropriate to adopt the approach taken by the other effective practice projects of looking for correlations between aspects of present practice and learning outcomes. Instead we took a more interventionist approach inspired to some degree by the design-based research perspective (The Design-Based Research Collective, 2003), developing approaches to using ICT, and also developing the skills of the tutors in using ICT for teaching.

Phase 1 (July 2003 to July 2004) -Development

In the first phase of the study we worked with a group of nine tutors as practitionerresearchers in devising uses of ICT to support learning in their classroom contexts. Tutors had one day a week release time and worked closely with two development officers who helped them set up the use of ICT in their classes. The tutors met on a regular monthly basis throughout the year for one-day workshops, in which they examined uses of ICT in learning from both a practical and theoretical viewpoint, and presented and refined with the group their designs for teaching with ICT. The development of each project went through three iterations of design and informal evaluation. At the end of this development phase, the tutors created case studies of their approaches, which we hope to make available soon.

Phase 2 (October 2004 to March 2005) -Evaluation

Based on the development work of the previous phase, seven scenarios of ICT use (referred to as 'schemes of work' in this report) covering a range of technologies, software and pedagogical approaches (video, WebQuests, collaborative learning, etc) were identified to take forward for evaluation. Each project was led by the practitioner-researcher who developed the approach and usually also included another tutor (a 'buddy') who implemented the same approach.

At the start of this phase the learners were given a series of tests -on ICT skills, ICT confidence and ESOL listening skills and/or the NFER GO reading test. The teaching sessions were approximately 40 hours in duration, spread over two terms. Each classroom was observed on three occasions, and a sample of individual learners was additionally observed carrying out the tasks. Learners were again tested at the end of the teaching sessions.

There were 51 class observations, and 123 20-minute observations of individual students. Some 180 students undertook the tests, and we have both pre-and post-test on the ICT skills for 81, on the ICT confidence for 91, on the GO reading test for 60, and on the ESOL listening test for 67, with complete data (i.e. ESOL and/or GO, ICT skills and ICT confidence, and Profile) for just 61 learners.

These are not large numbers of learners, and our study does not seek to make generalisations about learners, but rather seeks to identify the features of the ICT interventions which were most effective with these learners. The sample size is sufficient to do this, as it is large enough to allow tests of statistical significance to be usefully applied, as we will see in the analysis of the data. To what degree our sample is representative of other groups of learners is best determined by an examination of the demographic data about the learners that we present, and the case study accounts of the classrooms.

Structure of this report

The rest of the report is organised as follows:

■ Chapter 3 describes the ideas behind the design of the intervention studies; ■ Chapter 4 describes the case studies, summarising the classroom observations, and so presenting a picture of the tutors' practices; ■ Chapter 5 looks at the quantitative aspects of the classroom observational data; ■ Chapter 6 describes the learners who took part in the study, specifically their access to ICT and confidence in the use of ICT, and the progress that they made as shown by tests of language and ICT skills, and ICT confidence; ■ Chapter 7 brings together the work of the previous three chapters and asks: What is the relationship between the learners' experience, the teachers' practice and the learners' progress? This is explored partly through correlations between the quantitative measures, and partly through a consideration of the qualitative data derived from the classroom and student observations;

■ Chapter 8 looks at the development of the tutors themselves through working within the project, through a process of inventing new practice, and testing it out. We also examine the way in which the tutors worked with their buddies and ask to what degree the lessons learned by the tutors were transferable to their buddies; ■ Finally, Chapter 9 sets out our conclusions and the implications for effective practice in the use of ICT for supporting other Skills for Life and for teaching ICT skills.

The full project report, which will be available on the NRDC website, additionally contains a series of appendices giving details of the assessment instruments, observational schedules and other research instruments and background data on the participants.

3 Intervention design

Previous work

The starting point for our design of effective interventions was the recommendations that we had arrived at the end of the Stage 1 study (Mellar et al., 2004). There we identified a range of what seemed to us to be good practice in the use of ICT in adult literacy and numeracy.

■ Clear lesson aims, explained and/or negotiated with learners at the start of the session, coupled with a review of what has been learnt at the end of the session. (This has particular importance in directing the learners' attention to what they are expected to learn from the use of the technology and how they are expected to do this). ■ Multiple ways of providing information. ■ Providing opportunities for peer learning both through small group work and through projection of learners' work for class discussion. ■ Flexibility in classroom management: accepting collaboration when it occurs naturally, directing learners to specific reinforcement activities if necessary, and sometimes avoiding intervention as part of a process of encouraging student independence and autonomy.

We also identified a number of areas where development was needed:

Approaches to using technology ■ Tutors need to reflect on why they are using ICT and to match the way that they use the technology to their aims. ■ Teaching needs to address the changing nature of literacies in the digital age: new types of relationships between media are being created and new genres of writing are developing.

Teaching

■ A wider range of technologies and ways of using them needs to be explored. ■ Greater experimentation with teaching styles and forms of classroom management should be encouraged. ■ Work is needed to develop appropriate ways of learners working effectively together using ICT. This needs to go beyond simply allowing learners to work together, and to involve defining structures and roles for collaborative work.

Teaching ICT ■ The issue of how to teach ICT skills needs to be addressed more explicitly. Tutors who adopt a wide range of strategies for developing learners' literacy and numeracy skills sometimes adopt a purely didactic form of teaching when approaching ICT skills, or alternatively adopt a time-intensive strategy of individual tuition.

ICT, teaching and learning

In order to structure our thinking about the role of ICT in teaching and learning we will use the framework developed by Ivanic and Tseng (2004) which is summarised in their diagram reproduced in Figure 3.1 below, and incorporate within this framework some of the insights derived from the literature on ICT and learning 1 .

Figure3.1 The relationship between teaching and learning: a conceptual framework (Ivanic v and Tseng)

In their discussion of this framework Ivanic and Tseng place the use of ICT under 'use of resources' in the context of teaching/learning events. However, in this study we are looking at the learning and teaching of ICT skills themselves as well as the learning of language supported by ICT. So each aspect of their framework has implications for our study, and we will argue that ICT is actually implied in most, if not all, of the factors that they have identified, and so has a much more significant role in learning and teaching than is suggested by describing it simply as an educational resource.

The following discussion indicates some of the ways that we believe that ICT further adds to the complexity of each of the factors that they have identified, and evidence from the case studies will provide additional illustration for these interactions. While many of the points that might be made about ICT and learning and teaching could be common for a wide range of learners, we will seek to highlight those aspects which we believe are of specific concern for adult learners. So within the following discussion we will indicate in marked boxes those key points that emerge that either specifically informed the design of our interventions, and/or which we believe are of specific concern for ICT in adult learning (and so will inform our analysis).

Factors affecting learning-teaching events Participants' beliefs, intentions and resources

Learners come with beliefs about the role of technology in learning which impact upon the way that they use technology. Earlier work on media more generally has shown that the impact of media on learning and on motivation for learning is itself related to learners' beliefs and expectations about their reactions to the media and not just the media themselves [START_REF] Salomon | Television is easy and print is "tough": The differential investment of mental effort in learning as a function of perceptions and attributions[END_REF].

Learners also have different intentions relating to what they want to get from the courses in terms of the balance of ICT skills and literacy skills; rarely (in these courses at least) is ICT seen purely instrumentally, as simply the means to another learning objective. Access to ICT resources (ownership of technology and access to the internet), personal resources in terms of ICT experience and attitudes to technology, as well as supporting social networks of ICT knowledgeable family and friends all play important roles in learners' take-up and use of technology.

The ability of ICT to motivate adult learners, both to come into learning and to stick with learning has often been discussed. However, 'traditional' digital technologies, such as the desktop computer, may well be losing their appeal, and the project was keen to explore the motivational impact of the use of a variety of technologies, such as mobile phones, tablets, and digital video.

Much of what has been said about learners applies equally to tutors, who also bring differing beliefs about the role of technology in learning and the balance of skills that they are trying to teach, and who have differing access, skills and support from social networks. Tutors' familiarity and confidence with technology is an important factor in how easily they can integrate technology in their teaching.

■ Adults have differing expectations about ICT, and may also be lacking in confidence in the use of technology. ■ There may be differences between the ICT skills and attitudes to technology of younger and older adult learners, and particular care may need to be taken to develop the skills of older learners. ■ The use of a wide range of up-to-date digital technologies might be motivational for adults.

The political/institutional context Our earlier work (see for example Mellar et al., 2001;Mellar et al., 2004;[START_REF] Kambouri | Research into the effectiveness of learndirect materials and support system for learners with skills for life needs[END_REF][START_REF] Kambouri | Playing the literacy game: a case study in adult education[END_REF]NIACE, 2005) has made us aware that the policy context for ICT use in education has a strong influence on the learners. They are often very aware of the curriculum push on ICT, of the government's e-learning agendas, the rhetoric of the knowledge society and claims that ICT is important for employment. This work has also shown that these very agendas sometimes mean that institutional resources are targeted at groups of learners working at Level 2 and above rather than at Skills for Life learners.

■ Adequate access to resources is something that Skills for Life learners sometimes struggle to obtain. ■ The relationship of ICT to employment is one of great significance for many adults, and is often part of the reason they are doing these courses.

Socio-cultural factors and issues of inequity

It has been argued that ICT changes the power relations within the classroom (for example, [START_REF] Scanlon | Activity Theory and Higher Education: evaluating learning technologies[END_REF]. While this is clearly an important factor, we have chosen to concentrate in this project principally on the changing role of the tutor rather than wider issues of changing power relationships in the classroom.

The nature of learning-teaching events

The context ICT is an important element of the context of learning and teaching events, though ICT is not a single technology, and it is important to look at the ways in which a wide range of digital technologies enter into teaching and learning. There is also an interaction between ICT and other elements of the context, for example issues arising from the physical location of technology (often restricting movement in the classroom) and the liberating effects of mobile technologies (at least in terms of space).

■ Use of mobile technologies for learning may have particular importance for adult learners.

Approaches to teaching

Using ICT in their teaching often causes tutors to reflect upon, and hence change, their pedagogy. However, there is no single best way of using technology in teaching, and a wide range of approaches are adopted. In attempting to put some structure on the discussion of this range of approaches [START_REF] Conole | Mapping pedagogy and tools for effective learning design[END_REF] suggest three axes along which to describe uses of technology in education: 'Information -Experience', 'Reflection -Non-reflection' (discussed later under the heading 'How to learn'), and 'Individual -Social' (discussed later under the heading 'Social interaction').

The dimension 'Information -Experience' highlights the use of ICT for repetitive practice at the one extreme, and at the other learning involving the construction of concepts through the construction of artefacts (see for example [START_REF] Papert | Constructionism[END_REF]. In our projects tutors rarely used ICT for repetitive practice (though this was sometimes done, particularly in the use of websites for the practice of literacy skills), but commonly took the construction of an artefact as the central part of their activity. Another way in which ICT was used was to allow the learners to experience a variety of roles (for example as news interviewer) ■ Ways of employing ICT in order to develop experiential learning that may be of importance to adult learners include: -the (often joint) construction of artefacts -enabling learners to act out new roles.

Much discussion of learning in recent years has centred on Lave and Wenger's concept of 'communities of practice' [START_REF] Wenger | Communities of practice: Learning, meaning, and identity[END_REF], and some work in adult and community learning has put the idea of learning communities centre stage (see [START_REF] Cook | Beyond formal learning: informal community e-learning[END_REF]. This idea was discussed at some length with the tutors in the project, but interestingly most of them did not integrate this concept into their views of how they wanted to change their teaching practice. One possible explanation of this was that the degree of learner movement in and out of class groups sometimes made a sense of community hard to achieve. One tutor, however, was already working with this idea before his involvement with the project and incorporated this thinking within his project (some of this work is reported in [START_REF] Harris | Beyond Communities of Practice in Adult Basic Education[END_REF].

Social interaction

Tutors did readily accept the value for learning of social interaction between learners in the classroom. Earlier work has shown the difficulties with children in developing successful collaborative activities around ICT [START_REF] Crook | Computers and the collaborative experience of learning[END_REF][START_REF] Hoyles | Interdependence and Autonomy: Aspects of Group Work with Computers[END_REF][START_REF] Hoyles | Group work with computers: An overview of findings[END_REF], and more recent work by [START_REF] Mercer | Words and Minds[END_REF] looking at modes of talking in learning contexts (including the use of ICT) has also concluded that careful preparation is necessary for this to work.

■ The encouragement of collaboration between learners is important, but this can be difficult to achieve in groups whose composition changes quickly, and where learners have different learning goals, as is commonly the case in adult learning contexts.

The creation of learning opportunities

The management of learning One of the vital elements for us in thinking about the management of the learning was the way that tutors thought about the role that ICT had in this learning. [START_REF] Ginsburg | Integrating Technology into Adult Learning[END_REF] identified four rationales for the use of ICT in adult literacy: technology as curriculum; technology as complement to instruction; technology as instructional tool; and technology as delivery mechanism. We did not set out to champion a particular approach, but we did see it as important that tutors were explicit about why they were using ICT.

Another strong element of our thinking about the management of learning related to the idea of learner autonomy and we strongly encouraged tutors to move from a view of the tutor as expert to a view of the tutor as facilitator and supporter of the development of learner autonomy (see [START_REF] Letertre | Multimedia pedagogical resource and learner autonomy[END_REF].

■ Tutors need to be clear about the role that they see ICT as having in the learning process. ■ Tutors should encourage learner autonomy -this is something of particular importance for adult learners, who need to take charge of their own learning outside the formal learning context.

Different types of outcomes Content

We were always concerned with two kinds of content, ESOL, literacy and numeracy on the one hand, and ICT skills on the other. We took as our pragmatic definition of this content the established Skills for Life Curricula for ESOL, literacy and numeracy, and the then emerging ICT Skill for Life Curriculum (QCA, 2005) and built our tests around these definitions. We will not here discuss the ESOL, literacy and numeracy curricula, but will offer some comments on the ICT Skills for Life Curriculum. The ICT Skills for Life standards appeared to embed a restrictive, rather technological, definition of ICT literacy, though the concept of 'purposeful use' that was indicated in those standards was very helpful, and became more explicit in the development of the curriculum. We also looked to a number of other definitions to help us to informally identify the development of a wider range of ICT skills. Two approaches which we found particularly useful were those of [START_REF] Carvin | More than Just Access[END_REF] and the International ICT Literacy Panel (2002). [START_REF] Carvin | More than Just Access[END_REF] distinguishes a number of elements of relevant new literacies, including:

■ Technological Literacy: the ability to utilise common ICT tools, including hardware, software, and internet tools like search engines. ■ Information Literacy: the skills to ascertain the veracity, reliability, bias, timeliness, and context of information. ■ Adaptive Literacy: the willingness to learn new tools and to apply previous ICT learning to new situations.

■ Tutors need to have clear learning objectives for language, literacy and numeracy, and for ICT. ■ ICT should be used for a purpose. ■ Tutors need to be aware of (and seek to develop) a wider range of important ICT skills going beyond simple 'technological literacy'.

How to learn

ICT has often been seen as allowing learners to reflect upon their own learning processes [START_REF] Wegerif | Literature Review in Thinking Skills, Technology and Learning[END_REF], and some of our tutors saw this as an important aspect of their work.

■ ICT can help learners to reflect upon their own learning processes, and so 'learn how to learn'. This may be an important element of learning for adult learners who have had little success in their previous learning experiences.

Language

The use of the language of ICT within the language classroom can be seen as an example of real language or, alternatively, as a distraction from more useful language. The use of this language clearly presented a challenge for literacy and ESOL tutors and they developed different ways for dealing with this challenge.

Digital technologies are changing the nature of literacies; in particular there is a changing relationship between graphic and textual elements. While this changing nature of literacies is not reflected in the literacy and ESOL tests that we were using it was nevertheless an element of changing practice that we wanted to observe.

■ The language and terminology of ICT may be a barrier to some learners, or it may offer useful opportunities for language learning. ■ ICT is changing the nature of literacy: new digital literacies involve a closer integration of graphical and textual elements.

Wider benefits of learning

For many learners, developing ICT skills plays a role in their everyday lives, whether it opens up work opportunities, enables them to help their children with their use of ICT, or communicate with distant relatives by email.

■ ICT often has a role in the wider life aims of learners outside both learning and work (for example, e-mailing for strengthening/maintaining social relationships).

Developing the interventions

We worked together with the tutors in developing approaches to the effective use of ICT, starting from the tutors' work in the classroom, incorporating the recommendations from the earlier Stage 1 study and reflecting on the resulting designs in terms of the research literature on teaching and learning in adult Skills for Life and on ICT and learning. Each approach was trialled and refined over three iterations. The next chapter describes each of the seven approaches that were taken forward into the second phase for trialling and testing.

Note on numeracy

During the year we trialled several approaches to using ICT within adult numeracy provision. Two approaches which showed great promise were the CyberLab and the Family Numeracy projects -these are briefly described below. Unfortunately, in both cases the tutors concerned were unable to take the work forward into the evaluation phase of the project because of other work commitments. At the time, we found it difficult to recruit additional numeracy tutors for this project; we have since carried out further studies on the use of ICT with adult numeracy [START_REF] Coben | The use of ICT in Adult Numeracy Teaching in Scotland[END_REF].

CyberLab

CyberLab was a course for literacy and numeracy embedded into topics including science, technology, history and culture in a framework of ICT-enabled activities. The intervention included a mixture of constructional, instructional and exploratory teaching and learning approaches. This included the use of Logo and robots when investigating algorithms, and when investigating fractals, learners used digital cameras to take pictures of real-life examples. For an account of this work see: [START_REF] Harris | Integrating ICT, Literacy and Numeracy: Cyberlab[END_REF], and also http://case/glam.ac.uk/ CASE/StaffPages/SteveHarris/SRHPubs/NRDC%20Presentation%20notes.pdf

Family numeracy

This course used website resources to allow parents to assess and practise numeracy skills. The workshop was tailored around the needs identified by the initial assessments. A tablet computer was used to enable parents to model how they would undertake calculations. Different strategies that parents could use with their children were discussed and practised using freely available online children's games and school software.

Case studies

In this chapter we describe the seven case studies. The studies are named by the type of technological application that tutors selected to use for each intervention. Background for each of the seven cases is presented, including a description of learner characteristics, their tutors' background and the context in which they worked. The first five cases are ESOL classes based in Further Education colleges and the last two are literacy classes within community-based provision. The National Test and mindmap classes consisted mainly of older, retired, white, UK-born learners with English as a first language who were coming back to education for the first time after a long break. Seventy-one per cent of this group were female. Thirty per cent of the group regarded themselves as having some form of disability.

The other classes consisted mainly of younger immigrants, coming from a wide range of countries: almost none had English as a first language; most had come to the UK in the last five years, and had been attending similar courses for some time. About 30 per cent were in full-time education, 30 per cent with full-time family responsibilities and the remainder evenly split between employment and unemployment. Forty-eight per cent of this group were female. Only 3 per cent regarded themselves as having some form of disability.

Each approach is discussed here in terms of the components of the theoretical model described in Chapter 2.

WebQuests

WebQuests are 'an inquiry-oriented activity in which some or all of the information that learners interact with comes from resources on the internet' (Dodge, at www.webquests.com).

Classes in this case study took place in a North of England FE college in an area with high numbers of ESOL learners.

The aims of this intervention were to improve both the ESOL and ICT skills of Entry level 1 ESOL learners.

Factors affecting learning-teaching events

The two groups involved were Entry level 1 classes meeting between three and five times a week. Numbers varied but up to 15 were enrolled. Group 1 met four mornings a week, with an additional option class, while Group 2 met four afternoons a week plus the same option class. Each group had a personal tutor who met them three times a week. The project tutor was an experienced ESOL practitioner and an Information and Learning Technology (ILT) champion. The buddy she chose to work with was an ICT tutor who was less experienced in language teaching. During the trials the buddy taught a very similar scheme of work to that designed by the tutor, and had her close support.

The personal ESOL tutors (assigned by the college to each learner but not part of the project) and the project tutors worked very closely with each other so that the language focus and the ICT focus were always very well matched. The scheme of work for each class was developed by the personal tutor, and the language focus and skills for the ICT and option class, which were part of the study, followed on from this.

As it was unclear how much ICT experience the students would have, the first half-term was spent developing their ICT skills, in particular using the internet, basic word processing skills and copy/cut and paste. Tasks were set up in such a way as to build the learners' confidence.

The tutor's main objective for this study was to introduce computer skills by giving students a chance to explore and experiment with computers and the internet and to move away from demonstrations or lengthy paper-based instructions and formal guidance. More specifically, she was interested to see if knowledge of a procedure or skill in using one program could transfer to another program without much guidance. She therefore planned to incorporate different strategies for using the PCs and introduced collaboration through pair work and small group work on single PCs. She also split classes in half, so half were using the PCs while the others were doing paper-based activities (then swapped groups around) in order to encourage the learners to ask each other questions before calling on the teacher.

The tutor and buddy worked together on the difficult task of finding appropriate materials, suitable for Entry level 1 ESOL learners. They started from websites that were very imagefocused, such as sites featuring national flags, Google images and the college intranet. From there they moved on to asking the learners to find basic information, such as opening times and prices for local sports events and courses. WebQuests were adapted for Entry levels using the aclearn toolkit (www.aclearn.net) into a four-page simplified WebQuest: Home, Introduction, Task and Resources. The first two pages were presented and discussed on the SMART Board™ by the whole class, leaving only two pages in which students had to read and navigate.

Learners in these two classes were motivated by the need to improve (or, for over half of them, learn for the first time) their English language skills to find a job and/or to function in their new community, and so they were keen to improve these skills quickly. They were between 20-40 years old. First languages were Urdu (half of them), Kurdish, Arabic, Portuguese, and Chinese. Sometimes an interpreter was present.

The nature of learning-teaching events

It is clear from observations and tutors' reports that learners had a strong expectation that the tutor would directly teach them -as the style normally adopted by the other tutors in the college was either whole-class teaching or individualised teaching. The learners expected to ask the tutor for help rather than seeking a peer's help first.

Assessment needs dictated that learners produce separate outcomes, and this meant that much of the class time had to be managed as a whole group or on an individual basis. The learners found it difficult to get or receive help other than in their first language (these were E1 learners) and so peers were not always able to help (there was one support tutor).

Although the tutor and her buddy wished to allow learners more control of their own learning, and they set out to encourage collaborative learning through a change in classroom practice by using a range of strategies to group learners, this did not always happen as planned. There were some very successful examples of collaboration. However, the shortness of the tasks (less than 10 minutes) did not allow learners the opportunity to learn from others and to experiment. The person in control of the keyboard (and often the more experienced ICT user) tended to ask questions directly to the tutor rather than discuss issues with the group.

So although there was the intention to put students in groups -neither the task (for which they were allowed a fairly short time) nor the seating plan encouraged that. Most of the class was therefore conducted as a large group (35 per cent of time observed) or one-to-one (50 per cent).

The creation of learning opportunities

The WebQuests were an unexpected way of working for these learners, and the approach intrigued them -in particular the fact they were able to get on with the task without needing to understand and read everything on the screen.

As well as using the internet, the learners used both PowerPoint and Word from the start of the course. By becoming familiar with two programs the learners are able to transfer skills learned from one program to the other, building confidence and allowing them to explore different applications. A SMART Board™ was also used, but this was mainly controlled by the tutor and used to demonstrate tasks.

Different types of outcomes

With these Entry level 1 learners, the tutors were particularly focused on learners gaining confidence in speaking in front of the class and answering questions.

Tutors were satisfied with overall progress, particularly in learners' confidence levels, and the tests confirmed that for ICT skills and confidence strong gains were made. However, the ESOL test which was given to learners was completed by few students the second time (many of the original group from the pre-test had progressed to another class), so although the results show improvement they are not statistically significant. Moreover the test was measuring listening and comprehension skills rather than speaking skills -which are more difficult to develop -and so even a small increase on the test is an important achievement in this case.

e-Portfolios

In this course electronic-Portfolios (OPEUS)2 were used. This is a web-based technology that allows users to design websites in which they can store and display their work, thus creating electronic books or portfolios. It allowed quick display of learners' work (learners can design a web page and email someone with a link to their website) and get feedback by email. This intervention, based at an East London FE college with a high percentage of ESOL students, was intended to develop both learners' language skills (writing and communication) and ICT skills (web design, word processing, email, etc).

Factors affecting learning-teaching events

The learners were an Entry 2 literacy group of 16 ESOL learners aged 16-20, who attended on a full-time basis (36 participated in these two classes). Learners had access to computers outside class hours in the learning centre.

The tutor, an ESOL practitioner and ILT champion, had chosen as a buddy a colleague who was an experienced ESOL tutor but had not used much ICT in the classroom. This tutor was also the coordinating tutor in a year where Ofsted was due, so there was little time for preparation and discussion of lessons. In this case, the tutor took his role as mentor in the strict sense and passed on a highly structured scheme of work to his colleague, who followed it very closely.

The nature of learning-teaching events

Peer collaboration was invited by the tutor at the short (15-minute) lesson reviews at the end of each session which allowed learners to discuss and evidence their learning achievements on a weekly basis. Learners became used to discussing their achievements during the lesson in order to write it down on their record of work. (This approach had evolved from a previous practice of the tutor where the learners had produced a record of work, based essentially on his dictation of the lesson aims and objectives).

However, in looking back on the course the tutor felt that he might not have placed as much emphasis on collaborative working as he had intended: I wonder if it's not my teaching style this year that is overlooking collaboration. Perhaps some learners need more direction in order to work together. This makes me think that the overall task of producing individual electronic portfolios makes learners more selfcentred, rather than the opposite.

The creation of learning opportunities

Learner autonomy was fostered by encouraging sharing of computers, encouraging learners to observe or work with a peer on the same task before attempting it on their own. Tasks included learning to insert a table, create a page or insert images in OPEUS. Learners were sometimes encouraged to take on the role of instructor: whenever one learner completed the task earlier than the others, they would be invited by the teacher to help others. There was a tendency for these 'instructors' to be too directive or to do the task for the other student, so the benefits were partial and often only language-related. An example: Learner S's computer wasn't working and he had to share a computer with another learner. F. S., who is a very confident PC user, simply took over F's computer and did most of the task on his own. The only example of seeking support from his classmate was when S. was in doubt about language skills (sentence construction or spelling). Every time he needed help with ICT functions he asked the teacher rather than consulted with F.

Scaffolding3 could only happen when competence levels matched; otherwise peers did not benefit equally from interaction and collaboration failed.

Different types of outcomes

Designing websites was quite a demanding task, but the ICT used was fairly simple for this group who had good ICT skills.

The test results may not accurately reflect the learning of ICT skills in this class. The tutor, reflecting on the course, wrote: I remember having an end of term assessment activity in which I asked students to work in groups and create from scratch, within the time available, a website showing one piece of work from each student in their small groups. Their performance showed to me that they had learned to work together, learned rather sophisticated ICT skills and developed good language skills.

However the tutor felt that students' progress has been mixed. It was identified that students who started with good ICT skills and a mature attitude to studying benefited most from the approach used. Other students, who displayed good ICT skills in the beginning of the project but with a negative attitude to studying, did not seem to benefit much from what was being offered to them.

The test results reflect this complex reality by showing good progress (significant gains) in ESOL tests of listening and reading comprehension and a non-significant increase in ICT skills.

Tablets

This project used tablet PCs whose key advantages are portability and mobility, and incorporating the use of a pen and handwriting recognition. The case study focused on two ESOL classes in an East London FE college and aimed at creating language learning opportunities through the use of innovative ICTs, thus enabling learning at any place at any time.

Factors affecting learning-teaching events

In this intervention ESOL Entry level 3 learners shared tablets in two classes which met for a 2.5 hour session a week. There were 36 learners in total ranging from 20 to 60 years old. The two main first languages were Bengali and Somali.

The first tutor, an experienced ESOL practitioner and ILT champion, chose a buddy with lesser ICT skills to collaborate with, and in this case they taught the classes together, complementing each other's skills well.

The tutors' aims included investigating whether use of tablets could have a positive impact on language learning through extending the classroom to the real world, exploring how collaborative working affected language learning, and helping the process of learning ICT skills (by incorporating ICT in classes where this would otherwise not be possible). It also laid emphasis on collaborative working in groups with the learners sharing use of the tablets. The learners were also actively involved with the local community through participation in a neighbourhood project producing artefacts to be shared with peers. However, as with other cases, assessment requirements were individual (there were no plans for group assessment) and so printouts were used for individual exercises.

The nature of learning-teaching events Tutors described their approach as learner-centred, employing a variety of techniques and learning strategies:

1. Kinaestheticbringing the class to a gallery, working on the bus, moving around in the college. 2. Visualdrawing was encouraged as well as handwriting and planning on the tablet. 3. Auraluse of music, voice recordings and songs. 4. Action learninglearners had to choose tasks (e.g. exhibits in a gallery to focus on) and make decisions as well as making sense of the activities proposed. 5. Collaborative -learners had to share the tablets in groups of three or four although the tasks were not necessarily designed for those numbers.

Because most of the work was carried out using the tablets (which had to be shared), the majority of the work was group-based, with little individual working.

ICT skills were mainly taught through demonstration -although there was some evidence of exploration through trial and error of the tablet's handwriting facilities, and file management.

There was a demand from learners to take the tablets home, and this was allowed from time to time.

Learners were allowed to choose who to work with and they commonly grouped themselves with others with the same nationality or language and so used their first language when quick explanations were required.

Peers helped each other with ICT and literacy tasks -they took turns to use the pen but the more confident ICT user (not necessarily the most proficient in English) was often seen to take over and/or teach the other. Gender also played a role in the groups' dynamics with women sometimes being less active participants. Independence from the tutor in this case was manifested in exploration (trial and error) of tablet's handwriting facilities, or opening and closing files or browsing various files.

The creation of learning opportunities

During a visit to the Victoria & Albert Museum the learners interpreted their task in a variety of ways, and as they went around the exhibits some copied labels, others took notes or attempted to draw. Learners showed enjoyment while using the tablets but not all were fluent in their use. However, those who had grasped the technology preferred to use the tablet to paper when drawing or taking notes, because they could save their work in a variety of formats, and perhaps because it gave special status to the learners while visiting a gallery.

Different types of outcomes

Tutors were looking to bolster self-confidence and independence amongst their learners and to use the mobility of the technology to exploit 'the educational and learning opportunities that exist in abundance in this rich and diverse city….' There were significant gains in ICT skills and ICT confidence but not so in language skills although some learners felt their writing skills had improved. Tutors also expected gains in language and felt these may not have been captured by the listening skills tests carried out for the project.

m-learning

A group of ESOL learners in an FE college in the South East of England used XDA2s (handheld computers with mobile phone functionality and built-in cameras) and simpler camera phones.

They sent text, images and sounds wirelessly (as MMS -multimedia messages) from various locations (around the campus, on a local history trail around town, museums, etc) to a website where they were displayed. They would then work at PCs in the classroom to edit and expand their work on the website, working on both their ICT and language skills.

Factors affecting learning-teaching events

The tutor was an ILT champion and ESOL practitioner who chose to work with three buddies, ESOL practitioners who had not used m-learning before as the college felt this would be a unique opportunity for professional development. The team met weekly for mutual support. There were 45 learners in total in the four classes involved, who ranged from 18-50 years old, mostly male, and whose first languages were predominantly Dari (Afghan), Kurdish, Pashto and Somali.

The tutors' aim was to transfer and extend learners' existing knowledge of ICT-based communications (text messaging and/or email) to use the mobile devices to send text, images and sounds as multimedia messages or emails to a website to create an online presentation, either as a 'moblog' -a mobile photolog/weblog (in which messages are organised in a linear/chronological way) or to a mediaBoard (in which messages are attached to locations on a main image or map).

The nature of learning-teaching events

The creation of moblogs and mediaBoards with ESOL learners was intended to allow the learners to develop fluency and competency through meaningful goal-oriented interaction in the target language. The tutor expected that through finding out how to use the technology and then using it for project work the learners would reinforce and gain automaticity in target structures (e.g. tenses) as well as learning some new vocabulary.

Observations indicated that the acting out of the role of interviewer was also a powerful motivator for these learners.

Learners who worked together in writing up their information on the mediaBoard were seen to discuss grammatical forms, and remind each other of what they had learned in class.

Those who wrote up individual contributions put their thoughts into writing but then called the tutor over to check it.

During the second term tutors started to download the data from the PDAs in advance of the class for the learners to use. This was so that learners in other classes could then use the phone in between without the potential for data getting lost (i.e. accidentally deleted).

The creation of learning opportunities

Two learners, who were asked to compare these classes with other ICT provision, said this was very different because it helped them with their spoken English, in that they were required to talk more and work with others.

Most use of mobile phones/PDAs was in activities outside the classroom, rather than in the classroom, and classroom use was often to download data previously captured with the mobile phone/PDA to the computer.

Tutors reduced the amount of explicit instructions in the use of the technology, and concentrated on exploration, helping the learners to work out procedures for themselves.

The tutor wanted to enable her learners to be autonomous by taking initiatives in the college's life. She wished to encourage them to make greater use of the services on offer at the college, and so she sent out learners in pairs to interview the people running the various services and take digital photographs, which they shared through the mediaBoard.

Different types of outcomes

The aims were both to use ICT-based communication (understand and use key vocabulary and use a handheld computer or a mobile phone) as well as language skills such as use of sentence structures which are fit for purpose.

In the mediaBoard work the main writing the learners were doing was captioning photographs -a relatively novel form of literacy outcome for these classes, arising from the combination of pictures and writing in digital media.

The tutor felt that ICT gains had been made but that it was unrealistic to expect to be able to measure any improvement in their spoken language over such a short period. Yet ESOL test results actually showed significant gains in listening skills. The ICT skills gains were also significant and confidence in ICT increased but not significantly.

Digital video

This course was titled 'ESOL and Computing'. Its aim was to develop both ICT and language skills, and it was run in a North London FE college. The learners used a digital video camera to create films, and they were encouraged to take turns and assume different roles in filming the college's new building.

Factors affecting learning-teaching events

This was an optional course for second language learners (mostly Entry level 3) who came to this course to get an introduction to ICT skills. The group consisted of thirteen 20-50 year olds, half of whom were mothers looking after families and the others in part-time work. First languages varied, including Somali, Arabic, Chinese and French.

The tutor was an experienced ESOL practitioner who had good ICT skills but had not used digital video before. Her proposed buddy was unable to take part in the project due to staff changes.

As part of the preparation for use of the camcorder, learners accessed Video Nation (http://www.bbc.co.uk/videonation) -a collection of amateur short streamed videos on the BBC website which served as a source of examples and material to work on -and looked at issues such as following instructions on using a camcorder and how to direct the scene to be filmed.

Most of the learners in this class had very few ICT skills at the start of the course, and for them the main reason for joining the course was to learn about the internet, email and word processing. However, they were also offered a more ambitious set of skills in the use of digital video, filming and editing, which they were not always keen about. Some felt that it distracted them from learning more ICT skills and in one of the filming sessions two people said they did not feel they had learned anything new -whereas the tutor's plans were for consolidation and practice of language skills.

The nature of learning-teaching events

The ICT tasks were more demanding than those generally taught on similar courses, and there was a less formal focus on language skills although the tutor reinforced grammar and speaking.

Through working together learners were able to follow the instructions without having seen a demonstration of the use of the equipment, and this encouraged greater practice of language skills and enabled the tutor to take on a more facilitative role.

A very successful aspect of the interviews carried out by the learners for the videoing was that learners had to understand and respond to the answers of people that they interviewed outside the class, thus reinforcing their language skills.

Facilitating the task of rotating roles as camera person, director, assistant director, interviewer and interviewee was important but not always successful (probably because it needed more mediation). Learners also took turns in connecting the camera to the computer and playing back the clips.

Problems in collaboration sometimes arose when learners' skills (both language and ICT skills) were different and stronger learners were unwilling to help the weaker learners.

The creation of learning opportunities

Learners watched and commented on Video Nation covering a range of topics and reflecting on views and experiences from a wide cross-section of society, making it rich material for ESOL learners. The videos were useful for listening practice and generating oral work but less so for common ICT skills.

The tutor found that asking learners to work together away from PCs for the development of interview questions and the production of the videos, followed by individual writing up on the computers worked best with these learners.

Different types of outcomes

This was an ambitious project in terms of technological demands (sound recording, editing) and at times the need to achieve outcomes dominated other aspects of the teaching-learning situation especially language outcomes. The results of the tests reflected this as there was a slight decrease in ESOL listening skills but an increase in reading skills and a significant increase in confidence and in ICT skills.

Mindmaps

In this course the tutor used a mindmapping program called Inspirations (http://www. inspiration.com/productinfo/inspiration/index.cfm) to support the development of planning for writing skills and email as a tool for communication between learners but also as writing practice. The course took place in an adult education college in the South East of England.

The tutor aimed equally at achieving literacy as well as ICT skills learning goals.

Factors affecting learning-teaching events

The tutor was an ICT practitioner with some TEFL teaching experience abroad and ESOL, literacy, numeracy and ICT teaching experience in the UK. Due to staffing demands the chance to work with a colleague (buddy) did not occur.

The course was advertised as 'ICT and Literacy' and the aims were both to give learners confidence in using computers and to improve their skills in written expression. It ran for two terms (one two-hour session per week).

This was a small group of 10 learners, all over 50 years old, mostly retired, ranging in ability from Entry level 3 to Level 2 literacy with an interest in acquiring ICT skills. There were two ESOL students whose first language was Chinese, and two who reported having disabilities.

The software was used to help improve the planning of written work through organising thoughts and structuring work into separate paragraphs (these maps could be exported directly into Word in order to be used as a basis for writing).

The nature of learning-teaching events

At the start of the course there was little collaboration between learners, though the degree of interaction increased over time. Sometimes learners needed to share machines, and the tutor wished to encourage peer interaction and more collaborative working.

The teaching style was principally didactic, although the tutor's role became more facilitative as she became more confident in her teaching. Learners were mostly encouraged to work through the exercises on their terminals and papers individually but the tutor's objective ultimately was to introduce peer interaction to improve overall class dynamics and to help her better manage the classroom demands. The interactions with the tutor usually consisted of extending the work or proposing another way of doing things.

The creation of learning opportunities

The tutor was making changes in her practice to encourage a degree of exploratory and independent work. Taking a step back and adopting a facilitating role, the tutor had more time to observe learners and help them more effectively by finding different ways to support language development. For instance, in teaching learners how to use email with attachments, the tutor put them into groups and had the groups email one another, thus both encouraging autonomy and reducing the time demands on the tutor.

The choice of task (learning organisational and writing skills through mindmapping) enabled differentiation.

Learners felt pairing up had helped them with language skills (particularly as there was a mixture of first and second language speakers) as well as boosting their confidence with the computer. The production of mindmaps was discussed in groups but executed mostly individually (each person making their own map and writing a piece at the end).

Different types of outcomes

The tutor was particularly concerned to develop learners' skills in thinking about the logical structure of their text, and also to encourage them to take more responsibility for their own learning.

Retention improved and one learner reported he did 'not want to miss a day!' Overall there seemed to be an improved atmosphere in the class and enhanced communication since learners worked together and interacted more. There was no handwriting involved (although not discouraged) but the tutor felt typing text facilitated correction of their text, enhanced fluency and provided a professional aspect, which brought a sense of achievement (as well as being easier to correct). She felt students' writing had improved.

Although writing was not assessed formally pre-and post-intervention (as hoped by the tutor), she carried out the college's reading assessment and also the literacy reading comprehension (GO) tests which showed positive gains in literacy; tests also showed significant gains in ICT skills.

National Test

This case study examined community-based provision which used computers as a hook to attract people into learning by encouraging residents who lived on an estate to join a basic ICT class with embedded basic literacy skills. The tutor's final objective was to support learners to take the National Test in literacy.

Factors affecting learning-teaching events

This intervention focused on two evening classes (11 participants) that took place once a week in a community centre hosting a UK online centre in the South East of England accommodating up to seven learners, with regular attendance of three to five. These were mainly older, retired, white, UK-born women with English as a first language who were coming back to education after a long time and had little experience of using a computer. Some of the learners had difficulty retaining new information between classes (two because of old age and one with special needs). Most were unable to continue with the work at home as they either lacked PCs or were not confident enough to use them.

Some of the learners remained nervous about using technology to the end of the course but they did become more positive about it. Most of them said that they disliked the tests, though the fact that the ICT Skills test was online was a definite attraction.

This course suffered from a number of physical and organisational barriers: the venue was remote; it was often locked when the tutor arrived and she sometimes had difficulty getting access; and there was no technical support at hand. The course used online Skills for Life resources such as the BBC Skillswise website alongside paper-based materials.

The tutor had considerable experience in administration in the commercial sector before moving into counselling, and then starting to teach basic skills and ICT about three years before the start of the project. Through the development phase there was a marked improvement in her class and resources management and her confidence as tutor grew. She chose to share her scheme of work with a colleague who was an ICT tutor experienced in working within the community setting, and they both benefited from the buddy system. During the two terms, they met once a week to discuss progress and make changes to lesson plans, and exchanged emails during the interim periods and often called the day after the lesson, particularly when a significant incident had occurred. The buddy followed the scheme of work as closely as possible, but needed to adapt the timings to take into account her learners' needs.

The nature of learning-teaching events

The teaching style adopted was mainly individual, one person per PC 'workshop style', with the tutor explaining tasks and instructing.

In this very small class there was some interaction between learners, as they asked one another for assistance but no planning for collaborative working.

The requirements of assessment and planning necessitated printed handouts to be used for individual exercises to be carried out or ILPs to be drawn up.

The tutor felt herself being encouraged to adopt more facilitative styles of teaching, but found direct teaching both more suitable to her own style and to what she perceived as the needs of the learners: 'I have to say that I find "discovery learning" very difficult to deliver. I want to save them time -by demonstration, with a handout to back up the learning point and then let them repeat it again and again.'

However, the tutor felt that using ICT helped learners work at their own level, and noted that positive feedback from the National Test in Literacy had motivated some to continue working on their own both at home and in class.

The creation of learning opportunities

Taking the National Test in Literacy boosted learners' confidence in using ICT and their desire to continue learning. Increased participation was observed towards the end of the course with learners gaining enough confidence to ask for more online work at their own pace (in the centre or at home for those who had computers).

Different types of outcomes

Several learners reported having started from zero in terms of ICT skills and making progress -visible to them and to the tutor. The learners also found the online grammar exercises useful. These results were confirmed through the tests and we found gains both in ICT skills and confidence and also in literacy although the numbers were too small for these to be statistically significant.

Classroom practices

In the last chapter we described each of the case studies separately in order to give a clear picture of the specifics of each scheme of work. This chapter first provides an account of the results of the quantitative element of the classroom observations, and then identifies some themes arising from the qualitative elements of the classroom observations. By pooling and comparing this data across the schemes of work it is possible to discern patterns that might otherwise not have been clear, and to provide supporting evidence for assertions that could not be justified on the basis of a single scheme of work because of the small number of observations.

Class organisation

Table 5.1 shows the percentage of the observed time in which the class was organised individually, in small groups or as a whole class. This table shows that while the learner working alone (the 'individual' category) predominates in most classes, there is significant use of small group and whole class work. The high levels of use of small group work in the tablets and m-learning groups are partly a result of learners sharing machines, and the lack of observed small group work in the National Test group is probably a result of the small numbers of students with disparate needs. The final row of the table shows the figures for similar observations from our Stage 1 observational study (Mellar et al., 2004), and this shows that there is much greater use of small group and whole class work than we had seen in that study, which is in accordance with our expectations as we had given considerable emphasis in our work with teachers to the idea of collaborative learning.

Teaching activities

Observed teaching activities were classified as:

■ Managing classroom activities -taking on a more supervisory and less didactic role, managing small group or individual activities, rather than direct teaching. ■ Questioning -dialogue between tutor and student(s) consisting mainly of tutor asking questions and the students answering. ■ Extending -building on/adding to material previously discussed or adding to and developing a comment by a learner. ■ Listening -actively listening to learners' comments or questions directed to the tutor. ■ Discussing -conversation between tutor and students not classified in other more specific categories. ■ Explaining -telling the learner 'how to' do something. ■ Instructing -telling learners what to do, what task they should undertake (i.e. organisational issues). ■ Repeating and reminding -repeating previously covered materials, or reminding students of material previously covered. ■ Modelling -showing the learner how to do something using the actual technology or its representation on a SMART Board™. ■ Social conversation -conversation between the tutor and student dealing with the student's social life but for a pedagogical purpose. ■ Presenting -using some form of media e.g. whiteboard, SMART Board™, etc. to present a topic. ■ Organising another activity -carrying out an activity not related to the current task.

Table 5.2 shows the distribution of time devoted to each of the observed teaching activities while the teacher was present for each of the schemes of work and the average for the whole group. The table shows that a large part of the time was devoted to 'managing activities'that is the tutor was acting as a guide, supporting activities that the learners were involved in. This is a significant change from observed behaviour in our Stage 1 study (though we have used a different classification of categories, and so cannot make a direct comparison), and in accordance with expectations, as this was the role we had tried to encourage tutors to adopt more frequently.

Use of media

Table 5.3 shows the use of media both by tutors and by learners. The percentages are of the total class time. The upper part of the table enables comparisons between schemes of work, whereas the lower part of the table enables us to compare teacher and learner use of media. Most learners spent the great majority of their time using either desktop PCs, with the tablets group and the m-learning groups also using tablets and mobile phones respectively. A relatively small amount of time was devoted to paper-based activities. The use by tutors was more varied, some using a projector, and some the whiteboard a great deal, others not. It is interesting to note that there is relatively little relationship between the media used by the tutors and those used by the learners: high levels of usage by tutors are not reflected in high levels of usage by learners, nor are high levels of tutor use reflected in low levels of learner use (which might happen if the tutor was dominating the use of the technology).

Classroom observations

An analysis of the qualitative elements of the classroom observational data highlighted three areas: autonomous learning, collaborative learning and differentiation.

Autonomous learning was one of the main pedagogic goals of the tutors. There is evidence in the classroom observations that this goal was often met through using ICT to develop appropriate tasks that could be used by learners of varying skill levels and ability to learn.

Examples of this are:

■ Learners were intrigued by the use of WebQuests, particularly by the fact that they were able to get on with the task without needing to understand everything on the screen, and so were not held back by their limited ability to read, nor did they have to call on the tutor all the time. One tutor opted to provide no support at all for the learners in learning how to navigate around online shopping sites, and the learners were still able to complete the task successfully. ■ In the area of ICT skills, particularly, learners were often seen to be making clear progress and improvement in skills with relatively little input from the tutors. ■ In the mindmap class, the tutor set out to allow more space for exploratory and independent work than she had previously done. Autonomous learning was made possible by the appropriate choice of task, including the organisational skills taught through mindmapping. However, it is clear that some learners still expected to be taught on a one-to-one basis, and were unwilling to work collaboratively with others, or to work independently with ICT. We therefore find a lot of one-to-one instruction in these classrooms. Often users are working independently on the PC and the tutor comes around to help, and the tutor then interacts with the learner to extend the work carried out or propose another way of approaching it, though they try to maintain a facilitating rather than a demonstrating role.

Most of the groups began with little in the way of collaborative learning, though in some contexts (for example, digital video) learners worked with one another even though no specific collaborative tasks had been set by the tutor. Many of the tutors sought to incorporate a greater element of collaborative learning into their teaching over time, but this was not always successful, and a number of particular issues with collaborative learning were observed:

■ Sometimes the task the tutor set (e.g. the construction of e-Portfolios) was actually pushing the learners towards a more individualised approach. ■ Discussion in the groups was often positive and pairing up helped with language skills and could boost confidence with the use of the computer. However, some learners clearly did not trust their partner enough to forgo the teacher's intervention. ■ Collaboration often happened because learners had to share a machine, for example learners had to share the use of the tablets, taking it in turns to use the pen. Some learners seemed happy to watch while others needed to be in control of the technology, and this seems usually to have been left by the tutors to the learners to negotiate. When collaborative work was forced by the need to share technology it was not as successful as when tutors developed tasks that required peer interaction. ■ Commonly the more confident ICT user (not necessarily the most proficient in English) took over and/or taught the other learners. In one observation of learners collaborating to produce a mindmap, the younger one (who was more comfortable with the PC) was in control, but the other seemed happy to observe and they both managed to produce individual maps. However the test results show that the learner who was in control of the technology made somewhat greater gains in ICT confidence than the other. This situation of the more confident ICT user taking over from the less confident user in group work was seen frequently across most of the groups. Occasionally a tutor would intervene to ensure that this did not happen.

The construction of an artefact -often jointly -was often a useful focus, generating motivation, collaboration and purposeful action. In certain contexts role-play also played an important part in these activities. This was particularly the case where learners presented themselves as journalists, photojournalists, or TV interviewers, though there were other circumstances where a change of role was evident, for example the use of tablets in the museum context enabled learners to project themselves as 'experts'.

The classroom observations highlight the advantages that ICT has to allow for differentiation. This is particularly the case where ICT is used to construct an artefact. Some learners preferred to work either more quickly or more slowly than their classmates on tasks, and the organisation of technology-based activities could often make this possible. When learners completed tasks ahead of others, it was also often relatively easy for the tutor to generate extension activities using ICT.

Classes used a range of hardware and software, and this variety was important in maintaining interest and developing skills. The use of mobile technologies (tablets and mobile phones) enabled greater flexibility, and learners in both of these schemes of work spent significant amounts of time on museum visits, or other activities outside the classroom.

6 Learners' progress

ICT background

Learners were asked both before and after the trials about their experience and use of ICT and about their confidence in using ICT for a range of tasks. Table 6.1 shows the numbers of those who had access to a computer at home at the start of the study. This is higher within the National Test and mindmap groups and lower for the other groups. Most of these figures are lower than the national average -ownership of home computers was 58 per cent of all households in 20024 . However, this is not necessarily a reflection of the fact that these learners were Skills for Life learners, as the analysis we carried out for the ICT Skill for Life -Action Research Project (NIACE 2005) showed 70 per cent of those learners in Skills for Life courses had access to a computer at home. Table 6.2 shows the distribution by age. More younger people than older reported having a computer at home, and more women than men, though for the youngest age group the distribution between the genders is roughly equal. Those who had a computer at home were also asked whether they actually used it, and a number of women over 40 in particular reported having a computer but not using it. We had pre-and post-study data about access and use for 90 learners, and this showed (see Table 6.3) a marked rise (12 per cent) in the number of learners reporting access to a computer at home, although two reported they no longer had a computer. Of the 15 learners who at the start of the study reported they had a computer at home but did not use it, 9 (60 per cent) reported that they had started to do so. The overall picture then is one of increased levels of ownership and use between pre-and post-trial surveys. This in itself is an important indicator of the impact of the trials on the learners' lives.

Tests

Introduction

The overall changes on each test, for all the groups combined, are quite large -the effect size5 for the ICT-skills assessment is 0.84, for the ICT-confidence questionnaire 0.58, for the reading test (GO) 0.52, and for the listening test (ESOL) 0.356 .

The scores on the tests in Tables 6.5-6.9 below are all given as percentages for ease of comparison, and statistically significant results (p < 0.05) are marked with an asterisk (*). Very small probabilities are expressed in e format, so 7.6e -6 is 0.0000076. Some of the sample sizes for individual schemes of work are very small and it would be unlikely that one would find statistical significance in these cases.

The test results probably understate the learning gains made as the tests were general, rather than specific tests of what was being taught in the class. Many of the tutors said that they thought their learners had made greater progress in language than the tests were likely to show because the tests did not relate closely to the learning objectives of the course. It was necessary to adopt these tests, however, for comparison with the other Effective Practice Studies and in order to create sample sizes that were large enough to allow statistically significant results to be found. The ICT-skills assessment and ICT-confidence questionnaire were more closely aligned with the skills taught in the classes, which meant that the gains on the tests were greater and hence correlations between the ICT test data and other measures were often clearer than for the literacy/ESOL tests, as we will see in Chapter 7.

ICT-skills assessment

Table 6.5 shows that all groups improved in ICT skills: significant changes were noted on the digital video, m-learning, tablets, WebQuests and mindmap schemes of work. All groups improved in ICT skills. Significant changes were noted on the digital video, mlearning, tablets, WebQuests and mindmap schemes of work. The change in ICT skills on the National Test scheme of work was not found to be significant, despite the notable differences between the pre-and post-means (probably a result of the small sample size). In the prestudy ICT skills test learners in the e-Portfolio scheme of work have the highest scores and learners in the mindmaps scheme of work have the lowest scores -this was probably related to the fact that the learners in the e-Portfolio group were considerably younger than those in the mindmaps group.

There appears to be a somewhat uniform level of ICT skills levels across the schemes of work after the study. It is possible that this could be due to a test ceiling effect, but no learner scored full marks and so it may rather be a reflection of the level of ICT literacy learners were taught to.

In the case of the e-Portfolio scheme of work the lack of significant measured overall improvement in ICT skills may be because learners in this scheme of work were taught ICT skills (e.g. web page authoring) which were not directly measured by the test. Indeed Table 5.4 shows that this group spent a lot of time using FrontPage (a web-authoring tool) and relatively little using other forms of software.

ICT-confidence questionnaire

Table 6.6 shows that although some learners expressed negative changes in confidence, there was an overall positive change in confidence within each scheme of work. Although some learners expressed negative changes in confidence, there was an overall positive change in confidence within each scheme of work. One learner expressed no overall change in confidence, 18 (23 per cent) decreased and 59 (76 per cent) increased in overall mean confidence. Learners who took the mindmaps scheme of work showed the greatest overall change, while learners in the e-Portfolio scheme of work demonstrated the least change in confidence. However, as noted earlier, learners in the e-Portfolio scheme of work were younger and had higher levels of confidence at the start of the course. So, each scheme of work was having a positive impact on most learners' confidence in using ICT.

Reading (GO test)

Five of the seven schemes of work used the 'GO' reading assessment. Table 6.7 shows that there was an improvement in the mean reading level within each scheme of work, and although these were relatively large, they were only statistically significant for the e-Portfolio and m-learning schemes of work. Five of the seven schemes of work used the GO reading assessment. There was an improvement in the mean reading level within each scheme of work, and although these were relatively large, they were only statistically significant for the e-Portfolio and m-learning ■ Digital video -successful in meeting its ICT goals, large effect sizes, literacy gains not significant but very positive and ESOL losses. ■ e-Portfolio -somewhat weak on ICT skills but very effective in delivering literacy and ESOL goals. There is a confounding factor here in that the ICT skills concentrated on in this course (HTML authoring) did not appear in the test. ■ Mindmaps -strong impact on ICT, and strong positive impact on literacy though not statistically significant. ■ m-learning -strong impact on ICT skills and on the literacy and ESOL scores. ■ National Test -big improvement in ICT scores, though not statistically significant, but little impact on literacy. ■ Tablets -strong improvements in ICT scores, but little impact on ESOL. ■ WebQuests -strong impact on ICT skills, and possibly on ESOL scores, but this is not statistically significant.

ICT

Additional indicators

Motivation

There are many indicators within the observational and interview data of the motivational impact of using ICT in the classroom. On their own these observations may mean little, but seen together and against the backdrop of the quantitative data presented above, they strengthen the argument that ICT had a strong motivational impact on these learners.

On a number of occasions learners said that they found the ICT aspect of their work intriguing and exciting, that they were highly motivated to come to class, and even to buy a PC to continue learning at home. In a discussion after a visit to a museum using the tablets, one of the learners said firmly that this was the class he learned how to write best and that he had not expected this given the 'playful' nature ICT brought to it.

In the community-based class (National Test scheme of work) several learners reported having started from zero and made good progress, even though they were using a PC for the first time. Two of them had received very positive feedback through the tests and this motivated them to continue on their own (either at home or in class). Tutors often reported positive outcomes in terms of motivation in the classroom following change in their pedagogic style and this was confirmed by our observations. However, in a number of courses younger learners who were very confident at using the internet to play games, video and music struggled to follow the taught programme. In one class, the tutor suggested that learners who had started with good ICT skills and a mature attitude to studying benefited most from the approach used, whereas those who displayed good ICT skills in the beginning of the project but had a negative attitude to studying did not benefit.

Learning experience

A small number of learners ( 19) who had been specifically observed during the classroom observations were briefly interviewed at the end of the trials.

Learners were positive about learning ICT, ESOL, and/or literacy, with just one learner being unhappy about his ICT learning experience.

Of the learners who responded to a question comparing the course with other courses they had taken, 11 commented that they thought the course better in some way to previous experiences. One learner who had been on the course using tablets commented that they thought another ICT course they were taking at the time was better. Half of the responses commented specifically on finding the teaching style better; the reasons given were centred around learning being at their own pace and the small, friendly, open structure. In general, learners' comments were positive about the teaching and about the use of ICT in their learning.

Most learners reported the course had had some impact on their lives: the changes mentioned principally related to changing their confidence or knowledge, influencing their work, and improved social interactions (mainly using email to contact friends and relatives).

Teaching and learning

This chapter explores the relationships between some elements of the student background, classroom practices and the test results, through an examination of correlations. The first section explores the relationship between the classroom observations and the performance measures, the second examines the relationship between the test scores themselves, and the third section examines the relationship between the test scores and age, class size, and dropout and attendance.

Relationship between classroom observations and test scores

This section explores the relationship between the classroom observation variables and the performance measures. For each class, the mean length of time for each observation category over the three or four observations for that class was determined, and the relationship between these and the change in performance on the literacy (GO) listening (ESOL), ICT-skills and the ICT-confidence tests was explored.

Use of technology

A notable significant positive correlation was found (see Table 7.1) between the total amount of time students spent using technology and improvement in the ICT-skills score (n=88, r=0.433, p=2.5e-05). There was also a corresponding positive correlation with change in the ICT-confidence score (n=78, r=0.359, p=0.001). So, unsurprisingly, increased ICT performance and confidence were related to the amount of time students spent using the technology within the classroom. More specifically, student use of PowerPoint and word processing were found to be positively correlated with gains in the ICT-skills score, though of these only the use of word processing was also correlated with increase in the ICT-confidence score.

As for tutors, it is interesting to note that tutor use of PowerPoint correlated negatively with learners' change in ICT-skills, but use of a whiteboard correlated with improvements in learners' scores on both ICT-skills and ICT-confidence. ICT long LIVE 17/1/07 17:44 Page 51

Class organisation

The time spent working as a whole class correlated significantly with the change in the ESOL listening scores (n = 67, r = 0.281, p = 0.021). Time spent working in small groups (2-5) was negatively correlated with the change in the ICT-skills score (n=88, r= -0.216, p=0.043), and the amount of time that learners spent working individually with the technology was significantly correlated with gains in r=0.344,p=0.001). This indicates that those classes where more time was spent working as individuals had better gains on the ICTskills assessment, while those classes spending more time working in small groups did less well.

Teaching activities

A specific teaching practice that had been encouraged in the development phase was for tutors to take on a more supervisory and less didactic role, managing small group or individual activities rather than direct teaching -this was coded as managing activities.

Whilst there were no relationships found between any of the teaching practices and change in performance in literacy and ESOL, there was a significant correlation noted between the amount of time spent by tutors 'managing activities' with gains made in both ICT-skills (n=88, r=0.359, p=0.001) and r=0.255,p=0.024). The correlation between 'managing activities' and learners' gains in ICT-skills is still significant when the total amount of time learners spent using the technology is controlled for (n=67, r=0.366, p=-0.002).

Extending was defined as the tutor building on/adding to material previously discussed or adding to and developing a comment by a learner. There was a strong statistically significant correlation between this teaching activity and changes in both ICT-skills (n=84, r = 0.539, p=<0.0005) and ICT-confidence scores (n=76, r=0.475, p=<0.0005). There was a strong positive correlation between tutors' use of the whiteboard and the use of extending (n=14, r=0.739, p=0.015), and as noted earlier there is a correlation between tutor's use of a whiteboard and gains in ICT-skills (n=71, r=0.582, p=1.02e-07) and ICT-confidence scores (n=61 r=0.578, p=1.09e-06). It is possible therefore that we are seeing here a tutor behaviour in which they are using the whiteboard to support them in extending an activity, and it is this which is having a beneficial impact on ICT skills and confidence.

The following teaching activities had significant positive correlations with improvements in the ICT-skills scores, though the sizes of the correlations were not large (around 0.3):

Discussing: Tutor engaged in two-way dialogue with learners about a topic.

Instructing: Tutor telling learners what to do, what task they should undertake (i.e. organisational issues).

Listening: Tutor actively listening to learners' comments or questions directed to the tutor.

Modelling: Tutor showing the learner how to do something using the actual technology or its representation on a SMART Board™.

On the negative side, explaining (which we defined as the tutor telling the learner how to do something) was found to be significantly negatively correlated with improvements in ICT-skills (n=88, r=-0.397, p=1.26e-04) and ICT-confidence scores (n=78, r=-0.281, p=0.013).

Summary

The classroom observations provide the following picture of activities which were positively correlated with changes in ICT skills and confidence:

■ greater use of technology by learners ■ learners working individually ■ tutor using the whiteboard ■ tutor managing the classroom activities, extending, discussing, instructing, listening and modelling.

The following were negatively correlated:

■ tutor use of PowerPoint ■ tutor 'explaining'.

The 'explaining' category refers specifically to the tutor telling the learner 'how to do something'. Where the tutor, for example, was demonstrating what was to be done ('modelling') this was much more successful.

Relationships between test scores

Table 7.2 shows the relationship between the test scores before the trials start. The ESOL and GO scores are quite strongly correlated to one another as are the ICT-skills and ICTconfidence scores, probably indicating that there is some degree of overlap in each case in what is being measured. So the tests measuring improvements in language are correlated and the tests measuring improvements in ICT are correlated. This expected result is further evidence of the construct validity of the measures. The ESOL and ICT-skills scores are also positively correlated -those with higher ESOL scores had better ICT skills.

The ICT-confidence score is positively correlated with the ESOL score -those with better ESOL scores tended to be more confident in ICT use, but negatively correlated with the GO reading score -the higher the reading score the lower the level of ICT skill. However, neither of these relationships survives as statistically significant when age and class size are controlled for.

Table 7.3 shows the relationship between the initial scores and the changes in scores; we are looking to see whether any of the initial test scores 'predicts' changes in the scores. For all four tests there is a negative correlation between the initial level on the test and the gains on the test -those who scored low in the initial test improved more than those who initially scored highly. The initial GO score correlates positively with changes in both ICT-skills and ICT-confidence scores, indicating that those with better reading skills improved more than those with lower reading skills. Initial ESOL skills correlate negatively with changes in ICT-skills, but not with changes in ICT-confidence scores; however, these relationships between ICT and language scores do not survive controlling for age and class size.

No relationship was found between initial scores on ICT-skills and ICT-confidence and change in the reading (GO) and listening (ESOL) tests. However, when age and class size are controlled for we find significant negative correlation between initial ICT-skills and change in ESOL (r=-0.445, p=0.029) and initial ICT-confidence and change in p=0.015). This suggests that good initial ICT skills and confidence may be associated with smaller language gains.

Table 7.4 looks at the relationship between changes in the scores. There was no correlation found between change in ICT-skills and ICT-confidence scores and changes in reading (GO) and listening (ESOL) scores. This suggests that the two areas of skills are being learned independently; the acquisition of one is not affecting the acquisition of the other. However, as we saw earlier there are some suggestions in the data that initial language skills may impact on ICT learning, and initial ICT skills on language learning, and this is an area worth further investigation. The relation between change in ICT-confidence scores and change in performance on ICTskills is shown in Figure 7.1. This illustrates the clear relationship, but with quite a degree of variability.

-10.00 Initial scores on ICT-skills levels were related to the age of the participants and there was a significant negative correlation between age and ICT skills for both male (n=69, r spearman = -0.556, p=1.00e-06) and female learners (n=77, r spearman = -0.365, p=0.001). Younger learners had higher ICT skills than older learners.

Correlations between age and change in test scores are shown in Table 7.5. There is a negative correlation for ESOL (older learners made least progress), and positive for ICT-skills and ICT-confidence (older learners made most progress). There was a significant positive correlation for males between change in ICT-confidence and age (r spearman = 0.475, p = 0.003), though the relationship for females was not significant (r spearman = 0.041, p = 0.804). It may be that younger males at the start of the courses overestimated their ability and therefore had greater confidence in undertaking these ICT tasks, but at the end they had a more realistic view of their skills. A significant negative correlation was found between class size and overall improvement in performance ICT skills and change in ICT confidence (see Table 7.6), indicating that the larger the class size the less the overall improvement. However, class size was itself related to age (r spearman = -0.528, p=1e-006*, n=168): the older learners were in smaller classes, and it is not clear whether the relationship between class size and ICT is a reflection of the age of individuals in the class.

Test scores and drop-out and attendance

Learners who had left a course because they had moved up or completed a course were counted as continuing learners; all others who left a course for whatever reason were classified as withdrawn. Learners who had withdrawn were compared with those who continued but there were no common demographic factors that related to learners withdrawing from the classes.

The pre-course ICT-confidence scores of learners who had withdrawn from the course were compared to those of learners who completed the course, and it was found that learners who had withdrawn had lower scores than those who had continued. However learners' scores on the ICT-skills assessment at the start of the course were not found to be significantly correlated with whether they withdrew from the course.

The only statistically significant relationship between attendance and the test results was with the ICT-confidence initial score (n = 116, r = 0.210, p = 0.024) -indicating that those with confidence in their ICT skills were more likely to attend regularly.

In summary, learners with lower levels of ICT confidence were likely to attend less frequently, and were more likely eventually to drop out of the course.

8 Tutor development

A model

We developed a model of support and development for participating tutors based upon approaches drawn from action research. Tutors subscribed to this model through a common understanding that they would participate in an experimental, interventional and developmental project where self-development was a core component.

Tutors were initially familiar with the basic skills curriculum as well as being skilled at ICT. In the first (development) phase nine tutors participated: eight tutors were qualified basic skills practitioners, experienced in mapping competencies to the National Standard; the ninth tutor was given additional support in this area. The development officers worked with the tutors to identify appropriate pedagogical approaches for individual contexts, and were able to help focus teaching aims and learning objectives.

The tutor support and development model consisted in the first phase of three cycles (each of one term) through a process of reflection on practice followed by action and change, and then another period of reflection, leading in the second (evaluation) phase to the development of a scheme of work for transmission to a colleague, followed by reflection on this work. During Phase 1 we had monthly team meetings, in which we discussed pedagogic issues such as ways of structuring the classroom, grouping learners, constructing tasks for a purpose etc., and the tutors kept weekly research diaries, describing their teaching, reflecting on the processes involved and feeding this back to the group. Tutors were also encouraged to spend more time observing students learning (so that they were more able to promote independent learning). The development officers and researchers visited classes and observed and discussed progress with the tutors. In phase 2 the tutors recruited a 'buddy' to repeat the teaching approaches they had designed, and the diaries were now focused around 'critical incidents' or significant events, which were reported monthly.

Reflection on teaching practice

Diaries demonstrated that tutors were making decisions about their teaching in response to their learners and the technology they were using.

The warm-up task was too long and complicated. I think it was too many instructions for this task to be followed by more instructions for the actual WebQuest task. I repeated this activity with the afternoon entry 1 group but introduced it differently. The activity ran much smoother.

The two female students both thanked me profusely. One said it was really interesting looking at the language on the net ('real' language) from the point of view of how it was put together rather than just trying to understand what it means (so she appreciated this sort of inductive approach to grammar, I guess). The other lady said she'd really enjoyed working together on the computers more than she normally likes working on the computer (actually at the beginning of the lesson she had requested not to go and work on the computers today!) They both said they'd learnt a lot. Did it help that we'd discussed how they learn beforehand, I wonder?

Consideration of pedagogical approaches were also apparent in the diaries:

Once again, students worked in pairs to select/delete the appropriate item in order to write their evaluation. I'm aware that this was a highly controlled task, but this was done with the less confident students in mind. I told those who wished to write their own text [they] could do so. Or they should feel free to write/modify/add to the suggested text.

The following extract documents the thinking process that informed the lesson taught and the many decisions that were made in its delivery, including the managing of support staff:

Started with a quick warmer, in groups they wrote the alphabet and had to think of as many words for shops and clothes for each letter in five minutes. This allowed for two latecomers to join the class.

We talked about the task (researching the internet, writing a PP [PowerPoint], coming to front to present to class), the class last week and the problem with the internet and I explained there had been a problem saving their PPs so they needed to start again. Once in groups they went onto the internet, every group found the WebQuest on the intranet with no support or even a review from the SMART Board or orally. I directed one group to the links page on the WebQuest but both other groups found it independently. The bilingual support gave some single step instructions for group three for an extended period and I asked her to move away as I felt that the initial question they had asked had been answered and they didn't need her anymore.

Signs of development

The diaries showed that important changes in pedagogy were taking place. Tutors were moving away from a didactic style to enable learners to be more independent, and were also experimenting with different groupings of students to promote collaborative learning. Tutors were often taking considerable risks with this change of role and they needed the support of the development officers, their organisations and the monthly workshop sessions where they could present successes and challenges. The development seen was linked to an increased understanding of pedagogy, increased knowledge of the learners and a growing understanding of the potential of the chosen technology.

The development of learner autonomy was a common theme. Towards the beginning of the first phase a tutor noted about her learners:

At the moment I feel that they are rather 'spoon-fed' by me and I realise that it may be necessary for me to take a step back and allow their own learning to develop by whatever method they individually come up with. I think that I need to be more assertive in my approach.

As the project went on tutors demanded more independent learning:

There will be no traditional presentation of the language they will need for the project. Learners will be challenged to understand authentic language and produce correct language by reflecting on it, discussing it together, and asking their teacher for an explanation if that is their preferred strategy.

My role will be as a facilitator offering guidance. Technology will be used as an instructional tool for finding and presenting the information.

I want to develop/promote learner independence by taking an approach that instigates learners to find out by themselves what computers can do for them.

Another theme was the development of collaboration, and the organisation of groups:

Group one -The two with strongest IT skills dominated the PC, I gave them a five-minute warning that they would need to switch around. This they did when I asked and again throughout the session without comment from me.

The aim of this was to encourage them to reflect on what they were doing and how they were learning. In this project ICT will be used as a tool for engaging learners in competing teams to collaborate to solve problems and find information using people and artefacts around them and then develop that information together.

There was a developing awareness of how specific technologies might enhance learning:

Enable collaborative or communicative homework tasks as opposed to just individual work, even if learners are unable to do homework in the same place at the same time.

The product itself (a webpage on CTAD's MediaBoard multimedia messaging platform) will contain evidence of learners' notes and edited writing about the locality … learners will engage in exploratory activities and create their own content.

I was interested in exploring the claims that the use of portable ICT devices could help increase collaborative learning, communication and independent learning amongst students. The claims were attributed to the mobility and capacity of the machines. That they did not dominate in the same way as desktops and could be more readily integrated into classroom use across the curriculum with the minimum of disruption to existing practices.

Tutors also began to reflect on how best to teach ICT skills:

I am interested in which IT skills are learnt through embedding ICT into the language outcomes. This was driven by the desire to give the learners transferable skills through encouraging a 'trial and error' approach to ICT. I achieved this by moving away from demonstrating step-by-step instructions.

Effective practice

By the end of the first phase of the project the tutors had become more flexible and reflective in their teaching approaches moving away from a didactic style to enable students to become more independent learners, as tutors became more experienced in promoting collaborative practices. This widened the variety of methodologies used within lessons and provided opportunities to expand learning outside the classroom e.g. the use of technologies at home, email, digital cameras and tablet PCs.

Another very significant change was that the tutors became more practised in observing learning and developing systematic assessment skills. At the beginning of the project there was a lack of consideration given to the assessment of learning; either specified learning outcomes were not being assessed and recorded or there was a failure even to recognise learning outcomes to assess against. By the end of the first phase, tutors were all producing considered schemes of work with achievable learning outcomes and assessing learners against these. There was also an awareness of 'softer' outcomes such as increases in confidence and self-esteem and the inclusion agenda.

The following comment by a tutor in his end-of-year report reflects a common feeling about being part of the project:

Buddies

All buddies were able to take on the schemes of work and lesson plans, and with the appropriate technology training to deliver according to the tutors' original plans, only making alterations to suit the literacy/ESOL skills levels of their groups of learners. Some required more support than others in order to achieve this, but all developed their own skills and confidence as a result of the project. By the end of the project, most of the buddies were working more collaboratively with their tutors and consequently became more confident and independent, and were more willing to input their own ideas and strategies. Tutors also benefited by developing new skills as well as consolidating those gained in the first phase. An analysis of the test scores of learners taught by buddies and those taught by the mentors showed no significant differences in any of the tests, indicating that the buddies were effectively delivering the schemes of work.

The buddying system had additional benefits: it motivated the buddies to adopt the technologies within their own teaching practice and to start to use new ones. It also provided practitioners with the time to discuss their teaching practice, to share ideas and in the end to allow collaborative working and peer support.

Research Report 62 9 Conclusions and implications

In this project, we developed and evaluated approaches to using ICT in adult literacy and ESOL classes, and alongside this we also developed approaches to training tutors in using ICT in their teaching. In Section 9.1 we present our main results and conclusions about learning and pedagogy, and in Section 9.2 our main results and conclusions related to tutor development.

In presenting these conclusions we will seek also to draw out recommendations for ongoing work in implementing ICT in Skills for Life teaching and ICT as a skill for life, and for the development of appropriate forms of staff development in this area.

ICT, teaching and learning

In the evaluation trials, classes improved in almost all cases in both literacy/ESOL skills and ICT skills and confidence, though different classes had different balances between improvements in the two sets of skills (see Section 6.2). The learning gains for classes were often statistically significant and the effect sizes were large. This lends strong support to the claim in the Moser Report (DfEE, 1999) that: 'Learners who use ICT for basic skills double the value of their study time acquiring two sets of skills at the same time.' As we have shown earlier, we had encouraged tutors to change their style of teaching, and as a result tutors sometimes saw themselves as taking risks with their pedagogy and were often pleasantly surprised that the changes worked, typically noting that their classes became more social and 'gelled' together better, that pressure on the tutor was reduced and so they were able to devote more attention to understanding their learners' needs.

We will first discuss the results within the framework we have adopted from Ivanic ˇ and Tseng (2004), and then present some conclusions about effective practice.

Factors affecting learning-teaching events Participants' beliefs, intentions and resources

The learners' age was a significant factor in predicting learning gains (see Section 7.3.1). There was a negative correlation for ESOL skills (older learners made least progress), and for men (but not for women) there were positive correlations for ICT skills and confidence (older learners made most progress). Initial ICT-confidence scores were found to be correlated with learners' persistence, those with lower scores were likely to attend less frequently, and were more likely to eventually drop out (see Section 7.3.2). Although not unexpected, this result does highlight the issue that while ICT-based teaching can be very successful for many learners, there are others who do not have the necessary levels of ICT confidence to be able to take full advantage of the approach.

There was no correlation found between changes in ICT-skills and ICT-confidence scores and changes in reading (GO) and listening (ESOL) scores (see Section 7.2). This suggests that the two areas of skills are being learned independently, that is the acquisition of one is not affecting the acquisition of the other. However there are some suggestions in the data that initial language skills may impact on ICT learning, and also that initial ICT skills may impact on language learning, and this is an area worth further investigation.

Learning and teaching resources

Most users found the use of ICT motivating (see Section 6.3.1). A very small number of learners said that they found the technology a distraction from their language work. Sometimes this seemed to be a result of the learners' lack of interest in technology, and sometimes to the tutor not making sufficiently clear the relationship of the ICT work to the language work. Mobile technologies (tablets, PDAs, mobile phones) were found to be particularly motivating, and enabled greater flexibility in teaching, with tutors taking advantage of the mobility of the technology to move outside the classroom. Within the context of this project relatively small amounts of mobile technology were made available and it is likely that increased access would be even more effective (see the case studies in Section 4.3 and 4.4).

The nature of learning-teaching events

The context -use of technology Increased ICT skills and confidence (as evidenced by the tests that we used) were positively correlated with the amount of time learners spent using technology within the classroom (see Section 7.1). More specifically learner use of the internet, PowerPoint and word processing were found to be positively correlated with gains in ICT skills. (As we shall see below, tutor use of PowerPoint is negatively correlated with learners' gains in ICT skills -pointing up the need for active involvement on the part of the learner, not passive observation).

Approaches to teaching Collaboration

We laid particular emphasis on learner collaboration, and observations indicated that tutors were often successful in the classroom management of collaboration. Some tutors carefully structured collaborative activities, though others simply allowed learners to work together in their own ways.

However, classes where individual learners spent more time working on their own showed better gains in ICT skills (though not ICT confidence) than those classes which spent more time working in small groups (see Section 7.1.2).

Examination of the classroom observation data (see the case studies in Chapter 4) showed that there were many occasions where collaboration seemed to be effective, and that, in particular, it could be important in bolstering learners' confidence in using the technology. However, this data also showed some of the problems about the way that work was organised in groups. When collaborative work was forced by the need to share technology it was not as successful as when tutors developed tasks that required peer interaction. When technology was shared, one person sometimes dominated the use of the technology, and it is possible that this domination undermined the usefulness of collaborative work for developing ICT skills. In one scheme of work collaborative work was carefully planned, but seems to have been given insufficient time, being pushed out by other classroom activities.

Collaborative learning faces challenges in many contexts, but adult learning presents specific challenges -in particular membership of classes changes quickly making continuity difficult. At least one tutor found it very difficult to organise collaborative working because of the continual changes in class membership.

There is sufficient evidence of the potential of collaborative working from the observational data to support the continued exploration of collaborative approaches, but further work is needed to determine the best way of using collaborative work with technology.

Construction of artefacts and role-play

Most of the schemes of work involved learners in the construction of an artefact -often jointly (see the case studies in Chapter 4). Evidence from classroom observations indicates that this was often a useful focus, generating motivation, collaboration and purposeful action, and also pointed up the value of ICT used in this way in allowing differentiation within the classroom. In certain contexts role-play was also an important part of these activities -this was the case, for example, where learners presented themselves as journalists, photojournalists, or TV interviewers.

The creation of learning opportunities

The management of learning There were no correlations found between any of the observed teaching strategies and changes in performance in the reading (GO) and listening (ESOL) tests, but a number of correlations were found for ICT skills and ICT confidence (see Section 7.1.3).

We encouraged teaching strategies that aimed to increase the autonomy of learners so that they were able to engage in self-directed learning. There was much evidence in the observational data that tutors were in fact doing this to a greater degree than previously. The observational category managing activities gives us some indicator of this changed role of the tutor, and there was a positive correlation between the amount of time spent by tutors managing activities with gains made in both ICT skills and confidence.

The most effective teaching strategy was extending, where the tutor built on/added to material previously introduced by them, or added to a comment by a learner. There was a strong correlation between this type of teaching activity and changes in ICT skills and confidence scores. There was a strong positive correlation found between tutors' use of a conventional whiteboard and the use of the extending strategy, and they seem to have used the whiteboard to support this strategy. Other teaching activities that positively correlated with improvements in ICT skills were: discussing, instructing (telling learners what to do -i.e. organisational issues), listening, and modelling (showing the learner how to do something using the actual technology or its representation on a SMART Board™).

One strategy was found to be negatively correlated with changes in ICT skills and confidence, and that was explaining (where the tutor tells the learner 'how to do something'). Where the tutor, for example, was demonstrating what was to be done ('modelling') this was much more successful. Tutor use of PowerPoint also correlated negatively with change in ICT skill.

Different types of outcomes

ICT can change the focus on the knowledge to be learned: this was evidenced by tutors talking more about managing information, and perhaps less about learning it, and talking more about browsing and scanning and less about reading as comprehension. The nature of literacy tasks was also often changed in subtle ways, for example in the m-learning scheme of work the learners took photographs outside the classroom, and then back in the classroom their task was to write captions for the pictures -so what had been previously a task conceptualised as writing short sentences now involved issues around relating image to text.

Going beyond the skills described in the ICT Skills for Life Curriculum and those tested in the ICT-skills assessment, tutors also sought to promote another range of ICT skills that they saw of value, for example being able to evaluate the value of resources, willingness to learn new skills and transfer of skills from one context to another.

Effective practice and implications for pedagogy

The concept of purposeful use within the ICT Skills for Life Curriculum is an important concept that can help to change ICT teaching, but it does not in itself, of course, constitute a pedagogy. The work of this project can perhaps help to inform the sort of pedagogy that might enable this curriculum to be effectively implemented.

The classroom observations provide the following general picture of activities which were positively correlated with changes in ICT skills and confidence:

■ greater use of technology by learners ■ learners working individually ■ tutor using the whiteboard ■ tutor managing the classroom activities, extending, discussing, instructing, listening and modelling.

The following were negatively correlated: ■ tutor use of PowerPoint ■ tutor 'explaining' how to do something.

The specific components of our approach to changing classroom dynamics for the use of ICT in adult literacy and ESOL classes were shown to be effective in supporting the development of ICT skills and confidence: ■ encouraging learner autonomy, and consequently releasing teacher time to get to know their learners better and more carefully to adapt their teaching to learners' needs ■ the use of technology to construct (usually shared) artefacts -which often allowed learners to experiment with a variety of roles, and allowed for differentiation in activities ■ use of a wide range of technologies, and in particular mobile technologies to support greater flexibility ■ encouraging collaborative learning (which was shown to be effective to some degree, though where it restricted learners' access to the use of the technology it could be counterproductive).

Tutor development

The development of tutors and buddies within the project

Our approach to development was based on reflection, group meetings, iterative design, and on-site support from development officers. Evidence from diaries and interviews shows that teachers developed in their understanding of pedagogy with ICT, in particular: ■ Increased confidence in teaching with technology (awareness that ICT encompasses more than just computers in teaching)

ICT long LIVE 17/1/07 17:44 Page 65 -Increased awareness that the use of modern technologies in teaching implies the need to identify new ways of teaching -Strengthened belief in a learning methodology that puts learner autonomy at its heart -Strengthened belief in changing pedagogies to foster peer learning through collaboration -More confidence in experimenting and trying new technologies with learners to achieve specific goals.

This process took a lot of time and effort, but did succeed in enabling tutors to develop into confident innovative users of technology in the classroom. Several of the tutors went on to be mentors within the LSDA-led project 'Exploring E-Learning for Literacy, Numeracy and ESOL Teachers'.

In the first phase of our project, the tutors had developed robust models of ICT use in literacy and ESOL, and the tutors were then able to induct their buddies into using these models relatively quickly. The mentors worked with the buddies over two terms and talked through the creation of around six critical incident reports, but nevertheless the investment in time on the part of the buddies was considerably less than their mentors had put in the previous year, and yet the tests showed no significant differences between the scores of learners taught by the tutors and those taught by their buddies. The robust models of ICT use developed by the tutors were able to support the buddies' performance in an unfamiliar area.

Implications for professional development

In work that we did with NIACE in an action research project looking at the implementation of the ICT Skills for Life Curriculum (NIACE, 2005) we were able to obtain data on the backgrounds of quite a large number of tutors delivering ICT Skill for Life. Indications from the NIACE work suggest that ICT tutors are sometimes poorly prepared for teaching Skills for Life, and Skills for Life tutors sometimes have a rather limited view of ICT.

The tutors we worked with in this project started as individuals with special interest in the area, and went through an intensive period of training and reflection within the project8 , which for some has consolidated their positions as leaders in the field. This intensive form of staff development clearly has an important role to play.

Within the NIACE project there was considerable discussion about the level of ICT knowledge required by ICT Skills for Life tutors. The tutors in this project were delivering both language and ICT skills, most were not ICT specialists, but neither were they novices in ICT. They did take the technology seriously, were often very skilled in particular aspects (usually selftaught), and did not treat ICT as 'merely a tool'.

Another issue that came out strongly from the NIACE study was the need for good exemplar materials. As we argued above, given robust models of ICT use, the practices of the tutors could then be relatively efficiently picked up and used by their buddies.

There are two forms of staff development required -that needed by those who have to generate and develop models of ICT use, and that needed by those who wish to pick up and adapt tried and tested models.

Glossary of terms

e-learning: a general term referring to the use of digital technologies to support learning and teaching.

e-Portfolio: a collection of electronic evidence (e.g. Word files, images, blog entries, web pages) assembled and managed by a user as a demonstrations of the user's abilities and as a platform for self-expression.

Mindmap: is a diagram used to represent ideas linked to and arranged radially around a central key word or idea.

m-learning: the delivery of learning to students through the use of mobile or portable technology including mobile phones, PDAs and tablet PCs.

PDA: a personal digital assistant is a small hand held electronic device which includes some of the functionality of a computer, a mobile phone, a music player and a camera.

Standard deviation: the average deviation between the individual scores in a distribution and the mean of the distribution.

Spearman rho coefficient: the correlation coefficient used to measure the association between two ordinal scale variables, i.e. numerical variables where the numbers are meaningful but the distance between them is not constant, such as ranked data.

Tablet: a notebook-shaped mobile computer with a touch screen which enables the user to operate the computer with a digital pen instead of a keyboard or mouse.

SMART Board™: an interactive whiteboard connected to a computer. The computer screen image is projected onto the whiteboard, and the user can interact with the computer through the use of a pen on the whiteboard.

WebQuests: an inquiry-oriented activity in which some or all of the information that learners interact with comes from resources on the internet.

Figure 7 . 1

 71 Figure 7.1 Changes ICT-Skills and ICT-Confidence scores

Table 5 .1 Classroom organisation Individual Small group Whole class

 5

	ICT long LIVE 17/1/07 17:44 Page 37
	WebQuests	49%	16%	35%
	e-Portfolio	52%	14%	34%
	Tablets	4%	68%	28%
	m-learning	33%	40%	27%
	Digital video	60%	12%	27%
	Mindmaps	64%	10%	26%
	National Test	81%	0%	19%
	Average	46%	26%	28%
	Comparison with Stage 1 study	79%	10%	2%

Table 5 .2 Teaching activities

 5

	WebQuests	66%	2%	1%	3%	5%	2%
	e-Portfolio	72%	1%	0%	0%	1%	11%
	Tablets	66%	4%	2%	0%	10%	5%
	m-learning	67%	2%	1%	3%	8%	2%
	Digital video	65%	6%	1%	0%	4%	7%
	Mindmaps	74%	0%	6%	3%	7%	4%
	National Test	66%	1%	0%	0%	5%	6%
	Average	68%	2%	1%	2%	6%	5%
	WebQuests	9%	3%	4%	0%	1%	3%
	e-Portfolio	6%	4%	3%	0%	0%	1%
	Tablets	8%	2%	4%	0%	0%	0%
	m-learning	8%	2%	0%	0%	0%	5%
	Digital video	7%	5%	3%	1%	1%	0%
	Mindmaps	2%	0%	4%	0%	0%	0%
	National Test	20%	1%	0%	0%	0%	0%
	Average	9%	2%	2%	0%	0%	2%

  Table5.4 shows the use of various pieces of software by both tutors and learners. The percentages in this table are of the time actually spent using PCs or tablets. WinJournal is a package available only on the tablets allowing the capturing of handwriting and diagrams, and so its use is naturally restricted to the tablet users. The e-Portfolio group made a lot of use of FrontPage in order to construct their web pages, but consequently made little use of other software packages. Internet Explorer and Word were amongst the most commonly used applications for both tutors and learners, with tutors also using PowerPoint a great deal.

	Table 5.3 Use of media 5.4 Use of software
	TUTOR	PC	Laptop	Tablet	SMART	Board™	Mobile	phone	PC	projector	Managing	activities Whiteboard		Questioning Other technology	Extending Paper-based activity	Listening to Total media use students	Discussing	Explaining
	WebQuests	2%	0%	0%	6%	0%	1%		4%		1%	1%	15%
	e-Portfolio	0%	0%	0%	0%	0%	26%		4%		4%	0%	34%
	Tablets	0%	0%	4%	0%	0%	4%		1%		7%	0%	16%
	m-learning	0%	0%	0%	0%	0%	0%		3%		2%	0%	5%
	Digital video	2% Table 5.4 Use of software 0% 0%	8%	0%	34%		0%		5%	0%	48%
	Mindmaps	6% TUTOR	0%	0%	0%	0%	0%	10%		0%	0%	16%
	National Test	1%	0%	0%	0%	0%	0%		0%		0%	0%	1%
	WebQuests e-Portfolio ALL LEARNER	54% 59% WebQuests e-Portfolio 1%	0% 0% 0%	0% 0% 1%	2% 0% 2%	0% 0% 27% 5% 0% Internet Explorer	0% 0% 0% 0% 6% Outlook (Email)	Instructing 0% 0% 2% 63% 3% Word	Repeating	and 0% reminding 4% 62% 25% 3% PowerPoint	Modelling 11% 4% 0% 6% 0% FrontPage	Social 66% conversation 67% 0% 0% 15% Win Journal	Presenting 0% 0% Other educational	software	other 9% activitiy 0% Organising Other
	Tablets m-learning	0% Tablets 45% m-learning Digital video 0% 0%	78% 0%	0% 0%	0% 38% 13% 100% 17%	0% 0% 0% 0% 0%	0% 4% 0% 0% 47%		0% 0% 0% 0% 37%	1% 0% 13% 0% 0%	57% 80% 0% 72% 0%	2% 0% 0%	0% 0% 0%
	Digital video	65% Mindmaps	0%	0%	0%	0% 0%	0% 44%	0% 56%		4% 0%	4% 0%	73% 0%	0%	0%
	Mindmaps	66% National Test 0%	0%	0%	0% 27%	0% 0%	0% 52%		0% 21%	0% 0%	66% 0%	0%	0%
	National Test	94% ALL	0%	0%	0%	0% 15%	0% 3%	0% 43%		0% 29%	0% 2%	95% 6%	0%	1%
	ALL	52%	0%	10%	0%	4%	0%		0%		1%	7%	75%
		LEARNER
		WebQuests					37%	8%	25%		29%	0%	0%	0%	1%
	WebQuests	e-Portfolio					1%	0%	27%		0%	66%	0%	1%	5%
	TUTOR	2% Tablets	0%	0%	6%	0% 17%	1% 0%	4% 10%		1% 0%	1% 0%	15% 26%	48%	0%
	LEARNER	54% m-learning	0%	0%	2%	0% 87%	0% 0%		0% 7%		0% 0%	11% 0%	66% 0%	0%	6%
	e-Portfolio	Digital video				24%	0%	63%		12%	0%	0%	0%	1%
	TUTOR	0% Mindmaps	0%	0%	0%	0% 4%	26% 26%	4% 53%		4% 0%	0% 0%	34% 0%	17%	0%
	LEARNER	59% National Test 0%	0%	0%	0% 39%	0% 0%	0% 57%		4% 0%	4% 0%	67% 0%	0%	3%
	Tablets	ALL					29%	5%	36%		5%	9%	4%	10%	2%
	TUTOR	0%	0%	4%	0%	0%	4%		1%		7%	0%	16%
	LEARNER	0% WebQuests	0%	78% TUTOR	0%	0% 27%	0% 0%		0% 2%		0% 62%	1% 0%	80% 0%	0%	9%
	m-learning			LEARNER			37%	8%	25%		29%	0%	0%	0%	1%
	TUTOR	0% e-Portfolio	0%	0% TUTOR	0%	0% 5%	0% 0%	3% 63%		2% 25%	0% 6%	5% 0%	0%	0%
	LEARNER	45%	0%	0% LEARNER	0%	13% 1%	0% 0%	0% 27%		0% 0%	13% 66%	72% 0%	1%	5%
	Digital video	Tablets		TUTOR			38%	0%		4%		0%	0%	57%	2%	0%
	TUTOR	2%	0%	0% LEARNER	8%	0% 17%	34% 0%	0% 10%		5% 0%	0% 0%	48% 26%	48%	0%
	LEARNER	65% m-learning	0%	0% TUTOR	0%	0% 100%	0% 0%		0% 0%		4% 0%	4% 0%	73% 0%	0%	0%
	Mindmaps TUTOR LEARNER National Test	6% Digital video 0% 66% 0% Mindmaps	LEARNER 0% TUTOR 0% LEARNER TUTOR LEARNER	0% 0%	87% 0% 17% 0% 24% 0% 4%	0% 0% 0% 0% 0% 44% 26%	7% 10% 47% 0% 63% 56% 53%		0% 0% 37% 0% 12% 0% 0%	0% 0% 0% 0% 0% 0% 0%	0% 0% 16% 0% 66% 0% 0%	0% 0% 0% 0% 17%	6% 0% 1% 0% 0%
	TUTOR	1% National Test 0%	0% TUTOR	0%	0% 27%	0% 0%	0% 52%		0% 21%	0% 0%	1% 0%	0%	0%
	LEARNER	94%	0%	0% LEARNER	0%	0% 39%	0% 0%	0% 57%		0% 0%	0% 0%	95% 0%	0%	3%
	ALL	ALL		TUTOR			15%	3%	43%		29%	2%	6%	0%	1%
	TUTOR	1%	0%	1% LEARNER	2%	0% 29%	6% 5%	3% 36%		3% 5%	0% 9%	15% 4%	10%	2%
	LEARNER	52%	0%	10%	0%	4%	0%		0%		1%	7%	75%

Table 6 .1 Access to a computer at home Web-e-Portfolios Tablets m-learning Digital

 6

	Mindmaps National	TOTAL

Table 6 .2 Access to and use of a computer at home

 6

		16-19	20-29	30-39	40-49	50-59	Over 59	TOTAL
	Access
	Male	10	7	3	1	1	4	26
	Female	8	12	5	7	4	5	41
	Total	18	19	8	8	5	9	67
	Use
	Male	10	7	1	1	0	3	22
	Female	8	11	5	3	3	1	31
	Total	18	18	6	4	3	4	53

Table 6 .3 Changes in access to a computer at home Web-e-Portfolios Tablets m-learning Digital Mindmaps National TOTAL Quests video test

 6

	No longer have a computer 0	1	0	1	0	0	0	2
	No change	7	23	16	17	5	6	3	77
	Obtained a computer	0	2	4	0	2	2	1	11

Table 6 .4 Changes in use of a computer at home Web-e-Portfolios Tablets m-learning Digital Mindmaps National TOTAL Quests video test

 6

	Stopped using	0	2	0	1	0	0	0	3
	No change	7	23	19	17	5	4	3	78
	Started using	0	1	1	0	2	4	1	9

Table 6 .5 Changes in ICT-skills assessment scores

 6

			Pre-study	Post-study	Change	Significance
	Scheme of Work	N	Mean	SD	Mean	SD	Mean	SD
	WebQuests	7	13	13.57	46	9.41	33	16.24	0.016*
	e-Portfolio	25	52	14.78	57	13.30	5	15.57	0.142
	Tablets	20	29	20.97	42	14.51	13	16.84	0.004*
	m-learning	19	43	18.49	57	13.89	14	9.70	7.6e-6*
	Digital video	6	46	11.22	55	3.32	9	6.08	0.031*
	Mindmaps	7	9	8.81	57	9.62	47	16.43	0.016*
	National Test	4	30	34.62	63	7.76	33	28.38	0.25
	ALL	88	37	22.32	53	14.14	16	19.35	2.86e-11*

N = number of learners, SD = standard deviation * indicates statistical significance at at least the 5% level.

Table 6 .6 Changes in ICT-confidence questionnaire scores

 6

			Pre-study	Post-study	Change		Significance
	Scheme of Work	N	Mean	SD	Mean	SD	Mean	SD
	WebQuests	8	30.00 25.89	46.89 26.78	19.56 23.89	0.109
	e-Portfolio	26	74.22 18.44	78.00 20.89	3.00 18.22	0.124
	Tablets	19	34.67 25.11	54.89 26.33	21.33 28.78	0.018*
	m-learning	8	60.33 28.67	66.44 26.78	7.78	7.78	0.078
	Digital video	7	42.22 21.00	65.67 14.56	19.33 21.11	0.043*
	Mindmaps	7	11.11	7.22	54.11 19.78	42.78 21.56	0.008*
	National Test	2	30.00	2.67	52.22	4.67	22.22	7.33	0.5
	ALL	78	48.22 30.00	64.22 24.67	15.67 24.22	1.60e-07*

N = number of learners, SD = standard deviation * indicates statistical significance at at least the 5% level.

Table 6 .7 Changes in reading (GO) test scores Pre-study Post-study Change Significance

 6

	Scheme of Work	N	Mean	SD	Mean	SD	Mean	SD
	WebQuests	0
	e-Portfolio	27	23	18.21	33	12.35	10	11.90	7.5e-5*
	Tablets	0
	m-learning	14	23	11.69	33	13.49	10	9.14	0.001*
	Digital video	5	29	10.29	45	11.87	16	9.54	0.063
	Mindmaps	6	57	19.15	67	5.27	10	13.96	0.156
	National Test	4	58	24.60	61	19.10	4	17.00	0.875
	ALL	56	29	20.91	40	17.34	10	11.54	2.03e-08*
	N = number of learners, SD = standard deviation
	* indicates statistical significance at at least the 5% level.

  long LIVE 17/1/07 17:44 Page 47

Table 6 .9 Comparisons between the schemes of work

 6

	WebQuests	7	33* 2.68* 8	20	0.58				5	27	1.08
	e-Portfolio	25	5	0.33 26	3	0.18 27	10* 0.66* 23	8* 0.72*
	Tablets	20	13* 0.72* 19	21* 0.75*				15	1	0.05
	m-learning	19	14* 0.84* 8	8	0.21 14	10* 0.76* 19	7* 0.24*
	Digital video	6	9* 0.87* 7	19* 1.24* 5	16	1.49	5	-2 -0.33
	Mindmaps	7	47* 4.56* 7	43* 2.06* 6	10	0.54
	National Test	4	33	1.02	2	22	4.83	4	4	0.14
	N = number of learners
	* indicates statistical significance at at least the 5% level.

Table 7 .1 Use of technology -statistically significant correlations with ICT use

 7

	Number of		Change in		Change in
		Classes		ICT-skills score		ICT-conf score
			r	p	n	r	p	n
	Student -total PC use	14	0.433	2.5e-5*	88	0.359	0.001*	78
	Student -PowerPoint	3	0.7	0.008*	13	-0.04	0.888	15
	Student -Word processor 11	0.383	0.001*	68	0.401	0.001*	60
	Tutor -PowerPoint	6	-0.363	0.021*	40	0.008	0.961	41
	Tutor -Whiteboard	10	0.582	1.02e-7*	71	0.578	1.09e-6* 61
	* indicates statistical significance at at least the 5% level.

Table 7 .2 Correlations between initial scores Initial GO Initial ESOL Initial Initial ICT- (non parametric) ICT-skills confidence

 7

	Initial GO	r	-	0.704	0.186	-0.317
		p	-	1e-06*	0.052	0.005*
		n	-	91	110	78
	Initial ESOL (non parametric)	r	0.704	-	0.601	0.319
		p	1e-06*	-	1e-06*	0.002*
		n	91	-	125	93
	Initial ICT-skills	r	0.186	0.601	-	0.587
		p	0.052	1e-06*	-	1.2e-012*
		n	110	125	-	122
	Initial ICT-confidence	r	-0.317	0.319	0.587	-
		p	0.005*	0.002*	1.2e-012*	-
		n	78	93	122
	r = correlation coefficient, p = probability that the result would have been found by chance (i.e. statistical significance), n =
	number of learners.
	* indicates statistical significance at at least the 5% level.

Table 7 .3 Correlations between initial scores and improvement in scores

 7

	Change in	Change in ESOL	Change in Change in ICT-
	GO	(non parametric)	ICT-skills	confidence

Table 7 .4 Correlations between changes in scores Change in Change in ESOL Change in Change in ICT- GO (non parametric) ICT-skills confidence

 7

	Change in GO	r	-	0.317	0.081	0.044
		p	-	0.041*	0.565	0.786
		n	42	53	40
	Change in ESOL	r	0.317	-	0.056	0.021
	(non parametric)	p	0.041*	-	0.663	0.885
		n	42		64	52
	Change in ICT-skills	r	0.081	0.056	-	0.511
		p	0.565	0.663	-	8.7e-06*
		n	5 3	6 4		68
	Change in ICT-confidence	r	0.044	0.021	0.511	-
		p	0.786	0.885	8.7e-06*	-
		n	4 0	5 2	68
	r = correlation coefficient, p = probability that the result would have been found by chance (i.e. statistical significance), n =
	number of learners.
	* indicates statistical significance at at least the 5% level.

1 Test scores and age and class size

	56	Research Report
		7.3 Other relationships
		7.3.
		-8.00 -6.00	-4.00 -2.00	0.00	2.00	4.00	6.00	8.00	10.00
			Change in ICT confidence

Table 7 .5 Correlations between change in test scores and age range

 7

	Change in	Change in	Change in Change in ICT-
	GO	ESOL	ICT-skills	confidence

Table 7 .6 Correlations between test scores and class size Change in Change in Change in Change in GO ESOL ICT-skills ICT-confidence

 7

	Mean class size r spearman	0.117	0.052	-0.505	-0.264
	p	0.390	0.673	5.33e-07*	0.021*
	n	5 6	6 7	8 8	7 6
	* indicates statistical significance at at least the 5% level.

Effective Teaching and Learning : Using ICT

See: http://www.opeus.com

'Teachers can use scaffolding techniques to teach a specific aspect of basic skills. They construct a "scaffold" around the area so that learners have direct access to the chosen focus, with nothing allowed to get in the way. To be of benefit, scaffolding must be temporary. When the learner shows signs of handling the task in question, the "scaffolding" can then be removed gradually until it is no longer needed.' -from http://www.dfes.gov.uk/curriculum_literacy/tree/speaklisten/listenrespond/ accessguidance/3/

ICT long LIVE 17/1/07 17:44 Page 32Effective Teaching and Learning : Using ICT

Office of National Statistics http://www.statistics.gov.uk/ [accessed 15 May 2006] ICT long LIVE 17/1/07 17:44 Page 44 Effective Teaching and Learning : Using ICT

The effect size is the difference between the two means (in this case the pre and post-test means) divided by the pooled standard deviation. Roughly speaking a figure of 0.2 indicates a small effect, 0.5 a medium effect and 0.8 a large effect. For comparison: an effect size of 0.2 corresponds to the difference between the heights of 15 and 16 year old girls in the US, an 0.5 effect size corresponds to the difference in heights between 14 and 18 year old girls and an effect size of 0.8 equates to the difference in heights between 13 and 18 year old girls. An improvement of one GCSE grade in Maths or English would represent an effect size of 0.5-0.7. These examples are taken from What is an 'Effect Size'? A brief introduction Robert Coe, CEM Centre, Durham University March 2000 available at: http://www.cemcentre.org/renderpage.asp?linkid=30325016

The listening assessment (ESOL) used ordinal scale data whereas the reading test (GO) is measured on a continuous scale, and as a result small changes are less notable than they would have been with continuous scale data.

ICT long LIVE 17/1/07 17:44 Page 46Effective Teaching and Learning : Using ICT

ESOL listening had a separate test for each level. We combined the scores/levels on each test into a single continuous scale, but as this is levels rather than scalar scores this score been treated as ordinal data.

We met with the tutors on a monthly basis, and between these meetings two development officers visited the tutors and worked with them in developing their practice. Each week the tutors completed an online reflective diary, and each term they drew up an intervention plan.

Acknowledgements

We would like to express our thanks to the practioner-researchers for all their insights, hard work, enthusiasm and good humour despite everything: John Budis, Elizabeth Caush, Cathy ICT long LIVE 17/1/07 17:44 Page 73 ICT long LIVE 17/1/07 17:44 Page 74 ICT long LIVE 17/1/07 17:44 Page 75

Effective Teaching and Learning : Using ICT schemes of work. Lack of significance in the other schemes of work could be due to the low number of participants who took both sets of assessments.

ESOL 7

Five of the seven schemes of work used the ESOL listening assessment. Table 6.8 shows that the learners participating in the WebQuests made the greatest improvement but this change was not found to be significant (possibly due to the small number who took both assessments), while learners in the e-Portfolio and m-learning schemes of work made statistically significant improvements. Five of the seven schemes of work used the ESOL listening assessment. Learners participating in the WebQuests made the greatest improvement but this change was not found to be significant (possibly due to the small number who took both assessments), while learners in the e-Portfolio and m-learning schemes of work made statistically significant improvements. All tutors commented that they had not included English listening literacy skills explicitly in their schemes of work.

Comparison between schemes of work