Agnieszka Lenko-Szymanska

Alex Boulton
email: alex.boulton@atilf.fr

HAL is a

Corpora, large textual databases in the modern sense, first really became available for systematic analysis of language in the 1960s. Since then they have been growing in size and complexity, as have the tools used to process them. Fifty years later we have at our disposal a plethora of corpora of different kinds: monolingual and multilingual collections (the latter containing either original but comparable or translated texts); small highly specialised corpora of a few tens of thousands of words to large reference or monitor corpora of hundreds of millions or even billions of words; corpora containing just text (plain or annotated) or multimodal corpora aligned with sound or video. Present-day tools also go beyond presenting a word in its immediate contexts (KWIC) and providing a wordlist, and offer complex statistical analyses enabling automatic identification of collocations and grammatical patterns, text keywords or stylistic profiles. It can be safely claimed that since their introduction, corpora have changed for ever many research areas: linguistic analyses in the first place, which now are based on attested examples of language use and account for frequency phenomena, but also applied disciplines such as machine translation, speech recognition or forensic linguistics. They are also used outside linguistics, for example in historical or applied social studies, and are commonplace on many websites for the general public (for search functions in particular). One of the areas in which corpora have had a considerable influence is language pedagogy.

In the discussion of the applications of corpora for language teaching we frequently come across the notion of affordances (cf. Hafner & Cadlin 2007;[START_REF] Boulton | Beyond concordancing: Multiple affordances of corpora in university language degrees[END_REF]. This notion originally comes from the field of perception psychology and was first proposed by the American psychologist James [START_REF] Gibson | The theory of affordances[END_REF][START_REF] Gibson | The Ecological Approach to Visual Perception[END_REF]. According to this theory, it is a person's perception of the environment that prompts some course of action. Affordances thus refer to the properties of an object in the environment enabling any kind of activity. However, the theory also implies a relationship between the object and the person, and what he or she can do with that object. For example, a book enables reading but can also be used as support for a leg of a wobbling table and countless other purposes. The realisation of these affordances depends on the person's ability to read or his or her ingenuity to use the book to support the wobbling table, etc. In other words, an affordance can be defined as any use of an object that a person can perceive. This theory has been taken up in various fields including design, human-computer interaction, ergonomics, and so on. Douglas Arthur 'Don' Norman, a well-known researcher in cognitive science, human-centred design and usability engineering, is credited with the observation that "the value of a well-designed object is when it has such a rich set of affordances that the people who use it can do things with it that the designer never imagined" (1994: n.p.).

Corpora have multiple affordances for language teaching which were not immediately apparent to the pioneering corpus compilers, including Randolph [START_REF] Quirk | Towards a description of English usage[END_REF] or Henry Kučera and W. Nelson Francis (1967), who created the first one-million-word corpora of British and American English respectively. However, while more and more researchers carried out projects aiming at improved corpus-based linguistic description (e.g. John Sinclair, the director of the COBUILD project at the University of Birmingham), the implications of corpus analyses for language teaching started slowly to emerge. In 1994 the first conference on Teaching and Language Corpora (TaLC) was held at the University of Lancaster. Its aim was to bring together researchers and language teachers who were involved in pedagogical activities making use of corpora. The proceedings from this conference (Wichmann et al. 1997) were the first published attempt to tackle explicitly and in a wide-ranging way the use of corpora in language education.

In the first chapter of that volume, Geoffrey [START_REF] Leech | Teaching and language corpora: A convergence[END_REF] proposed a distinction between direct and indirect uses of corpora in language education. The indirect uses involve situations in which learners come into contact with the results of corpus investigations through the mediation of reference works and teaching materials; though corpora inform such products, the end-user need not even be aware of this. The direct uses of corpora, on the other hand, involve students exploiting corpus data themselves to learn to perform linguistic analyses and, more importantly, to learn the language. Tim Johns (1991: 30), one of the most influential pioneers of the approach, advocated "cut[ting] out the middleman as far as possible and… giv[ing] the learner direct access to the data." He called this kind of activity classroom concordancing, the title of the seminal collection of papers he co-edited with Philip King in 1991, and also coined the term data-driven learning (DDL) as students were supposed to learn directly from linguistic data, following the motto: "Every student a Sherlock Holmes" (1997: 101). According to Leech, a data-driven approach to learning should be central application of corpora in education, not least since language itself and language learning a fortiori are themselves usage based (e.g. [START_REF] Beckner | Language is a complex adaptive system: Position paper[END_REF]).

Since the early days of the first TaLC conference, corpora have been increasingly used in language pedagogy. A personal account of the growing influence of linguistic databases in language teaching and systematic evidence to support this observation is offered by Tribble (this volume). Over the last twenty years or so, numerous books, articles and conference presentations have discussed a variety of corpus affordances in education, from more accurate corpus-based descriptions of the target and learner language, to the creation of new corpus-based resources and tools for language teaching and learning (e.g. Ghadessy et al. 2001;Sinclair 2004;Braun et al. 2006;Aijmer 2009;Campoy-Cubillo et al. 2010;[START_REF] Boulton | Beyond concordancing: Multiple affordances of corpora in university language degrees[END_REF], to name just a small sample of collected volumes on these topics). The academic debate has been followed by real-life pedagogical endeavours. Currently, corpora are present in the production of almost all learner dictionaries (e.g. the Macmillan English Dictionary for Advanced Learners -Rundell 2007), reference grammars (e.g. the Longman Grammar of Spoken and Written English - [START_REF] Biber | Longman Grammar of Spoken and Written English[END_REF] However, the direct uses of corpora in language teaching are treated rather marginally in the literature in the field. This sadly reflects real classroom practice, in particular outside higher education, where explicit data-driven techniques are rarely incorporated into regular teaching procedures, as observed by a number of researchers [START_REF] Mukherjee | Bridging the gap between applied corpus linguistics and the reality of English language teaching in Germany[END_REF][START_REF] Römer | Corpus research and practice: What help do teachers need and what can we offer? In Corpora and Language Teaching[END_REF][START_REF] Römer | Using general and specialized corpora in English language teaching: Past, present and future[END_REF]Tribble, this volume). Despite the enthusiasm of a handful of specialists who point to the numerous and varied advantages of the approach, language learners rarely have hands-on experience with corpora in mainstream education, at least in prototypical DDL form. This is regrettable because learners can benefit tremendously from the direct use of corpora. They gain access to authentic language, which they can query in a variety of ways for the information which is interesting and relevant to them at a particular moment and which allows them to refine their understanding of how language really behaves. Frequently, the information available in a corpus is richer than any account offered by reference and teaching materials. Moreover, the very act of formulating an appropriate query and making sense of its results contributes to the development not only of their linguistic awareness but also of their general learning skills and autonomy, which are at the forefront of the recent educational movements (Lamb & Reinders 2007).

A very good reason for teachers' reluctance to introduce corpus-based activities into their work (assuming they are aware of them) is the fact that DDL is perceived to be a demanding and challenging instructional technique. It requires from learners good technical skills in manipulating corpus-analysis software and formulating appropriate queries. These skills need time and effort to develop, and many teachers feel that they do not have space for this in their already-packed curricula. In addition, corpus data are often criticised as being unsuitable for lower-proficiency students, due to their linguistic difficulty and the limited relevance of topics covered (cf. [START_REF] Breyer | Learning and teaching with corpora: Reflections by student teachers[END_REF]Chujo et al. this volume). And since the majority of language learners worldwide are below the advanced stage of proficiency, their teachers may not find it appropriate to expose them to corpus data containing large quantities of authentic language. Still another hurdle to DDL is its perceived incompatibility with the prevalent communicative language teaching methodology, which advocates focus on fluency rather than accuracy, with priority given to linguistic skills and functions rather than form [START_REF] Widdowson | Aspects of Language Teaching[END_REF]. DDL, on the other hand, promotes focus on language patterns which are frequently, if not exclusively, equated with grammatical structures. Perhaps the most important obstacle is that DDL implies an inductive approach to learning (as discussed in detail by Flowerdew, this volume). In order for DDL to be a valuable learning experience, learners should be able to take an active attitude to language analysis and interpret the results of corpus queries on their own. Such autonomy, although widely advocated by recent educational trends, is not always in tune with institutions' educational traditions or individual students' learning styles (see [START_REF] Boulton | Corpora for all? Learning styles and data-driven learning[END_REF]Flowerdew, this volume).

Nevertheless, these limitations can easily be reduced, as suggested by several researchers, including the authors of the chapters in this volume. The technical challenges are addressed by a suggestion to replace students' access to corpus analysis software with printouts which still enable learners to deal with relevant language data for independent analysis but "takes the computer out of equation" [START_REF] Boulton | Data-driven learning: Taking the computer out of the equation[END_REF]; this goes right back to [START_REF] Johns | From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning[END_REF], and is reprised in Curado Fuentes (this volume). Other solutions advanced here include replacing standard corpus analysis software with purpose-built learning environments (cf. Chujo et al., this volume; Babych, this volume) or engaging learners in DDL through the use of websearch engines -tools which they most certainly are familiar with (Boulton, this volume).

The inappropriate nature of corpus data for lower-level students is also challenged by the papers in this book. The creation of level-suitable or simplified corpora can be one way of overcoming the obstacle (Chujo at al., this volume); another is building corpora containing text types and topics which are familiar to students from their first language, for example news items. In such scenarios even pre-intermediate learners can benefit from working with corpora (Curado Fuentes, this volume). In some special pedagogical contexts corpora can be used with learners who have very limited command of a new language, provided that they can draw on their knowledge of a cognate language (Babych, this volume).

The alleged divergence of communicative language teaching and data-driven learning has largely been invalidated over the years, by researchers both within DDL and further afield. Recent developments within the communicative approach have demonstrated that focus on form is indispensable in language learning, especially if learners are to reach higher levels of proficiency (Doughty & Williams 1998;[START_REF] Norris | Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis[END_REF]. However, form should always be studied in context so that students can associate it with meaning and use. This is exactly what DDL offers [START_REF] Gaskell | Can learners use concordance feedback for writing errors[END_REF]. For example, by studying words in concordance lines, learners can observe their characteristic phraseologies (Thomas, this volume) or pragmatic functions of frequent lexical bundles (Aston, this volume). Yet DDL activities can also help to develop language skills, one of the priorities of communicative language teaching, in particular writing (Charles, this volume) and reading (Babych, this volume; Curado Fuentes, this volume).

The cognitive demands of DDL can be solved by using corpus data more proactively, as suggested again by [START_REF] Johns | From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning[END_REF]. This means that corpus data can be used in a more deductive approach to teaching whereby corpus data is not used to elicit a rule or pattern but to exemplify it. Yet another way, proposed by Molés-Cases and Oster (this volume), is for students to work on highly structured and collaborative corpus-based tasks which can provide greater scaffolding and enable them to gain more confidence from working in a group.

However, it has to be acknowledged that problems with incorporating corpora into language instruction can originate in teachers themselves who frequently complain about a lack of guidance on how to take advantage of advances in corpus-based technologies [START_REF] Heather | Evaluating corpus literacy training for pre-service language teachers: Six case studies[END_REF]; Leńko-Szymańska 2014, forthcoming). As discussed above, an affordance of an object (a corpus in this case) is not inherent in the object itself, but depends on the perception and ability of the user. There has thus been a genuine need for publications reporting on various corpus-related projects around the world, promoting new corpus resources and tools, and demonstrating how to apply them not only in language instruction, but also in the professional training of applied language specialists such as translators (Marco & van Lawick, this volume; Sotelo, this volume).

One forum for promoting various affordances of DDL have been offered by TaLC conferences, which, after the success of the first meeting in Lancaster in 1994, have been taking place every other year, originally in the UK (Lancaster again in 1996, Oxford in 1998) and then in Western Europe (Graz, Austria, in 2000;Bertinoro, Italy, in 2002;Granada, Spain, in 2004;Paris, France, in 2006;and Lisbon, Portugal, in 2008). In 2010 TaLC moved further afield to Central Europe with the conference organised in Brno, Czech Republic; and the University of Warsaw, Poland, had the privilege to host the 10 th edition of the conference in 2012. Each of these conferences resulted in the publication of a volume of selected papers (Wichmann et al. 1997;Botley et al. 1996;Burnard & McEnery 2000;Kettemann & Marco 2002;Aston et al. 2004;Hidalgo et al. 2007;Kübler 2011;Frankenberg-Garcia et al. 2011;Thomas & Boulton 2012), each with important sections devoted to reporting a variety of DDL projects around the world.

The papers in this volume arise from the TaLC10 Conference in Warsaw; all those selected for publication here have a specific focus on one particular affordance of corpora in language pedagogy -data-driven learning. This focus on corpus use by learners intends to fill a gap in the literature, as identified for example in a recent review of a book in the same Studies in Corpus Linguistics series in which Cheong (2013: 2) regrets there are not "more articles focused on pedagogical implications of how to use corpus-informed research for classroom uses." By concentrating on learners' own use of corpus tools and techniques, and by providing accounts of the authors' first-hand experiences from such projects, our volume addresses the need for relevance to various language teaching practitioners.

The book comprises a collection of 12 main papers by corpus linguists and applied linguists, all with considerable experience in language and translation teaching and research in higher education. Its aim is to present recent advances in DDL, addressing issues involving different types of corpora, for different learner profiles (lower-level learners, specialist and nonspecialist students, etc.), in different ways for different purposes (exposing students to concordance lines in their native language, finding solutions to the cognitive and technical demands), and using a variety of different research methodologies and perspectives.

The first two chapters set the stage for more specific accounts of DDL projects covered in this book by discussing the theoretical underpinnings of corpus-driven language pedagogy and its effectiveness, as well as by presenting the development of DDL from its conception to the present date, discussing the range of current uses and possible future lines of development.

Lynne Flowerdew (Chapter 2) provides a theoretical foundation for the applications of corpus data described in the volume by discussing several language learning theories underpinning DDL: the noticing hypothesis, constructivist learning and Vygotskyan sociocultural theories -theories which, remarkably, are rarely discussed in depth or made explicit in accounts of DDL projects. Thus, this paper illustrates how the tenets of the three language learning theories support and are realized in DDL, drawing on examples of key studies in this field. Flowerdew reports on the small number of existing experimental investigations which address questions of the effectiveness and affordances of DDL by examining learners' performance (rather than attitudes or evaluations) in reference to these three theories, as well as in relation to the concept of learning styles. The conclusion she draws from the review of these studies is that while DDL shows promise, a larger number of large-scale and longitudinal studies are needed to confirm its beneficial effects.

In Chapter 3, Chris Tribble provides an overview of the history of the application of computers and corpora in language education from a personal perspective. He remarks on how the keystones in his career as an applied and corpus linguist were marked by rapid changes in technology and the growing availability and functionality of hardware, software and textual databases. The second part of the paper reports on the results of a questionnaire distributed on various Internet-based discussion lists in 2012 addressing the practices of language teachers in relation to DDL. The questionnaires provide information on the teachers' and their students' profiles, their reasons for using or not using DDL in their teaching, the resources and books used and found useful in the area, and respondents' perceptions of the benefits and disadvantages of DDL tools. Comparing the results with a similar questionnaire circulated four years earlier, Tribble concludes that the results are encouraging given the "emerging maturity of corpus-informed language teaching", and he offers predictions for future developments in this area.

Each of the 9 chapters in the main body of the book describes a project designed to place learners or teachers in direct contact with corpora, and encourage them to analyse corpus data for a variety of purposes. The papers are grouped into three sections: Corpora for Language Learning (Chapters 4, 5 and 6); Corpora for Skills Development (Chapters 7, 8 and 9); and Corpora for Translation Training (Chapters 10, 11 and 12).

Section I examines how corpora can help learners to improve their language knowledge, describing projects where learners exploit corpora as an aid to gaining better control over linguistic systems (phraseology and grammar of the target language). The focus of the first chapter in this section (Chapter 4) is on using DDL techniques for teaching foreign language phraseology to future interpreters. Guy Aston subscribes to the view that phraseology is key to fluency in both speech production and reception, particularly in cognitively and affectively demanding contexts like interpreting. The originality of this project lies in the use of a speech corpus for this purpose: a one million-word collection of talks from the TED -Ideas Worth Spreading website which aligns transcripts with audio/video files. The activities proposed in this chapter aim not only to enrich learners' limited repertoires of phraseological items, but also to improve their command of the associated lexicogrammatical, functional, and prosodic aspects -the last of these being impossible when exploiting transcriptions alone without access to audio material. The examples discussed in the chapter provide some ideas as to how phraseological items and their relevant linguistic properties can be identified, interpreted and made salient through DDL.

A similar claim about the importance of phraseology is made by James Thomas (Chapter 5), who advocates the use of full texts as a starting point for tackling phraseology. This allows learners to observe collocations associated with salient words in the text, and the typical meanings and uses of these collocations can be further explored in a corpus. However, Thomas argues that DDL cannot stop there as such activities do not guarantee active knowledge or appropriate use of observed collocations. He proposes Collocation Plus as a procedure whereby learners explore the lexical and grammatical environments of collocations in the contexts in which they meet them in corpora, and then record their findings in 'word templates' which will later be available for production. Thomas argues that this procedure will help learners produce more accurate, fluent, idiomatic and sophisticated language, whilst developing their autonomy in using available corpus resources and tools.

The last chapter of this section (Chapter 6) addresses the need for corpora and corpus tools which are suitable for low-proficiency learners of English as a foreign language (EFL). Kiyomi Chujo, Kathryn Oghigian and Shiro Akasegawa describe the creation of two new resources designed especially for such students in Japan in order to help them improve their knowledge of grammatical structures which they find particularly problematic. The first resource is a database of example sentences called the Sentence Corpus of Remedial English (SCoRE), which contains sentences featuring target grammatical structures. Although the sentences are specially written by the researchers, their creation was preceded by a careful analysis of lexical and structural characteristics of a 30-million-word corpus comprising various authentic educational materials relevant for lower-level learners. The second resource is a new DDL tool, the Grammatical Pattern Profiling System, which is a browsing program with a simple interface in which the results appear as complete sentences rather than in KWIC format. Teachers and students can search the database by grammatical category or target word to retrieve examples of a target structure from SCoRE. The authors hope that the browsing system will promote DDL among lower-level learners.

Section II concentrates on the development of language skills. The projects described here contain DDL activities whose aim is to give learners the opportunity of becoming more fluent language users by developing their skills in reading and writing.

Maggie Charles (Chapter 7) advocates the use of self-compiled specialized corpora by advanced learners of English for academic purposes (EAP). Her paper presents an approach to implementing DDL in a multidisciplinary EAP course with the goal of helping students improve their writing skills in their individual academic disciplines. In class students work with their own corpora of academic articles from their own disciplines, but they all complete the same teacher-guided tasks, designed to investigate key features of written academic discourse. The advantage of this approach is that students' discoveries of the features characteristic of their own disciplines are enhanced by the comparison with other students' discipline-specific results. The paper presents the use of several corpus tools (concordancer, collocation finder, wordlist and distribution plot) for exploiting EAP corpora and suggests that all have valuable pedagogical applications that go beyond the traditional KWIC format of DDL. The author concludes by suggesting that the opportunity to explore different sets of data and to compare the results brings an additional benefit to DDL.

In the case of cognate languages, DDL can help learners use an L2 as a springboard to an L3, as demonstrated by Svitlana Babych (Chapter 8) for reading skills. This chapter proposes a methodology which can help students develop top-down processing strategies, in particular the awareness of text organization, by focusing on textual connectors in the L1, L2 and L3. This methodology has been embedded in an on-line environment to support text comprehension in L3 Ukrainian and L2 Russian, which incorporates two elements. One is a comparable corpus of online newspaper articles in English, Russian and Ukrainian, providing a source of texts of similar type or genre for teaching reading. The other is a multilingual electronic thesaurus of textual connectors for the same three languages, systematically derived from large corpora and classified according to Babych's own semantic classification scheme. Babych discusses the ways in which the environment can be used for developing reading skills, and suggests that although this methodology was developed primarily for specific languages, it can be applied to the development of both reading and writing skills in any L2 as well as an L3 which is closely related to the L2.

Alejandro Curado Fuentes (Chapter 9) also describes an application of DDL to developing reading comprehension of news articles. However, this chapter focuses on exploiting news keywords in various DDL activities which aim to raise students' awareness of linguistic properties of the target words, such as parts of speech, collocations and colligations. This is based on the assumption that exploiting news keywords through various DDL tasks can aid learners in subsequent decoding of information in a text. Curado Fuentes conducts an experimental study with business English students at B1 level in which DDL is compared with more conventional paper-based activities. Pre-tests and post-tests were used to compare the progress of experimental and control groups, while direct observation, logs and questionnaires provide further insight. The results indicate that DDL activities focusing on news keywords work favourably for the students' comprehension of news articles.

Section III presents various ways in which corpora can be used in translation training. It describes data-driven tasks promoting general language awareness as well as skills in exploiting corpora as a reference resource, both of which are particularly relevant for future translators. It also demonstrates how corpora can help trainees to develop their professional expertise by giving access to multiple examples of various translation phenomena such as interference, as well as established conventions and practices.

Teresa Molés-Cases and Ulrike Oster (Chapter 10) observe that DDL has proved to be a valuable methodology not only in language learning and skills development but also in translator training. Their chapter reviews various applications of corpora as pedagogical resources in the translation teaching context. However, the authors remark that corpora are not always fully appreciated by translation trainees because they are considered excessively complicated in terms both of the abundance of linguistic information they provide and of their technical complexity. As one possible solution to this problem, the authors propose corpus-based webquests: this scaffolded learning task format, with links to various resources on the web, involves learners in the investigation of open-ended questions and provides the opportunity to share and discuss the results with their peers. The chapter presents some examples of corpus-based tasks promoting student-student collaboration and studentteacher interaction. It also offers a detailed account of two webquests designed for future translators, including the results of a short survey eliciting students' reactions to these tasks. The authors conclude that due to their accessibility, flexibility, and ease of use, webquests offer students a favourable context in which to access corpus-based resources and improve their linguistic and translation competence, and help to improve their autonomy and collaborative skills.

In Chapter 11, Josep Marco and Heike van Lawick describe the use of comparable corpora to enhance translator trainees' awareness of source text interference on their translations. The authors present a procedure in which learners study multiple Catalan translations of an English or German text produced by students from previous years. The trainees analyse these translations in relation to possible instances of interference from the source language that surface as unidiomatic or incorrect phrases. They verify their intuitions in a comparable corpus, which in this case refers to a collection of published texts in Catalan containing both non-translations and translations from English and German, Next, the students are provided with the original of the translations they have been analysing to establish whether the confirmed instances of interference may be justified by the stylistic profile of the source texts. The paper reports on a small-scale study measuring the impact of this procedure on the trainees' own translations. The results tentatively confirm the benefit of exposure to comparable corpora as a way of enhancing students' awareness of the dangers of source text interference.

Chapter 12 reports on a project in which a corpus is used to foster specific professional skills among translation trainees. Patricia Sotelo describes the creation of a multimedia corpus of subtitles containing several English-language films subtitled in Galician. The corpus comprises subtitles in both languages aligned with the video material. Such a rich resource gives trainees multiple opportunities to explore translation phenomena and conventions characteristic of subtitling that cannot be studied using more traditional corpora. Sotelo also presents a series of DDL activities carried out by translation students taking a course in audio-visual translation. While awaiting a formal study to evaluate the effect of using the corpus, based on her own observations and the students' comments on the activities the author concludes that the integration of corpus-driven tasks in the audio-visual translation classroom enhances students' professional competence as subtitlers.

The book ends with Chapter 13; this is set apart from the rest, as it suggests a form of DDL that builds bridges with existing practices of Internet use as a language learning or reference resource. Alex Boulton observes that the use of dedicated concordancers to explore corpora requires significant training in the use of technology, which can be discouraging for both teachers and learners. He proposes a way around this problem which equates the Internet with a corpus, and search engines such as Google with a concordancer. This allows him to compare DDL techniques with web searches, undoubtedly a very frequent practice but currently underreported in the L2 literature. Boulton points out that although web searches are not rigorous enough for much academic research, they may be sufficient or even preferable for pedagogical purposes. The chapter also demonstrates how the advanced options available in standard search engines can enhance web searches in order to make them more similar to corpus queries and to make their results more valuable for learners. Boulton reviews several previous studies which exploit web resources for data-driven pedagogy, and ends with the pragmatic and practical conclusion that the web and search engines may constitute one possible form of DDL, depending on local circumstances.

The research papers presented here are based on multiple resources: monolingual general and specialized corpora, speech corpora, multilingual (comparable and parallel) corpora, multimodal corpora, corpora of simplified texts, learner-compiled corpora and the web as a corpus. These text collections are exploited either with popular off-the shelf concordancers (WordSmith Tools, AntConc, Sketch Engine, and even Google), or with software tailored to the specific needs of a project. The chapters also describe other tools which foster the use of corpus data in the classroom, integrating DDL into a more comprehensive learning environment or via activities such as webquests.

The editors hope that such a diversity of projects involving so many different corpus resources and tools as well as the whole range of different instructional contexts will highlight the versatility of DDL in language pedagogy, and be an inspiration to the readers of this volume on how to incorporate DDL techniques in their classrooms and how the multiple affordances of language corpora can be of benefit to their students.

Agnieszka

  Leńko-Szymańska & Alex Boulton. (2015). Data-driven learning in language pedagogy. Introduction to A. Leńko-Szymańska & A. Boulton (eds), Multiple Affordances of Language Corpora for Data-driven Learning. Amsterdam: John Benjamins, p. 1-14.

This is a pre-publication version. For the version of record, please see https://benjamins.com/catalog/scl.69 (or email me at alex.boulton@atilf.fr)


Alex Boulton
email: boulton@atilf.fr

Hands-on / hands-off: Alternative approaches to data-driven learning

One of the most apparent obstacles to DDL is the use of the technology itself -the computer with its query software and interfaces for accessing electronic corpora. Where this is the case, the obvious question is whether the computer can be successfully removed from the equation without losing the benefits of the overall approach. In the present study, two groups of lower-intermediate level learners at an architectural college were introduced to an English language corpus both 'hands on' and 'hands off'. The learning outcomes of the two approaches were tested, and compared against feedback obtained at the start and again at the end of the course. The general conclusion is that paper-based DDL does represent a viable option: there is unlikely to be a single best version of DDL for all learners in all situations at all times, and variety and sensitivity to learner variation and local conditions is crucial.

B. USE THE ON-LINE CORPUS

Find concordances for the following verbs, and note frequent patterns of usage in the boxes.

advise:

recommend:

• Each verb in A has similar patterns to one verb in B -which corresponds to which?

Introduction

One striking feature of research into corpus use for language learning is the considerable variety of practices and techniques covered under the blanket heading of "data-driven learning" (DDL), a term borrowed from computer science by Tim [START_REF] Johns | From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning[END_REF]. The choice of term itself is not uncontroversial -possibly deliberately so, as the initial intention seems to have been to present the underlying concepts as "radical" (Johns 1988: 21), "innovative and possibly revolutionary" (Johns 1991: 27). This perhaps reflected an attempt to encourage people to question existing practice, and to return learner and language to centre stage, as in one of his most quoted phrases: "What distinguishes the DDL approach is the attempt to cut out the middleman as far as possible and to give the learner direct access to the data" (Johns 1991: 30). There is some indication that Johns (2002: 107) may have regretted the expression later on as DDL became, in the minds of some people, associated exclusively with its most extreme form of hands-on, autonomous, serendipitous corpus exploration, which may be difficult to implement in "the reality of ELT classrooms" (Mukherjee 2006: 14). Further, the radical label may even have contributed to a lack of takeup among a wider community of language teachers and learners (Boulton forthcoming). To adopt Römer's (2010: 28) analogy of the "corpus mission", to win new converts there may therefore be an argument for presenting DDL as "ordinary practice" (Boulton 2010c: 20). Johns (2002: 107) shows how DDL grew from his long-standing interest in "the recovery of linguistic information from authentic text", and how it builds on many existing practices and recommendations for language learning -the use of authentic language in a learner-centred approach with emphasis on induction, noticing, consciousness raising, and so on (Boulton 2009b). None of these elements is exclusive to DDL -indeed, it is difficult to pinpoint any single feature that is necessary or sufficient to define DDL (Boulton forthcoming).

While Johns was responsible for highlighting the potentially radical nature of DDL, his own "experience in using concordance data reactively […] indicated that it could be used proactively also in a more traditional teacher-centred setting " (1991: 31). His use of generic materials in the form of print-outs of corpus data accompanied by detailed instructions and guidance from the teacher meant that DDL could be introduced to learners at lower levels of language proficiency and with minimal training, with immediate pay-offs. Paper-based materials of course do not have the full potential of hands-on corpus work, though it should be noted that hands-on corpus consultation does not necessarily live up to its alleged openness: the teacher may decide the points to cover, devising step-by-step tasks with set queries leading to pre-established outcomes, and monitoring performance at all stages. However, they do have a number of advantages which suggest they may be useful in some contexts at least, which is why Johns (1991: 36) even wanted to publish his in-house DDL materials on paper for a wider audience.

First, extensive training may be required to reap the full benefits of hands-on concordancing, a drain on time that can be difficult to justify in an already full syllabus [START_REF] Turnbull | Towards independent concordance work for students: Lessons from a case study[END_REF]. The motivating factor of ICT is frequently overstated in education [START_REF] Jarvis | Investigating the classroom applications of computers on EFL courses at higher education institutions in UK[END_REF], and the technology can be unappealing for many learners [START_REF] Bernardini | Exploring new directions for discovery learning[END_REF] as well as teachers [START_REF] Farr | Evaluating the use of corpus-based instruction in a language teacher education context: Perspectives from the users[END_REF]. Even if suitably equipped computers are available on a regular basis, lack of technical expertise and back-up can lead to considerable waste of valuable classroom time, quite apart from time lost in changing rooms and logging on, finding passwords, struggling with the interface and query syntax, conducting inappropriate searches, misinterpreting data, and so on. Whether such difficulties are real or only presumed, they can be off-putting in many cases for learners and teachers having to come to terms with "new material (the corpora), new technology (the software), and a new approach (DDL) all at once" (Boulton 2010a: 539). In sum, the computer is one of the most significant barriers to wider uptake of DDL (e.g. [START_REF] Yoon | ESL student attitudes toward corpus use in L2[END_REF].

Use of prepared materials allows the teacher to decide the points to cover, conduct the searches in advance, select and sort small amounts of data appropriately, and present them in controlled activities tailored to the learners' needs and abilities. The print medium can use familiar activity types (gap-fills, matching, etc.), thus presenting a more realistic goal in the short term, and may therefore be more motivating than hands-on work, as well as more appropriate for some learning styles. Paper-based materials can be more efficient as much of the background 'noise' is weeded out in advance, so it should be possible to use them immediately with little if any prior training. Although the teacher is still present, this is not necessarily a bad thing in all contexts, especially for novice corpus users (see Boulton 2010c for discussion). Printed materials can thus provide a convenient way in to DDL, and the scaffolding "can be gradually reduced until students can be presented with concordance output to investigate independently and unaided" (Johns et al. 2008: 495). This is not to say that printed DDL materials are 'better' than hands-on corpus consultation; inevitably, they do not allow the full potential in terms of autonomy, flexibility, individualisation, life-long learning, and so on. These arguments are put forward here not to promote paper-based DDL over hands-on consultation, but to redress the balance as the former is something of a poor relation -at least as far as empirical research is concerned. In a recent survey of 80 evaluations of DDL, Boulton (2010b, web supplement) found less than a fifth using printed materials, as researchers inevitably attempt to push the boundaries as far as they can, favouring computer technology over pencil and paper.

Learners can also extract significant benefit from concordances, whether for the presentation of new items [START_REF] Boulton | Looking for empirical evidence of data-driven learning at lower levels[END_REF], in test situations [START_REF] Stevens | Concordance-based vocabulary exercises: A viable alternative to gap-filling[END_REF] or for longer term recall (Boulton 2009a). When paper-based DDL is compared against a control group, [START_REF] Allan | Data-driven learning and vocabulary: Investigating the use of concordances with advanced learners of English[END_REF] and [START_REF] Koosha | Data-driven learning and teaching collocation of prepositions: The case of Iranian EFL adult learners[END_REF] report statistically greater gains for DDL. [START_REF] Sripicharn | Evaluating classroom concordancing: The use of corpus-based materials by a group of Thai students[END_REF], [START_REF] Tian | The impact of learning tasks and learner proficiency on the effectiveness of data-driven learning[END_REF] and Boulton (2010a[START_REF] Boulton | But where's the proof? The need for empirical evidence for data-driven learning[END_REF] also give the advantage to DDL using printed materials, though the differences are not significant in these cases. Where reported, learner feedback to paper-based DDL is generally favourable in this and other research (e.g. [START_REF] Johns | Data-driven learning: The perpetual challenge[END_REF][START_REF] Hadley | Sensing the winds of change: An introduction to data-driven learning[END_REF][START_REF] Ciesielska-Ciupek | Teaching with the internet and corpus materials: Preparation of the ELT materials using the internet and corpus resources[END_REF]. The evidence from most individual studies might seem somewhat mitigated, potentially providing ammunition to sceptics. But there are two main reasons for optimism. First, individual studies should be treated with caution: what counts is the overall picture. Inevitably, limited data sets achieve significance with difficulty (cf. the current debate on complex / dynamic systems, e.g. [START_REF] De Bot | Introduction: Second language development as a dynamic process[END_REF][START_REF] Larsen-Freeman | Complex Systems and Applied Linguistics[END_REF], but if a meta-analysis were possible, it seems highly probable that the data would go well beyond the usual levels of acceptability (Boulton 2010b). Second, the objective is not necessarily to show that DDL is better than traditional techniques and methodologies that have received unparalleled research and practical development in classrooms around the world. Rather, it is to show that DDL can provide a complement in some cases. In other words, it is enough to show that DDL is as good as traditional treatment; if this is the case, then individual differences will inevitably mean that it is more appropriate for at least some learners some of the time.

The original motivation for the present study stems specifically from a pair of previous experiments with learners in the same institution, in one case using paper-based materials, in the other hands-on corpus consultation (Boulton 2010a(Boulton , 2009c respectively) respectively). Feedback suggested considerable enthusiasm for the former, less for the latter, though the differences may be attributed to a number of other factors (especially a novelty effect). Such comparisons of two different studies out of context can be misleading, hence the need for a controlled experiment setting up the two paradigms in parallel, something no study has yet attempted explicitly. This paper describes an experiment where learners had experience of paper and computer-based DDL over an extended period; it looks at learning outcomes from the final activity, and relates this to the learners' appreciation of the activities, as well as proficiency levels. The main research questions are thus:

• Is there a difference in learning outcomes following DDL on paper and on computer?

• Do learners prefer DDL on paper or on computer?

• Do the outcomes or the preferences relate to general levels of language proficiency? • How do learners' reactions to DDL change over a 10-week course?

Method

The underlying rationale was for two comparable groups of learners to experience DDL both on computer ('hands-on') and on paper ('hands-off') on alternate weeks for an extended period at the end of each class. In this way, all learners would be familiar with both approaches and in a position to compare them, and would cover the same language points for the final test.

Two groups of learners had 20 English classes in the year of 90 minutes each, most of which time was devoted to covering the syllabus common to all groups, and due to be tested at the end of each semester. Ten of the sessions also involved a short DDL activity for approximately 20 minutes at the end of the class, focusing on language areas the students had found difficult during an earlier in-house practice TOEIC. In the first week, the two groups completed the same activity featuring both paper-based and computer-based corpus work; subsequently they alternated between the two approaches. In this way, each group experienced both types of presentation and covered the same language points. The activities themselves were typical of those discussed in DDL research, involving induction from authentic concordances, sorting and interpreting data, testing and matching rules, and so on. The activities allowed learners to experience a range of typical DDL tasks: amongst other things, printed concordances featured matching and gap-fill exercises, while computer searches became rather freer over the course. For example, one seasonal activity asked learners to compare differences between Christmas and Xmas, find compound nouns including Christmas and New Year, and find adjectives for each. The students were then encouraged to extend their searches to other American national holidays, and discuss their findings in relation to customs in France.

Learning outcomes comparing paper-based and computer-based DDL were assessed for the final session only; here it was necessary to harmonise the activities to reduce variables and ensure comparability of the two situations. The Appendix gives the worksheet used for the final test session, and may be taken as indicative of some of the activities used throughout the course.

Population

The participants were all students in their second year of study at a French architectural college. The 108 students enrolled were assigned to five classes on a basis unrelated to their ability in English; the present data are taken from the 49 students in the two classes outlined below. Attendance is nominally compulsory but highly variable, and dependent on commitments in other courses which require projects to be submitted at frequent intervals. On average, 13 students out of 22 were present in Group A each week, 19 out of 27 in Group B. Data from 5 of the 49 students enrolled were rejected as they participated in less than 5 of the 10 sessions involving DDL. The final data set concerns 28 students for the DDL test, 38 for the first questionnaire, 36 for the second. Of the 40 who participated in at least one of these data sets, 27 were female and 13 male, the average age was 20 years, and the mother tongue French with the exception of two Arabic speakers and one Bulgarian.

English is a required course for these students, with marks each semester counting towards their degree. Following common practice in France, graduation from the school is subject to an attested level of competence in a foreign language, equivalent to B2 on the Common European Framework of Reference for Languages (Council of Europe 2001). External certification, typically from the TOEIC 1 , is required for students to graduate. The participants had been studying English for an average of 7½ years, but the level of proficiency is not high: on a start-of-year TOEIC conducted in-house, the mean scores were around 500-550 points on the TOEIC, equivalent to A2 or B1 on the CEFR. These relatively low levels of proficiency might partly be explained through lack of motivation for foreign languages, with the English classes often perceived at best as an entertaining interlude, at worst as a distraction from their main academic interests and concerns as they relate to architecture.

Corpus

All corpus data was taken from the Corpus of Contemporary American English (CoCA), currently standing at over 400 million words (see [START_REF] Davies | The 385+ million word Corpus of Contemporary American English (1990-2008+): Design, architecture, and linguistic insights[END_REF]. This corpus was chosen partly for practical reasons, especially that it is stable, fast and reliable, available free on line with no download (students registered for a free password for the hands-on sessions), and the interface is sufficiently clear and intuitive for basic functions not to require substantial training. A further consideration is that simple, free, on-line tools and corpora allow the students to work on their own outside class if they wish, or even after the end of the course, and can be used immediately by other teachers -essential if DDL is to reach out to a wider audience. It is not possible here to give a full run-down of all the features offered on the BYU site,2 but various activities required the students to use wild cards and part-of-speech tags, to look at frequencies and word lists, to create collocates lists and interpret random samples of concordances, to search by register, use the 'chart' function, and compare usage in various ways.

Data collection and test instruments

The data were collected from: a) a test on the target language items of the final session to compare learning outcomes from paper-based and computer-based DDL activities; b) a questionnaire completed after the first and final sessions to gain feedback on learners' reactions to DDL.

Test of learning outcomes

The language point in the final session concerned four verbs largely associated with two distinct types of verb structure: 'infinitive' and 'subjunctive' (Biber et al. 1999: §9.4, §3.9 respectively), and can be summarised traditionally as:

• infinitive:

encourage someone to do something advise someone to do something • subjunctive: recommend someone do something demand someone do something

These verbs and structures were chosen not for their communicative importance, but because they are frequently tested in the TOEIC, and correspond to questions which had presented difficulty for these students in the start-of-year TOEIC practice. Although these were the main target structures, other types of structure associated with the verbs were not neglected, as can be seen in the worksheet provided in the Appendix. The materials themselves were devised to be representative of much current DDL practice, albeit in a fairly controlled fashion -a 'soft' version, to borrow Gabrielatos' (2005) term. Following a short introduction and an example using a different verb, students worked in pairs, using printed concordances to find the patterns associated with the first two verbs, then working on-line for the second two, and finally comparing the two verbs they had in each phase to detect similarities and differences. The teacher monitored work in progress, directing students' attention where relevant and, in the hands-on phase, occasionally making suggestions to help with technical or search difficulties. Once the students had completed the tasks, whole-group feedback was instigated to ensure that all students came away with essentially the same information -the key point being that they had worked it out for themselves. The procedure was thus essentially the same for the data on paper and on computer, and each group did both types of activity for both main types of verb structure. In each case, the paper-based work was done first for practical reasons, which if anything should have given the advantage to the computer-based work as students would already have become familiar with the procedure and solved some problems in the previous phase.

The test instrument was presented as TOIEC practice to be completed at the end of the learning period, and lasted about 15 minutes (see Figure 1 for examples). It consisted of 20 questions based on clear contexts taken from elsewhere in the corpus (i.e. authentic contexts similar to those worked on, though here featuring only full sentences). There were five forced-choice questions for each verb covered, in standard single-sentence gap-fill format. This has the advantage of being familiar to students through years of practice (the exact format was based on the TOEIC Part 5 -incomplete sentences), and ensured only one 'correct' response per item to produce quantifiable data. The results can thus be sorted according to treatment, to verb type, and to group.

Feedback questionnaires

In addition to the test for learning outcomes, a questionnaire was completed after the first and the last DDL sessions to obtain feedback from the learners. The questionnaires were short and simple in the learners' mother tongue, requiring responses on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), with the opportunity to provide extra information where they desired, although this opportunity was rarely taken up. The two questionnaires were essentially identical, with minimal changes in form. The first five questions sought to measure the learners general 'receptivity' to DDL as a whole; further questions looked at their perceptions of potential applications of corpora and how they might be used. In addition, the second questionnaire also sought to compare the students' reactions to the use of paper-and computer-based DDL.

Results and discussion

Tests

It is important to remember that the objective here is not to show that DDL 'works' (as there is no pre-test and no control group), but merely to compare paper-and computer-based use of corpus data. Table 2 gives the mean scores for each group on the test of learning outcomes from DDL: 10 paper-based and 10 corpus-based items; 10 'infinitive' and 10 'subjunctive' structures. Although there is no a priori reason to expect one group to perform better than the other (as both experienced both types of treatment), Group A does score higher than Group B, but the result is not significant (p>.05); see Table 3. There was a statistically significant difference between the two types of structures, with the infinitive structures being used more correctly than the subjunctive structures (p<.0001). The crucial comparison is between paper-based and computer-based treatments: here the former resulted in slightly better results for both groups, but the difference is not significant (p>.05). These results support the original hypothesis that using prepared corpus data in printed materials can be as effective as hands-on concordancing, at least for short-term learning outcomes with students such as these -i.e. with comparatively low levels of proficiency and motivation following several short sessions of corpus work, both on paper and on computer.

It is possible that the lack of difference between the paper-based and computer-based treatments might be in part a factor of the experiment design itself. For the two presentations to be comparable, it was necessary to reduce the potential variables involved, which in turn reduces the likelihood of finding a significant difference. This is not a new dilemma, as reported for example in the work by [START_REF] Sripicharn | Evaluating classroom concordancing: The use of corpus-based materials by a group of Thai students[END_REF] in another DDL experiment. Thus both paper-based and computer-based presentations involved essentially the same instructions and procedures, the main difference being that work on computer also required the students to formulate the queries themselves, to sort the results and analyse more data than for the paper-based activity. The learners therefore have more to do in this case, which may reduce their chances of success in the short-term.

On the other hand, they also have more opportunity to adapt the task as they see fit, appropriating it for their own ends: they can by-pass obvious things to focus on language that is unclear to them, adapt the queries appropriately, refine and extend them as necessary, select data which are relevant to them, and see more contexts than were available in the printed concordances. All of these things are more typical of hands-on concordancing, and although the learners had become used to conducting such procedures themselves over the course, this was not enough to give them the advantage. The difference between hands-on and hands-off corpus work is about more than merely the medium of delivery. Only computer-based work can make the most of long-term benefits such as noticing, language awareness and autonomy. If the results reported here are any indication, a 10-week period is not sufficient for such advantages to accrue, and support the use of paper-based materials in the short term.

DDL questionnaires

Though 36 and 38 students responded to the two DDL questionnaires, only 32 completed both due to the inevitable absences. The data in Table 4 show that the students as a body consistently responded favourably (i.e. attributing a score of at least 3 out of 5) to all items in both questionnaires. Inevitably the average tendencies conceal a certain amount of variation. This is particularly relevant among the first five questions (whether corpora are easy, useful and interesting to use, and whether the respondents felt they learned something and would like to do more such activities in class), which may be taken as some indication of the students' general 'receptivity' to DDL. For these questions, five students were responsible for the eight lowest scores of 1 out of 5 in the second questionnaire, while 14 of them gave the highest score of 5 out of 5 at least once. This suggests that there are some strong opinions, though most responses are of course more neutral, as indicated by the relatively low standard deviations. To borrow Kaszubski's (2008: 174) terminology, there are some "adopters" and some "refusers" but most rate the various functions in the middle, somewhere above the mean possible score. Moreover, the adopters in the first questionnaire tended to remain positive in the second, and vice versa, as Pearson's correlation coefficient for each student's average receptivity between the two questionnaires is very high (R=.61). Overall, however, the average scores for general receptivity to DDL decreased (from 3.77 to 3.46), suggesting a drop over the course; moreover, this decrease is statistically significant (p<.0001, see Table 5). Yet this is perhaps not unduly worrying. First, though significant, the decrease is relatively small (-.31). Secondly, it is perhaps to be expected, as the first questionnaires likely reflected a certain 'novelty effect' which would inevitably diminish over time. Given learners whose main motivation is not English, anything new may seem desirable; after a period of time, the new activities are no longer new, and are thus by definition less desirable -clearly this is not specific to DDL. However, it could also reflect a more realistic appreciation of what corpora can bring to the learning process, or a general perception that the work is particularly mechanical or laborious, as other researchers have found (e.g. [START_REF] Chambers | Integrating corpus consultation in language studies[END_REF]) -especially in the present context where these non-specialist learners expect to be entertained during their English courses. The final pair of items, possible in the second questionnaire only, asked students for their opinions on the crucial issue of using computer-based and paper-based materials. As the results show, these students do have a preference for working on computer (3.28 vs. 3.08), though the scores are high (interestingly, their 'liking' for each type of activity is lower than their overall 'interest', gauged at 3.36 in question 3 of this questionnaire), and the difference not statistically significant (p=.36). The question did not explicitly ask students to say if they preferred one to the other, and they could equally like (or dislike) both. However, the correlation coefficient is not particularly high (R=.18) and not statistically significant (p=.28), suggesting that their feelings towards the two styles of presentation are independent: some prefer working on paper, others on computer.

Relating the data

Both test results and representations are interesting in their own right, but it might also be revealing to relate them to each other. Table 6 shows the correlation between the students' general receptivity to DDL (as measured by their average responses to questions 1 -5 in the second questionnaire), and the outcomes of the DDL test (following the computer and paper-based presentations, and the two combined). The correlations do suggest that, overall, the more receptive a student is to DDL, the more likely he or she is to perform well, but the results are not statistically significant. It is also possible to compare preferences and outcomes for using corpora on paper or on computer, but here Table 7 shows negative correlations all round. The more they like doing one type of DDL activity (on computer or on paper), the less well they actually do on the other. This seems to make sense, except that one would also expect them to do better on paper if they prefer paper, and vice versa, an expectation that is not borne out by the data. The more they like doing paper-based DDL activities in particular (where the correlation reaches significance), the lower their learning outcomes. Finally, the data presented in Table 8 relate the learning outcomes (for paper-and computer-based treatments, as well as overall) to three possible measures of the learners' level of proficiency in English: a) their scores in the start-of-year TOEIC; b) their marks in the end-of-year exam based on the syllabus common to the entire cohort; c) the number of years of study (data available for 26 students in all cases). It should be remembered that the population as a whole is of roughly lower-intermediate level, ranging from 'elementary' to 'intermediate' only.

The only significant results are for the paper-based treatment, which correlate positively with two of the three indicators of proficiency. One way of looking at this is that the more advanced students have achieved their higher level of proficiency from their previous experience of language teaching, which tends to be fairly close to the traditional knowledge transmission model in France (cf. [START_REF] Brown | Language learner motivation and the role of choice in ESP listening engagement[END_REF]. It might then seem that paper-based DDL corresponds more closely to this, and offers a way in to DDL which does not represent an unsettling break with the past. On the other hand, there is no significant correlation between proficiency and the computer-based activities, nor with learning outcomes taken as a whole. This suggests that DDL as an approach can be useful to learners at different levels of proficiency within the band represented here, and that a hands-on approach in particular may open it up to lower-level learners. This result may contradict received wisdom whereby only advanced learners can be entrusted with hands-on corpus work (e.g. [START_REF] Granath | Who benefits from learning how to use corpora?[END_REF]), but is in line with [START_REF] Yoon | ESL student attitudes toward corpus use in L2[END_REF], who also found their intermediate group more favourably disposed to DDL than the advanced group.

Conclusion

Feedback from the learners suggests the majority are generally favourable to the DDL approach as a whole, with a sizeable minority very receptive, and only a small number overtly unconvinced. Such a distribution seems inevitable: few activities or approaches are likely to appeal to all learners, and appeasing the refusers would mean rejecting virtually any novel practice. Receptivity to DDL also dropped slightly over the duration of the course, but not significantly. Again this is to be expected for any new practice given the present context where the learners are not particularly motivated to L2 work as a whole, and any novelty appeal will diminish over time.

The main objective was to see how a DDL approach using paper-based materials can fare when compared to hands-on concordancing. The results show a slight advantage for printed materials in terms of learning outcomes, though the difference is not significant. The implication is that printed materials do have a role to play, especially in producing immediate results with learners at lower levels of proficiency and with limited time available for training. Further, highly controlled activities on paper also provide a gentle lead-in to more open-ended individual work (cf. Gabrielatos 2005), a claim which is supported by the present study. First, the learners' level of language ability was found to be positively correlated with outcomes from the paper-based treatment; this suggests that these materials are more familiar and compatible with their prior learning experiences, and useful as a first step. On the other hand, there was no correlation between language proficiency and hands-on concordancing, which leaves open the possibility that computer-based corpus consultation may be accessible to all at a later stage. Additionally, the learners had a slight preference for computer-based work overall, which is encouraging in this respect.

The claim is not that paper-based DDL is 'better' than working directly on an electronic corpus, especially in the long term. The most important conclusion is that prepared, controlled DDL activities should not automatically be ruled out: they represent one option which may be applied in different ways according to local conditions. Various factors need to be taken into account -practical and logistical (e.g. the accessibility and reliability of computer rooms and support staff), cultural and individual (especially in terms of learning styles and expectations from past experience), as well as pedagogical. Both hands-on and paper-based DDL have their own advantages and limitations; each may be more appropriate for some learners, teachers or contexts than the other.

Figure

  Figure 1. Examples of test items 1. The U.N. Security Council has _____ all enrichment activity because of suspicions the program's aim is to make weapons. (A) demanded that Iran stops (B) demanded that Iran to stop (C) demanded that Iran stop (D) demanded Iran to stop

  Table 1 outlines the basic lesson plan for the final session, showing how each group used both techniques and covered all four verbs.

	Table 1. Final lesson plan
		introduction	phase 1	phase 2	reflection	feedback
			paper-based	computer-based
			DDL	DDL
	Gp A Gp B	introductory examples	encourage demand advise recommend	advise recommend encourage demand	pair the verbs according to typical patterns	whole-class, student-initiated

1. Examples of test items

	1. The U.N. Security Council has _____ all enrichment	2. Mubarak also used his visit yesterday to
	activity because of suspicions the program's aim is to	_____ out of Iraq.
	make weapons.
	(A) demanded that Iran stops	(A) encourage the United States get
	(B) demanded that Iran to stop	(B) encourage the United States to get
	(C) demanded that Iran stop	(C) encourage that the United States get
	(D) demanded Iran to stop	(D) encourage that the United States to get

Table 2 . Learning outcomes

 2

		Group A	Group B	ALL
		(n=11)	(n=17)	(n=28)
	paper treatment	5.73	5.29	5.46
	computer treatment	5.55	5.18	5.32
	infinitive structures	7.27	6.24	6.64
	subjunctive structures	4.00	4.24	4.14
	MEAN	11.27	10.47	10.79

Table 3 . Comparative analysis of different data types

 3

		paper vs.	infinitive vs.	group A vs.
		computer a	subjunctive b	group B c
		Gp A Gp B	Gp A Gp B	Gp A Gp B
	Mean	2.73	2.66	3.32	2.07	2.82	2.62
	SD	1.41	1.64	1.32	1.46	1.5	1.55
	N	56	56	56	56	44	68
	a) two-tailed paired t-test:	p=.7421 (t=.33; df=55; SE=.22)
	b) two-tailed paired t-test:	p<.0001 (t=5.7; df=55; SE=.22)
	c) two-tailed unpaired t-test: p=.4988 (t=.68; df=110; SE=.30)

Table 4 .

 4

	Questionnaire responses
		Q'aire 1	Q'aire 2
	A. General receptivity to corpus use	mean SD mean SD
	1. I think corpora [will be / are] easy to use.	3.55 1.03 3.69 1.04
	2. I think corpora [will be / are] useful.	3.87 0.70 3.50 0.85
	3. I think corpus work [will be / is] interesting.	3.87 0.88 3.36 0.87
	4. I think I [will learn / learned] things from corpora.	3.89 0.73 3.69 0.86
	5. I would like to do other corpus activities in class.	3.66 0.94 3.06 0.98
	MEAN: 3.77 0.86 3.46 0.94
	B. Using corpora
	6a. I liked doing corpus activities on computer.		3.28 1.06
	6b. I liked doing corpus activities on paper.		3.08 0.91

Table 5 . Changing attitudes to corpus use (questions

 5

	1-5)

Table 6 . Learning outcomes and general receptivity

 6

			receptivity
			to DDL
		computer	0.20
	outcomes	paper	0.26
		ALL	0.27

Table 7 . Learning outcomes and learning preferences

 7

			preferences
			paper	computer
	outcomes	computer paper	-0.27 -0.51 a	-0.24 -0.36
	a: two-tailed t-test: p=0.0216 (t=2.5; df=18)

Table 8 . Learning outcomes and proficiency

 8

				proficiency
			TOEIC	EOY	years
		computer	-0.01	-0.09	0.02
	outcomes	paper	0.55 a	0.37 b	0.19
		combined	-0.32	-0.16	0.13
	a: two-tailed t-test: p=0.0024 (t=3.36; df=26)
	b: two-tailed t-test: p=0.0263 (t=2.03; df=26)

This is a pre-publication version. For the version of record, please email me at alex.boulton@atilf.fr.

The TOEIC (Test of English for International Communication) consists of

multiple choice questions (100 listening, 100 reading), with brute scores converted to points out of 990 under guarded formulae which differ from test to test. See http://www.ets.org/toeic.

The reader is referred to the 'more information' documents on the CoCA website: http://www.americancorpus.org/.

 Boulton (eds), Input, Process and Product: Developments in Teaching and Language Corpora. Brno: Masaryk University Press, p. 152-168.

Appendix

Final DDL activity: paper and computer-based worksheets for verb structures NB This worksheet is for Group A only; Group B had essentially the same worksheet and tasks but alternated the treatment for each pair of items.

CORPORA 10 -VERB STRUCTURES Verbs occur in different structures in French, e.g.:

• demander à quelqu'un de faire quelque chose on his armor-toed boot, and he caught her by the hand to stop her falling. She thought once more of the golden retriever pup, who had d.

. " In some places, " Ms. Raje said, " women were stopped from going to the film. " # Ms. Raje laments the rigidity of popular e.

# The day Topsy died was a cold, cold day, which didn't stop the people from coming out to watch. Electricity was a novelty, and movies

The main patterns of usage here are:

• to stop someone (from) doing something • to be stopped from doing something

A. USE THE PRINTED CONCORDANCES BELOW

Examine the 5 concordance lines below for each verb, and note patterns of usage in the boxes. encourage: demand: