Alex Boulton
email: alex.boulton@atilf.fr

Pascual Pérez-Paredes

Researching New Uses of Corpora

Obviously these applications were indebted to lexicographical work carried out by linguists with an interest in what we call today corpus linguistics. However, a separate current emerged in the 1980s where language teachers, often close to a research team working with corpus data, saw a connection between research and teaching and began tentatively to explore how corpora could be of more direct benefit to their learners. Such applications are closely associated with the work of the late Tim Johns, but the first academic publication seems to be from [START_REF] Mckay | Teaching the syntactic, semantic and pragmatic dimensions of verbs[END_REF] in San Francisco, whose students explored printed corpus materials to observe verb patterns in context. [START_REF] Ahmad | Using computers with advanced language learners: An example[END_REF] took this further, describing how their students benefited from direct access to an electronic corpus to pursue their own questions -a surprisingly ambitious approach given the capabilities of computers, corpora and software at the time. This is the scene to which [START_REF] Johns | From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning[END_REF] introduced the term data-driven learning or DDL (see also Johns & King 1991). It is the same scene which saw the founding of the biennial Teaching and Language Corpora (TaLC) conference series in 1994, each giving rise to a high-quality volume of selected papers (Wilson & McEnery, 1994;Wichmann et al., 1997;Botley et al., 1996;Burnard & McEnery, 2000;Kettemann & Marko, 2002;Aston et al., 2004;Hidalgo et al., 2007;Kübler, 2011;Frankenberg-Garcia et al., 2011;Thomas & Boulton, 2012;Leńko-Szymańska & Boulton, forthcoming).

The first TaLC Conference and accompanying publications not only brought together researchers and teachers interested in the pedagogic applications of corpora, but also linguists who were already familiar with language description and corpus methodology. They outlined the main reasons why corpora, and corpus-derived data, were already perceived as an asset in language teaching and learning all those years ago: (1) computers and storage at the time were improving dramatically; (2) there was a new interest in authentic data and usage in language education; and (3) there was a consensus that learners were adopting new, more active roles in their learning process. These three aspects have remained at the heart of the academic debate and the professional practice of teachers using corpora ever since. They also established much of the research and teaching agenda of the next two decades, including direct and indirect uses of corpora for teaching, learning, testing and as a reference resource, the suitability of existing corpora for pedagogical purposes, the development of LSP corpora, multilingual corpora, parallel corpora, learner corpora, ad hoc disposable corpora, and more user-friendly tools for learners and teachers alike. This special issue underlines the continued relevance of these issues while also showcasing new developments in researching uses of corpora.

Though the field has yet to reach full maturity (Thomas & Boulton, 2012: 8), there is today a tradition of research and practice which provides a reassuring background to uses of corpora in language teaching and learning. Researchers feel less need to explain the field and justify the benefits of corpora for pedagogical purposes, and are broadening their interests by applying research methodologies already in place in applied linguistics, particularly in second language acquisition (SLA) and fields such as error analysis (e.g. Granger et al., 2002). The focus is switching from corpus linguistics to language pedagogy; in other words, rather than trying to bring corpus linguistics into the language classroom, the emphasis is increasingly placed on the L2 user and how he or she might benefit from corpus linguistic tools and techniques. Much of the impetus for this comes from applied linguists and language teachers, as opposed to specialists in corpus linguistics; to paraphrase [START_REF] Widdowson | On the limitations of linguistics applied[END_REF], we might thus talk of applied corpus linguistics rather than corpus linguistics applied. This change can be seen partly as greater attention is accorded to the learner rather than the tools and techniques themselves; emic studies, for example, have highlighted a number of reservations (e.g. in deciding points to develop, formulating appropriate queries, and interpreting output in the unusual form of truncated KWIC concordances). Other work has focused on developing more pedagogically relevant corpora and software, as new generations of learners became more computer-literate too.

One alleged problem is the repeated allegation of a dearth of empirical studies in DDL and associated areas. The first review of empirical studies was provided by Chambers (2007a), who concluded that researchers and teachers typically overlapped their roles, that empirical research was mostly small-scale and quantitative, and confined to the university context. Though today it is possible to collate a hundred or more academic papers which seek to evaluate some aspect of corpus use for L2 users [START_REF] Boulton | Learning outcomes from corpus consultation[END_REF], it is clear that more diverse and rigorous research designs are needed to focus on the complex phenomena covered. As Pérez-Paredes (2010: 54) puts it: "Can we really claim that direct transfers from [linguistic] research will be successfully implemented in a learning environment?" Our intention for this volume was therefore not to repeat a previous special issue of ReCALL (Integrating corpora in language learning and teaching) edited by Angela Chambers (2007b), but to focus specifically on empirical research and evaluation in DDL-like approaches. This special issue is rather longer than usual, reflecting the high quality of papers received. The order of presentation is inevitably difficult, but generally goes from the more experimental to the more observational, from short-term to longer term, from etic to emic, from controlled to open, with each study bringing something new to the table. Many of the papers in this volume continue existing research traditions in DDL, with much of the work at university level among relatively advanced learners for writing purposes. However, some are conducted in less well-researched areas in secondary schools, at lower levels of proficiency, or for speaking purposes. Some of the papers are rigorous experimental / laboratory studies lasting just a few minutes (including one semi-replication study), generally on a specific (lexicogrammar) language point, while others are more ecological and cover an entire semester with a much more open language focus; these are generally backed up by questionnaires and other instruments to gather feedback from participants. Much use is made of control and experimental group comparisons to explore corpus use vs. no treatment or traditional treatment (teaching, dictionary use, etc.), or variations of DDL (e.g. inductive vs. deductive; for comprehension vs. production); several also feature delayed post-tests in addition to the more common immediate post-tests. The tests themselves may be highly controlled, but several look at more open-ended language use, especially in the form of writing. The corpora range from the large, publically available British National Corpus (BNC) or Corpus of Contemporary American English (COCA) to smaller, purpose-built corpora; some involve the learners' own productions, and are used in quite different ways, from learning aids to reference tools.

Ana Frankenberg-Garcia provides an extension or semi-replication (rather than a single-variable replication) of an earlier study comparing single and multiple concordance lines and dictionary definitions as a language reference resource. One crucial difference is that the present paper deals with secondary-school students, a hugely under-researched population in DDL work. Students were randomly assigned to one of four groups using materials featuring a single corpus example, multiple corpus examples, dictionary definitions, or no materials in a control group. The experiment clearly distinguished between comprehension and production in both materials and tests. The examples were selected from large corpora and featured relevant contextual clues to meaning (for the decoding task) and appropriate colligations / collocations (for the encoding task). The dictionary definitions were similarly chosen (mainly from Cobuild) to include the relevant senses only, and also provide the necessary colligational / collocational clues. In the test itself, the decoding task featured unfamiliar items in a multiple-choice gap-fill format; the encoding task required translation from L1 Portuguese of 'known' but error-prone items using lexical prompts. In the comprehension test, the group with dictionary definitions fared best, but not significantly better than the multiple corpus examples group; even the group with a single corpus example significantly outperformed the control. In the production test, all three experimental groups scored higher than the control, but only those with multiple corpus examples significantly so. The results support the findings of the earlier study, and are taken to mean that learners can derive useful information on both meaning and usage from corpus examples, that multiple examples are better than one, and that different types of examples are needed for encoding and decoding. There are clear implications for compilers of learners' dictionaries as well as for language teachers and materials developers.

Yukio Tono, Yoshiho Satake and Aika Miura report on how Japanese learners of English can use corpora to help revise their own writing. Texts from 93 undergraduates were error-coded, with each error being broadly classified as omission, addition or misformation, and two errors selected from each essay: one appropriate for correcting using corpus data, one without. Three weeks later, the participants were given a 20minute initiation to the corpus tool and then required to revise their first drafts, using the tool for one of the errors as indicated by the codes. The results show that learners can make useful revisions to texts using corpus data, and that there is a significant difference in the accuracy rate among the three error types: omission and addition errors were more easily identified and corrected than were misformation errors. The analysis discusses error types in much more detail, and also compares learners at different levels of language proficiency.

Zeping Huang focuses on awareness of the patterning of abstract nouns among 40 Chinese students majoring in English. Following the pre-test writing assignment, an experimental group was provided with paper-based concordance lines to study the collocations of five abstract nouns, while the control group was allowed to consult dictionaries for the usage of the words involved. The results of the immediate posttests showed that the writing of the experimental group contained not only fewer linguistic errors among the target abstract nouns, but also greater variety of collocational and colligational patterns; this tendency continued in delayed post-tests two weeks later. Data from questionnaires and learning journals show the learners have favourable opinions of the approach overall and are able to adapt to the inductive approach involved in such corpus work, but do bring to light some reservations.

Jonathan Smart compares different types of corpus-based instruction, using paperbased materials derived from Mark Davies' corpora2 for the passive voice. The first corpus group experienced inductive DDL, analysing printed concordances in collaborative tasks; the second also used corpus-based materials but followed a deductive presentation-practice-production paradigm (PPP). The PPP format was also used with a third group based on conventional teaching materials. Students of L1 Arabic and Chinese participated in the 4-hour experiment during an intensive language course in the US, completing various tasks on form, meaning and use in immediate and delayed post-tests in the same format as a pre-test. All three groups improved their performance in the immediate post-test, but the two deductive groups returned essentially to their pre-test scores in the delayed post-test. The inductive DDL group, however, experienced the greatest gains (+51% in the immediate post-test), an increase which remained significant in the delayed post-test (+43%). This is taken to suggest that the main benefit of DDL lies in its inductive nature rather than just the use of authentic language samples derived from corpora. It may of course still be that deductive DDL or traditional instruction would be of particular benefit to some students according to individual learning styles and preferences.

Elena Cotos examines the effects of corpora in noticing, exploring and reusing linking adverbials among graduate students from various disciplines enrolled on a 10-week writing course at a university in the US. One group used a 40K-word corpus of research articles relating to their own discipline; the other additionally had access to a similar sized corpus of academic writing produced by the participants themselves. Both groups virtually doubled their use of target items in their writing after the course (88%), an increase sustained and even extended four weeks later (93%); they also used a greater variety of linkers, and to better effect semantically and syntactically in context. Both groups also improved significantly from the pre-to post tests with a very large effect size. Between groups, access to a local learner corpus gave rise to more frequent, varied and appropriate use of linkers than just using a native corpus, and provided significantly better scores on the post-test. While questionnaires elicited positive reactions from both groups, work on their own and peers' texts was received particularly favourably, and seems to have led to greater cognitive processing.

Joe Geluso and Atsumi Yamaguchi attempt to integrate corpus use into an original course design with the focus on spoken fluency. Lower-intermediate Japanese students were introduced to COCA over three weeks, then investigated it largely for formulaic sequences (FSs). Activities were designed around student-led lessons on favourite FSs, and preparation of a 'speaking journal' which encouraged them to use the FSs in unpredictable encounters with native speakers. Recordings of these interactions suggest the students were generally able to use the FSs appropriately in context. Questionnaires and interviews revealed largely positive reactions, though some reservations emerged about unfamiliar vocabulary and truncated concordance lines, as well as possible mistrust of corpus data (some students preferring to have a dictionary on hand to check their findings). Some reported that the class period was insufficient to complete their corpus queries, but that they used the corpus enough overall.

Ji-Yeon Chang provided an introduction to corpus use for post-graduates in engineering and computer science in Korea to help with writing. The students consulted corpora on their own for their individual needs, submitting their writing regularly and meeting weekly with the researcher over 22 weeks for corpus-based feedback (cf. Johns' kibitzers 3 ). The work featured both a general corpus (COCA) and a local corpus of research articles in the relevant fields using AntConc 4 . The analysis uses grounded theory in an emic approach to examining recordings of the weekly individual sessions and the final questionnaires. Overall, the participants appreciated the use of corpora as examples of writing, especially at the level of lexicogrammar, though they did find it time-consuming and linguistically quite demanding. Some deemed the local corpus to be more relevant to their specific needs, though less credible as it included non-native writing (albeit in successfully published papers). COCA scored points from its mere size, as more results could be retrieved even in micro-registers, but was found difficult to use by some. A further worry concerned the risk of plagiarism. Overall, support is found for students being able to use corpora successfully on their own for their specific writing needs, and for corpora as a useful complement to web search engines for linguist purposes, even in an ambitiously autonomous programme such as this. However, it is suggested that further training would be useful in deciding what to search for and how to find it, and in helping students to compile their own corpora relevant to their specific needs.

Agnieszka Leńko-Szymańska describes a course designed for Master's students including trainee teachers in Poland, the objective being to promote corpus literacy for a variety of uses. A pre-course questionnaire shows almost no prior awareness of corpora, and limited motivation. The course itself ran over one semester (14 weeks) in a combination of lectures and hands-on work to introduce a wide range of corpora and associated tools (all simple, free, and readily available), as well as corpus-related materials and techniques from lexicogrammar to genre awareness; a useful link is provided to the online course page. A final questionnaire found generally promising evolution in opinions and positive reactions, especially for content geared towards the final project to compile a corpus and present a lesson plan for exploiting it. Some comments are more ambivalent, notably claims that there was too much to do but that they would have liked to do more to really familiarise themselves with the tools. The general conclusion is that an introduction to corpora can require far more than the 22 hours allotted, and that cross-fertilisation would be beneficial if students could recycle the tools and techniques in other courses.

http://www.routledge.com/books/series/RFD/

This is a pre-publication version. For the version of record, please see DOI: 10.1017/S0958344014000068(or email me at alex.boulton@atilf.fr)

http://corpus.byu.edu

A sample is available at http://lexically.net/TimJohns

http://www.antlab.sci.waseda.ac.jp/antconc_index.html

Acknowledgements

The editors are indebted to the following scholars who reviewed papers for this special issue: Guy Aston, Angela Chambers, Maggie Charles, Frederik Cornillie, Fiona Farr, Robert Fischer, Ana Frankenberg-Garcia, John Gillespie, Michael Goethals, Detmar Meurers, Hilary Nesi, Hans Paulussen, Simon Smith and Chris Tribble. We would also like to thank the ReCALL editors June Thompson and Françoise Blin for their confidence in us, and would like to acknowledge the work of the Cambridge Journals Online team for making this special issue possible.


. The implication is that these frequent items need to be learned -and to be learned thoroughly. However, diminishing returns set in, and accessible tools are required for the many tens of thousands of other items. Dictionaries are among the most widely-used tools for foreign and second (L2) language learning, and can help with both frequent and infrequent items. This chapter begins with a discussion of dictionaries themselves in relation to L2 use, and the main issues affecting their development over the last few decades in particular. These include the increasing use of empirical data, especially in the form of corpora, and the appearance of monolingual learner dictionaries in addition to bilingual and other dictionary types. It then moves on to a discussion of research into how dictionaries are used by non-native speakers (NNSs), especially as a reference resource for encoding and decoding, but also their impact on language learning itself.

Introduction: Historical overview and major issues

As will have become clear in this Handbook, dictionaries have a long and varied history. In essence, their intention is to inform people unsure of the meanings or uses of 'words', such people often being non-native speakers (NNSs). From antiquity, word repositories included ad hoc collections of translations intended to help with commerce and conquest; from the Middle Ages, the margin glosses produced by monks gave rise to more principled collections for deciphering religious texts in Latin, Greek and other classical languages. The Enlightenment saw the development of dictionaries for native speakers (NSs), thus giving rise to a preoccupation with 'good usage', a trend which was only questioned in the 20 th century with a more descriptive approach to lexicography, which has particular implications for learners and other NNSs. The first part of this chapter looks at recent (i.e. 20 th and 21 st century) developments in the design, compilation and use of various types of dictionaries and associated resources for and by NNSs. The second part moves on to research conducted on dictionary use and how this informs lexicography as well as informing good practice for dictionary use by teachers, learners and other users.

A descriptive approach to word lists in language learning

The descriptive approach to NNS lexicography predates the modern corpus era. The first notable event was the appearance of Thorndike and Lorge's Teacher's Wordbook of 30,000 Words which was published in 1944Words which was published in , though earlier versions had appeared in 1921Words which was published in and 1931. . Based on a collection of 18 million words, the book contained 30,000 entries listed according to frequency bands per million words in different categories, including juvenile literature and magazines. Lorge went on to collaborate in West's General Service List of English Words, the definitive version of which was published in 1953 (earlier versions date back to 1936). West's work was based on a much smaller corpus (2.5 million words in the main) and provided far fewer target items, but the volume included not just frequencies but also meanings. The 2,000 entries include lexical 'families', such that the various meanings and forms of present and presence as a verb, noun and adjective (i.e. inflections and transparent derivations) are included under the same heading. The overall frequency is given, along with percentage frequencies for each main meaning of each main form, and the list is still available and used today (e.g. at http://www.lextutor.ca/freq/lists_download). While this frequency approach was common in English, substantial work was also conducted in some other languages, notably French with the Dictionnaire Fondamental de la Langue Française [START_REF] Gougenheim | Dictionnaire fondamental de la langue française[END_REF], which marked a turning point by making significant use of spoken language. But it is only recently that computer tools have made the task considerably easier, giving rise for example to the Routledge Frequency Dictionaries currently available for 14 major languages. As these span nearly a decade, there are a number of differences between volumes; but the French version, as an example [START_REF] Lonsdale | A frequency dictionary of French: Core vocabulary for learners[END_REF], gives the 5,000 most frequent lemmas in rank order derived from a corpus of over 20 million words (evenly divided between speech and writing), along with part of speech, an example sentence from the corpus plus English translation, and thematic selections and alphabetic lists for ease of use, etc. Though the series is called 'dictionaries', it is rather far from the prototypical dictionary many learners are used to -though not as distant as some other frequency lists such as the Academic Word List [START_REF] Coxhead | A new academic word list[END_REF] which was based on the GSL, the Academic Formulas List [START_REF] Simpson-Vlach | An academic formulas list: New methods in phraseology research[END_REF], or the Phrasal Expressions List [START_REF] Martinez | A phrasal expressions list[END_REF], all of which serve to complement dictionaries in aiding the learner to decide what items (and which meanings and uses) to give priority to in different areas.

Intuition and data in learner dictionaries

Returning to more standard dictionaries, the first major innovation of the 20 th century was probably the appearance of Hornby's seminal work which is now in its eighth edition as the Oxford Advanced Learner's Dictionary, accompanied by an American edition and various spinoffs for phrasal verbs, idioms, and other language areas. This derived from the insight that dictionaries intended for NSs often were of limited relevance to language learners, an insight that affected choices of what items to include (the most frequent, but also the most useful and with the widest coverage), what information to provide (as regards meaning, form and usage), as well as how to present it (with more comprehensible definitions and suitable examples). The world of monolingual learner's dictionaries (MLDs) in particular, however, was to undergo a fundamental revolution [START_REF] Hanks | The corpus revolution in lexicography[END_REF] in the 1980s with the appearance of the Collins Cobuild English Language Dictionary, based on the Bank of English corpus as conceived by John Sinclair, which has developed from the original 7 million words to over 650 million today as a subset of the 4.5 billion-word Collins Corpus. The basic idea was that lexicographers should work with systematic collections of language data (corpora) rather than their intuitions or fortuitous examples of salient items, hence its motto "helping learners with real English" (see the papers in Sinclair, 1987, for an overview of the original project). This was essential in deciding what words to include, as well as in ordering entries by the most frequent meanings and uses, but beyond this it notably featured definitions in full sentences without abbreviations, and examples extracted directly from the corpus itself. (See also de Schryver, 2012, for a history of lexicography based on a corpus of academic publications.) Even today, Hanks (2013, p. 145) can argue that definitions in many dictionaries "owe more to art than to science" as lexicographers seek to come to terms with the fuzzy nature of meaning and the probabilistic nature of usage. The very fact that dictionaries select different items to which they attribute different meaning classifications presented in different orders is testimony to the fact that entirely objective answers have yet to be achieved; the orderly presentation of dictionaries may even mislead the user into believing that language itself is orderly. Nonetheless, other publishers soon followed the corpus lead with their own MLDs, to the extent that the major ones for English today are all corpus-based to a greater or lesser degree: Cobuild, Oxford, MacMillan, Longman and Cambridge, as well as Merriam-Webster for American English (several of the British publishers also now have American editions, and all include information on major varieties of English). Indeed, attempts to avoid corpora in major lexicographical enterprises today would be "laughed out of court" (McCarthy, 2008, p. 564). Current MLDs generally feature more-or-less authentic examples, information about frequency, help boxes for close synonyms and confusing items, and special entries for common errors or problems, thematic lists, colour pictures and maps, and appendices ranging from verb tables to thesauruses to writing guides. Research involving learner corpora (i.e. corpora that contain written and/or spoken productions by language learners) has also fed into some of the materials included in a number of MLDs, typically in the form of error notes and writing guides. User-friendliness is key as lexicographers are well aware (cf. [START_REF] Nielsen | The effect of lexicographical information cost on dictionary making and use[END_REF], and is reflected in limited metalanguage and a core defining vocabulary (first rigorously introduced by Longman), as well as in attempts to signpost longer entries (cf. [START_REF] Lew | Users take shortcuts: Navigating dictionary entries[END_REF]. They come in all shapes and sizes, from small pocket dictionaries to desk volumes of 100,000 entries or more. Most include a CD/DVD-ROM providing extra information (e.g. further examples from the source corpus), and have an online version which is usually accessible free (if without some of the extras available in the print purchase), as well as editions for mobile phones and whiteboards for classroom use; pocket dictionaries are still in vogue in some parts of the world (e.g. Asia; [START_REF] Jian | The role of electronic pocket dictionaries as an English learning tool among Chinese students[END_REF]. These new interfaces have vastly increased the flexibility and mobility of dictionary consultation, though in the case of most major dictionaries the changes are mainly limited to the quantity of information available (especially access to more examples plus some extra features such as study guides), and speed and ease of access to the same contents as in a paper dictionary (see [START_REF] Rizo-Rodríguez | Review of five English learners' dictionaries on CD-ROM[END_REF], for a review).

Though the main electronic dictionaries tend to resemble their printed versions closely, individuals or small teams have worked on a number of entirely new formats. Major initiatives tend to be research-driven rather than learner-focused (e.g. WordNet -see Piasecki et al., 2013), but worthy of a special mention in this chapter are 'pattern dictionaries', where recurring syntactic patterns tend to reveal common meanings [START_REF] Hanks | A pattern dictionary for natural language processing[END_REF]; 'organic dictionaries' derived from self-selecting collocations arising from a specialised corpus [START_REF] Williams | Bringing data and dictionary together: Real science in real dictionaries[END_REF]; and various tools for creating personal or collaborative 'word banks' or wiktionaries which can be suitable for general or LSP classes [START_REF] Horst | Growing academic vocabulary with a collaborative on-line database[END_REF]. Online dictionaries can also be integrated directly into learning platforms: with Hypertext (http://www.lextutor.ca/hypertext), for example, after submitting a text the user can click on a word to bring up the dictionary entry for that item [START_REF] Cobb | A resource wish-list for data-driven learning in French[END_REF].

Monolingual and bilingual dictionaries

The discussion so far has mainly focused on English and on monolingual dictionaries, and not without reason. Hanks (2013, p. 104, p. 355) happily acknowledges that lexicography for languages in other countries may be more advanced in terms of theoretical foundations (e.g. Russia), or phraseology in dictionaries (e.g. Czech), but it is no secret that dictionary publishing is not a purely altruistic venture -it is also a business in search of profits. As dictionary production is highly competitive (with several major dictionaries in the English MLD market where once there was only the OALD), it has been "marked by rapid and constant change, technological advance, innovative and creative development and response to users' needs" (Kirkness 2004, p. 68). An opposing force though is inertia: given the expense in producing a dictionary, publishers also tend to be conservative and to provide customers with slightly improved versions of tools they are already familiar with [START_REF] Hanks | The lexicographical legacy of John Sinclair[END_REF]. The market for dictionaries of English is enormous compared to that for most other languages, and monolingual versions can be sold around the world while bilingual dictionaries have a much more restricted market. Nonetheless, the bilingual dictionary in one form or another is probably the single most common language tool for NNSs, preferred by learners in almost all studies (e.g. [START_REF] Atkins | Monitoring dictionary use[END_REF]but cf. Frankenberg-Garcia, 2011), and topping the list of favourite Internet resources for language [START_REF] Scheffler | When intuition fails us: The world wide web as a corpus[END_REF][START_REF] Todd | Computer-assisted language use: An internet survey[END_REF]. Familiarity and staying within the "comfort zone" are no doubt contributing factors (Kaur & Hegelheimer 2005, p. 298), but in some cases bilingual dictionaries may actually be more effective than MLDs (e.g. [START_REF] Lew | Which dictionary for whom? Receptive use of bilingual, monolingual and semi-bilingual dictionaries by Polish learners of English[END_REF].

In bilingual dictionaries, an entry from the target language (L2) into the mother tongue (L1) is mainly intended as an aid in decoding meaning, while an L1=>L2 entry is useful for encoding -though as such dictionaries are usually intended for learners of either language, the English=>French section of a bilingual dictionary needs to include information relevant for speakers of both languages and hence for both encoding and decoding purposes. A similar problem exists in MLDs, though greater space can be allotted to each individual entry since the focus is often more on encoding, and equivalent information does not need to be repeated in both bilingual directions. This is not a minor issue, as [START_REF] Frankenberg-Garcia | Learners' use of corpus examples[END_REF], 2014) shows how example sentences which are appropriate for decoding may not be helpful for encoding purposes, and vice versa. It is particularly problematic given the dictionary's focus on individual words rather than contextualised uses, especially in translation where the bilingual dictionary is the staple tool even though it is no easy task to use it given that we do not translate one word at a time (cf. [START_REF] Teubert | Units of meaning, parallel corpora, and their implications for language teaching[END_REF]. More recently, corpus work has shown that all normal language use relies heavily on 'chunks', and more recently still that many chunks have psycholinguistic reality (e.g. [START_REF] Conklin | The processing of formulaic language[END_REF]. If words really only have 'meaning potentials' which are realised in actual use, how is a learner to distinguish between possible meanings when the dictionary is limited in the sample contexts it offers? The problem is confounded in production: how is a learner to choose between all the options which dictionaries fail to rule out, i.e. to distinguish normal uses from creative and exceptional but rare exploitations [START_REF] Hanks | Lexical analysis: Norms and exploitations[END_REF]. As Taylor (2012, p. 100) puts it: "A person armed only with the dictionary and the grammar book could well come up with expressions which are fully grammatical and with meanings which can easily be worked out but which happen not to correspond to what speakers of a language would normally say." 1.4. From dictionary to user So much for dictionaries themselves, but what of the learners? Clearly it is important to improve dictionaries, but it is also worth asking whether we can "improve the users" (Atkins & Varantola, 1997, p. 1). Most dictionaries intended for NNSs, whether monolingual or bilingual, paper or electronic, are very easy to use on first encounter -at least to the extent that the untrained user is able to derive some benefit from them at a very basic level. It may however be precisely because of this that training in dictionary optimisation is rare even though it is widely acknowledged to be extremely valuable: "The dictionary is an excellent tool in the hands of a skilled learner. An unskilled user wastes time and comes away frustrated from dictionary consultations" (Roby, 2006, p. 59). Ideally, given the ubiquity of dictionaries of various types, training should be integrated into lessons [START_REF] Béjoint | The place of the dictionary in an EFL programme[END_REF] early and regularly. Considerable evidence suggests that such training can indeed be effective (e.g. [START_REF] Ranalli | Online strategy instruction for integrating dictionary skills and language awareness[END_REF], found a large and significant increase in the number of look-ups following dictionary instruction), but rather than treating it as an add-on activity separate from regular work, Frankenberg-Garcia ( 2011) and others propose dictionary training to help learners on an ad hoc basis whenever the need arises. In this view, we begin not with the dictionary and its hypothetical uses but with the learners themselves and the questions they genuinely have.

Teacher and learner representations of vocabulary in general and dictionary use in particular are not always well founded (cf. [START_REF] Folse | Vocabulary myths: Applying second language research to classroom teaching[END_REF]. One prevalent assumption is that dictionaries themselves contain all and only 'right' answers (hence the comment lament: "But I found it in the dictionary"; cf. the 'blind faith' learners have in dictionaries reported by [START_REF] Bowker | Exploring the potential of corpora for raising language awareness in student translators[END_REF], such that any error following dictionary consultation is reputedly the learner's fault. Clearly though, lexicographers have to make choices about what items to include, condensing a huge wealth of data into short entries on the most common features which may therefore not contain the answer to a specific question -especially for phraseology and collocation, or genre-specific uses [START_REF] Hanks | Lexical analysis: Norms and exploitations[END_REF]. Another common assumption is that dictionaries are easy for any literate person to use, such that any specific training is seen as a waste of classroom time and almost an insult to the learner's intelligence -again putting the onus on the learner. A third is that dictionary consultation leads to learning: though there is some evidence for this (e.g. [START_REF] Hulstijn | Incidental vocabulary learning by advanced foreign language students: The influence of marginal glosses, dictionary use, and reoccurrence of unknown words[END_REF][START_REF] Ronald | L2 lexical growth through extensive reading and dictionary use: A case study[END_REF], there is no fundamental reason to suppose that a dictionary look-up will necessarily lead to retention, or that an excellent reference tool will necessarily make the best learning aid. At the same time, it seems clear that learners generally look only for 'unknown' words and ignore frequent items which they in fact know only partially and for which dictionaries could be of help; that they tend to seek to add to the number of words they know (overwhelmingly nouns) rather than increasing depth of knowledge of the smaller number of common items which have much more complex meaning potentials and uses; that they stop at the first part of the entry or the first familiar item and do not explore longer entries (the so-called 'kidrule'); that they do not exploit phraseological information (such as it is) in the dictionary; that they find it difficult to understand and thus tend to ignore guides to pronunciation, syntax, register, and so on -let alone the introduction and other supplementary material (see [START_REF] Nesi | A study of dictionary use by international students at a British university[END_REF]. But all of these representations can be subject to research of one type or another, which is the topic of the next section.

Description of research to date

Because dictionary making is a commercial enterprise, "research conducted by dictionary publishers is not generally made public" (Nesi, 2014, p. 39). There is however an increasing quantity of empirical studies on dictionary use by learners. These are highly diverse and target various aspects of dictionary use such as learners' beliefs and attitudes, their look-up behaviour and preferences (e.g. menus vs signposts in long entries for polysemous items), how successful they are with dictionaries as a reference resource (for decoding and/or encoding), and whether dictionary use does indeed translate into learning. Investigating dictionary use is a complex and challenging undertaking in view of the many variables involved (see [START_REF] Nesi | Dictionary use by English language learners[END_REF], for a historical overview). Most studies are therefore carried out in experimental settings which are often highly controlled, some even making use of purposebuilt dictionary entries or eye-tracking technology, for example.

Because of the complexity involved, findings from different investigations are seldom directly comparable and often raise more questions than they answer [START_REF] Tono | Research on dictionary use in the context of foreign language learning: Focus on reading comprehension[END_REF][START_REF] Abecassis | Is lexicography making progress? On dictionary use and language learners' needs[END_REF][START_REF] Chan | The use of a monolingual dictionary for meaning determination by advanced Cantonese ESL learners in Hong Kong[END_REF]. The contribution of dictionaries is often analysed by comparing learners' performance on a series of tasks with or without access to dictionaries or dictionary entries; contrasts are sometimes extended to other reference tools, such as comparing performance using dictionaries or concordances derived from computerised corpora (e.g. [START_REF] Boulton | Data-driven learning: Taking the computer out of the equation[END_REF]. It is notable that there is as yet no meta-analysis of learners' use of dictionaries per se, neither as a learning aid nor as a reference resource for encoding or decoding (the nearest thing to date seems to be Abraham, 2008, with a meta-analysis of computer-mediated glosses). From a synthetic reading though, what clearly emerges from the research is that many aspects of dictionary use are influenced by learner variables such as proficiency level in the L2, whether or not the L2 is also the medium of instruction, the degree scheme (e.g. language majors or students needing languages for specific or academic purposes), familiarity with certain types of dictionaries, cultural and educational environment, and so on [START_REF] Abecassis | Is lexicography making progress? On dictionary use and language learners' needs[END_REF][START_REF] Laufer | What lexical information do L2 learners select in a CALL dictionary and how does it affect word retention?[END_REF][START_REF] Nesi | Dictionary use by English language learners[END_REF][START_REF] Tono | Research on dictionary use in the context of foreign language learning: Focus on reading comprehension[END_REF]. The focus of the rest of this section is on insights from studies that explore the use of dictionaries as a resource for decoding and/or encoding, and the role they may have in learning vocabulary or other aspects of the L2.

Dictionaries for decoding

One of the key roles of dictionaries as a reference tool is to help learners understand the meaning of words or expressions in decoding activities (e.g. reading a text in the target language). On the whole, research suggests that learners tend to be rather reluctant to turn to dictionaries in receptive activities [START_REF] Atkins | Monitoring dictionary use[END_REF][START_REF] Hulstijn | Incidental vocabulary learning by advanced foreign language students: The influence of marginal glosses, dictionary use, and reoccurrence of unknown words[END_REF][START_REF] Lew | Which dictionary for whom? Receptive use of bilingual, monolingual and semi-bilingual dictionaries by Polish learners of English[END_REF]. This finding has a very practical impact on studies that specifically set out to analyse the effectiveness of (various types of) dictionaries in receptive use as they presuppose actual dictionary use. As a result, researchers have had to resort to experimental set-ups that require the participants to use dictionaries (e.g. the use of pseudo-words and made-up entries and/or careful monitoring: [START_REF] Tono | Research on dictionary use in the context of foreign language learning: Focus on reading comprehension[END_REF][START_REF] Lew | Which dictionary for whom? Receptive use of bilingual, monolingual and semi-bilingual dictionaries by Polish learners of English[END_REF]. After reviewing findings from empirical studies that compare learners' vocabulary comprehension with and without dictionaries, Chen (2011b, p. 218) concludes that there is "no simple yes-or-no answer to the question whether the dictionary is useful for vocabulary comprehension or not". That said, studies that rely on tests that target actual vocabulary comprehension rather than general text comprehension (where not understanding some vocabulary items might not be a problem; see also [START_REF] Nesi | How using dictionaries affects performance in multiple-choice EFL tests[END_REF] point to the positive influence of dictionary consultation. The effectiveness of dictionary use has been connected with individual learners' proficiency levels and dictionary reference skills, the relevance, complexity and quality of the information provided, the type of dictionary consulted and the accessibility of information in (long) entries [START_REF] Tono | Research on dictionary use in the context of foreign language learning: Focus on reading comprehension[END_REF][START_REF] Lew | Which dictionary for whom? Receptive use of bilingual, monolingual and semi-bilingual dictionaries by Polish learners of English[END_REF][START_REF] Chen | Dictionary use and vocabulary learning in the context of reading[END_REF]. Another factor is the type of dictionary used. For example, [START_REF] Lew | Which dictionary for whom? Receptive use of bilingual, monolingual and semi-bilingual dictionaries by Polish learners of English[END_REF] study suggests that bilingual dictionaries are both favoured by the learners and more effective than MLDs as they provide an immediate solution in the L1. It is worth noting though that this preference may arguably have been induced -at least in part -by the use of translation tasks in the experimental setup. [START_REF] Chen | Dictionary use and EFL learning: A contrastive study of pocket electronic dictionaries and paper dictionaries[END_REF] and [START_REF] Dziemianko | Paper or electronic? The role of dictionary form in language reception, production and the retention of meaning and collocations[END_REF] have shown that the electronic medium can speed up and boost the reference support learners get from dictionaries in vocabulary comprehension. Two examples will exemplify dictionary studies of this type. [START_REF] Chan | The use of a monolingual dictionary for meaning determination by advanced Cantonese ESL learners in Hong Kong[END_REF] investigates advanced ESL learners' use of MLDs to determine the meaning of familiar polysemous words used in less familiar contexts. The participants were asked to select the most appropriate meaning of italicised strings of words in sentences from a list of five choices, first without and then with access to an MLD (the target lexical items for consultation were underlined, e.g. The minister is alive to the responsibility of his position). Overall, the results of the multiple-choice tests reveal an improvement in test results after dictionary consultation. The study incorporated self-reporting protocols to uncover learners' strategies and problems when extracting information from the MLDs. For example, the participants frequently reported that there were "too many definitions and examples in the entries" and that "the arrangement of the entries was too long and confusing" (Chan, 2012, p. 128). Chan also found that learners tend to be rather insensitive to grammatical information in the entries when decoding, which sometimes hindered the comprehension of the familiar words in unfamiliar grammatical contexts.

The definition component of entries in MLDs is often presented as the key element in vocabulary comprehension [START_REF] Laufer | The effect of dictionary definitions and examples on the use and comprehension of new L2 words[END_REF]. To test this, Frankenberg-Garcia (2012) set out to compare Portuguese EFL learners' comprehension of low-frequency words (1) without any reference consultation, (2) with the help of a dictionary definition only, or with (3) a single or (4) multiple well-chosen corpus examples. The results of the multiple-choice test (which required the participants to select the best Portuguese translation of the items under study) show that the performance of the learners who used multiple examples was very similar to that of the learners who consulted the definition. A quasi-replication study (Frankenberg-Garcia, 2014) with secondary-school learners came to similar conclusions, additionally showing that the types of examples needed for encoding and decoding purposes are very different, but that multiple examples can in fact be of greater help than dictionary definitions when it comes to language production. Though these examples were manually selected, it should be possible to establish criteria for appropriate examples (cf. Hanks, 2013, p. 410), and to implement these at least partly automatically; steps have been taken in this direction by the SketchEngine team for Macmillan (see [START_REF] Rundell | Automating the creation of dictionaries: Where will it all end?[END_REF].

Dictionaries for encoding

Although MLDs have often been presented as encoding tools par excellence by lexicographers and researchers [START_REF] Rundell | Dictionary use in production[END_REF][START_REF] Mcenery | Corpus-based language studies: An advanced resource book[END_REF][START_REF] De Cock | Computer learner corpora and monolingual learners' dictionaries: The perfect match[END_REF], learners' actual use of these dictionaries for production can appear to be rather limited or unsuccessful [START_REF] Lew | Which dictionary for whom? Receptive use of bilingual, monolingual and semi-bilingual dictionaries by Polish learners of English[END_REF]. Frequently cited explanations for this include learners' lack of awareness of their own needs and problems when encoding (e.g. Frankenberg-Garcia, 2011), including and perhaps especially in terms of collocation (e.g. [START_REF] Laufer | The contribution of dictionary use to the production and retention of collocations in a second language[END_REF]. They similarly lack awareness of the wealth of useful encoding information in this type of dictionary and have poor dictionary skills, both connected with a general lack of dictionary training. And of course, sometimes the dictionaries themselves may be responsible if they do not include the necessary words or information, or if the answers are difficult to find or buried among a mass of information which is poorly signposted (e.g. Chan 2012).

One of the typical features of corpus-informed MLDs is the inclusion in entries of a series of example sentences or phrases taken from computerised corpora, either directly or subject to editing in some cases. These examples, which illustrate the use of words or strings of words in different contexts, appear to be particularly useful to learners in production activities provided the focus is on examples specifically meant to help production [START_REF] Frankenberg-Garcia | Learners' use of corpus examples[END_REF], 2014), and favour a usage-based model of language learning [START_REF] Taylor | The mental corpus: How language is represented in the mind[END_REF]. Research also suggests that learners might benefit from direct or mediated access to computerised corpora and other online resources when producing texts (although there appear to be strong individual differences; e.g. [START_REF] Charles | Proper vocabulary and juicy collocations': EAP students evaluate do-ityourself corpus-building[END_REF], as this provides them with access to the use of target words in more, different and even highly specialised contexts which might be close to the contexts in which they need to use the words. As a result, the combination of an electronic dictionary with other electronic resources, typically corpora or concordances, has been widely advocated and occasionally implemented [START_REF] Paquot | The LEAD dictionary-cum-writing aid: An integrated dictionary and corpus tool[END_REF][START_REF] Verlinde | Data access revisited: The Interactive Language Toolbox[END_REF]. Some degree of corpus mediation may be required, since learners (especially at lower levels of proficiency and with limited needs and ambitions) cannot be expected to be corpus linguists any more than they can be expected to be lexicographers (Chambers, 2005, p. 114). Further, direct corpus consultation can be a challenge even for experienced corpus users when they need immediate answers to production questions [START_REF] Frankenberg-Garcia | Learners' use of corpus examples[END_REF][START_REF] Chambers | Corpora and concordancing: Changing the paradigm in language learning and teaching?[END_REF]).

As mentioned above, dictionary use and learner variables are closely intertwined, and the role of learners' L1 in L2 production cannot be ignored [START_REF] Bogaards | Dictionaries and second language acquisition[END_REF]. Learners can often be seen to access the L2 via their L1 when encoding, and this is arguably reflected in their reported general preference for bilingual dictionaries. Using bilingual dictionaries for encoding is generally not regarded as desirable, however, insofar as bilingual dictionaries tend to lack the rich production-oriented lexico-grammatical information typically included in MLDs. Bilingualised dictionaries (cf. [START_REF] Bogaards | Towards a new type of bilingual dictionary[END_REF]) attempt a solution to this as they seek to "combine the best of both worlds" by bringing together "the extensive knowledge that is condensed in modern monolingual learner's dictionaries and the exploitation of bilingual equivalence that are so well established in the learner's mental lexicon" (Bogaards, 2010, p. 119). [START_REF] Laufer | Examining the effectiveness of 'Bilingual Dictionary Plus': A dictionary for production in a foreign language[END_REF] compare the 'usefulness' of four types of dictionaries in a productive Hebrew into English translation task performed by Hebrew EFL learners: (1) a printed bilingualised (L1-L1-L2) dictionary; (2) a printed L1-L2 bilingual dictionary; (3) a printed L1-L2-L2 bilingualised dictionary (referred to as bilingual dictionary plus or BD+); (4) a computerised BD+. The translations were scored taking both semantic and syntactic appropriateness into consideration. The results suggest that the paper and electronic BD+ can have advantages over other dictionaries. It would be interesting to see if this advantage is confirmed in future studies focusing on encoding tasks that do not directly involve translation (e.g. writing).

Although bilingualised dictionaries appear to be commonly used in some countries (e.g. China: Chen, 2011a), they are still generally less easily accessible than traditional bilingual dictionaries and MLDs. An alternative is the production dictionary (e.g. the Longman Language Activator) which is specifically designed to help with writing. More recently, the Louvain English for Academic Purposes Dictionary is presented as a "dictionary-cum-writing aid" (Paquot, 2012, p. 163), and makes full use of the flexibility of the electronic medium to admit considerable customisation. French-speaking users have access to L2 entries via their L1 and language use needs are taken into consideration (e.g. for EAP learners and in disciplines such as medicine, economics, linguistics or political sciences). Within the framework of the project, an examination of the log files is currently under way to determine how the tool could be improved to better meet the users' needs.

Dictionaries and language learning

As pointed out by Lew and Doroszewska (2009, p. 239), learning new vocabulary can be regarded as "a useful by-product of dictionary consultation". Any learning resulting from the use of dictionaries as reference tools has been labelled as incidental learning (as opposed to intentional learning [START_REF] Hulstijn | Incidental and intentional learning[END_REF][START_REF] Bogaards | Dictionaries and second language acquisition[END_REF]. In empirical studies of vocabulary learning and dictionary use, learning tends to be operationalised as the extent to which new words, meanings or collocations are remembered by the learners in immediate and/or delayed (unexpected) vocabulary tests following decoding or encoding tasks involving the use of a dictionary. Besides comparing learners' retention of words and collocations with and without dictionary use, various studies also explore the impact of a series of dictionaryrelated factors on retention: types of dictionary used, dictionary medium (paper vs electronic), use of various entry components (e.g. L1 definitions, L2 translations, examples of use, animated pictures, etc.), look-up behaviour or number of look-ups. On the whole, dictionary consultation seems to have a positive impact on the retention of the items concerned (e.g. [START_REF] Laufer | Incidental vocabulary acquisition in a second language: The construct of task-induced involvement[END_REF]. This impact appears to be enhanced by factors such as the presence of word-focused activities and the highlighting of the target items in the tasks carried out by the learners, or the availability of an L1 translation in the dictionary entries [START_REF] Hill | Type of task, time-on-task and electronic dictionaries in incidental vocabulary acquisition[END_REF][START_REF] Lew | Electronic dictionary entries with animated pictures: Lookup preferences and word retention[END_REF][START_REF] Chen | Dictionary use and vocabulary learning in the context of reading[END_REF]. [START_REF] Hill | Type of task, time-on-task and electronic dictionaries in incidental vocabulary acquisition[END_REF] investigated the effect of task type on incidental L2 vocabulary learning. The Chinese ESL learners in the study had to (1) read and understand a text (containing unfamiliar target words, which were highlighted in the text and whose meanings could be looked up in a dictionary); (2) perform one of three tasks (a form-oriented production task -selecting one of four target words that means the same as a given synonym or paraphrase of the target word; a form-oriented comprehension task -selecting the meaning of a given target word from a list of four options; or a meaning-oriented taskanswering yes/no text comprehension questions); and then (3) take immediate and delayed vocabulary tests (they had to provide the meaning of the target words in English or Chinese). The results reveal that the two form-oriented tasks (and especially the productive task) were more conducive to vocabulary learning (i.e. better retention) than the meaning-oriented task because they were more demanding for the learners and led to more look-ups. Hill and Laufer conclude that "an important factor determining task effectiveness for vocabulary learning is the amount of word-related activity that the task induces" (p. 87).

One recurrent theme in research into dictionary use and vocabulary learning is the connection between depth of processing and retention. Findings seem to support the idea that the deeper the processing, the better the retention of new vocabulary (cf. the Involvement Load Hypothesis: [START_REF] Laufer | Incidental vocabulary acquisition in a second language: The construct of task-induced involvement[END_REF]. However, as pointed out by [START_REF] Dziemianko | On the use(fulness) of paper and electronic dictionaries[END_REF] in a discussion of the effects of paper vs electronic dictionaries on word retention, deeper processing should not be equated with laborious look-up processes typically associated with printed dictionaries, as retention can be higher when using electronic rather than paper-based dictionaries. Dziemianko argues that depth of processing is triggered by attention, which "does not necessarily correlate with the effort put into dictionary look-up, which is often performed quite automatically. Instead, the conspicuousness of headwords and entries on the computer screen might arrest users' attention" (p. 331). This is arguably in line with Schmidt's noticing hypothesis (1990). Although he and others in the area have abandoned the original 'strong' form of the hypothesis (whereby "noticing is the necessary and sufficient condition for converting input to intake"; p. 129), it does seem that conscious awareness of new language can promote learning. And since dictionary use involves conscious effort, it may indeed contribute to language learning. As we saw above, the electronic versions of some MLDs (e.g. Longman, Oxford, Macmillan) have been transformed into veritable language learning environments featuring a whole host of resources, exercises, (learner-corpus informed) error notes, and grammar and writing sections. To our knowledge, however, no empirical research has been published into the use of this extra material by learners (or teachers) and its impact on actual language learning. The web-based Interactive Language Toolbox [START_REF] Verlinde | Data access revisited: The Interactive Language Toolbox[END_REF] is an example of what Abel (2010) calls dictionary-cum-CALL systems, which combine a dictionary and a learning tool (for French and Dutch vocabulary in this case). [START_REF] Verlinde | Data access revisited: The Interactive Language Toolbox[END_REF] explain how insights from a usability study (based on a think-aloud experiment and a users' questionnaire) were taken into consideration to enhance and simplify data access and search facilities. They argue that "users' expectations and habits are still insufficiently taken into account by lexicographers" and highlight the importance of implementing a "more useroriented lexicography" (Verlinde & Peeters, 2012, p. 161-162).

Conclusion

We have seen in this chapter that dictionaries are a perennial staple for language learners and foreign or second language users of all types. Bilingual dictionaries remain the basis for this, though curiously they have been less innovative perhaps than MLDs. Though there may be a move from print to electronic formats, this reliance on dictionaries seems unlikely to disappear any time soon as they also top the list of students' preferred language technologies [START_REF] Steel | Language students and their technologies: Charting the evolution 2006-2011[END_REF]. However, they are inevitably complemented by other resources which are likely to become even more widely accessed as users become more internet-literate and resort to on-line translators and corpora, or indeed general-purpose search engines which enable them to consult the 'web-as-corpus' (see [START_REF] Boulton | Applying data-driven learning to the web[END_REF].

Though research into dictionary use is a surprisingly recent phenomenon [START_REF] Nesi | The use and abuse of EFL Dictionaries: How learners of English as a foreign language read and interpret dictionary entries[END_REF], the results seem to point consistently in the same direction: dictionaries (if appropriately chosen and appropriately used) can contribute substantially to enhancing language performance in terms of both encoding and decoding, and can lead to retention of new items (meaning, form and usage). A key term however is 'appropriate' -dictionaries tend to be taken for granted in many instructional settings, and training in their efficient use is arguably one of the unaccountably missing ingredients in language teaching today.

It would be an unusual paper in applied linguistics which did not call for more research, but this is perhaps particularly true in the area of dictionary use by L2 users. Target populations need to be extended beyond the favoured audience of university students, and include younger secondary-school learners as well as adult users who need an L2 on a more or less regular basis in their professional or social lives. There is also a particular lack of ecological studies on actual dictionary use over an extended period (as opposed to controlled, experimental or laboratory designs where the participants perform an imposed task, often in just a few minutes). Such studies may be more difficult to organise rigorously given the dynamic and complex nature of language learning (cf. Verspoor et al., 2011), but should become increasingly accessible as more learners turn to electronic resources which enable their look-ups to be unobtrusively tracked systematically at distance and evaluated in terms of outcomes in language use.

This is a pre-publication version. For the version of record, please see DOI: 10.1007/978-3-642-45369-4_25-1 (or email me at alex.boulton@atilf.fr)


Keywords: Corpus, Data-driven learning, DDL, Corpus-based language learning

Introduction

The essential condition for any language learning is exposure to the language itself. In foreign language contexts in particular, such exposure may be inadequate: Zahar et al. (2001: 558) estimate that an hour's reading may lead to the incidental learning of just two words; at that rate, it would take decades to build up a sizeable vocabulary. Clearly exposure alone is not enough in such cases and may be complemented by formal instruction intended to help speed up the process by drawing learners' attention to important points, explaining, demonstrating, providing examples, and so on. Fortunately, syntheses show that instruction does make a difference (e.g., [START_REF] Norris | Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis[END_REF], though that does not mean that any type of instruction works equally well for all learners in all contexts [START_REF] Hattie | Visible learning: A synthesis of over 800 meta-analyses relating to achievement[END_REF]. Formal teaching may however oversimplify things: the contrived language that is presented, the all-purpose definitions provided, the abstract rules given, as well as the structured tasks to be completed. These can all have their uses, but if that is all there is then they may lead to dependence on the teacher and an inability to work with authentic language -i.e., to make the most of any real exposure. This is where language corpora can be of use in what has come to be known as data-driven learning (DDL). The basic concept is to allow massive exposure that is still organized and focused. Using the power of computer software, learners can query large collections of texts relevant to their needs, looking at frequencies and distributions and multiple occurrences of target items in context. This essentially constructivist, inductive approach means they can then reach their own conclusions that are meaningful to them individually, and the cognitive processing should lead to longer retention than simply "being taught." This may be quite time consuming at the start, but the real advantages lie not so much in the explicit knowledge gained as in the processes involved -ability to deal with authentic texts in different genres; awareness of frequency, chunking, and collocation; noticing forms and variation; formulating hypotheses and inferring meanings; and so on. In other words, it should help students become better language learners and users.

Early Developments

The word "corpus " can mean different things to different people in a variety of disciplines. In corpus linguistics, it is a large collection of authentic texts that has been deliberately sampled to be representative of the type of language one is interested in; it is accessed by software often called a concordancer, though it can usually do more than just concordancing as we shall see below. This is, however, a prototypical definition, and corpus linguistic tools can be used with just a few thousand words (wherever repeated searches can beneficially be conducted by computer rather than regular reading), a single text (e.g., a novel), "non-authentic" text (arguably simplified readers or textbooks, or learner essays), and collected automatically (in the case of web-compiled corpora) or at least partly serendipitously (depending on resources available). In language teaching, the overriding criteria are pedagogical rather than theoretical, and the ad hoc creation of a small, specialized corpus of texts can be much more relevant to learners' needs than some of the large, general-purpose corpora that are publicly available.

The first modern corpus is no doubt the Brown corpus, a million words carefully sampled from 500 extracts of texts that had been published in 1961 [START_REF] Kučera | Computational analysis of present-day American English[END_REF]. This was partly motivated by dissatisfaction with the tools then available for describing English which derived largely from intuition and fortuitous examples. The goal here was to introduce greater scientific rigor from a more systematic base. Later, the Bank of English at Birmingham University was designed for linguistic purposes but also with pedagogical aims in mind (see Sinclair 1987, for a review). This monitor corpus was designed to increase over time to account for developments in British English, but originally just over 7 million words were used to produce the first Cobuild dictionary. The corpus could be sorted on the computer and then the short contexts (concordances) were printed out for every occurrence of every item; the lexicographical work took place entirely on paper, similar to earlier hand-compiled concordances from the Bible or Shakespeare, for example. The 100 million words of the British National Corpus (see [START_REF] Aston | The BNC handbook: Exploring the British National Corpus[END_REF] represented a truly monumental undertaking when it was built in the early 1990s, but later advances made it possible for a single person to create the Corpus of Contemporary American English semiautomatically from the Internet, currently standing at 520 million words (see [START_REF] Davies | The 385+ million word Corpus of Contemporary American English (1990-2008+): Design, architecture, and linguistic insights[END_REF]. Entirely automated procedures now mean that billion-word corpora are regularly compiled (e.g., Baroni and Bernardini 2006). At the top end of the scale, the searchable Internet has debatable status as a corpus but can be usefully queried via regular search engines or more specialized software for pedagogical purposes.

In the hands of experts, corpora can be useful in preparing all kinds of pedagogical materials and resources, from general and specialized dictionaries to grammar books and usage manuals, from syllabus design to testing, from wordlists to coursebooks. Such uses are beyond the scope of this chapter, which is concerned with how learners can use corpora directly.

Many of the early attempts at learner corpus consultation are based around Birmingham with teachers in contact with the Cobuild project. McEnery and Wilson (1997: 12) mention uses dating back to 1969, though the earliest publication seems to be from San Francisco where [START_REF] Mckay | Teaching the syntactic, semantic and pragmatic dimensions of verbs[END_REF] describes activities encouraging learners to identify grammatical patterns to distinguish semantically similar verbs, based on sentences printed out from a corpus. In Surrey, [START_REF] Ahmad | Using computers with advanced language learners: An example[END_REF] had their advanced learners using a computer to query a corpus directly, though such early software could be exasperatingly slow. Things really took off in the late 1980s back in Birmingham, with Tim Johns as a leading pioneer often cited as the founding father of DDL. He created a concordancer specifically designed with language learners in mind (MicroConcord, which later morphed into WordSmith Tools; see http://www.lexically.net/wordsmith), and published a number of papers explaining many different ways in which he used corpora for and with his students, with many widelycited sound bites and frequent citations, especially from the seminal collection of papers he coedited (Johns and King 1991) and which included two papers of his own. Johns created or used different types of corpora -scientific texts, parallel corpora of translations, a single novel -to be relevant to the learners' needs. Authenticity was important in terms not just of text but also of needs and indeed the task, since corpus consultation involves exploring and thinking about language -crucial to any language learning. In this way the learner was seen as a researcher with direct access to the data, and the teacher as guide rather than dispenser of linguistic knowledge. Proactive materials could be printed out for repeated use with lower-level students for general purposes, while more advanced students could explore the corpus individually or collaboratively, using the concordancer themselves for serendipitous browsing or focused querying. Johns would leave the concordancer on in his classes as an informant and used it in his one-to-one advising sessions to help with academic writing. For him, DDL was not just learner centered but also provided a means to keep language (especially lexicogrammar) firmly center stage. All of this was argued to lead to greater autonomy; indeed, his final paper [START_REF] Johns | Integrating corpus-based CALL programs and teaching English through children's literature[END_REF] provides some evidence that the DDL participants outperformed the control group even on items that had not been explicitly covered, suggesting that the processes may improve language skills as a whole. The tremendous variety of uses of corpora promoted by Johns set the agenda for years to come, though of course he was not alone, especially in the UK and Europe. Of particular note is the biennial Teaching and Language Corpora (TaLC) conference series inaugurated in Lancaster in 1994, each event giving rise to a selected volume of papers; further information can be found in Thomas and Boulton (2012: 17-34).

Major Contributions

Before going any further, it may be useful to see what DDL actually looks like. Traditionally the user sees corpus data in the form of a concordance, typically in KWIC (key word in context) format. Fig. 1 shows a random selection of 20 concordance lines taken from a corpus of academic writing (110 papers focusing on DDL, 600 k running words) for the search term, "e.g." here centered and in bold. This very simple formal example highlights a number of features which may be useful to learners. On the left, it is immediately apparent that most occurrences occur within brackets (other searches show that this is true in 85% of cases), the implication being that it is unusual in the syntax of the main sentence and should not be overused in this way in this type of writing. On the right, the presence or absence of a comma owes more to individual journal style guides than any generalizable pattern. Beyond that, it is often used to introduce references, which can lead to further searches for citation practices and discussion of whether in other disciplines or other languages, research cited is thus typically relegated to brackets or not. Most corpus analysis software offers this basic concordancing function. Other features include frequency counts of individual words or clusters, collocates, distributions, and so on, all of which can prove useful to L2 learners. It is in the nature of innovations in computer-assisted language learning (CALL) that early publications tend to be descriptive of classroom practices and software developments. The situation gradually evolves, and [START_REF] Boulton | Corpus use in language learning: A meta-analysis[END_REF] identified over 200 publications attempting some kind of evaluation of corpus use by L2 users; some of the more widely cited are briefly outlined below.

Much of the initial interest lay in emic studies to find out what learners thought about DDL. The data were often gathered through interviews, diaries, or especially questionnaires; the latter are still frequently used but often now as a complement to other aspects. Long-term ecological studies are particularly valuable here, such as [START_REF] Baten | The use of concordances in vocabulary acquisition[END_REF] who received overwhelmingly positive feedback from 400 Dutch economics students after 4 months. More recent is the frequently cited paper by [START_REF] Yoon | ESL student attitudes toward corpus use in L2[END_REF], who introduced corpora to their ESL students in the USA over several weeks. The questionnaires again revealed considerable enthusiasm, especially among those with comparatively low levels of linguistic proficiency, which opens up the question of who DDL is most appropriate for. Across all studies, the response is overwhelmingly positive which no doubt owes something to the novelty factor and the Hawthorne effect, given that most researchers/teachers were themselves enthusiastic. Nonetheless, it seems that DDL can appeal to a wide variety of learners, though [START_REF] Turnbull | Towards independent concordance work for students: Lessons from a case study[END_REF] provide a detailed case study of two students needing English for a master's degree in Australia: one was found to be field independent and took to corpora very quickly; the other was field dependent and found it largely a waste of time.

Another focus has been on the uses learners make of corpora, again mostly by asking learners about their practices; a notable exception is [START_REF] Pérez-Paredes | Learners' search patterns during corpus-based focus-on-form activities: A study on hands-on concordancing[END_REF] in Spain who tracked their learners' searches. They found that lack of training led to fairly unsophisticated queries, with learners approaching corpora in much the same way as they did Internet searches; indeed, the most successful outcomes were found to be combinations of corpus and web searches. The types of queries formulated are analyzed by [START_REF] Kennedy | Corpus-assisted creative writing: Introducing intermediate Italian learners to a corpus as a reference resource[END_REF], who usefully distinguish pattern hunting (i.e., search for inspiration) and pattern defining (i.e., checking specific questions); success was linked to trial and error, among other things. [START_REF] Charles | Getting the corpus habit: EAP students' long-term use of personal corpora[END_REF] had her graduate students compile their own disciplinespecific corpora to help with academic writing and followed up use a year later. Eighty six percent of the respondents continued to use corpora at least occasionally in drafting or revising their academic writing and 38% of them regularly. The overall picture that emerges is that most students can use concordancers directly, though it remains controversial how much training is needed. Where time or resources are limited, or for students with lower levels of L2 proficiency, linguistic sophistication, or motivation, work with printed data can provide one solution (e.g., [START_REF] Boulton | Data-driven learning: Taking the computer out of the equation[END_REF].

Others have attempted to see whether DDL leads to measurable outcomes from a more etic perspective, i.e., whether it "works" or not. These again split into two groups, the first evaluating the use of corpora as a learning aid, focusing on learning outcomes usually of specified target items. The results generally derive from some kind of language test, whether pre/post or control/experimental designs. Among the earliest and most ambitious, [START_REF] Cobb | From concord to lexicon: Development and test of a corpus-based lexical tutor[END_REF] PhD thesis and papers derived from it showed that lower-level Arabic students were able to learn large numbers of words using DDL over a long period of time and were significantly more likely to retain them long term than control groups with word lists and dictionaries only. Most other studies come to similar conclusions for vocabulary and lexicogrammar in general, which may be what DDL is most suitable for, whatever the level of the learners [START_REF] Lee | A study of using web concordancing for English vocabulary learning in a Taiwanese high school context[END_REF]. Much of the work here is relatively ecological, being based on a regular course over several weeks or a semester. Chujo and Oghigian and their colleagues in Japan run a semester-long DDL course on a regular basis enabling different types of data collection and analysis, especially as they tweak the course each time. In a 2012 paper looking at noun and verb phrases over two semesters, the experimental group made significant gains in most areas compared to the control group; the results are found to be particularly promising when printouts and hands-on concordancing are combined.

The other group of studies interested in outcomes looks at the impact of corpus use not as a learning aid but as a reference resource, especially while writing (drafting or revising texts or translations). Some of it is short-term experimental work such as by Frankenberg-Garcia (2014), who provided her Portuguese high-school learners of English with dictionary definitions and multiple concordance lines. Both were found to be useful for comprehension, but as few as three carefully-chosen corpus examples proved significantly more effective in production. O' Sullivan and Chambers (2006) got their Irish students of French to correct their own essays; following training, they successfully corrected many underlined errors of grammar and lexis in particular, as well as syntax and even formal things such as spelling where dictionaries or other resources would have been quicker and just as effective. Geluso (2013) also had his learners produce essays especially for the study, but then got them to use Google frequencies as a test of formulaicity for sequences in inverted commas which the students themselves chose as dubious. Four native English speakers rated the results as being significantly more "natural." Search engines were also used by [START_REF] Todd | Induction from self-selected concordances and self-correction[END_REF], but here with the snippets as an equivalent to concordance lines to help correct errors; again, the results suggest that learners can indeed make significant use of such self-selected data.

Work in Progress

The essential ingredients in DDL are corpora and the software to query them, and users today have access to tremendous numbers of both. More and larger corpora can be compiled quickly and easily and distributed free or at small charge via the Internet for many different languages: SketchEngine alone currently lists over 50 languages, some with many different corpora (www.sketchengine.co.uk). However, the prevalence of (semi)automatic compilation aids means that few corpora are as rigorously compiled as the BNC, for example, and care inevitably needs to be taken in interpreting the results. Some tools such as BootCaT (bootcat.sslmit.unibo.it) are publicly available and mean that ordinary users can compile rough-and-ready corpora in a few minutes for specific purposes: all that is needed is to input a handful of "seed" words which are characteristic of the type of language required; the tool does the rest. The availability of large quantities of text via the Internet also means that teachers or learners can manually identify and download texts to build their own corpora for local use. These are often far smaller, which can be an advantage when the needs are highly specific. Software development has also led to increasing numbers of query tools often freely available on the web or for download, which again helps to make DDL much more accessible. Some of these are highly specific, some are intended for experienced researchers; others though are extremely simple and sufficiently general for ordinary L2 learners to be able to work with. AntConc (www.laurenceanthony.net/software/antconc) deserves a special mention here as it has been among the most widely used in recent DDL studies, including some of those mentioned above.

Technological advances have made DDL faster, simpler, more intuitive, prettier, more accessible, and so on. But in terms of methodologies, the essential aspects of DDL remain largely unchanged, typically featuring induction from multiple occurrences in context, augmented with lists and charts of frequencies, collocates, wordsketches, etc. This means that much of the research has been in piloting specific corpora or software, or in testing the basic approach with different learners in different contexts with different needs and questions in mind -all the while doing quite similar things. The advantage of this is that enough evidence has accumulated to be able to take stock. [START_REF] Boulton | Corpus use in language learning: A meta-analysis[END_REF] have undertaken the first systematic meta-analysis of DDL with 88 unique samples from 64 separate studies. The results show large effect sizes overall, both within and between groups. Moderator analyses reveal gaps in the research agenda, including for languages other than English, spoken skills, long-term uptake and occupational uses, etc. Three trends in recent years are of particular note. First, a number of studies apply essentially DDL-like practices to the web as corpus. The value here is that the web itself is large and varied enough to contain almost anything the user might want; the challenge of course lies in finding it using regular search engines as surrogate concordancers (cf. [START_REF] Boulton | Applying data-driven learning to the web[END_REF]. At the same time, users are already familiar with the web and with search engines, which may go some way toward countering objections of technical difficulties, and further training in their use is more likely to be taken up long term precisely because the tools are so general purpose. A second way to help integrate DDL into learning is to graft them into CALL packages. Cobb's Compleat Lexical Tutor (www.lextutor.ca) provides a number of tools in addition to regular concordancing, allowing learners or teachers to create gap-fills automatically from multiple concordance lines, to visualize the frequency bands of words in a particular text, to click on a word in their own text for a concordance to pop up, to consult and share concordances during writing or error-correction, among other things. This is a way of bringing DDL to the learners rather than expecting them to come to corpus linguistics. Finally, the traditional interest in lexicogrammar is being complemented by more work at the level of discourse, especially using corpora as a reference resource for academic writing. This is not necessarily obvious, since many features of interest may be difficult to search for at surface level; having the students build their own small, specialized corpora increases ownership and familiarity and is one way forward suggested by [START_REF] Charles | Getting the corpus habit: EAP students' long-term use of personal corpora[END_REF].

Problems and Difficulties

The advantages of DDL notwithstanding, the fact that it is not mainstream practice suggests that there are difficulties involved. Various questionnaires have noted problems from the learner's perspective, but many of these have dissipated over time, and solutions exist for others. Despite copyright issues and questions of ownership, lack of access to appropriate data is far less a problem today with the increasing availability of large numbers of corpora, as well as the Internet itself. Technical problems can be eliminated if the teacher prepares printed handouts for activities, and software and interfaces have become far more userfriendly in recent years. The ubiquity of Internet search engines have gone a long way towards bridging the gap between everyday practice and DDL: the concept of data searching is familiar and the techniques are largely transferable; users are able to read concordance lines nonlinearly just as they are Google snippets and are less concerned with "drowning in data." Some learners may find the language in corpora difficult: smaller, more relevant corpora may make them more approachable, especially where learners are involved in choosing familiar texts. At lower levels of proficiency, learners may be more comfortable with parallel corpora of translations (see below) or even with corpora of simplified texts or graded readers (available for English on www.lextutor.ca/conc/eng).

Perhaps the biggest problem lies in simply knowing what to query in the first place: much work with error-correction, for example, relies on teachers indicating problem areas (e.g., O'Sullivan and Chambers 2006). One possibility is to rely on frequency data from the web as an indication, focusing on rare items except where they include proper names or highly technical items, as suggested by [START_REF] Geluso | Phraseology and frequency of occurrence on the web: Native speakers' perceptions of Google-informed second language writing[END_REF]. To the extent that DDL enhances language awareness, increased practice is likely to make this easier and more intuitive over time. There is still the problem of formulating the question as a query that the software can understand, and then interpreting the results. Training is recommended by many just to get the most out of Internet search engines, and more may be required for dedicated concordancers and other corpus software. How much training is needed for hands-on concordancing is a controversial issue, though it will ultimately depend on the learners' own needs and preferences, and how much they are likely to want to use corpora in the future. This raises the further question of the types of learners that DDL is likely to suit best, given that there is considerable variation in their appreciation of the approach and the benefits they derive from it. By far the majority of studies to date have focused on university students, though there is no intrinsic reason why younger learners cannot also benefit. On the other hand, there has been considerable work with learners at lower-intermediate level who are majoring in disciplines other than language, suggesting that language proficiency and sophistication may not be insurmountable barriers. It may even be that DDL is more appropriate in such cases for learners whose previous experience with more teachercentered, deductive approaches has left them uninterested or struggling (cf. [START_REF] Yoon | ESL student attitudes toward corpus use in L2[END_REF]. All that can really be said at the moment is that further work is needed in a number of areas -which leads us to the final section.

Future Directions

Empirical DDL research has largely focused on university students with intermediate to advanced levels of English as a foreign or second language. It may be that this is where it is most useful and appropriate, though for a more comprehensive picture we would expect more work with younger learners, in secondary or even primary schooling, in private language schools, and outside formal education. This last point seems particularly important: if corpus consultation is argued to be useful for real needs, then we know to know what it can bring to professional situations. Interest in long-term uptake of DDL is at present limited (though see [START_REF] Charles | Getting the corpus habit: EAP students' long-term use of personal corpora[END_REF], and introducing it to the workplace seems to be nonexistent except for academic writing.

As far as the corpora themselves are concerned, English is likely to remain the major preoccupation for the foreseeable future, but we would expect more work on other languages too. More important, concordancers work only with written text (including transcriptions); since many learners are primarily interested in spoken language, we would expect the next few years to see development of aligned corpora with sound and even video. It is extremely time consuming to collect spoken data, and the few that currently exist tend to consist largely of interviews (e.g., www.uni-tuebingen.de/elisa/html/elisa_ index.html or www.um.es/sacodeyl). An obvious bypath would be to use existing subtitled documents which are already aligned, albeit imperfectly: [START_REF] Aston | Learning phraseology from speech corpora[END_REF] describes such uses of the TED talks using WordSmith Tools; [START_REF] Quaglio | Television dialogue: The sitcom Friends vs. natural conversation[END_REF], among others, has shown that scripted dialogues are closer to "authentic" spontaneous conversation than might be thought, and thus also have their place in a spoken program of DDL for general language learning purposes.

Parallel corpora of translated texts may also be further developed: at the moment, they are relatively rare outside specialist translation courses, despite their obvious uses in many areas, as well as for learners at lower levels of proficiency. There are currently very few that are freely available and easy to use, and they often have their limits: of note is EuroParl, the proceedings of the European Parliament in 21 languages (www.statmt.org/europarl). While the status of Linguee (www.linguee.com) as a parallel corpus may be debatable, it can be used in ways compatible with DDL but with more than one language. Other initiatives can be expected as it becomes easier to align translations for analysis with free parallel concordancers (e.g., www.laurenceanthony.net/software/antpconc).

Technological advances have helped to bring DDL closer to its potential users, with numerous corpora and software designed with L2 learners in mind. At the same time, as technology and the Internet in particular become second nature, learners are already involved in everyday practices that bring them closer to DDL. Johns was originally determined to present DDL as radically different to traditional teaching; the time may have come for it to be seen as an extension of ordinary practice. It will be interesting to see if and to what extent web searches and DDL merge. Finally on the technological front, smart phones and other mobile devices may also bring about substantial changes, but interfaces will need to adapt to allow for screen size and processing speed in particular; entirely new practices may emerge. For the most part, the basic shape of DDL was formed quite early on: recent studies can in many cases be considered replications of earlier work. DDL is in line with a number of theories of language, learning, and use, some of which derive from insights gleaned from corpus linguistics, but this is largely a one-way relationship. The future may usefully see more empirical studies explicitly designed to analyze the theoretical foundations in more detail. Among other things, we know that language consists of regular overlapping sequences in the form of chunks that are processed, stored, and retrieved as wholes rather than being constructed bottom-up from grammar "rules" as traditionally thought, meaning that any individual item is typically found in a limited number of contexts (Sinclair 1991, on the idiom principle; Hoey 2005, on lexical priming; Millar 2011, on psycholinguistic evidence for chunking). This breaks down the grammar/lexis divide suggesting that our language knowledge is the sum of the encounters we have with it, both receptively and productively, in line with emergentist, usage-based theories [START_REF] Tomasello | Constructing a language: A usage-based theory of language acquisition[END_REF]. [START_REF] Taylor | The mental corpus: How language is represented in the mind[END_REF] talks by analogy of the "mental corpus," highlighting that many of these theories not only support corpus linguistics and DDL but owe much of their origins to them.

Finally, new research practices are needed to test the real benefits of DDL -not just for learning specific items but in helping users to become better language learners, more sensitive to language as a whole. This is the central claim, but so far the only evidence is incidental and at best suggestive [START_REF] Johns | Integrating corpus-based CALL programs and teaching English through children's literature[END_REF][START_REF] Allan | Data-driven learning and vocabulary: Investigating the use of concordances with advanced learners of English[END_REF]. What is needed are careful longitudinal studies that specifically focus on this. Ideally, for any technology or approach to become really useful, it needs to be taken up outside the context of a single course -with teachers of other and subsequent courses, and after the end of the instruction period.

Fig. 1

 1 Fig. 1 KWIC concordance of, "e.g.", in a corpus of academic writing

This is a pre-publication version. For the version of record, please see DOI: 10.1007/978-3-319-02328-1_15-1 (or email me at alex.boulton@atilf.fr)