Alex Boulton
email: alex.boulton@univ-lorraine.fr

Corpora in language teaching and learning

This timeline looks at explicit uses of corpora in foreign or second language (L2) teaching and learning, i.e. what happens when end-users explore corpus data, whether directly via concordancers or integrated into CALL programs, or indirectly with prepared printed materials. The underlying rationale is that such contact provides the massive contextualised exposure needed for language learning, but in a more controlled way than purely haphazard exposure via regular reading or listening, thus promoting or enhancing noticing, language awareness, autonomy, and ultimately producing 'better learners'. It interweaves with many other notions in language teaching, from authenticity and autonomy to induction and constructivism, taking in much of what we know about language from corpus linguistics itselfits fuzzy, probabilistic nature and the importance of lexical patterns, collocations, chunks, frequencies and distributions. Commonly known as DATA-DRIVEN LEARNING or DDL (Johns 1990), the approach goes back several decades; yet despite its impressive pedigree, a frequent lament is that it remains a marginal practice. Teachers may not be aware of DDL if it is absent from their initial training, or may see it as a research activity confined to higher education. While corpus consultation may appear too demanding for school-age learners, there are connections with the web searches that are already a frequent practice for many. Though lack of uptake is sometimes attributed at least in part to a dearth of empirical research, this claim is increasingly difficult to defend, as witnessed by the number of entries in this timeline and the hundreds of others not included. For a balanced discussion of the limits of DDL as well as what it can reasonably achieve, see Kaltenböck & Mehlmauer-Larcher (2005) 1 .

The goal here has been to provide an overview of the evolution of the field from the earliest publications up until the present day. The first objective was to focus on empirical evaluations of DDL based on an extensive trawl of empirical research which identified over 200 publications. In some cases, the choice is quite straightforward, especially where a name is associated with a single widelycited and influential paper; in others, it is difficult to choose among an author's cumulative output of a dozen or more publications which could almost merit an individual timeline of their own. Few authors have more than one entry here as the aim is to provide as broad a sweep as possible. On occasion, an intermediate solution was adopted, with the same authors contributing a base study along with a follow-up or replication study in the same entry. As a failsafe, the reference sections of the various papers were analysed to identify authors of more than one empirical DDL study; the 25 most frequently cited are all included here.

This procedure on its own would not do justice to the full range of activity in the field, and some key players would be excluded since their most widely cited work is non-empirical. Since research is a fundamentally human activity, a second consideration was to focus on the individuals and their most inspirational work. Empirical research is thus complemented with a number of descriptive, argumentative or position papers, especially from the early days, along with a handful of more theoretical works that provide some foundation to DDL that would otherwise be missing, as well as the main surveys reported in the area. Only manuals and textbooks were ultimately excluded from the timeline itself, though some of these provide additional theoretical support for the approach; selected references for further reading are provided as supplementary materials on the Language Teaching website. Some difficult decisions still needed to be made, and though the final list inevitably has an element of subjective choice, some of my personal favourites have had to be omitted for reasons of space and coherence according to the various inclusion criteria.

DDL draws on many different strands of research, and some of the most frequently cited names from neighbouring fields could only be included in a heterogeneous timeline of several hundred entries. These range from corpus linguistics itself and the development of corpora, tools and methodologies that are widely used in DDL, including work on language for specific purposes, frequency lists and learner corpora, error-correction and contrastive analysis, to corpus uses in syllabus design, materials and resources, testing and translation, as well as fields from lexis to acquisition studies and more general language learning theory.

While much work has been reported in a few major journals (notably Computer Assisted Language Learning, ReCALL, Language Learning & Technology, System and CALICO Journal), the bulk appears as chapters in various collections, often arising from conferences or in less well-known journals or conference proceedings, often available on line. In favouring the most accessible and prestigious sources here, the result is inevitably biased towards publications in and on English and from certain regions of the world, and does not address reports in other languages, or using corpora or software specific to other languages.

Ultimately, the timeline papers fall into four main categories as outlined below. Since there is no overarching programme for DDL research, the balance is not perfect between the various categories, which often overlap.

A

Theoretical underpinnings. As mentioned above, this category does not attempt to outline all the general language teaching and learning theory that the approach draws on (e.g. cognitive skills or autonomisation), and is limited to those landmark papers most directly connected to DDL itself. Much of this derives from research in corpus linguistics, which has influenced our understanding of language (e.g. in the nature of lexicogrammar, patterning or chunks).

B Descriptive papers.

A potentially enormous category, so largely limited here to a few influential papers from the relatively early days.

C

Empirical evaluations are divided into sub-sections.

First, mainly emic papers, often featuring questionnaires, interviews, logs etc., which focus on:

C1 learners' attitudes and representations about corpus use;

C2 learners' behaviour and processes in corpus use.

Second, largely etic papers, often with a pre/post-test or control/experimental design from tests or analyses of learner productions, which look at:

C3 effects of corpus use as a learning aid;

C4 effects of corpus use as a reference resource.

D

Surveys and syntheses. This category includes papers that attempt to make sense of the increasing body of research in the field.

Several features emerge. In chronological terms, it is in the nature of the developmental curve in applied linguistics research that a new field is developed by a small number of visionaries first in descriptive and speculative papers (beginning around 1990), followed by increasing numbers of ever more focused and sophisticated empirical evaluations (especially from 2000 on, with a dozen or more such papers every year since 2005) until such time as enough are available for various types of research syntheses, with theoretical underpinnings appearing at various points along the way. Curiously, perhaps, while many empirical studies refer to theoretical and pedagogical foundations, few seek directly to test them, and theory has not been a major driving force leading to new practices. The main initial impetus came largely from different spheres in the UK (Birmingham, Lancaster and Nottingham, among others), but it has since widened into much of the rest of the world: other Englishspeaking countries, western Europe, the Middle East and Asia in particular. The relative paucity of DDL work in the United States of America, however, would bear further examination.

The methodologies have become increasingly rigorous and sophisticated in designnot to mention complex in the case of the many PhDs not covered herewith a range of data collection instruments. Interestingly, there remains a balance of emic and etic studies with questionnaires ranking alongside pre/post-tests or control/experimental designs as the instruments of choice. This allows considerable insight not just into the outcomes but also into the participants' representations and the processes involved. There has been a move over time from purely lexicogrammatical to more discourse-oriented studies, and from use of corpora as a learning aid towards their use as a reference resource, especially in language for specific or academic purposes, with a predictable dominance of English as the target language. Some issues prove controversial, such as the need for training, or the potential with lower proficiency levels or for oral skills. More longitudinal work is needed outside university classrooms: in autonomous contexts, in secondary education, in language schools, or in the workplace for 'genuine' life-long needs; as well as the possibilities of exploiting more everyday tools for DDLcompatible web searches. Surprisingly undercited, the first empirical evaluation of DDL exhibits considerable ecology and complexity, allowing 400 Dutch economics students to select and explore their own texts in conjunction with the Brown corpus for English according to their own interests over four months.

Their written work and a prepared oral exam showed remarkable sophistication in corpus use. Feedback is overwhelmingly positive regarding motivation and the possibilities for individualisation and autonomy in contextualised vocabulary work; but Baten et al. also note negative points relating mainly to the limitations of the corpus and availability of the concordancer.

C1, C2

1990 Johns, T. (1990). From printout to handout: Grammar and vocabulary teaching in the context of data-driven learning. CALL Austria 10, 14-34.

One of Johns' most frequently cited papers, especially in the 1991 version in Johns & King (eds.) 3 along with its sister paper there ('Should you be persuaded: Two samples of data-driven learning materials'), this contains the first occurrence of the term 'data-driven learning', variously referred to elsewhere as 'classroom concordancing' or 'corpus-based learning'. Given the technological limitations of the time, much of his early work concentrated on use of generic or 'proactive' printed materials derived from corpora. Johns describes the approach as 'revolutionary', promoting the authenticity of corpus data and the role of learner as researcher with direct access to that data rather than being dependent on the teacher. Various activities are discussed, with examples in the appendix. Sinclair, J. (1991). Corpus, concordance, collocation. Oxford: Oxford University Press.

A tremendously influential figure in moving linguistics back towards empirical investigation, Sinclair is often considered the father of modern corpus linguistics. Founder of the Cobuild project in Birmingham (cf. JOHNS 1990), his research was mainly descriptive/lexicographical but had a practical interest in learning and teaching. This widely-cited book presents the case for combining lexis and grammar, suggesting that language use is based on the 'idiom principle' as opposed to the traditional teaching emphasis on grammar, such that learning happens by deconstructing chunks of input which are later used to build up utterances. This is supported by recent psycholinguistic evidence beyond the scope of this timeline (e.g. The only paper that goes beyond description or suggestion in this seminal volume edited by Johns & King, Stevens tests students' lexicogrammar in a traditional gap-fill text or multiple concordance lines. Each participant served as control and experimental participant, completing half the questions using traditional gap-fills, half using concordances. The concordance questions derived from a corpus of the students' textbooks were answered significantly better in the second session, suggesting the importance of training.

C3

Fligelstone, S. (1993). Some reflections on the question of teaching, from a corpus linguistics perspective. ICAME Journal 17, 97-109.

This paper is widely cited (see Leech 1997 for a follow-up) 5 for its three-way characterisation of corpora in teaching. Fligelstone first noted 'teaching about' corpus linguistics as an academic subject in its own right, and 'teaching to exploit' corpora in giving students training in the mechanics of hands-on corpus use for various questions. Most importantly here, 'exploiting to teach' included both indirect applications (i.e. selection and sequencing of items to teach in accordance with corpus findings) and hands-on concordancing for language learning purposesi.e. DDL. This study mainly relies on a specially compiled corpus of chapters from computer manuals for a 10hour computing course in Hong Kong to help correct written work. Students' search queries are analysed and compared against questionnaires and interviews with randomly selected participants. Despite some success, Ma's main conclusion is that, left to their own devices, students do not always use their time efficiently, formulating inappropriate queries and underusing some functions, and concentrate on corpus use for error-correction only.

A

C1, C2

1997 Aston, G. (1997) Frequently cited for their innovative workbook published the previous year, Thurstun & Candlin here provide the rationale underpinning it. Designed for university students requiring academic English, the book presents a small number of rhetorically related items using overtly corpus-derived materials following a four-stage process akin to 'the three Ps' but initially inductive: look, familiarize, practise, create (see also Carter & McCarthy 1995 6 for 'the three Is': illustration, interaction, induction). The obvious objection is the amount of work involved for so few target items, but the authors defend this as exposing learners to far more language through concordance contexts, and promoting deep processing and cognitive skills which should have longer-term benefits.

A, B

1998 Turnbull, J. & J. Burston (1998).

Towards independent concordance work for students: Lessons from a case study. ON-CALL 12.2, 10-21.

Case studies allow more in-depth analysis of learners' concordancing activities, as reported by Turnbull & Burston for two students enrolled for a master's degree in Australia. Radically different motivations and learning styles meant that one (field independent) student used the concordancer frequently and effectively, and felt she learned a great deal, while the other (field dependent) showed less interest and was less successful, and found it a waste of time. It is perhaps surprising that more work has not been conducted relating DDL and learning styles, strategies and motivations.

C1, C2

2000 Bernardini, S. (2000). Systematising serendipity: Proposals for concordancing large corpora with language learners. In L. Burnard & T. McEnery (eds.), Rethinking language pedagogy from a corpus perspective.

Frankfurt: Peter Lang, 225-234.

Bernardini's take on autonomous corpus browsing has become an accepted definition, popular especially for corpus-driven post-graduate academic writing needs. Here she attempts to balance it with greater guidance in a 'pedagogy of discovery' (e.g. ASTON 1997). Third-year Italian translation students were required to analyse agreed questions on their own out of class; reporting back, their conclusions were generally found to be correct, and their enthusiasm is clear as they indicated using the BNC for other courses too. Problems are noted, such as concentration on the unusual, frustration in formulating search strings, and underuse of some software functions. Following a disappointing first attempt with relatively 'technophobic' students, Seidlhofer aimed to increase motivation and involvement by asking her English students in Austria to examine corpora of their own writing using WordSmith Tools. Despite the small corpus of very short texts, learners were able to discover many of the elements of corpus linguistics and compare their texts against larger corpora. Allowing learners greater personal involvement in the corpus and in the questions to explore led to far greater enthusiasm and motivation (cf. ASTON 1997 These two studies have language students in Australia consult the authors' own corpus of Italian, with detailed analysis of individual student behaviours. The 2001 students proceeded from controlled activities, to correcting their own work, to exploring their own questions; most found it useful, though difficulties are described in the main stages of formulating the question, devising a search strategy, observing and selecting examples, and drawing conclusions. The 2010 learners used the corpus and bilingual dictionaries to revise their own creative writing. Kennedy & Miceli's characterisation of 'pattern hunting' (looking for content or language ideas) vs. 'pattern defining' (checking usage) has achieved widespread recognition. The results, along with interviews and questionnaires, show diverse levels of uptake, and successful corpus use is linked to trial and error and ownership with further training as the main way forward. Lee & Liou's 17-year-old high-school students spent 10 weekly sessions using DDL for vocabulary, with a variety of corpus activities based around their textbook contents. All three levels improved in the post-test, with the differences between them decreasing; this may suggest that DDL can be accessible at lower levels. Learners with an inductive preference tended to perform better with DDL, and even those with a deductive profile were quite enthusiastic despite alleged cultural orientation towards knowledge-transmission in Taiwan, where considerable DDL work has been successfully carried out. (2006) programmes. In both studies, the learners first wrote an essay in the L2 with access to traditional resources. Following training in corpus use, they were allowed two hours to use concordances to correct their original texts, changes being marked as positive, negative or no effect. Improvement was noted in all areas, even for items which could have been more easily checked in dictionaries; reactions were positive overall, with most claiming they would use corpora in the future.

C1, C2

C1, C2

C1, C3

C1, C4

Curado Fuentes, A. ( 2004). The use of corpora and IT in evaluating oral task competence for tourism English. CALICO Journal 22.1, 5-22.

Most corpus work features written texts; Curado Fuentes is notable for his use of corpora of transcribed speech, here for academic and professional texts with upper-intermediate students in tourism. Over two weeks, the course moved from identifying recurrent forms in short texts to online concordancing and related activities, and was evaluated via prepared and spontaneous spoken tasks. The experimental group showed considerably more effective use of items that occurred in the corpus in the prepared task; they also exhibited more effective use of cohesive markers in the spontaneous discussion, and claimed greater confidence. Though the experimental group made as many errors, this is attributed to their overall faster pace of delivery and longer presentations.

C3, C4

Mukherjee, J. ( 2004). Bridging the gap between applied corpus linguistics and the reality of English language teaching in Germany. In U. Connor & T. Upton (eds.), Applied corpus linguistics: A multidimensional perspective. Amsterdam: Rodopi, 239-250.

Arguing that the teacher's perspective is crucial in promoting DDL, Mukherjee reports on a series of four-hour workshops in Germany. In all, 248 experienced English teachers enrolled; questionnaires show that they overwhelmingly intend to use corpora in the future, but mainly see the relevance for themselves as learners and, as teachers, for preparing paper-based DDL materials, with only 12% expecting to have their learners consult corpora directly. This is perhaps not surprising in a secondary school context, and age was found to be a factor with younger teachers more willing to experiment. However, it does suggest some limitations to the fully autonomous, serendipitous type of corpus consultation often equated with DDL (cf. BERNARDINI 2000).

C1

Yoon, H. & A. Hirvela (2004). ESL student attitudes toward corpus use in L2. Journal of Second Language Writing 13.4, 257-283.

Much work in DDL has focused not on outcomes but on learner beliefs and attitudes to corpus use, a trend crystallised in this paper by Yoon & Hirvela. They introduced corpora to their learners in the US over several weeks, providing feedback on writing assignments conducted out of class. Detailed questionnaires show considerable enthusiasm, with the overwhelming majority claiming they would use corpora in the future, though the authors follow up negative reactions too. The paper is also notable for its practical approach and bringing DDL to lower proficiency levels (cf. LEE & LIOU 2003), who seemed more favourable than the advanced group, perhaps as they had received more guidance. Braun, S. (2005). From pedagogically relevant corpora to authentic language learning contents. ReCALL 17.1, 47-64.

C1, C2

Written corpora are comparatively easy to collect, while many learners may have greater need of oral skills (cf. CURADO FUENTES 2004). A series of papers by Braun is important for its focus on multimedia corpora, beginning with ELISAa small collection of interviews which can be watched individually or searched through a concordancer aligning text with sound and video, and are enriched with word lists etc. The corpus is specifically designed with pedagogical rather than corpus-linguistic aims in mind (cf. MCENERY & WILSON 1997). Small-scale testing with secondary-school learners features in later papers. Much work with Asian learners has recourse to parallel corpora, often in conjunction with the authors' own software and related CALL materials as in this paper by Chan & Liou. Their first-year EFL course covered verb-noun collocates either inductively using a corpus, or deductively without, via various online activities. An immediate post-test showed significantly greater improvement for collocates via DDL than in the control group, an advantage maintained in a delayed post-test 10 weeks later; it is also argued that an inductive approach takes time to produce its maximum effect. There was no significant correlation between proficiency level and post-test scores (cf. LEE & LIOU 2003); a questionnaire showed generally favourable attitudes to DDL, which did correlate with performance.

C1, C3

Hoey, M. ( 2005). Lexical priming: A new theory of words and language. London: Routledge.

Hoey's lexical priming draws on usage-based theories in claiming that grammar is essentially the product of innumerable encounters with words in context, thus following SINCLAIR's (1991) focus on lexicogrammar. Mastery of a language (L1 or L2) is mastery of primings (i.e. expectations of meaning and use in context), and learning crucially consists of repeated exposure to items in context; the role of teaching is to fast-track the process so the learner can notice different uses.

A Kaur, J. & V. Hegelheimer (2005). ESL students' use of concordance in the transfer of academic word knowledge: An exploratory study. Computer Assisted Language Learning 18.4, 287-310.

While vocabulary lists have their uses, they also have their limitations (cf. COBB 1997/1999), as highlighted in this study on academic vocabulary by Kaur & Hegelheimer. Their ESL students in the US were introduced to concordancing over two weeks, and used the BNC via LexTutor for an out-of-class writing assignment in addition to dictionaries. While no significant difference was found in terms of overall performance (attributed to the time limitations in this experimental study), the DDL students did use the target items significantly more frequently and more accurately in the written assignment.

C1, C2, C3

Lee, D. & J. Swales (2006). A corpus-based EAP course for NNS doctoral students: Moving from available specialized corpora to self-compiled corpora. English for Specific Purposes 25.1, 56-75.

Lee & Swales join forces to bring corpus-based discourse/genre work to the classroom; this paper is particularly cited for its detailed notes on a 15-week course designed to help research students write academic papers. Work covered a number of language points decided by the teachers, though learners compiled corpora of their own and had freedom of choice in the final project, presented as a conference paper. The students were largely successful, appreciating their increased autonomy with language specific to their needs; this is reflected in the fact that most of them bought WordSmith Tools for their own future use.

B, C1, C2

Chambers, A. ( 2007). Popularising corpus consultation by language learners and teachers. In E. Hidalgo, L. Quereda & J. Santana (eds.), Corpora in the foreign language classroom. Amsterdam: Rodopi, 3-16.

Chambers provides one of the earliest surveys of empirical research in DDL. Though she covers only 12 studies, she manages to paint a broad picture of research up to that time, noting that it is mostly conducted by practising teachers/researchers in an individual in-class experimental approach; research outside the classroom is almost non-existent. The studies are broadly action-research in design, with greater reporting of qualitative results (learner attitudes and representations, processes and behaviours) than quantified learning outcomes. Hafner, C. & C. Candlin. (2007). Corpus tools as an affordance to learning in professional legal education. Journal of English for Academic Purposes,6.4,[303][304][305][306][307][308][309][310][311][312][313][314][315][316][317][318] The question addressed by Hafner & Candlin is what students would make of a corpus on their own initiative. The first cohort had just a brief demonstration supplemented by online tutorials, but with disappointing uptake. The introduction was expanded to a full hour in the second year, doubling the number of students who used the corpus to 40%. Interviews with selected students showed that this may still not be enough: some 'adopters' made little use of the corpus for its intended linguistic function, but browsed it for its legal content -an alternative 'affordance'. In Johns' final paper, with colleagues in Taiwan, the 'corpus' consists of a single novel to be read traditionally by a control group and accompanied by various CALL tools and DDL activities in the experimental group. Questionnaires showed very positive attitudes and greater motivation in the experimental group; they also performed significantly better than the control in the post-test for reading comprehension, and read twice as fast. End-of-term exam results suggest improvement extending beyond the specific tasks covereda tantalising discovery and one sorely in need of following up, since a major claim of DDL is that it can help students become better learners. Gilmore, A. (2009). Using online corpora to develop students' writing skills. ELT Journal 63.4,[363][364][365][366][367][368][369][370][371][372] Corpora are often used to help with writing, as in Gilmore's study with lower-level Japanese learners of English for academic purposes. Following a brief introduction to large on-line corpora, they then worked out of class to improve items marked as wrong on an earlier assignment. The outcomes were rated by four judges, who found the revisions produced more 'natural' text in nearly two thirds of cases; feedback from the learners was also overwhelmingly positive. (2003), DDL was particularly useful for students at lower levels of proficiency , while the more advanced learners maintained their advantage using the traditional approach. A questionnaire completed after the experimental condition showed very positive reactions. This study introduced concordancing alongside advanced Google searches, on the assumption that the two involve similar techniques. The students at this Australian university found the training useful, especially concordancers. Nearly half already used the internet for language learning; while the initial figures were far lower for corpus work, they increased dramatically by the end of the course. A case study of three learners found them subsequently making more and better use of Google in error-correction (cf. TODD 2001), which Conroy takes to mean that accessibility, familiarity and ease of use are crucial, with a promising future for 'Google-driven language learning'. Graduate students of mixed language and disciplinary backgrounds who need English for academic writing feature in several studies by Charles in the UK. The 2012 paper outlines how students create their own disciplinary corpora of research articles for analysis using AntConc; the 50 questionnaires reveal considerable enthusiasm and use outside class, and highlight potential problems. The 2014 paper provides a rare look at continued usage one year on; of the 40 respondents, 86% used their corpora in some way (38% regularly) for writing or revising their academic writing. Though the initial course had focused on discourse, most continued use was for lexicogrammar.

D

C1, C2

C1, C3

C1, C4

C1, C3

  Ma, B. K. C. (1994). Learning strategies in ESP classroom concordancing: An initial investigation into data-driven learning. In L. Flowerdew & A. Tong (eds.), Entering text. Hong Kong: Hong Kong University of Science and Technology Language Centre, 197-214.

  Millar 2011) 4 .

	A
	Stevens, V. (1991). Concordance-
	based vocabulary exercises: A
	viable alternative to gap-fillers. In
	T. Johns & P. King (eds.),
	Classroom concordancing.
	English Language Research
	Journal 4, 47-61.

  Aston's initial attempt at DDL with his advanced Italian learners of English was only partly successful as the tasks were too open (cf. MA 1994), and learners tended to focus on isolated examples and miss patterns and tendencies. A second class provided greater focus with a small corpus of medical research articles where learners were encouraged to look first for non-linguistic information (i.e. content). Working in small groups and still with a substantial degree of autonomy, they successfully pursued their own questions in browsing mode, reporting their findings clearly to others. Ph.D. using his own software, since developed into the highly popular Compleat Lexical Tutor (http://www.lextutor.ca/). The software included corpus data, the objective being to learn hundreds of high-frequency words in a single year. Though control groups using word lists and dictionaries were successful at acquiring definitional knowledge in the short term, the experimental group did significantly better in retaining it long-term and in transferring it to new texts, and rated the materials highly.

		. Involving	C2
		learners in developing learning
		methods: Exploiting text corpora
		in self-access. In P. Benson & P.
		Voller (eds.), Autonomy and
		independence in language
		learning. London: Longman,
		204-214.
	1997,	Cobb, T. (1997). Is there any	These two studies derive from Cobb's A, C1,
	1999	measurable learning from hands-	C3
		on concordancing? System 25.3,
		301-315.
		Cobb, T. (1999). Applying
		constructivism: A test for the
		learner-as-scientist. Educational
		Technology Research &
		Development 47.3, 15-33.
	1997	McEnery, T. & A. Wilson (1997).	A, B
		Teaching and language corpora.
		ReCALL 9.1, 5-14.
	1998	Thurstun, J. & C. N. Candlin
		(1998). Concordancing and the
		teaching of the vocabulary of
		academic English. English for
		Specific Purposes 17.3, 267-280.

Best known for their corpus linguistics work, McEnery & Wilson had considerable interest in early DDL, and their influence continues to this day. This paper outlines key concepts from autonomy, discovery learning and individualisation to self-assessment, correction and translation, with the teacher as guide (cf. JOHNS 1990). It is often cited for the 'percolation' of corpus data and methodologies into language learning and its call for more focused empirical research, as well as a reprise of the trichotomy: learning about corpora, teaching to exploit corpora, and exploiting to teach with corpora. It also argues for pedagogical corpora, and the need for DDL to break out of higher education.

  ).

		B
	2001,	Kennedy, C. & T. Miceli (2001).
	2010	An evaluation of intermediate
		students' approaches to corpus
		investigation. Language Learning
		& Technology, 5.3, 77-90.
		Kennedy, C. & T. Miceli (2010).
		Corpus-assisted creative writing:
		Introducing intermediate Italian
		learners to a corpus as a reference
		resource. Language Learning &
		Technology 14.1, 28-44.

  Some difficulties in DDL relate to the unfamiliar nature of the technology and how to use it; Todd attempts to circumvent these by using regular search engines to provide 'concordances' from the web. Two lexical errors were highlighted in the work of each of 23 postgraduate students in Thailand; most successfully derived 'rules' from their web searches and produced valid corrections. The most difficult items were those with multiple meanings or patterns of use. For an update on DDL-like use of the web as concordancer, see Boulton (2015). 7

	2001	Todd, R. W. (2001). Induction	C4
		from self-selected concordances
		and self-correction. System 29.1,
		91-102.
	2003	Lee, C.-Y. & H.-C. Liou (2003).
		A study of using web
		concordancing for English
		vocabulary learning in a
		Taiwanese high school context.
		English Teaching and Learning
		27.3, 35-56.

B

  Chan, T.-P. & H.-C. Liou (2005).

	Effects of web-based
	concordancing instruction on EFL
	students' learning of verb-noun
	collocations. Computer Assisted
	Language Learning 18.3, 231-
	251.

  Boulton shows here that even lower-level learners are capable of detecting patterns in carefully prepared corpus-derived data after only a five-minute introduction. As with LEE & LIOU

	Boulton, A. (2010a). Data-driven	While many studies have suggested extensive training is crucial (e.g. KENNEDY & MICELI
	learning: Taking the computer out	2001/2010),
	of the equation. Language
	Learning 60.3, 534-572.

  Boulton here reports on 27 evaluations of learning outcomes from DDL. The weight of evidence is positive overall, even if the conclusion at this stage is that the variety of research designs and questions would make a formal meta-analysis virtually impossible (but cf. COBB & BOULTON 2015). The 'real' benefits of DDL may be only partially assessable by short-term experimental studies; more longitudinal studies of cognitive development in particular are called for.

	2010	Conroy, M. A. (2010). Internet
		tools for language learning:
		University students taking control
		of their writing. Australasian
		Journal of Educational
		Technology 26.6, 861-882.
		Boulton, A. (2010b). Learning	Despite frequent claims that DDL lacks empirical support, D
		outcomes from corpus
		consultation. In M. Moreno Jaén,
		F. Serrano Valverde & M.
		Calzada Pérez (eds.), Exploring
		new paths in language pedagogy:
		Lexis and corpus-based language
		teaching. London: Equinox, 129-
		144.

  This case study (cf. TURNBULL & BURSTON 1998) had a single advanced learner in the US using a corpus for writing. Park & Kinginger track all corpus queries and record the computer screen; the participant later reviews these and comments. The results suggest that the writing process is divided into 'transactions' on specific questions, a fifth of which are complex in requiring several related queries. This study is important in providing an insight to the cognitive processes involved in corpus consultation.Pérez-Paredes and colleagues in Spain have been notable for rigorous tracking of how learners use corpora instead of relying solely on emic feedback. The 2011 study provided corpus guidance for the experimental group, who were found to perform better but also to make greater use of other internet resources and complete more activities, despite having less time available. Though the 2012 study found many queries were relatively unsophisticated, it also reports a combination of corpus and other online searches producing the most successful outcomes. These papers further highlight the comparability of corpus and internet searches (cf. CONROY 2010), and suggest that concordance designers have much to learn from search engines.

	2012,	Charles, M. (2012). Proper	C1, C2,
	2014	vocabulary and juicy collocations:	C4
		EAP students evaluate do-it-
		yourself corpus-building. English
		for Specific Purposes 31.2, 93-
		102.
		Charles, M. (2014). Getting the
	2010	Park, K. & C. Kinginger (2010). corpus habit: EAP students' long-	C1, C2
		Writing/thinking in real time: term use of personal corpora.
		Digital video and corpus query English for Specific Purposes
		analysis. Language Learning & 35.1, 30-40.
		Technology 14.3, 31-50.
	2011,	Pérez-Paredes, P., M. Sánchez-
	2012	Tornel, J. M. Alcaraz Calero & P.
		Aguada Jiménez (2011). Tracking
		learners' actual uses of corpora:
		Guided vs non-guided corpus
		consultation. Computer Assisted
		Language Learning 24.3, 233-
		253.
		Pérez-Paredes, P., M. Sánchez-
		Tornel & J. M. Alcaraz Calero
		(2012). Learners' search patterns
		during corpus-based focus-on-
		form activities: A study on hands-
		on concordancing. International
		Journal of Corpus Linguistics
		17.4, 483-515.

  Work on English in Japan is well represented by a series of papers by Chujo & Oghigian and their colleagues, who are particularly interested in the use of DDL at lower levels of proficiency (cf. BOULTON 2010a). This paper has their engineering students use a parallel news corpus to study vocabulary, noun phrases and verb phrases over two semesters, mainly hands-on. Pre-and post-tests show the experimental group making significant gains in almost all areas, while the control group made very limited improvement. The following year, a comparable group did most activities on paper; a mix of paper-based and hands-on DDL is therefore recommended. These two experimental studies by Frankenberg-Garcia compare the ability of Portuguese learners of English to derive information from single or multiple (3-line) corpus examples vs dictionary definitions (cf. BOULTON 2010a) at university level (2012) and in secondary education (2014). Neither study found a significant difference between multiple examples and dictionary definitions in comprehension, but multiple examples were significantly more helpful in production. The 2014 paper selected different examples for receptive and productive purposes, giving still greater advantage to multiple examples on traditional problem items, with implications for lexicography as well as teaching.

			C1, C2
	2012	Chujo, K. & K. Oghigian (2012).	C3
		DDL for EFL beginners: A report
		on student gains and views on
		paper-based concordancing and
		the role of L1. In J. Thomas & A.
		Boulton (eds.), Input, process and
		product: Developments in
		teaching and language corpora.
		Brno: Masaryk University Press,
		170-183.
	2012,	Frankenberg-Garcia, A. (2012).	C4
	2014	Learners' use of corpus examples.
		International Journal of
		Lexicography 25.3, 273-296.
		Frankenberg-Garcia, A. (2014).
		The use of corpus examples for
		language comprehension and
		production. ReCALL 26.2, 128-
		146.

 Taylor, J. R. (2012). The mental corpus: How language is represented in the mind. Oxford: Oxford University Press. Boulton, A. & Cobb, T. (2017). Corpus use in language learning: A meta-analysis. Language Learning 67.2.

Taking HOEY (2005) a step further, Taylor claims that language in the mind is itself analogous to a mental corpus in terms of input, processing and output. Knowledge of a language is the cumulative experience of multiple encounters with different items, with repeated patterns reinforcing each other in normal usage. This accounts for the probabilistic rather than rule-generated nature of language, with its inherent fuzziness and messiness. Learning can be promoted by focus on exposure in context with noticing of exemplars, in line with usage-based models.

A Geluso, J. ( 2013). Phraseology and frequency of occurrence on the web: Native speakers' perceptions of Google-informed second language writing. Computer Assisted Language Learning 26.2, 144-157.

Learners' frequent use of Google as a language aid (cf. PÉREZ-PAREDES et al. 2011, 2012) is exploited here as a route into DDL with only a brief introduction and three short sessions. Geluso had 25 Japanese students search for formulaic sequences using double quotation marks to improve self-selected 'questionable phrases' in their own writing, revising according to frequency. Using a similar procedure to GILMORE (2009), four raters compared the 167 phrases before and after, finding a significant increase in 'naturalness'.

C4

Chang, J.-Y. ( 2014). The use of general and specialized corpora as reference sources for academic English writing: A case study. ReCALL 26.2, 243-259.

Chang's study has two main objectives: to compare a general and a specialised corpus, and to see how learners use them in their own time for genuine writing needs. Ten master's and doctoral students with varying degrees of publication experience used the corpora over 22 weeks; one-to-one advising sessions were recorded and transcribed, and interview data collected. Both types of corpora were found to be useful, sometimes in different ways: the general corpus (COCA) was more 'credible' due to its size and inclusion mainly of native-speaker data, while the specialised corpus of research articles was considered more relevant. In a rare study comparing use of printed corpus data and hands-on concordancing (cf. CHUJO & OGHIGIAN 2012), Yoon & Jo report a case study with Korean learners exposed to five weeks of each treatment. The same writing task performed before and after the course showed substantial improvement in accuracy rates. The correction rate was higher in the indirect use, but the participants appreciated the direct use more and appreciated formulating their own searches. They also used more (and more different) strategies working hands-on; these are discussed in detail based on existing taxonomies. 2015). Teaching and language corpora: Perspectives from a personal journey. In A.

C1, C2

C1, C2, C3

Leńko-Szymańska & A. Boulton (eds.), Multiple affordances of language corpora for data-driven learning. Amsterdam: John Benjamins, 37-62.

Tribble is well placed to offer an overview of how the field has developed over the decades. In addition to a personal retrospective, of interest here is the latest in his series of surveys into corpus use. While the respondents are partially self-selecting, this nevertheless provides a useful profile of the types of people who use corpora in their teaching, the corpora and tools they use, and the purposes they put them to for themselves and for their students


Alex Boulton
email: alex.boulton@atilf.frorcid.org

Data-driven learning for younger learners: obstacles and optimism

Data-driven learning for the next generation: Corpora and DDL for pre-tertiary learners

That DDL means different things to different people is not new. The approach is closely associated with work by Tim Johns who coined the term in 1990; in his various publications, he gave examples of open-ended serendipitous corpus exploration, one-to-one writing correction, class use of hands-on concordancing on pre-determined language points, reusable paper-based materials for remedial grammar, and even 'blackboard concordancing' whereby each learner is given a page of a text, directed to find examples of a particular feature and write them on the blackboard in concordance-like format. So it should come as no surprise that there are many definitions of DDL. At the narrow end, we might adopt a prototype definition, i.e. a kind of core which everyone can agree on. I once suggested this might be something like "the hands-on use of authentic corpus data (concordances) by advanced, sophisticated foreign or second language learners in higher education for inductive, self-directed language learning of advanced usage" (Boulton, 2011, p. 572). With a prototype definition like this, "the further an activity is from the central, prototypical core, the less DDL-like it is, but any cut-off point beyond which we might like to say 'this is no longer DDL' seems likely to be arbitrary rather than empirically grounded or based on a coherent, hermetic definition" (p. 563). While a prototype definition has its uses, in many cases a more general definition is preferable such as that proposed by Gilquin and Granger: "Data-driven learning (DDL) consists in using the tools and techniques of corpus linguistics for pedagogical purposes" (2010, p. 359).

DDL is not a theory of language or language learning, though it can be argued that corpus linguistics has radically changed our understanding of language, giving rise to a number of theories such as the idiom principle [START_REF] Sinclair | Corpus, concordance, collocation[END_REF], lexical priming [START_REF] Hoey | Lexical priming: A new theory of words and language[END_REF], the mental corpus [START_REF] Taylor | The mental corpus: How language is represented in the mind[END_REF], and norms and exploitations [START_REF] Hanks | Lexical analysis: Norms and exploitations[END_REF]. Further, it aligns with several important principles and theories in current thinking, such as authenticity, autonomy, chunking, consciousness-raising, constructivism, critical thinking, complexity and dynamic systems theory, discovery learning, focus on form, individualisation, induction, learner-centredness, (meta-)cognitive skills, noticing, salience, task-based learning, usagebased learning, etc. It is beyond the scope of this short text to discuss these alleged advantages in detail; various aspects are raised in the papers present here (see also [START_REF] Flowerdew | Data-driven learning and language learning theories: Whither the twain shall meet[END_REF]. But if DDL is so wonderful, why isn't it more widespread? And in particular, why has it been comparatively under-researched in the case of younger learners? First, it could be simply that the majority of researchers in the field are themselves university academics using DDL in their own teaching (cf. [START_REF] Chambers | Towards the corpus revolution? Bridging the research-practice gap[END_REF]. This certainly includes me, but the case is not exclusive to DDL, with university students being by far the most extensively studied population in second language acquisition (SLA) and applied linguistics as a whole. Although languages are of course widely taught to younger learners, research in that context tends to be more the domain of education rather than applied linguistics, each field having its own journals and cultures. Unfortunately, the two do not talk to each other very much, with the result that DDL work in applied linguistics remains relatively unknown in educational circles. The consequence of this is not only that DDL is understudied with younger learners, but also that DDL and corpus linguistics tend not to form a major part of teacher training programmes (see Hirata, this volume). Consequently, trainee teachers (outside applied linguistics programmes) are rarely introduced to DDL and cannot themselves use its tools or techniques in their own teaching. As Conrad (2000, p. 556) notes, "the strongest force for change could be a new generation of ESL teachers who were introduced to corpus-based research in their training programs"; until such time as researchers in applied linguistics talk to researchers and decision-makers in education, the current situation is unlikely to change much and DDL will remain on the margins, rarely integrated (Wicher, this volume), and never "normalised" [START_REF] Chambers | Towards the corpus revolution? Bridging the research-practice gap[END_REF]. Of course, the same can be said of the students themselves: school exams and teaching practices need to be constructively aligned for any innovation to have the opportunity to penetrate (Meunier, this volume), and it is understandable that language students (just like student teachers) might resent time spent on practices that do not directly help with exam results (Szudarski, this volume).

A second reason why DDL is less studied among younger learners might be that researchers and practitioners expect it not to work very well in the first place (see also Schaeffer-Lacriox, this volume). This could be because their young age is taken to be synonymous with insufficient levels of language proficiency, an issue much debated even in university contexts, where few projects have explored DDL among lower-level learners. One solution is to use prepared materials of appropriately selected items, and even for hands-on work the texts can be selected to be appropriate for the target users, or graded for level (e.g. [START_REF] Hadley | Enhancing extensive reading with data-driven learning[END_REF]. But it is certainly true that there is a lack of appropriate corpora and tools beyond academic corpus linguistics, as argued by Pérez-Paredes (this volume) who looks at uses of SACODEYL, a corpus of teenage interviews. If existing corpora tend to be geared to older users (or linguists), it is still possible for teachers to create their own corpora that are specifically relevant to younger learners, as shown in several other papers in this volume: corpora of children's news in Di Vito; corpora of the learners' textbooks in Papaioannou et al.; multimodal corpora in Hirata.

The language points discussed in DDL also tend to be relatively advanced, but this is not in itself an indication that DDL is inaccessible for other features appropriate to lower levels. Liontou (this volume), for example, provides successful results in the "notoriously difficult" area of idiomatic phrases. Indeed, most of the studies with lower levels have found promising results, although two recent meta-analyses of DDL come to slightly differing conclusions. For DDL as a whole, [START_REF] Boulton | Corpus use in language learning: A meta-analysis[END_REF] analysed 88 unique samples from 64 studies and found that proficiency did not systematically correlate with effect sizes, suggesting that it is open to all levels. [START_REF] Lee | The effects of corpus use on second language vocabulary learning: A multilevel meta-analysis[END_REF] focused on DDL just for vocabulary in 38 samples from 29 studies, and did find a medium difference between advanced and lowlevel learners, but there was still a significant effect in all cases. Caution should always be a watchword in dealing with such findings since proficiency is not only difficult to assess, but is reported in so many ways that what is 'advanced' in one study might be considered 'intermediate' or even 'low' in another context. Of studies apparently with advanced learners in major CALL journals, [START_REF] Burston | The contribution of CALL to advanced-level foreign/second language instruction[END_REF] found that fully 50% were at best B1 level on the Common European Framework of Reference for Languages, which is commonly thought of as lower intermediate. But coming back to the main point: "DDL works pretty well in almost any context where it has been extensively tried" (Boulton & Cobb, 2017, p. 386).

Another explanation might be that children lack the cognitive development necessary for sophisticated language work. A number of studies have addressed DDL with children in exploring their mother tongue (L1); some of these are mentioned in Crosthwaite and Stell (this volume). Over ten years ago, John Sinclair, often considered the father of modern corpus linguistics with the COBUILD project, planned the wide-ranging introduction of corpora to primary schools in Scotland via a tool known as PhraseBox:

PhraseBox… is like giving each pupil real-time access to a huge memory of all the different ways in which thousands of people have expressed themselves over several years, all instantly available in a highly organised presentation. Gradually, students are expected to internalise what they need of the resource and gather confidence in their ability to express themselves publicly; but the resource will always be available when it is needed. (Sinclair, 2006, n.p.) Sadly, the resource never did become available, as Sinclair died in 2007, and the project along with him. I was fortunate to see him demonstrate it with Ana Mauranen at the IVACS conference in Nottingham in 2006, but he wrote little about it (see also [START_REF] Stubbs | A tribute to John McHardy Sinclair[END_REF]. Nonetheless, the very fact that he and people in a national education system believed in it is revealing in itself: there is no a priori reason to assume that younger learners are unable to use the technology and analyse language in ways compatible with DDL. This brings us specifically to the issue of ICT (information and communication technology), and whether younger learners have the skills necessary for DDL. To counter that, it might be pointed out that DDL can be conducted using prepared, paper-based print-outs which still give medium effect sizes [START_REF] Boulton | Corpus use in language learning: A meta-analysis[END_REF]. Further, younger learners who have grown up with ICT may actually be more relaxed about it than older learners, as discussed by Gatto (this volume). A case in point: some of my students in an English master's degree have difficulty with the wildcard function which so impressed a ten-year-old in Crosthwaite and Stell (this volume).

Finally, the fact that there are few published studies of DDL for children does not necessarily mean that it is absent from common classroom practice. Johns and King (1991, p. iii) situated DDL "within the overall aim of developing students' ability to puzzle things out for themselves." For our purposes, this has to involve language data in one form or another, but otherwise that leaves the field very open. So in particular, while the internet is not a 'corpus' and Google is not a 'concordancer', it is quite possible that teachers are encouraging their learners to explore language without ever having heard of DDL. As McCarthy (2008, p. 566) puts it, "we are, all of us, corpus users, because we use the internet." And if we are using the internet as a source of repeated language examples, we are all doing DDL (Gatto, this volume; see also [START_REF] Boulton | Applying data-driven learning to the web[END_REF]. As one student remarked of COCA in the study by Papaioannou et al. (this volume): "This is just another tool the internet offers us to learn new words". And of course it is possible that learners are doing this without their teachers' knowledge, or even against their recommendations. Meunier (this volume) discusses several tools that provide opportunities for very DDL-like activities, using accessible corpora (e.g. TV series such as Breaking Bad or Game of Thrones in www.playphrase.me), and with no mention of 'corpus' or 'concordance' or other problematic terms.

Research can be based on actual practice, and can certainly inform it, but should never be 'applied'. The main consumers of research are other researchers rather than teachers or decision-makers (e.g. [START_REF] Borg | English language teachers' conceptions of research[END_REF], but when teachers do read research, it is essential that they interpret and adapt research findings to their own local contexts and needs: existing reports of DDL aimed at students and adults should not be simply transformed into a 'corpus approach for kids'. If the search for a single 'best method' has been largely abandoned in recognition of the diversity of teaching goals, contexts, needs and profiles of the individuals involved [START_REF] Brown | Forty years of language teaching[END_REF], it cannot be desirable to impose DDL as the sole form of language instruction. Researchers thus also need to remain open to a variety of approaches, and not be dogmatic about what is or is not DDL and how it should be implemented across the board. As Meunier (this volume) argues, we need to think outside the box and go off the beaten path to promote creativity in DDL. It is refreshing that some of the examples given in this volume (e.g. Liontou; Papaioannou et al.) are quite different from usual expectations of DDL. What we have is a host of variants under the umbrella term of DDL, which have their place alongside others for some learners in some contexts and for some language questions. I hope this short foreword has shown that the common assumption that DDL is not suitable for younger learners is just an assumption -an absence of research evidence merely means that it has not been widely tested, not that it is not or cannot be effective. DDL might even be more appropriate for children than for older learners, as their expectations are less fixed and the approach lends itself to the inquisitive; the children in Di Vito (this volume) remembered the methodology well because they felt involved and valued and encouraged to think for themselves. There certainly are obstacles and we should not be blasé about them (Schaeffer-Lacroix, this volume), but there are any number of reasons to think that DDL, appropriately implemented, can bring something to the table in a variety of contexts. The present volume thus addresses a genuine need for DDL research among younger

'Empirical' here is defined as studies that "subject some aspect of DDL to observation or experimentation with some kind of externally validated evaluation other than the researchers' own intuition"(Boulton,

2010, p. 130).


Rodrigo Dantas Da Silva

Jean Jar Pereira De Araújo

Álvaro Ferreira
email: alvaro.ferreira@lais.huol.ufrn.br

Pires De Paiva

Ricardo Alexsandro

Medeiros Valentim

Dantas Karilany
email: karilany@ufrnet.br

Jailton Carlos De Paiva

Azim Roussanaly
email: azim.roussanaly@loria.fr

Anne Boyer
email: anne.boyer@loria.fr

Jean Jar Pereira

Jean Jar
email: jean.jar@lais.huol.ufrn.br

Pereira De Araújo

Coutinho

Ricardo Alexsandro De Medeiros Valentim

A Big Data Architecture to a Multiple Purpose in Healthcare Surveillance: The Brazilian Syphilis Case

Keywords: Information systems → Mediators and data integration; Enterprise application integration tools big data, healthcare surveillance, syphilis, epidemiology ACM

établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

The use of technologies for analyzing big data is not new, but it has been drawing attention especially in recent years. With the arrival of the 2000s, the strengthening of social networks, mobile devices and web 2.0, some players from the technological world had to adapt to meet the demands. Conservative studies estimate that enterprise server systems in the world have processed 9.57 zettabytes of data in 2008, this number is expected to have doubled every two years from that point, these examples provide a small glimpse into the rapidly expanding ecosystem of diverse sources of massive datasets currently in existence [START_REF] Kambatla | Trends in big data analytics[END_REF]. In 2004, the world introduced the big data processing model called MapReduce, developed by Google. The major contributions of this work is a simple and powerful interface that enables automatic parallelization and distribution of large-scale computations, combined with an implementation of this interface that achieves high performance on large clusters of commodity PCs [START_REF] Dean | MapReduce: Simplified Data Processing on Large Clusters[END_REF] [START_REF] Ghemawat | The Google File System[END_REF]. This processing model not only made room For its creator, other companies took advantage of this new technology and embraced the idea, developing new applications. Apache Hadoop is an open source software framework that dramatically simplifies writing distributed data intensive applications since the primary primitive of map / reduce is a distributed sort, most of the custom code is glue to get the desired behavior [START_REF] White | Hadoop: The Definitive Guide: The Definitive Guide[END_REF]. This kind of approach did an effect not only within the virtual world, on social networks, but also on people's real life, in their daily lives. At the same time as technological evolution already directing us to generate much more data. In a few years we will experience a significant increase in connected devices, thanks to the advent of IPv6 and the 5G connection standard. Big Data, like Big Oil, is big precisely because it can control access to data as well as extraction of information and knowledge, and we need to pay attention to that fact because knowledge, like energy, is not just a passive, inert resource. It is fuel: fuel for our ideas, our actions, everything. And the power that comes with control over that fuel is therefore formidable [START_REF] Lynch | The Internet of Us: Knowing More and Understanding Less in the Age of Big Data[END_REF].

One of the most prominent areas in the adoption of this technology, which has already presented results, is the health area. This is an area that is basic to anyone, where a single medical procedure can contain structured and unstructured data, images, audio and digital signals. By scaling this scenario to the reality of a country where thousands of people use health services a day, we then bring dialogue into the scope of big data technologies. Using health data from a population or even an individual can help reduce costs, fight endemics, treat rare diseases, and so on. By analyzing a patient's medical data it is possible to see the patient's clinical picture broadly.

One subarea of health that proves to be a major investment field is epidemiology. It is the branch of medicine that studies the different factors that influence the spread of diseases, their frequency, their mode of distribution, their evolution and the means necessary for their prevention. In other words, it is the area of science responsible for analyzing the health-disease relationship with social, environmental and biological dimensions. In the 1950s the World Health Organization (WHO) convened a committee that could propose a method to define and assess the standard of living of a population. However, it was concluded that it would be impossible to construct a single index. It was then suggested 12 items that should be evaluated separately, the first of which being "Health and Demography", focused on what is expressed as the health level of a population [START_REF] Laurenti | Estatísticas de saúde[END_REF]. Since then, the assessment of the standard of living of human populations has been a matter of interest to public and private institutions for a long time. We currently monitor every small step of the patient within the system from birth to death, detailed by his or her passage in the public health system.

For epidemiology, the variety of sources and diversity of data types is nothing new. Equally unsurprisingly for epidemiologists is the work of uniting datasets of different natures to obtain answers. However, the large increase in data volume presents new challenges, such as data validation, to avoid, for example, the reduction in data quality. The process of data surveillance is essential for the development of analyzes, as they, if low quality, can bias the whole results. In our current society, one person is capable of generating hundreds of data per minute. We make use of our smartphones as essential for living, we increasingly use wearable devices that monitor our health [START_REF] Young | A "big data" approach to HIV epidemiology and prevention[END_REF] [START_REF] Mooney | Commentary: Epidemiology in the era of big data[END_REF]. We are increasingly adopting an automated lifestyle with smart devices in our homes, cars and workplaces. Many of us are addicted to social networks and we open the door to our virtual world.

In Brazil, in 2016, the Ministry of Health created an agenda of strategic actions for the qualification of health professionals in the diagnosis and treatment of sexually transmitted infections. The following year, the Union Court of Auditors realized after an extensive audit that the number of syphilis in the country had skyrocketed, growing by over 5000% in less than a decade. Given this scenario, the Ministry of Health has launched a project to combat and rapid response to syphilis, a campaign called "Syphilis No!" Given these facts, the present work proposes an architecture for data storage and processing that, in its conformation, allows the standardized availability of data from the areas of health, social data and social network. It also allows researchers from different institutions to use this data to study theories and revalidate studies with the objective of meeting public health needs, such as epidemiological surveillance and the current situation of syphilis in the country.

The next sections will better present the project "Syphilis No!", The architectural models for big data solutions and data manipulation, the methodology adopted in this work for its development and what results have already been achieved.

THE PROJECT "SYPHILIS NO!"

Syphilis is a secular disease, present in various passages of history, which is once again becoming endemic. But not only in Brazil, case numbers have also grown in countries in North America, Europe, Asia and Oceania. Despite being a disease of simple treatment and diagnosis, it is having room to spread in society. Therefore, the Ministry of Health understood that there was no space for traditional methods, a communication campaign that reached all ages was necessary, a campaign to strengthen the capacities of health professionals, as well as the development of new techniques of monitoring and technologies for the most accurate and effective diagnosis. There was an indispensable need to create a horizontal process between the various areas of modern society so that we could indeed overcome the barrier of casual habit and indeed create technical and technological capacities to break the primacy of a disease that could already have been eradicated. The Ministry of Health Rapid Response to Syphilis in the Healthcare project that received the popular name "Syphilis No!" It is an action that goes beyond the classic campaigns of testing, prevention and treatment.

This project, started in 2018, is a partnership between the Brazilian Ministry of Health, the Pan American Health Organization (PAHO / WHO) and the Federal University of Rio Grande do Norte (UFRN). The project already operates in all states of Brazil through the work of supporters hired to create an interface between the project and the locations with the worst indicators. The project also provides for ongoing training activities for health professionals, public awareness campaigns, in addition to promoting academic and medical research focused on new technologies to support and combat the disease.

THE ARCHITECTURE FOR BIG DATA

When we talk about data we need to go back to several basic concepts. Data is the smallest unit of information, and knowledge is the aggregate set of information that promotes state change. In the computing world the most traditional way of storing data is through databases. The CAP Theorem, or Brewer's Theorem, tells us that it is impossible for distributed data storage to simultaneously provide more than two of the three guarantees that are:

• Consistency: Each partition receives the most recent write or an error. • Availability: The system continues to function as expected even if nodes fail. • Fault-Tolerant Partition: The system continues to function despite an arbitrary number of messages dropped (or delayed) by the network between nodes.

In other words the theorem states that for a fault tolerant partitioned system, it would be necessary to choose between consistency or availability. In terms of large data processing can take a long time to execute a simple query. These queries could not be executed in real time and the processing delay would eventually return a result from hours ago.

Lambda Architecture

Seeking to solve these problems Nathan Marz published in his personal blog in 2011 a possible solution that allowed all three guarantees for distributed data storage systems, he called Lambda Architecture [START_REF] Hausenblas | Lambda architecture[END_REF] [6] [3]. Marz's proposal solves this problem by creating two paths to the data flow, as shown in the Figure 1. All data received by the system goes through both data streams. • Batch Layer or Cold Layer: stores all input data in its raw form and makes the data available for batch processing.

Results are stored in another area for consumption. • Quick Layer or Hot Layer: analyzes data in real time. This data is constrained by latency requirements so that it can be processed as quickly as possible. This usually requires some disadvantage, which in this case is the level of accuracy. Both layers serve a service layer, which indexes the batch data and receives incremental updates of the hot data. Raw data stored in the batch layer is unchanging. That is, new data is always added and old data is never overwritten. Any change to a specific data is taken as a new record with a new timestamp. These rules allow recalculation at any point in time in the data history. The ability to recalculate raw data is important as it allows new views to be created as the system evolves.

A disadvantage of Lambda architecture is its complexity. Data processing logic appears in two paths (hot and cold) using different structures. The complexity of administration and the risk of duplication in calculations can be a serious problem.

Kappa Architecture

Kappa architecture comes as a simplification of Lambda. The proposal is similar, just delete the cold path and to process batch data simply send it as a stream and process it by the hot path [START_REF] Lin | The lambda and the kappa[END_REF], as shown in the Figure 2. The Kappa architecture comes with the proposal to make data canonical, that is, to standardize data so that it is possible to select and dynamically shape the execution of one or more business services. Simply put, the way Kappa works is as if all data were logged as a log of a system or service, and this log is provided for the architecture to process and store through a continuous stream of ingestion [START_REF] Leadbetter | Where big data meets linked data: applying standard data models to Figure 4: Sankey diagram depicting the profiles of people who oppose syphilis between 2007 and 2017. Image of the author.. environmental data streams[END_REF]. This architecture was designed by Jay Kreps in 2014. An evaluation by [START_REF] Sanla | A Comparative Performance of Real-time Big Data Analytic Architectures[END_REF] to compare the architectures showed that Lambda uses 2.2 times longer than Kappa architecture, and that Lambda architecture needs about 10 -20% more CPU and 0.5 GB of RAM.

ETL vs ELT

Since the emergence of Business Intelligence solutions and the entire area of Analytics, some techniques and concepts have been created, developed and gained their space, especially when we talk about the main source: data. One of the main structures in these areas is called Data Warehouses (DW), large data warehouses that seek to structure data in order to answer the operational questions of a business area. With the advent of DWs a process has evolved and become essential, the Extract-Transform-Load (ETL).

ETL is a three-step data processing technique which in practice means extracting data from a source, transforming it to correct any abnormalities and suit business needs, and ultimately carries it in a new framework that supports the queries you want. Although ETL provides great solutions to many problems, it does generate some other problems for itself. When we talk about large volumes and data diversity the ETL technique no longer meets the needs. Suppose you need to process thousands of files with many GB of data and that this data needs to be made available to countless applications. According to [START_REF] Wibowo | Problems and available solutions on the stage of Extract, Transform, and Loading in near real-time data warehousing (a literature study)[END_REF] and [START_REF] Petrova | Puzzle Methods for Automatic Selection of Data Cleansing Techniques[END_REF] the ETL process can be easily complex, underperforming and facing data availability issues. Also according to the authors, the ETL process was conceived in such a way that the Extraction and Loading steps must occur at times when the source and destination systems of the data are in their maintenance windows, so that it does not occur completely. of services.

For authors [START_REF] Michel Marín-Ortega | ELTA: New Approach in Designing Business Intelligence Solutions in Era of Big Data[END_REF] while the database technologies used for DW have evolved in performance and scalability in recent years, the ETL process has not evolved at the same pace. As a result, most BI infrastructures are facing bottlenecks: they can't easily get data on demand. The authors further state that, to eliminate the disadvantages of ETL, the adoption of new storage media. They suggest that the Extract-Load-Transform (ELT) approach may overcome these needs. The basic idea behind the process is to extract the data, store it as captured, and transform it only at the time of use, tailoring the transformations to suit different needs. The authors suggest four advantages of ELT over ETL:

• Flexibility to add new data (EL part of the process).

• Aggregation can be applied countless times on the same data (part T of the process). • The transformation can be retrofitted even on legacy data.

• Accelerated process of implementation.

METHODOLOGY

The main objective of this work is to build a data architecture that allows other researchers to have access to data, concentrating not only data regarding the health of a population, but also data from areas that indirectly influence the shaping of a society and thus reflect how public health policies are implemented. The work is thus divided in the construction of a cluster for storage and processing of this data, a parallel step of curating the historical data of society in the areas of public health, demographic, economic and education. We chose to use the ELT process because the ultimate purpose of this architecture is to provide data from multiple sources to various systems and researchers, that is allowing the end user to decide which structure is best for the data they want to consume. It is also guaranteed that the data will be exactly the same for any purpose.

The Cluster

A cluster capable of storing and processing 8TB (terabytes) of data was built. This cluster was structured according to Lambda architecture, using Apache Hadoop version 3.0.3 as its development and management platform. For data repository was implemented Apache Hive version 3.1.2 on Hadoop. For cluster administration, Apache Ambari 2.7.4 is used. There are also two separate servers, one for storing the original data and one for data manipulation. The latter is used only for conformation of variables, such as gender normalization, as well as file format conversion. The choice for this structure was due to the scalability of the storage and processing infrastructure at a viable project cost.

Over the cluster was implemented an inverted indexing service, in order to obtain greater efficiency in the process of searching and processing data. Reverse indexing works like a book's index, which shows which pages occur in a given term, unlike traditional ways of storing data where an id is associated with a given data set. This service is used as a "gateway" by the data ingestor, thus allowing a prior analysis of the data before storing it in the repository. The pre-parse process allows you to map all values that dataset has, after this analysis the system creates a unique hash code for this dataset by inserting it into the repository. The service also provides a RESTfull service so that other applications can consume data as desired. An abstraction about the architecture can be found in the Figure 3 below.

Data Curation

Data from Ministry of Health systems are available in monthly batches, representing all consolidated data of the corresponding competence. Data regarding outpatient production, hospital production, mortality, disease notification, health facility infrastructure, health teams, health professionals and health financing are captured. Data on basic education, higher education and education funding are also being collected in batch format. Other annual frequency data were also collected, such as municipalities' Gini indicator, municipalities' GDP and population. All data are previously studied by the development team of this project, with the support of public health experts. The databases that have data dictionary, go through a parameter check, in order to see if the collected data actually correspond to what theoretically should, according to the dictionary. Other data, such as notifiable diseases, do not have a dictionary. In this case the notification forms are used to construct a dictionary so that the parameter matches can be checked. In this last scenario, the construction is accompanied by nurses and epidemiologists for technical support. None of the bases are discarded if they fail to match variables and the dictionary. This routine is only for mapping out what needs a particular source can meet.

The curation process also guides how data should be normalized before it is inserted into the cluster and made available to researchers. Variables corresponding to gender, education, locality, procedures and others, change their insertion according to the origin system. Thus, an organizational pattern of the data is created so that all data can correspond to the same universe of options, as shown in the table 1.

RESULTS

The project is still in its first months of development, however some experiments have already been performed and some previous results have been obtained. Approximately 2TB (terabyte) of public data on health, education and economics, commensurate with the past decade, has already been collected.

One of the first experiments performed using the built data architecture was to understand who are the syphilis patients in Brazil. For this, we used data corresponding to the last 10 years of syphilis notifications provided by the Ministry of Health. A simple clustering technique was used observing some social variables of the patients, such as ethnicity, education and age group. We found 130 distinct groups of characteristics using the k-means algorithm with 97.41% accuracy. A representation can be found in the Figure 4 which shows in the first column the gender, at the second one the ethnicity, at the third the education level and last but not least, the fourth column shows the age range.

This result reinforces the need and objectives of these projects. First, we must observe the dimensionality of the different groups found. Brazil is a country with just over 200 million inhabitants and there are 130 different representations of syphilis patients. Second, understanding the behavior of this population in their social contexts will help to see how to create actions for these 130 groups. We realized, for example, that a portion of young people between 20 and 29 years old had complete medical education, what was missing in the training of these young people to create awareness about STIs? In other words, what actions are necessary to reach this segment of the population?

DISCUSSION

For many years, some governments have been developing public health policies without seeing the causal facts and consequences of these actions in a given society. Another point is that some public policies do not consider the country's dimensionality, as is the case in Brazil, which often presents geographical, cultural, financial, educational and social diversity. In view of these points, it is essential that we understand the various locoregional structures so that we can build targeted strategies, adapted to the real needs of a city, state or region. This allows for much more efficient responses and better use of the budget. But this whole vision had never been taken from paper.

The project proposed here brings to light new possibilities for public health, going beyond the traditional clinical view. The ability to look at data from different areas and understand the social behavior of a disease enables health managers to act in a preventive way, but not in a generic and standardized way for the whole country, but observing the peculiarities of each location. Some of the studies in progress will also result in the statistical mapping of the health care network, such as the relationship between the technical capacity of health professionals and the capacity to respond to a certain condition. Allowing, for example, to understand the network of hospital beds in a given region and allowing to assist in the best order of use.

It is worth mentioning that a web platform for researchers' access management is under development, which can find in this space the database catalog, with the description of its variables, relationship between databases, as well as the option to download a sample for develop your research work according to the filtering options. This platform is really a project concern with the rules of the General Data Protection Law (LGPD / Brazil) and the General Data Protection Regulation (GDPR / Euro).

Figure 1 :

 1 Figure 1: Representation of Lambda Architecture. Author's image.

Figure 2 :

 2 Figure 2: Representation of Kappa Architecture. Author's image.

Figure 3 :

 3 Figure 3: Abstraction of the implemented architecture focused on layers of interaction from the process of data acquisition, indexing, storage and retrieval to provide service. Author's image.

Table 1 :

 1 Sample codes found used for gender

	Samples	Standardization
	Man, Woman, Male, Female, M, F, 1, 2	Man, Woman

ACKNOWLEDGMENTS

I would like to acknowledge the research team at the Laboratory of Technological Innovation in Health and the Federal University of Rio Grande do Norte as well as the Brazilian Ministry of Health for funding this research.