-Individual support to ESFRI and other world-class research infrastructures

Executive Summary

The current report is based on in-depth interview with representatives from six DARIAH Accession countries (DACs: the Czech Republic, Finland, Israel, Spain, Switzerland and the United Kingdom) as well as members of the DARIAH Board and Chair of the DARIAH VCC2.

The following summary presents key findings and recommendations that were identified in a qualitative assessment of the interviews.

• Investing time and effort into training and educating users is a way of securing the social sustainability of a research infrastructure.

• DARIAH is, among other things, a social infrastructure: DARIAH users need instruction on how to use DARIAH services. But training and education is more than that: it is a way of community building.

• DARIAH should be seen as complementing, not replacing, the existing universitybased educational offerings, especially in terms of providing alternative educational measures such as hackathons, summer schools, masterclasses.

• DARIAH is seen as a consolidator of community-approved standards and baselines regarding the use of technology in the humanities as well as the skill sets needed to engage with technology.

• When it comes to DARIAH's role in the development of training materials, opinions differ, ranging from recommendations that DARIAH develops its own training materials to the argument that DARIAH as a meta-organization should only be responsible for consolidating existing resources. This is a key point of divergence that should be further explored by DARIAH as it develops its strategic vision for training and education.

• Developing DH and infrastructure-related skills in the DACs is facing numerous challenges:

o lack of established degree programs in Digital Humanities. While degree programs are important for the academic validation of a research field, they are not seen by the majority of our informants as the alpha and omega of DH skills development. o interdisciplinarity is both an asset and a challenge. Interpersonal skills are essential for the success of interdisciplinary teams and, by extension, of digital research infrastructures for the arts and humanities.

o even when students acquire new skills in dedicated workshops or DH summer schools, there is no guarantee that they will be able to use those skills in their regular university coursework. University instructors in the Humanities need to learn DH skills as well. It is easier to teach individual tools than it is to change the methodological tenets of the traditional humanities departments.

• DARIAH should, when possible: o communicate better about its infrastructural offerings, the organizational structure, as well as its members, their roles and contributions. This is important for all DACs: it affects equally the lesser-resourced countries that are in the process of creating national infrastructures but have fewer training options in Digital Humanities; as well as the well-resourced countries that need to understand the added value of joining DARIAH. In this sense, a taxonomy-based approach on what can be offered locally, nationally or, indeed, internationally has been suggested as a useful way to proceed.

o use workshops and other training measures also as venues for identifying specific requirements and missing skills for developing DH and using the DARIAH infrastructure in lesser-resourced countries.

o not only share tools and best practices but create opportunities for trainees to spend time working face-to-face in order to learn in a non-virtual context. Summer schools, workshops and masterclasses are not only places in which participants can acquire new technical skills, but, as importantly, can learn how to communicate better with colleagues from different disciplinary backgrounds and different skills.

o specifically support international workshops where participants and instructors do not all come from the same country. This is important not only for the sake of exposing participants to different academic cultures, but also for the sake of testing DARIAH tools and methods in different national and institutional settings.

Introduction

The Digital Infrastructure for Arts and Humanities (DARIAH) is both a knowledge network and a service provider. In both of these areas of activity, DARIAH plays an important social role: enabling smart research using digital tools, but also contributing to the development of the Digital Humanities as a transdisciplinary community of scholarly practice.

Infrastructures need users, yet catering to a very diverse user base, with varying competences and interests, across many national borders, is a challenge that should not be underestimated. That is why training and education is one of the priorities for DARIAH. Infrastructures need users, and those users who are well-trained in the service provided by the infrastructure will be in the position to benefit the most from the infrastructure itself.

The growth of a research infrastructure like DARIAH is dependent on the growth and liveliness of its communities. Expanding the existing user base in DARIAH Member Countries (DMCs) and attracting new countries to join DARIAH are therefore essential. Investing time and effort into training and educating users is a way of securing the social sustainability of a research infrastructure.

Aims

The aims of this report are two-fold:

1. to provide a general overview of the skills base in the so-called DARIAH Accession Countries (DACs), i.e. six countries whose institutions are represented in the DESIR project: the Czech Republic, Finland, Israel, Spain, Switzerland and the United Kingdom1 ; and 2. to contribute to the ongoing discussions within DARIAH about the role and modalities of training and education in the development, consolidation and sustainability of digital research infrastructures.

The six DACs that are discussed in this report are very different in size, population, levels of digital literacy and use of digital tools in humanities research, as well as political context: four countries are members of the European Union, one is not (Israel) and one is on its way to exit the 28-member bloc.

Methodology, Structure and Perspectives

The main basis of this report are in-depth video interviews that were conducted with representatives from DACs. Additional resources (reports, articles, books) were used to inform the discussion of the educational offerings in each country, but the report does not pretend to be exhaustive in any way.

Our informants were selected based on two criteria: a) their knowledge of the local educational landscape; and b) their involvement in the DARIAH accession process.

Each DAC section consists of three sub-sections:

1. general background; 2. details about the informant; and 3. analytic summary of the main points raised in the interview.

While other methodological approaches (including surveys and statistical data) would without a doubt provide additional insights into the topic of this report, a qualitative assessment of the in-depth interviews has given us a unique chance to address some of the complexities and highlight some very specific needs in the DACs.

At the same time, in the Background section on DARIAH, we offer preliminary results from the DESIR Survey which included some relevant questions regarding training and education.

A detailed analysis of the survey results will be part of DESIR deliverable D6.2 "Empirical investigation of trust, gender and diversity in cross-disciplinary DARIAH communities", which is in preparation.

In addition to the interviews conducted with our DAC informants, we also interviewed members of the Training and Education has been recognized by the DARIAH Strategic Action Plan (STRAPL 3 ) as "an area where DARIAH has both momentum, unique expertise, and a track record of delivering high value assets for the community" (4) and associated with two key actions:

• Action 21: A systematic approach to capturing DARIAH events (workshops, summer schools, etc.) for future reuse through the DARIAH training channels by new audiences will be designed and implemented.

• Action 25: The suite of training materials developed through the various DARIAH projects and initiatives will be consolidated and promoted to users within the DARIAH ecosystem and in relevant wider communities (e.g. CLARIN, E-RIHS).

DESIR Survey

While the current report is based on qualitative assessment of in-depth interviews with informants from DARIAH and DACs, it is important to mention that questions regarding DARIAH's role in training and education have been included in the DESIR deliverable D6.2. The survey targeted researchers, professors, MA and PhD students within the arts and humanities community, users and potential users of Digital Humanities infrastructures, from universities or other research institutions, with or without any permanent institutional affiliation.

The questionnaire included sections on personal data, academic background, institutional characterization, use of Digital Humanities, experience and opinions on DARIAH services and tools. The results of this survey will be anonymous and strictly confidential.

The DESIR survey was conducted on-line using Limesurvey. It was disseminated in a number of universities and research institutions on arts and humanities in the 17 DARIAH members, as well as in entities from third countries. The survey was conducted from 11 th January to 18 th April 2018. It solicited 507 responses.

Based on the preliminary reading of the survey results, we can report the following.

Have The majority of the respondents didn't know DARIAH before this survey (54%). Only 12% of the respondents declare themselves as users of DARIAH services and tools, while 33% of the respondents who has already heard about DARIAH before this survey answered that they are not users of its services and tools. *Only if answered "yes" to "Are you a user of DARIAH services and tools?"

Among the respondents who are users of DARIAH services and tools, the quality of DARIAH training materials is rated mainly as 4 and 5 in a scale 1 to 6. DARIAH is a social infrastructure --or, as Laurent Romary calls it: a "people infrastructure", which is why training and education plays such an important role in it. It should, as Agiatis Bernardou points out, inspire researchers by exposing them to the state of the art and creating environments for intellectual exchange.

Informants

Even though Digital Humanities are becoming increasingly visible on the European landscape, the training and education in DH tools and methods is less established especially in terms of the more traditional, disciplinary university programs. This is why teaching DH has to be based on different pedagogical formats and engage with different disciplinary cultures. This means that a flexible approach is necessary: training needs to be adaptable because there is no one single form of training that fits all the communities and all the different levels of knowledge.

Learning a programming language, for instance, does not have to take place in a regular course at a university. Intensive summer schools are, according to Frank Fisher, often more suitable venues for that kind of work. "Doing DH" requires getting one's hand "dirty" on the keyboard, coding, and not only writing Word documents.

This is why research infrastructures could be seen as complementing the existing educational offerings, especially in terms of providing alternative educational methods in settings such as hackathons, summer schools, masterclasses and delivering reference training materials such as those in #dariahTeach, for instance. DH calls for this type of alternative education, and DARIAH can provide it.

DARIAH should offer researchers a stable ground to build upon and to prepare for the next technological leap. DARIAH should make it easier for researchers to make the leap to technology earlier, and for this reason, it has to be accessible and easy to use. Young researchers have to have an idea of where to find the tools, they have to understand the context and the community, including colleagues they could collaborate with. And all of these things don't have to be next door but can be anywhere and DARIAH should be able to provide access to them.

Accession Countries

3.1 Czech Republic

Background Information

Compared to many DARIAH member countries, the Digital Humanities landscape in the Czech Republic is quite young and not equally spread across a large number of institutions, research projects or communities. Instead, a small group of pioneering institutions is currently shaping Digital Humanities in the country.

The Library of the C zech Academy of Sciences, the Knihovna AV ČR (https://www.lib.cas.cz/en/), has played a prominent role in this respect for two reasons: 1)

The library -as a service provider for the Academy -is a holder of numerous collections of cultural heritage as well as research publications. 2) the library as a research infrastructure provider has itself been intensely involved in the digital transformation of the sciences. The Czech Digital Library (https://www.czechdigitallibrary.cz/en/) may serve a good example of this process. From the C zech perspectives, one may consider libraries as natural allies for the uptake of Digital Humanities.

The two largest universities of the Czech Republic, the Univerzita Karlova (C harles University Prague) and the Masarykova Univerzitet (Masaryk University), are the other important players in the field. According to our informant, they are currently working on implementing dedicated DH programs. This is important because currently nearly all DHrelated programs are embedded in media studies4 . Also the Západočeská univerzita v Plzni (University of West Bohemia) is considering implementing a DH programme. Currently researchers acquire DH skills on an individual basis. One the challenges in this respect is the fact that training materials are usually accessible only to students officially registered for the given course. Open Access policies currently cover research outputs such as scholarly publications or dissertations, but not necessarily training materials.

The funding provided by the European Commission is seen as an important catalyst in this regard as it is encouraging cooperation between various institutions on a national and European level. The Library of the Czech Academy of Sciences is applying to be included on the national roadmap for research infrastructures in order to secure financing of DARIAH-CZ from 2019 on.

Informant

Martin Lhotak5 has been a member of staff of the Library of Czech Academy of Sciences for over 21 years and is currently Library Director and also directly responsible for its IT and digitization department. The IT department of the library provides information systems, databases and software tools for digital humanities. The day-to-day work of the research and development department is focussing on digitisation projects and only to a smaller share on DH-related projects. Martin's own educational background is in information science.

Analytic Summary

Digital Humanities are not yet established as an academic field in the Czech Republic, although a number of researchers are using digital tools (not necessarily identifying themselves as DHers) and libraries provide access to numerous digital collections.

For the the development of a humanities research infrastructure in the Czech Republic and the country's accession to DARIAH, a handful of pioneering institutions (such as the Library of the Czech Academy of Sciences, Charles University and Masaryk University) play a decisive role.

Existing European research infrastructures and communities such as CLARIN or DARIAH are seen as essential hubs for promoting progress on the national level as well. They not only function as institutional frameworks for setting up projects but they are also seen as communities of experience and communication, which can provide valuable return on national investments. This correlation has been highlighted with the example of the LINDAT-CLARIN Consortium.

DARIAH should pay special attention to how it (re)presents itself to newcomers: both in terms of how it defines itself (a research infrastructure? a community) and in terms of how easy it makes it for newcomers to participate in it. The role of DARIAH as a social infrastructure and a learning framework needs to be made clearer for accession countries and new partners.

The vision of DARIAH as a data-sharing community is an attractive one, but DARIAH should do more to establish and promote common standards in terms of technical and research data interoperability. This is something that is evident even when looking at various national contributions to DARIAH.

Finland

Background Information

Finland has eight universities and the majority has its own minor programme related to subjects such as digital culture or information studies or information technology. Helsinki University is currently the only one with a Digital Humanities programme, with a minor subject study block and a master degree level.

This university also organizes an annual intensive course, in May, with students that have completed the minor study block or students from other universities, where the students can put their knowledge of Digital Humanities tools and methods into practice. These workshops are held with the support of other Finnish universities and cultural heritage organizations, and foster multidisciplinary approaches by bringing together students and scholars from the humanities and Computer Science. These courses frequently result in publications and new projects.

A DARIAH DiMPO survey conducted in 2016 found that Finnish researchers in Arts and Humanities are familiar with digital methods, but the majority of them use digital tools only at the beginning and the end of the research cycle for resource discovery and the publication of scholarly outputs. The percentage of those who use them in the central phases of the research cycle such as enriching or processing their research assets remains close to 50%. They also pointed out the need for support in choosing the best tools for their research, and also recommendations in terms of networking or people to work with when pursuing computational or more digitally-oriented tasks6 . This points to an important role that can be fuliflled by DARIAH.

Finland is already strong in language technologies and is a member of CLARIN7 .

Informant

Maija Paavolainen works with researchers from Helsinki Centre of Digital Humanities at Helsinki University Library (HELDIG). In the early 2000s, she obtained a philosophy degree from Helsinki University. In addition, she holds a master degree in Library Science. Mertens. Ever since then, she has been following various DARIAH activities. She took part in the DARIAH Winter School on Open Data Citation in Humanities and Social Sciences. She also served as a DARIAH ambassador in the Humanities at Scale (HAS) project.

Analytic Summary

For research in Digital Humanities and for the development of research infrastructures in the humanities, multidisciplinarity is both an asset and a challenge. Researchers from different disciplines should engage in open discussions about formulating their research questions and introducing their methods in a way that is understandable to everybody. Good communication builds trusts among researchers.

Interpersonal skills are therefore, arguably, the most important set of skills for budding DH scholars.

DH researchers need access to datasets for their work. Which is why, cultural heritage institutions are crucial in DH educational initiatives. DH is a data-hungry discipline: researchers (and students) need access to computationally exploitable data which is not restricted by copyright. This was for a long time a major challenge for cultural heritage institutions (CHIs), which were either unfamiliar with or under-resourced when it came to learning and Open Data, or were generally shy about making their collections freely available, even when they recognized the researchers' needs.

The University of Helsinki has played an important role in changing the attitudes of the Finnish CHIs by: 1) organizing training measures; 2) building a network; and 3) producing online materials to support institutions in providing open access. The University Library, for instance, organizes courses integrated in the digital humanities program, where the cultural heritage institutions present their data sets to students, in order to bring the open cultural data to students and researchers. This is an example of the need for active educational outreach. The mere availability of datasets is no guarantee that the datasets will find their users. Active promotion of datasets (or tools and methods, for that matter) should be considered by all the relevant stakeholders in the field.

Digital research infrastructures can help researchers develop their skills not only by sharing tools and best practices, but also by creating opportunities for people to spend time working face-to-face in order to learn in a non-virtual context. This is best achieved through short intensive courses and summer schools.

Israel

Background Information

Israel prides itself for being a "startup nation" and has strong high-tech culture and awareness, reflected also in government funding policy (see also [START_REF] Senor | Start-up Nation: The Story of Israel's Economic Miracle[END_REF]. This may raise expectations that the field of Digital Humanities would be just as advanced in the country, and yet, according to our informant, the accession process to DARIAH continues to be an uphill battle. In fact, competitive salaries in the high-tech industry already present a serious challenge to graduate programs in computer science and engineering, so it is all the more difficult to draw technologically savvy people in the direction of the more modestly funded humanities disciplines. This is not to say that there is no investment in digital collections. National foundations and philanthropic bodies such as the Yad Hanadiv foundation, the UK-based Polonsky Foundation and the Friedberg Jewish Manuscript Society (FJMS) have funded large digitization efforts. The National Library of Israel is entrusted with some of the most significant digitization projects in Israel and in the Jewish world. Its activities, however, are strongly leaning towards popular public outreach, rather than the smaller circles of advanced academic users, which may be the reason for its lack of investment so far in DH research projects. The tension between scholars and the NLI policies throughout this last decade has been covered in the Israeli press and continues to be a source of concern in academia beyond the more specific question of Digital Humanities.

DH Education in Israel

The first program for Digital Humanities in Israel opened in 2017 at Haifa University with 8 students studying for a BSc degree which combines a double major --in Information Systems and in a chosen department of the humanities --with a core of digital humanities courses.

Over Humanities which could provide a cross-institutional framework for support, collaboration and short-term dedicated training in DH subjects and methods. Most of the instructors of the above-mentioned courses are adjunct professors, whose choice of Digital Humanities as an area of specialty, may take a toll on their career prospects in Israel, since to this date no permanent positions have been created in the field. Since adjunct professors are far from influential decision makers, the progress of DH as a university and national priority is hampered.

Informant

Sinai Rusinek works as Adjunct Professor teaching courses in Digital Humanities and Digital Approaches to Discourse, Language and the History of Ideas at both the Graduate program for Science, Technology and Society at Bar Ilan University, and at the newly established BSc program for Digital Humanities at Haifa University. In addition, she recently starting working in OmiLab, the media and information lab at the Israel Open University, in a project dedicated to distant reading historical Hebrew newspapers. She is also an employee of the Haifa University Library for its involvement in the Desir project.

Having graduated in from the Hebrew University of Jerusalem with a degree in Philosophy, Sinai completed a doctorate in the field of the history of concepts. During her post-doctoral fellowship at the Polonsky Academy in Jerusalem she also served as editor, and later editor in chief, of the journal Contributions to the History of Concepts. It was during this time that she founded "Ruach Digitalit", an initiative and a group for digital humanities in Israel (nowadays active mostly in occasional meetups and as a facebook group) and became active both in national endeavours to advocate for the field in her country and in the international community, in centernet, ADHO, the COST Action "Reassembling the Republic of Letters" and Pelagios.

Analytic Summary

Despite the importance of Digital Humanities courses and university curricula in general, workshops are still considered by our informant to be the best way to gain new and improve existing skills. An important aspect of the DH workshop landscape is the element of international exchange, i.e. exposure to different academic and work cultures, as well as learning about various DH projects that are being implemented worldwide.

In when instructors from the DH international community would appreciate the agreeable weather.

DH and infrastructural skills should be developed in hands-on, projectaffiliated and Hackathon-like environments. It is essential for students and DH scholars to learn new skills through actual project work, side by side with people already trained in digital methods, in apprenticeship-like settings. According to Sinai, perhaps the most crucial skill for a digital humanist to learn is how to collaborate with peers of varying capabilities, disciplinary and professional backgrounds. Collaborative skills are best developed by working on actual projects.

From the Israeli perspective, DARIAH should above all invest in more international workshops where tools and methods are taught. For the reasons mentioned above, it would be especially useful if "hackathon" elements could be introduced to DARIAH workshops, where training would include an actual contribution to existing projects. Sinai's own experience in teaching shows that students can make meaningful contributions to small projects from the very beginning of their DH education.

A useful setup in this respect would be to establish mechanisms for organizing summer jobs for high school and pre-university students who would be trained and then employed to contribute data to projects (through OCR correction, tagging etc.) in which they would be remunerated for their literacy, which is an important asset and a skill that needs to be developed and supported.

Infrastructures should be environments that enable open scholarly workflows without "locking in" the scholars. In opposition to closed virtual environments which are based on user interfaces that isolate the scholar from the underlying technology, Sinai advocates for the creation of hybrid environments which cater to users of different level of technical expertise: As an illustration, Sinai mentioned the wikidata query service interface that has, beside a simple, user-friendly GUI, a SPARQL window, in which the users can see the code changing as they change their query. In this way, the interface does not lock the user in ignorance, but suggests in a non-daunting way that the code can be learned and manipulated.

It is important to remember that interfaces and infrastructures can also affect the relations between the various communities involved in DH efforts. A VRE can socially separate computer scientists from programmers, humanist scholars and designers, if each group works separately in a different coding language or a different mode of the given environment, yet it can also connect them when it is porous enough to enable knowledge exchange between its different facets.

Spain

Background Information

DH is a research area which is increasingly recognized in Spain even though its development started later than in many other European countries. This is not to say that Spain didn't have its own pioneers in the use of technology in the humanities --one could mention, for instance, projects like Philobiblon, a database of medieval manuscripts, whose beginnings date back to the 80s) --but most of these early initiatives were the result of autonomous work and without collective awareness about Digital Humanities as a line of work in itself or as a new discipline.

Since 2011, the situation has changed considerably, with various initiatives in the academic world leading to the consolidation of the Digital Humanities as a scholarly community (for more details on the evolution of the discipline in Spain see Spence and González-Blanco 2014).

Spain does not currently have a specific infrastructure devoted to the humanities, but there are several institutions that have invested time and effort in developing them. As an example, The National Library, The Royal Academy, and some cultural foundations like Fundación March, have created digital infrastructures for themselves, which they are starting to share with some projects.

In this sense, it is important to point out that DH initiatives in Spain have multiplied, leading to important collaborations, especially in the Spanish-speaking scientific community, and creating new ways of conducting research. An important testimony to this will be the the international conference DH2018 in Mexico City.

Only three Spanish universities offer master's degrees in Digital Humanities: University Pablo de Olvide in Seville, the University of Salamanca and the University of Castilla -La Mancha. more opportunities for training and knowledge exchange, and increasing the recognition of digital tools and methods among the wider humanities communities.

Informant

Our informant highlighted the difficulty of relying only on workshops as a source of training in digital methods for humanities students: if students learn how to use certain tools in a workshop, that still doesn't mean that they will be able to use digital methods as part of their official coursework in the university. Training students in extracurricular settings is not enough: for digital skills and digital methods to become anchored in Spanish universities, the teaching staff in humanities departments would need to learn and adopt new tools and methods as well.

It is important to keep in mind, however, that there is a plethora of tools available and that researchers need help in selecting the best tool for a given task. DARIAH could play an important role in recommending tools and best-practices by means of, for instance, white papers. This is especially important because there are many tools out there but many of them were created in specific settings, for specific purposes, and are often difficult to reuse.

Spanish researchers are increasingly aware of Digital Humanities, but the levels of infrastructural investment are still very low and in no way comparable to the situation in better-resourced countries like France, the Netherlands and Germany.

Even though universities are now populated by new generations of digitally literate students, the Spanish academic system, according to our informant, lacks the means to guide these young people toward opening new lines of research in DH. Spanish DH experts who live and work outside of Spain have difficulties returning because of the lack of job prospects in DH and a rigid, hierarchical university system. She is a member of several scientific committees (EADH, IGNTP, Humanistica, etc.) and editorial boards (Introduction to Digital Religion de Gruyter, etc.). She is co-leading a series with David Hamidović by Brill «Digital Biblical Studies», and research groups in DH (SBL, EABS). She is the author of an important bibliography with books and articles about the spreading of DH in biblical and ancient religions fields.

Analytic Summary

The topics of digital culture and training in digital skills is a prominent one in the Swiss educational landscape. Since 2017, the initiative digitalswitzerland.com have gathered together various stakeholders from academia, industry, politics and culture with the goal of fostering as much as possible the development of digital skills across the board. The general skills that are particularly needed are flexibility, employability, creativity, but also critical thinking and media literacy (for instance, the assessment of fake news and media reliability). Critical thinking and interpretative skills are core skills in the humanities. In the Swiss context, Digital Humanities develop at a crucial juncture between the long tradition of excellence in education and the present preoccupation with digital culture.

This general concern has, for example, lead to a pilot project in a high school in the Lausanne region, in which students do no longer use print-based materials in their classrooms. Their arrival at the University may lead to significant changes, for example regarding the format of the exams, etc. This general interest in digitisation and digital culture is fruitful ground for fostering DH training education.

  DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

o

  lack of high-quality, open-sourced training materials. In many countries, training materials are usually made available only to students officially registered in a given course. Open Access policies cover research outputs but not necessarily training materials. This is something that DARIAH may want to consider as its policy goal: to advocate Open Access more broadly, not only in research, but also in education. National research infrastructures could consider negotiating terms under which existing resources could be pooled to form a consolidate offering across institutions and open them to the general public. DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

o

  foster cooperation with cultural heritage institutions (CHIs) in general, but, more specifically, also because CHIs as dataset providers can substantially contribute to various DH training measures. o encourage the development of collaborative skills among researchers. Interpersonal and collaborative skills are nowadays crucial on the path to scholarly excellence. DH skills should be, whenever possible, honed in handson, project-affiliated and hackathon-like environments. DARIAH should consider how to best facilitate the link between various training measures and ongoing research projects. DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  The assessment of different training and education needs in the DACs and a reflection upon their diversity should inform DARIAH's long-term strategy in this field. DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

2

  [START_REF] Edmond | The DARIAH ERIC: Redefining Research Infrastructure for the Arts and Humanities in the Digital Age[END_REF]: The DARIAH ERIC redefining research infrastructure for the arts and humanities. 3 STRAPL is the unofficial abbreviation used within DARIAH for the document entitled DARIAH 2020: 25 Key Actions for a Stronger DARIAH by 2020, a "living document" that has been approved by the DARIAH's General Assembly in November 2017. DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

Four

  representatives of DARIAH were interviewed for this report: members of the DARIAH Board of Directors (BoD): Jennifer Edmond, Frank Fischer and Laurent Romary, as well as Chair of the DARIAH VCC2 Research and Education, Agiatis Bernardou. The BoD is responsible for the strategic direction of DARIAH and can -up to a certain extentallocate resources on particularly important topics. The VCC2 acts as the primary contact with the A+H research and teaching communities, providing the interface between the DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081. methods, communities and training at the European level making sure that the next generation of scholars start from a higher baseline.

  DARIAH plays an important part on the policy level of humanities research. It tries to speak on behalf of the humanities, to provide a voice in the discussion on crucial topics such as career evolution, open science or forms of scholarship. DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  In 2008, Maija started working in libraries, teaching students to locate sources for their work. Her interest in Digital Humanities started when the first seminars took place at her University. Maija first learned about DARIAH at a conference at which she met then DARIAH CEO Mike DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  the last five years, the other Israeli Universities: Ben Gurion University (Dr. Yael Netzer and Dr. Eliezer Baumgarten), Bar Ilan University (Dr. Gila Prebor and Dr. Sinai Rusinek), the Hebrew University (Dr. Sinai Rusinek and Dr. Zef Segal) and Tel Aviv University (Prof. Amy Singer and Dr. Yael Netzer) began offering introductory DH courses. The Open University of Israel is also planning to include a special track within its Humanities Bachelor program next year, and the Information Management Department of Bar Ilan University recently launched a "Digital Humanities Track" as part of their graduate program (though no humanities courses are offered in this program, or named as prerequisites for enrolment). Even though the field of DH has been gaining some traction in the Israeli academic landscape, the offerings are still limited. Israel still doesn't have a lab or center for Digital DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  terms of skills development in Israel, Sinai believes that two related institutional goals should be reached: 1. the establishment of an interdisciplinary DH center, where research and development projects are funded and supported; and 2. the creation of a Digital Humanities Winter School in Israel, which would complement the offerings of many summer schools in Europe with the added bonus of being offered at the time of year DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  in recent years, the current situation for researchers in Spain could be greatly improved by sharing best practices and standards, providing DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.3.5.2 InformantC laire C livaz works as Head of Digital Enhanced Learning at the Swiss Institute of Bioinformatics (Vital-IT, Lausanne CH). She leads interdisciplinary projects at the crossroads of New Testament Studies and Digital Humanities, with an interdisciplinary team under the label "Vital-DH".After a PhD in the New Testament in 2007 (Unil & Harvard), Claire was Assistant Professor at the University of Lausanne, before joining the Swiss Institute of Bioinformatics. Together with her team, she is working on a range of projects including etalks, a multimedia digital editing tool, or a Swiss National Fund project on a digital trilingual New Testament manuscript. She was a member of the Erasmus+ Strategic parte Partnership which produced the online platform with training materials #dariahteach.

  not leave "analogue" scholars behind. DARIAH tools should, when possible, be developed as hybrid environments which cater to users of different levels of expertise. This would potentially increase their adoption, which, in the long term, is essential for DARIAH's sustainability. At the same time, DARIAH should consider developing training materials for traditional humanists, introducing basic DH notions and tools. Horizon 2020 research and innovation programme under grant agreement No 731081. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

			Nature of the deliverable
	✓	R	Document, report
		DEM	Demonstrator, pilot, prototype
		DEC	Websites, patent fillings, videos, etc.
		OTHER
			Dissemination level
	✓	P	Public
		CO	Confidential only for members of the consortium (including the Commission Services)
		EU-RES Classified Information: RESTREINT UE (Commission Decision 2005/444/EC)
		EU-CON Classified Information: CONFIDENTIEL UE (Commission Decision 2005/444/EC)
		EU-SEC Classified Information: SECRET UE (Commission Decision 2005/444/EC)
	Disclaimer
	DESIR has received funding from the European Union's

o

  you ever participated in Digital Humanities training activities? Where did you participate in these Digital Humanities training activities?*In this open question, Germany appears as the country where a largest number of respondents have participated in DH training activities, closely followed by the United States and the United Kingdom. Another non-European country (Canada) appears in a high position, although most of the mentioned countries are European, namely France, Switzerland or the Netherlands. The dispersion of the results must be highlighted, since more than 10% of the responses correspond to countries that were only mentioned once. opportunities in their own institution (15%). The other reasons mentioned area related with the lack of availability and the absence of training activities that fit their needs.

	Why have you never participated in Digital Humanities training activities?*
	In my institution Are you a user of DARIAH services and tools? In another institution in the country where I live N % N % I never felt the need to improve my skills through Digital Humanities training activities N %	In another N % country N % 84 16,6
	Not selected What prompted you to seek training opportunities in another country? 340 67,1 385 75,9 421 There are no training opportunities in the institution where I currently 76 15,0 83 Yes 63 12,4
	Yes work/research/study No Total I have not been available to attend training activities in my institution 167 32,9 122 169 33,3 507 100 507 Total 232 45,8	24,1 100 N		86 507 %	17 100 54 10,7
	To get a better training Didn't know DARIAH (missing)	275	54,2		19		22,4 50	9,9
	N I have not been available to attend training activities in other institutions in my Connections and networking 14 country % Workshops available 14 I have not been available to attend training activities in other countries Total 507 100	16,5 16,5 46	9,1
	Yes Lack of training opportunities in their own country 236			11	46,5 12,9 41	8,1
	No Professional involvement in DH The existing training opportunities in my institution do not fit my needs 271	9	10,6 53,5
							25	4,9
	Total	507				100
	Funding opportunity Don't know training opportunities				5		5,9
							7	1,4
	Other The existing training opportunities in other countries do not fit my needs 13		15,3
	Total				85		100,0	8	1,6
	Other

The majority of the respondents have never participated in any Digital Humanities training activity (53,5%). DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081. *Only if answered "yes" to "Have you ever participated in Digital Humanities training activities?" Among the respondents who have participated in DH training activities, 33% have done it in the institution they work/study/research in, 24% have participated in these activities in another institution of the same country, and 17% have participated in DH training activities abroad. In this question, it was allowed to choose more than one option. In which country (or countries)? DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081. The most frequent reason why the respondents seek training opportunities in another country is to get a better training, followed by the availability of workshops and to establish connections and networking. DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081. *Only if never participated in Digital Humanities training activities. The most frequent reason why the respondents have never participated in Digital Humanities training activities is the lack of need to improve their skills (17%) and the lack of DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081. training *Only if answered "yes" to "Did you know about DARIAH before this survey?"

  How would you rate DARIAH's importance considering the provision of training and education to encourage the use of DARIAH tools and services?*

	C onsidering your study/research/work activity, how would you rate the
	DARIAH training materials?*
			N N	% %
	Not important at all Very bad	1	1	1 1	0,2 0,2
		2	2	3 2	0,6 0,4
		3	3	5 6	1 1,2
		4	4	6 17	1,2 3,4
		5	5	20 14	3,9 2,8
	Very important Very good	6	6	23 8	4,5 1,6
	No opinion Not applicable		5 15	1 3
	Missing	Missing	444 444	87,6 87,6
	Total	Total	507 507	100 100
	Which of the following DARIAH services do you most frequently use? Training
		materials*
		N		%
	Not selected		485		95,7
	Yes		22		4,3
	Total		507		100

*Only if answered "yes" to "Are you a user of DARIAH services and tools?" DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081. *Only if answered "yes" to "Are you a user of DARIAH services and tools?" Among the users of DARIAH services and tools (63 in total), 22 mentioned they frequently use the training materials provided by DARIAH, and 23 rate as "very important" DARIAH's role in providing training and education to encourage the use of its tools and services. Very few respondents consider it not very important or "not important at all". DESIR INFRADEV-03-2016-2017 -Individual support to ESFRI and other world-class research infrastructures, Grant Agreement no. 731081.

  Salvador Ros is Associate Professor at the UNED Computer Science School. He is Director of the Digital Humanities Master in the Life-Long Learning Program at UNED. He is Director of the BigdataLab at UNED, the PI of the POSDATA project devoted to poetry standardization and member of the UNED Governing Board.

DESIR project details: https://www.dariah.eu/activities/projects-and-affiliations/desir/

For instance, the New Media Studies Program: http://novamedia.ff.cuni.cz/o-oboru/charakteristikaoboru/ of the Filozofická Faculta of the Univerzita Karlova.

https://www.lib.cas.cz/en/about-the-library/organizacni-struktura/people/ing-martin-lhotak/

https://www.helsinki.fi/en/news/finnish-survey-on-digital-research-practice-in-the-arts-and-

humanities 7 See https://www.kielipankki.fi/language-bank/ and https://kitwiki.csc.fi/twiki/bin/view/FinCLARIN/KielipankkiIntroduction

Grant Agreement no. 731081.

Outputs

The current report is an official deliverable D7.1 of the DESIR project. The collected video material, however, will be edited and published as a series of interviews and distributed using various DESIR and DARIAH publication channels.

The videos complement the report but can also be viewed independently. DARIAH research infrastructure and researchers undertaking basic, applied and practice-led research across the A+H.

Two of the directors, Jennifer Edmond and Laurent Romary, as well as the VCC2 Chair, Agiatis Bernardou, come initially from a non-DH educational background.

Jennifer Edmond (Trinity College Dublin) was trained as a teacher for German language and culture in the United States and worked for some time in this affiliation. The traditional approach towards content and technology in this profession changed disruptively with the emergence and the steep uptake of the internet. Jennifer's involvement with digital technologies started with her use of digital resources for teaching the German language and culture, which over time lead to a deeper interest in the transformation of the humanities research by technology. She previously worked at the University of Nottingham as a humanities technology officer, and is currently at Trinity College Dublin, where the humanities are set as important part of the institution's strategy. Jennifer was the PI of a DARIAH-affiliated project: CENDARI (Collaborative European Digital Archival Research Infrastructure)

Laurent Romary (INRIA) has described his entrance into the Digital Humanities as "lateral". His educational background is in electrical engineering and computer science, but he quite early had the opportunity to lead a research team in computational linguistics. Working in this and similar affiliations for over a decade, he learned the importance of liaising between different worlds and scholars who come from different disciplinary traditions such as the humanities and computing.

Frank Fischer (Higher School of Economics Moscow) describes himself as a "natural born DHer" because he studied in parallel both literature and computer science. He previously worked as a DH coordinator at the Göttingen Centre for Digital Humanities. Similar to Laurent Romary, he also highlights the importance of the communication between the worlds of the computer scientists and humanists.

Analytic Summary

Infrastructures in general --and not only a research infrastructure --tend to disappear out of sight if they function well: the more efficient and productive an infrastructure is, the less will it be perceived by the users. Users take note of the infrastructure usually if they encounter problems accessing or using it.

A similar argument could be made about DARIAH: "We don't want in any way to feature the infrastructure as a great thing because then it's too much at the center. The process of preparing the Czech accession to DARIAH is dynamic and the potential partners in this process are still getting acquainted with one another. A DARIAH workshop in April 2018 in Prague will be an important asset in promoting this process of collaborative learning. The workshop will brings DH and DARIAH-interested researchers from the Czech Republic and the neighbouring countries. This and other similar workshops could be used as venues for identifying specific requirements and skills for developing DH and using the DARIAH research infrastructure in the Czech Republic. Successful infrastructures should bring together different technologies and methods. For the future, our informant envisions more platforms similar to what she sees as a paradigmatic case, READ's Transkribus platform. By enabling scholarly manual transcription and correction, as well as machine learning and training at the humanists' fingertips, the platforms enhances the dialogue between scholarly analysis and artificial intelligence while still giving enough weight to the human contribution and to the quality of data, a feature which is missing in most big data and deep learning endeavours these days. Pelagios' Recogito annotation platform is another telling example, in that it brings the automatic named entity recognition tools to constant check, verification and correction, thus preserving the crucial position of the scholar in the process of knowledge creation and enhancement.

A first overview of DH educational resources in 2013

The DH curricula are, like research and DH infrastructures, in full development. The SAHSS produced a report on Swiss educational offerings in DH in 2013 9 which revealed 3 PhD schools, 5 bachelors and 9 master curricula, as well as 4 continuous training and fifteen individual courses in Switzerland that were identified as DH or DH-related by the institutions themselves, and which covered a range of methodologies and fields, from computational linguistics and information science to digital critical editions.

The year 2013 was marked a milestone, with the first DH meeting organised by the SAHASS, and funds dedicated to data curation in Humanities and to develop the relationship between DARIAH and Switzerland (pilot project DDZ/CDP). This pilot project was developed as a center, the Data and Service Center for the Humanities, http://dasch.swiss/. In June 2013, the first Swiss DH summer school also took place, organised by Infoclio and diverse Universities in Bern, as an important starting point for DH Switzerland is also active in the DARIAH Working Group #dariahTeach, which in 2017 produced an online teaching platform, http://teach.dariah.eu, through an Erasmus+ Strategic Partnership, which involved seven partners, including the Swiss Institute of Bioinformatics and the University of Lausanne (see also http://dariah.eu/teach). Various degree programs and sommer schools in Switzerland (mentioned in the Background section above) demonstrate the potential of DH to overcome the traditional separations between the disciplines in the humanities.

Swiss academic and educational institutions are well-funded and have a long tradition of humanistic inquiry. Yet the humanities in Switzerland, too, are undergoing a transformation from a model focused on individual research to collaborative research. The advantages of cooperating with DARIAH should be framed in terms of networking and promoting excellence. DARIAH VCC2 presents several possibilities for collaboration, such as the development of new and re-use of the training materials already available on #dariahTeach. Some skills --like TEI for text encoding, for instance --are needed for all students and PhD students in the humanities who work on textual materials. Considering the size of Switzerland, and, in particular, the highly specialized fields in the humanities, collaboration and collaborative skills are crucial on the path to scholarly excellence. DARIAH can have an important networking impact on Swiss researchers, creating opportunities for knowledge exchange, for example around the topic of Data Management Plans, a mandatory point in all the projects submitted to the Swiss National Foundation.

United Kingdom

3.6.1Background Information

The United Kingdom has been a pioneer in humanities computing since since the early days. For instance, the Association for Literary and Linguistic Computing (ALLC), renamed in recent years as the European Association for Digital Humanities (EADH), was founded at a meeting in King's College London in 1973. Oxford University Computing Services played a pivotal role in the establishment (in 1987) and subsequent maintenance of the Text Encoding Initiative (TEI). Digital Humanities have solid institutional presence in institutions of higher learning, such as University of Glasgow, King's College London, University College London, Queen's University Belfast, to name just a few. Since 2012, the Digital Humanities Congress has been held biannually in Sheffield. Yet, as the DESIR D3.1 Deliverable: Country-Specific Strategy and Outline for Each Accession Process indicates: "national coordination at the level of infrastructural development and policy across High Education Institutions (HEI) is lacking. Focused lobbying and coordinated advocacy are needed now that the Digital Humanities has reached a higher level of institutionalisation in the country. Without these activities, the field risks not achieving its potential, and at worse could be undermined by a lack of cohesion and standards." (90).

Most research-intensive universities in the UK have established research data policies. Government and the industry also contribute to the establishment of research data; an E-Infrastructure Leadership Council has been established as well as services such as data.gov.uk, Government Digital Service and an Open Data Institute. Moreover a wide range of research data services are provided by generic commercial and not-for-profit providers. Research data management is understood as a shared responsibility, in which universities, libraries, ICT services, research offices and so on participate.

In the development of data service, the focus is on storage. Other types of services, such as platforms and tools, receive much less attention, although in some cases they may be connected to repository services. Re3data records some two dozen repositories run by UK universities. The OpenDOAR registry's focus is on open access as distinct from open data; but it records 252 repositories in the UK, of which 201 are institutional. The challenge here is that services are being provided for single projects in isolation from other services, and with no guarantees of sustainability for the future. for software development and reuse, and also career paths for those involved in software development and sustainability.

Apart from its online services, the Digital Curation Centre offers various training programmes and online resources. CODATA and several universities have developed Masters' programmes in data science, whereas the EDISON project has developed drafts of a science competence framework and a model curriculum for data science. The possibility of establishing a software seal of approval is currently being discussed, but also the long-term preservation of heritage software.

One challenge to be taken into account is the clear difference in culture and practice between the disciplines and subject areas, that are strongly influenced by differences in the kinds of data produced in different areas of research, by the level of funding available, the availability and accessibility of data infrastructures and services and by the closeness or otherwise of relationships between domain specialists and data specialists.

Notwithstanding the common principles, there is in addition no common policy template. That's why guidelines vary widely in scope and detail, as well as in the precise requirements and relationships they express with the guidelines of other actors such as sponsors and publishers. up documents in order to be able to answer their research questions. DARIAH's educational offerings should also focus more on methodological questions (echoing Unsworth's notion of scholarly primitives) rather than tools. Tools change, but the underlying methodological challenges (how do we annote, interpret or share data) remain.

Informant

Research infrastructures should consolidate existing training rather than reinvent the wheel. A research infrastructure is by its very nature a meta-organization: it has a bird's eye view and it can incorporate perspectives from different disciplines and different national communities. It can extrapolate best practices and serve as a onestop-shop for existing offerings rather than attempt to develop bespoke training.

Training in the UK, like in many other countries, is institutionally-based and often available only to students officially enrolled in a given course. For example, if a digital humanities research infrastructure existed in the UK, it could survey all the training that is offered through various doctoral training consortia in the UK and through institutions like the Institute of Historical Research and negotiate terms under which resources could be pooled to form a consolidated offering to people who are perhaps not within those communities or even to undergraduates or to master's students.

Humanities students, regardless of their discipline, need some core skills, generally speaking, in information and data literacy, and, more specifically, in dealing with the provenance and origins of source materials that are explored digitally. We often deal with digital materials that are not quite "our own", i.e. materials that have been digitized by national libraries or recovered from various data archives and data collections.

From the UK perspective --a country with a very lively Digital Humanities landscape --DARIAH should present a clear vision of what is the added value it offers to member countries: a taxonomy-based approach on what can be provided locally, nationally or, indeed, internationally would be helpful.


Thi-To-Quyen Tran
email: thi-to-quyen.tran@irisa.fr

Thuong-Cang Phan

Anne Laurent
email: anne.laurent@lirmm.fr

Laurent D ' Orazio
email: laurent.dorazio@univ-rennes1.fr

Improving Hamming distance-based fuzzy join in MapReduce using Bloom Filters

Keywords: Fuzzy join, Similarity join, MapReduce

Join operation is one of the key ones in databases, allowing to cross data from several tables. Two tuples are crossed when they share the same value on some attribute(s). A fuzzy or similarity join combines all pairs of tuples for which the distance is lower than or equal to a prespecified threshold ε from one or several relations. Fuzzy join has been studied by many researchers because its practical application. However, join is the most costly and may even not be possible to compute on large databases. In this paper, we thus propose the optimization for MapReduce algorithms to process fuzzy joins of binary strings using Hamming Distance. In particular we propose to use an extension of Bloom Filters to eliminate the redundant data, reduce the unnecessary comparisons, and avoid the duplicate output. We compare and evaluate analytically the algorithms with a cost model.

I. INTRODUCTION

Join is a critical operation within a data management system, making it possible to enrich data from a source with information stored outside of it. This is why literature is rich in working on join optimization, especially in parallel and distributed systems. In recent years, researches has focused on the problem of efficient joins in large-scale parallel environments. In other words, researchers toward their goals of limiting the use of resources in terms of bandwidth consumption or CPU usage. The first results, concerning the equi-join [START_REF] Phan | A Theoretical and Experimental Comparison of Filter-Based Equijoins in MapReduce[END_REF], impose strong constraints on the data (one of the sets having to be small enough to be distributed to all the machines used for the treatment) or their organization (sorting according to the join attribute, placement of data on specific nodes), leading to many data transfers (some unnecessary) and heavy workload on machines or requiring multiple (expensive) execution phases.

The problem is even more difficult when the equality constraint is released while this type of query is often necessary. A query in computer network defense for example: grouping information from a URL written in different ways (eg www.irisa.fr in a source and http://www.irisa.fr in the other), set of IP addresses (192.168.150.30, 192.168.150.31, etc.). As another query example [START_REF] Vernica | Efficient Parallel Set-similarity Joins Using MapReduce[END_REF] in mining social networking sites where user's preferences are stored as bit vectors (where a "1" bit means interest in a certain domain), applications wants to discover the similar interests of users. A user with preference bit vector "[1,0,0,1,1,0,1,0,0,1]" possibility has similar interests to a user with preferences "[1,0,0,0,1,0,1,0,0,1]". This query is defined as a fuzzy or similarity join and arosed in many applications, including detecting attacks from colluding attackers [START_REF] Metwally | Detectives: Detecting Coalition Hit Inflation Attacks in Advertising Networks Streams[END_REF], mining in social networking sites [START_REF] Spertus | Evaluating similarity measures: A large-scale study in the Orkut social network[END_REF], detecting near duplicate web-pages in web crawling [START_REF] Henzinger | Finding Near-duplicate Web Pages: A Large-scale Evaluation of Algorithms[END_REF], document clustering [START_REF] Broder | Syntactic Clustering of the Web[END_REF], master data management [START_REF] Sahami | A Web-based Kernel Function for Measuring the Similarity of Short Text Snippets[END_REF].

When dealing with a very large amount of data, fuzzy join becomes a challenging problem in a distributed parallel computing environment with the expensive cost of data shuffle. As a result, the data redundancy is very difficult to accept. Vernica et al. [START_REF] Vernica | Efficient Parallel Set-similarity Joins Using MapReduce[END_REF] proposed a similarity join method using 3stage MapReduce which utilized the prefix filtering method to support set-based similarity functions. Metwally et al. [START_REF] Metwally | V-SMART-Join: A Scalable MapReduce Framework for All-Pair Similarity Joins of Multisets and Vectors[END_REF] proposed a 2-stage algorithm VSMART join for similarity join on set, multisets and vector. Afrati et al. [START_REF] Afrati | Fuzzy Joins Using MapReduce[END_REF] proposed multiple algorithms to perform fuzzy join in a single MapReduce stage. While recent studies on the fuzzy join have the common limitations as redundancy and duplication of data, the filterbased approaches in our recent studies [START_REF] Phan | A Theoretical and Experimental Comparison of Filter-Based Equijoins in MapReduce[END_REF], [START_REF] Phan | Toward Intersection Filterbased Optimization for Joins in MapReduce[END_REF] can solve these problems. Our team was interested in using Bloom Filters [START_REF] Bloom | Space/Time Trade-offs in Hash Coding with Allowable Errors[END_REF], Intersection Filter [START_REF] Phan | Toward Intersection Filterbased Optimization for Joins in MapReduce[END_REF]. The idea is to filter irrelevant data as soon as possible to reduce data transfers and workload on different machines.

This study, therefore, focuses on a theoretical analysis of various Hamming distance-based similarity join algorithms in MapReduce, and their cost comparison in a map-reduceshuffle computation.

The remaining part of this paper is organized as follows. Section 2 presents the research background by the related works. Various Hamming distance-based similarity join algorithms are analysed as a research context in section 3. We propose the optimizations in section 4. We theoretically compare and evaluate the algorithms by an example in section 5. Finally, section 6 concludes and discusses future works.

II. BACKGROUND

A. MapReduce

MapReduce [START_REF] Dean | MapReduce: Simplified Data Processing on Large Clusters[END_REF] is a parallel and distributed programming model to process large amounts of data on data centers consisting of commodity hardware. This model allows users to focus on designing their applications regardless of the distributed aspects of the execution. Figure 1 illustrates MapReduce execution.

(k1, v1) map ---→ (k2, v2)
The intermediate values associated with the same key k2 are grouped together and then transmitted to the Reduce function which aggregates the values.

(k2, v2) reduce ----→ (k3, v3)
A MapReduce program is executed on multiple nodes. During the Map phase, each Map task reads a subset (called split) of an input dataset and applies the Map function for each key / value pair. The system supports the grouping of intermediate data and sends them to the relevant nodes to apply the Reduce phase. This communication process is called Shuffle. Each Reduce task collects the key / value pairs of all the Map tasks, sorts / merges the data with the same key and calls the Reduce function to generate the final results.

B. Fuzzy join

A fuzzy join aims to group data based on their similarity. It relies on a distance measure to find all pairs (x, y) in the input dataset(s) with a distance bellow some pre-specified threshold ε. Different solutions have been proposed for big data systems [START_REF] Vernica | Efficient Parallel Set-similarity Joins Using MapReduce[END_REF], [START_REF] Metwally | V-SMART-Join: A Scalable MapReduce Framework for All-Pair Similarity Joins of Multisets and Vectors[END_REF], [START_REF] Afrati | Fuzzy Joins Using MapReduce[END_REF], [START_REF] Silva | MapReduce-based Similarity Join for Metric Spaces[END_REF]- [START_REF] Xiao | Efficient Similarity Joins for Near-duplicate Detection[END_REF]. A survey has been written on MapReduce-based fuzzy join [START_REF] Silva | An Ex-perimental Survey of MapReduce-Based Similarity Joins[END_REF] studying supported data types (fixed-length string, variable-length string, numeric, vector, set) and distance functions (Hamming distance, Edit distance, Jaccard similarity, Tanimoto Coefficient, Cosine Coefficient, Ruzicka similarity, Dice Similarity, Set Cosine Sim, Vector Cosine Sim). In this paper, we focus on fuzzy join algorithms using Hamming distance [START_REF] Afrati | Fuzzy Joins Using MapReduce[END_REF] with fixed-length data inputs (b-bit strings).

Hamming distance (HD) between two strings s, t is the number of positions in which they differ. Given a set, S, of b-bit strings, a fuzzy join is stated using a Hamming distance used to define a similarity and a threshold ε is • d = 0, the ball of radius 0 around any given element B 000 (0) now is itself (000). • d = 1, the B 000 (1) now has 1 + 3 = 4 elements (000, 001, 010, 100). • d = 2, the B 000 (2) now has 1 + 3 + 3 = 7 elements (000, 001, 010, 100, 011, 101, 110).

{(s, t)|s, t ∈ S, HD(s, t) ≤ ε}

C. Bloom filter

A Bloom Filter (BF) [START_REF] Bloom | Space/Time Trade-offs in Hash Coding with Allowable Errors[END_REF] is a space-efficient randomized data structure used for testing membership in a set with a small rate of false positives. Figure 2 presents a Bloom Filter structure consisting of m bits, k independent hash functions, and a set S of n elements represented by BF (S). BF (S) can be described as follows:

• The set S = {x 1 , x 2 , ..., x n } of n elements is represented by an array of m bits, initially all set to 0.

• The filter uses k independent hash functions h 1 , h 2 , ..., h k with h i : x → {1..m}. • To insert an element x ∈ S, we compute h 1 (x), h 2 (x), ..., h k (x)
, and set the corresponding positions in the bit array to 1. Once this operation has been done for each element of S, the resulting bit array can be used as an approximate representation of the set. • To check if y ∈ S, we check whether for each of the k hash functions, the position h i (y) is set to 1 in the bit array. If at least one position is set to 0, this means that y / ∈ S. Otherwise, all positions are set to 1, that is to say that y may be a member of S with some probability. BF never returns false negatives. However, it can return false positives. A false positive element of BF is an element that does not belong to a set S while testing it on BF lead to the opposite result. Indeed, in some cases, a hash function can return the same value for multiple elements. As a consequence, an element that does not belong to S can also have a hash value at its position of 1. BF is a space-efficient structure to accelerate querying. The size of a filter is fixed, independently of the number n of elements. However, there is a relation between the size of the structure m and the false positive probability [START_REF] Guo | Theory and Network Applications of Dynamic Bloom Filters[END_REF]

f BF (S) = (1 -(1 - 1 m ) nk ) k

D. Motivation

This paper aims to improve fuzzy joins using Hamming distance in a MapReduce environment, relying on Bloom Filter. In order to compare the costs of different algorithms, it adapts a previous model (M, C, R) [START_REF] Afrati | Fuzzy Joins Using MapReduce[END_REF], where M , R, C are used to measure the effectiveness of an algorithm. The notations and parameters are described in Table I.

III. FUZZY JOIN ALGORITHMS IN MAPREDUCE

This paper studies hamming distance-based fuzzy join algorithms in MapReduce using the (M, C, R) cost model [START_REF] Afrati | Fuzzy Joins Using MapReduce[END_REF]. More precisely, it focuses on Naive join, Ball Hashing, Splitting and Anchor Points algorithms

A. Naive Algorithm

Naive algorithm can be used for any data type and distance function. It relies on a single MapReduce job. The main idea is to distribute each input record to a small set of reducers so that any two records be mapped to at least one common reducer for computing distance. The details of Naive algorithm for an input set S are specified as follows:

• With a constant J > 0, let K = J+1 2 = J(J + 1)/2 or J ≈ √ K be the number of reducers. • Each reducer is identified by a pair (i, j), such that 0 ≤ i ≤ j ≤ J • During the Map phase, all members X of S are hashed to J buckets so as to be sent to exactly J reducers (i, j) or (j, i) ∀i = [0, J) (key, value) pairs of the form ((i, j), X). X The challenge is to define K in order for every pairs of elements of S to be sent to exactly one reducer and thus avoid data duplication. Each input records must be compared with all others leading to data redundancy and inefficiency.

B. Ball Hashing Algorithms

Ball Hashing is a family of two algorithms BH 1 and BH 2 . These algorithms rely on the "ball of radius d" to reduce unnecessary comparisons. This means that each record is compared to the others within its similarity radius. To do this, there is one reducer for each of the n possible strings of length b. The number of reducers is thus n = 2 b .

1) BH 1 :

• The mappers generate all elements t in ball of radius d of each input record s (B s (d)) as (key, value) pairs of the form (s, -1) and (t, s) such that t = s and send them to the corresponding reducers. t is a string obtained from s by changing i ∈ [1, d) bits.

s map ---→ (s, -1) (t, s), ∀t ∈ B s (d), t = s
Thus the map cost is B(d) per input element. • Call a reducer that receives (s, -1) "active", it infers that s is in the input set and outputs all pairs of similar received strings. Assuming that it is not possible for multiple input records to have the same join value, the average number of strings to be sent to each reducer is |S|B(d)/n. The total cost of all |S| active reducers is |S| 2 B(d)/n. • A issue with BH 1 is data duplication due to ts and st similarity. A proposed solution is to proceed lexicographically [START_REF] Kimmett | Fuzzy Joins in MapReduce: An Experimental Study[END_REF]. A mapper only emits (t, s) if t < s. However, redundant data still exist in "inactive" reducers because similar records in B s (d) are sent to reducers although they are not elements in S. 2) BH 2 : BH 2 is an extension of BH 1 . The difference is that during the map phase, BH 2 generates ball of radius d/2. Because of this, every reducer is active and checks for the similarity between all the possible combinations of two strings it receives and eliminates the duplicate outputs.

C. Splitting Algorithm

Splitting algorithm is based on a principle of which have any of two similarity b-bit strings with a distance less than d, there exists at least one same substring of length b/(d + 1).

• Mappers decompose each input string s into d + 1 equallength substrings s 1 , s 2 , ..., s d+1 and emits (s i , s). s map ---→ ((i, s i ), s), i = 1..(d + 1), s i ⊂ s Each substring of length (d+1) has 2 b/(d+1) possible values. Therefore, the number of reducers is (d+1)2 b/(d+1) . The total communication cost is (d + 1)|S|.

• There is at least one reducer that will receive any two similar strings in S. Reducers test each string to see if it is within distance d of all other received strings, similar to the Naive algorithm. The processing cost is d+1) . To avoid duplicate results, when a reducer in the ith family finds that s and t are at distance d or less, it checks that there is no j < i for in which jth substrings are also equal and outputs s, t if there is no such j. However, the Splitting algorithm has also the same issue of redundant data as the Ball hashing algorithm.

(d + 1)|S| 2 /2 b/(

D. Anchor Points Algorithm

Anchor Points algorithm is the only randomized algorithm considered. The algorithm chooses a random universe. If the set is large enough, at least one string in the set can be expected to be within distance d/2 of any two strings in the input data. This algorithm will not be included in our research since the paper that introduced it showed that it is outperformed by the other algorithms [START_REF] Afrati | Fuzzy Joins Using MapReduce[END_REF].

IV. BLOOM FILTER-BASED FUZZY JOINS

The previous algorithms generate intermediate elements that may be not relevant to the join process in the map phase, because they do not match with any similar record in the input dataset. In this section, we propose to integrate BF into the join algorithms to improve performances. Naive algorithm will be used as a baseline for comparison with other solutions.

A. BF-BH1 Algorithms

During the map phase, BH 1 generates all elements within a distance d from s and sends to them to the reducers for combining with similar input records. It is easy to see that not all elements in the B s (d) belong to S. Our approach integrates BF (S) to remove elements in B s (d) that do not belong to S before sending it to the reducers. This solution consists of two stages:

• Stage 1 (Pre-processing) : A filter BF (S) is built on a join key value set of the input dataset S. Figure 3 describe an example of preprocessing stage of BF -BH1 for the fuzzy join with 3-bit string. • Stage 2 (Join processing) : BF (S) is distributed to all the computing nodes and used to eliminate non-similar elements of the input dataset in each ball of radius d during the map phase. An example of join stage of BF -BH1 for the fuzzy join with 3-bit string and threshold d = 1 is shown in Figure 4.

s map ----→ BF (S) (s, -1)
(t, s), ∀t ∈ B s (d) ∩ S, t < s After filtering none relevant data, the join algorithm then proceeds as in BH 1 .

Let us recall assumption that hash operation performs in unit time. With k hash functions, the pre-processing cost on all input records is k|S|. However, this cost can be amortized by streaming or caching techniques.

Each membership test also uses k hash functions, so the map cost for each record is kB(d).

In the shuffle phase, the number of intermediate elements for each record will be reduced, instead of B(d). Precisely, if we note δ S the ratio of similar records of S, f BF (S) the false positive probability of the BF of S, then the cost to transfer intermediate data from mappers to reducers is

D BF -BH1 = |S|[δ S B(d) + f BF (S) (1 -δ S )B(d)] D BF -BH1 < |S|B(d)
As a consequence, the processing cost in the reduce phase is also improved, the reduction being: D BF -BH1 |S|/n BF can also be used in BH2 and will be the subject of future works. In particular, we envision to address multiple inputs with Intersection Filters [START_REF] Phan | A Theoretical and Experimental Comparison of Filter-Based Equijoins in MapReduce[END_REF], [START_REF] Phan | Toward Intersection Filterbased Optimization for Joins in MapReduce[END_REF].

B. BF-Splitting algorithm

The Splitting algorithm generates redundant data by sending each record to d + 1 reducers. In fact, each record just need to be sent to some identified reducers if all its actual similar elements present in S and its substrings are known. As a solution we propose to combine Ball Hashing, Splitting and BF.

This approach also requires a pre-processing stage for building BF (S) with a cost of k|S|. The join stage is described as follows:

• A mapper generates all elements in the ball of radius d around each input record s. By the membership test in BF (S), it determines which elements {t = s} in B s (d) may actually be similar to s. Then each of them is divided in to d + 1 equal-length substrings {s i } and {t i }, i = 1..(d + 1). For each s i , if there exists a substring t i of t in the intersection of S and B s (d) that matches with s i , the pair (s i , s) will be outputs, and then t will never be considered again. V. SYNTHESIS Table II summarizes the costs of the different algorithms. According to the processing cost, Naive algorithm is the most expensive solution, but its cost is independent with the change of distance. With respect to the communication cost, Splitting algorithm is the best approach, while Ball Hashing is the most suitable solution to processing cost. However, Ball Hashing is sensitive to distance. With the greater the distance d, the number of elements in B(d) increases dramatically. Integrating BF in the algorithms implies the following changes according the (M, C, R) model:

0 J ≈ √ K K |S| √ K |S| 2 BH1 0 B(d) n = 2 b |S|B(d) |S| 2 B(d)/2 b BF-BH1 k|S| kB(d) n = 2 b D BF -BH1 < |S|B(d) D BF -BH1 |S|/2 b Splitting 0 d + 1 (d + 1)2 b/(d+1) (d + 1)|S| (d + 1)|S| 2 /2 b/(d+1) BF-Splitting k|S| kB(d)(d + 1) (d + 1)2 b/(d+1) D BF -Splitting < (d + 1)|S| D BF -Spitting |S|/2 b/(d+1)
• The pre-processing cost is incurred by reading the input to generate BF (S). However this cost can be amortized, especially using streaming or caching techniques (e.g Spark [START_REF]Apache SparkTM -Lightning-Fast Cluster Computing[END_REF]). • The map phase use k hash functions for the membership test. In the BF-Splitting, the map phase generates B s (d) for each input record. • The number of reducer does not change.

• Using BF (S), redundant elements are eliminated, thus the communication cost is reduced. This also leads to a decrease of the computation cost on reducers. Table III compares the costs of algorithms via a concrete example [START_REF] Afrati | Fuzzy Joins Using MapReduce[END_REF]. We choose b = 20, so n = 2 2 0 ≈ 10 6 . We use d = 4, so B(d) = 6226. We also take |S| be 10000. For the Naive algorithm, we take K = 10000. We assume that the the ratio of similar records of S is δ S = 1%, the small false positive of BF (S) is f BF (S) = 0.0001. As a conclusion, no algorithm is the best. Choosing a solution depends on the context. However, in a parallel and distributed environment, communication cost is one of the most important factors. Experiments in our previous studies [START_REF] Phan | A Theoretical and Experimental Comparison of Filter-Based Equijoins in MapReduce[END_REF], [START_REF] Phan | Toward Intersection Filterbased Optimization for Joins in MapReduce[END_REF] have proved that filtering can significantly improve execution times.

VI. CONCLUSIONS

In this paper, we study theoretical details for the fuzzy join algorithms based on Hamming distance measure in MapReduce, applied for b-bit strings input dataset. We propose the optimization for the Ball Hashing and Splitting algorithms, and show the comparison through the MapReduce cost model. Our approaches eliminate the redundant intermediate data, reduce the unnecessary comparisons and avoid the data duplication. For the fuzzy join of multiple input datasets, Intersection filter [START_REF] Phan | Toward Intersection Filterbased Optimization for Joins in MapReduce[END_REF] is applied instead of Bloom filter. Our optimizations may be extended in the cache or streaming supported framework to reuse the preprocessing cost. In the future works, we continue to validate our works, compare with other approachs and extend the research for other fuzzy join algorithms.

Fig. 1 .

 1 Fig. 1. MapReduce Execution

.

  The ball of radius d (B(d)) can be obtained by flipping the value of at most d bits of any given b-bit string. Thus, it is computed by the following formula [9]: B(d) = d k=0 b k ≈ b d /d! B s (d) consists of all similar elements in the ball of radius d around of s. In other words, ∀t ∈ B s (d), HD(s, t) ≤ d Example: Consider the 3-bit string (b = 3),

Fig. 2 .

 2 Fig. 2. A Bloom filter BF (S) with 3 hash functions.

2 =

 2 map---→ ((i, j), X) The total map cost and communication (data transfer from mappers to reducers) isM = C = O(|S|J) = O(|S| √ K) • Areducer receives all records with the same key (i, j), computes the distance between each pair of records and outputs the pairs satisfying the similarity, that is to say the threshold ε. Each reducer receives around |S|J/K = 2|S|(J + 1) elements, which requires 2|S|/(J+1) O(|S| 2 /K) comparisons. As a consequence, for K reducers, the total computation cost for all reducers R is O(|S| 2 )

Fig. 3 .Fig. 4 .Fig. 5 .

 345 Fig. 3. Pre-processing stage

TABLE II SUMMARY

 II OF COSTS FOR VARIOUS HAMMING DISTANCE-BASED JOIN ALGORITHMS

	Approche	Pre-processing	Map cost per element	# Reducers	Communication	Processing
	Naive

  TABLE II WHEN b = 20, d = 4, |S| = 10 5 , K = 10 4 , δ S = 1%, k = 8, f BF (S) = 10 -4

	Approche	Pre-processing Map cost per element # Reducers Communication Processing
	Naive	0	100	10 4	10 7	10 10
	BH1	0	6226	10 6	6.2 × 10 8	6.2 × 10 7
	BF-BH1	8 × 10 5	49808	10 6	6.26 × 10 6	6.26 × 10 5
	Splitting	0	5	80	5 × 10 5	3.1 × 10 9
	BF-Splitting 8 × 10 5	249040	80	10 3	6.2 × 10 3