Carlos Alario-Hoyos

Iria Estévez-Ayres

Mar Pérez Sanagustín

Derick Leony
email: dleony@it.uc3m.es

Carlos Delgado Kloos

MyLearningMentor: A Mobile App to Support Learners Participating in MOOCs

Keywords: MOOCs, planning, mentoring, study skills, work habits Categories:

MOOCs have brought a revolution to education. However, their impact is mainly benefiting people with Higher Education degrees. The lack of support and personalized advice in MOOCs is causing that many of the learners that have not developed work habits and selflearning skills give them up at the first obstacle, and do not see MOOCs as an alternative for their education and training. MyLearningMentor (MLM) is a mobile application that addresses the lack of support and personalized advice for learners in MOOCs. This paper presents the architecture of MLM and practical examples of use. The architecture of MLM is designed to provide MOOC participants with a personalized planning that facilitates them following up the MOOCs they enroll. This planning is adapted to learners' profiles, preferences, priorities and previous performance (measured in time devoted to each task). The architecture of MLM is also designed to provide tips and hints aimed at helping learners develop work habits and study skills, and eventually become self-learners.

Introduction

The advent of MOOCs (Massive Open Online Courses) has been followed by a revolution in traditional educational structures [O'Connor 2014]. Many universities and institutions across the globe are now including MOOCs in their catalogues. The affordances that stem from MOOCs are being used to enrich traditional courses under terms like "flipped classroom" [START_REF] Bergmann | Flip Your Classroom: Reach Every Student in Every Class Every Day[END_REF] or "SPOC" (Small Private Online Course) [Fox, 2013]. Much of the MOOC success can be attributed to initiatives such as edX (http://edx.org), Coursera (http://coursera.org) or MiríadaX (http://miriadax.net), which provide platforms for the deployment of MOOCs, as well as educational support to teachers and universities taking part of these initiatives.

The open nature of MOOCs is usually described as an opportunity for people with different origins, ages, classes, incomes and backgrounds to access high-quality education provided by a variety of institutions, including elite Universities [START_REF] Yuan | MOOCs and disruptive innovation: Implications for Higher Education[END_REF]. Some people have claimed that this "democratization" of education will bring positive effects in the reduction of the educational gap between those who can easily access Higher Education programs and those who cannot [Jaggars, 2014]. Nevertheless, the first studies that have been published based upon real data suggest quite the contrary: most learners enrolling and getting certificates in MOOCs have some previous academic degree [START_REF] Honeychurch | [END_REF]Draper, 2013] [Nesterko et al., 2014]. As an example, a recent paper from the University of Pennsylvania reports 83% of learners in MOOCs with post-secondary degrees and 44% with education beyond bachelor's degrees [Emanuel, 2013] [ Christensen et al., 2013]. Similar conclusions were extracted from the first MIT and Harvard MOOCs where, for instance, an average of 76% of the people getting a certificate in the first five Harvard MOOCs already had a Bachelor's Degree or above [Ho et al., 2014].

The lower percentages of people without previous qualifications taking and completing MOOCs can be attributed to the need for certain competencies and skills that MOOCs demand to learners [Laplante, 2013]. Examples of these competencies and skills are high responsibility, self-guided learning, or time management [Kay, 2013]. [START_REF] Hew | Students' and Instructors' Use of Massive Open Online Courses (MOOCs): Motivations and Challenges[END_REF] provide insights on this claim, concluding that the failure to understand the course contents and the lack of appropriate support are two of the main reasons behind most drop outs in MOOCs. There is thus a challenge in providing "personalized" support to learners when facing MOOCs ("personalized" is preferred to "customized" when referred to education and learning [START_REF] Freund | Mass Customization and Personalization in Adult Education and Training[END_REF]). This personalized support is especially encouraged in the case of learners without previous qualifications, who are the most vulnerable ones due to their lack of the abovementioned competences and skills. This support should help people without previous qualifications follow up and complete MOOCs, as well as develop and acquire these competencies and skills. In this way, drop outs derived from failure to understand course contents and from the lack of support when needing help would eventually decrease.

MyLearningMentor (MLM) is a mobile application designed to address the lack of support and personalized advice for learners in MOOCs. MLM provides recommended planning and advice adapted to MOOC learners' 4Ps: Profile, Preferences, Priorities and previous Performance (measured as the time devoted to complete previous tasks) [ [START_REF] Alario-Hoyos | Adaptive planner for facilitating the management of tasks in MOOCs[END_REF]] [Gutiérrez-Rojas et al., 2014a]. MLM aims to promote self-directed learning, as well as the development of the competences and skills demanded by MOOCs. Although anyone could use MLM, it is designed for people with less study know-how, who are the ones with more difficulties and higher drop-out rates in MOOCs. The design and implementation of MLM has followed an incremental process. After capturing the requirements, a first functional architecture was drafted. A set of mockups from the user interface showing the expected functionality were also sketched, and can be consulted in [Gutiérrez-Rojas et al., 2014a], for those that want to build a mental model of MLM at this point.

The remainder of this paper is divided into five sections. Section 2 describes the works related to mobile context-aware recommender systems and adaptive learning planners which are the basis for MLM. Section 3 introduces MLM, including the requirements analysis, design and current implementation. Section 4 illustrates the use of MLM with practical examples. Section 5 presents a discussion on MLM. Finally, the conclusions of the paper are drawn in Section 6.

Mobile context-aware recommender systems and adaptive learning planners

MLM is built upon research in two main fields (Figure 1). First, mobile context-aware recommender systems provide information about what aspects of the context should be considered to provide planning and advice to learners in MLM. Second, research in adaptive learning planners serves to define how the planner should be designed.

Mobile context-aware recommender systems

The advances on mobile learning and context-aware technologies in the last decade have been seized as an opportunity to provide context-aware recommender systems [Adomavicius andTuzhilin, 2011] [Lonsdale et al., 2004]. These systems take the knowledge acquired in the field of recommender systems in Technology Enhanced Learning, the research in mobile learning, ubiquitous learning and learning analytics [START_REF] Long | Penetrating the Fog: Analytics in Learning and Education[END_REF], and the developments related to mobile personal learning environments [García-Peñalvo andConde, 2014] [García-Peñalvo et al., 2011] to provide content-driven recommendations based on contextual information. Context-aware recommender systems rely primarily on the notion of context. However, recommender systems should also take into account the human dimensions since end users are human beings who have different and changing intentions, perceptions, motivations and daily habits [START_REF] Bellotti | [END_REF]Edwards, 2001] [Freed et al., 2014]. Moreover, human beings establish conversation and relationships of trust with peers who are also part of the context [Buckingham Shum and Ferguson, 2012].

In the particular area of mobile learning, "context" has been defined in an abstract way, as an artifact that is continuously created by people interacting with other people in their surroundings (establishing the aforementioned conversation and relationships of trust) and using everyday tools [START_REF] Sharples | Towards a theory of mobile learning[END_REF]. When implementing a system (such as MLM), context should be specified and defined to fit users' needs [START_REF] Yujie | Some Challenges for Context-aware Recommender Systems[END_REF]. Specifically, three issues of the context should be defined: (1) what contextual information is relevant to provide learners with scaffolding for advancing on their aims, (2) how contextual information is gathered, and ( 3) what mechanisms are implemented to deliver this content to the user. These three issues are also the base of learning analytics, defined as the measurement, collection and reporting of relevant information about learning and the environment where learning occurs [START_REF] Siemens | Learning analytics and educational data mining: towards communication and collaboration[END_REF].

Regarding what contextual information is relevant, a recent study by Verbert et al. [START_REF] Verbert | Context-Aware Recommender Systems for Learning: A Survey and Future Challenges[END_REF] provides an extensive review of context-aware recommender systems and analyzes the context features that are considered in them. The results are summarized into a framework with 8 dimensions, which are consistent with the works on learning analytics [START_REF] Siemens | Learning analytics and educational data mining: towards communication and collaboration[END_REF] and social learning analytics [Buckingham Shum and Ferguson, 2012]: (1) computing context, which comprises the software, hardware and network technologies used to retrieve information about the device; (2) location context, which helps to identify where the user is located; (3) time context, such as timestamp data or time intervals; (4) activity context, which includes tracking explicit user interactions to capture information in the context (e.g., scanning a QR code or an RFID tag); (5) physical conditions, which includes data from the environment; (6) resource context, which collects information from learning resources; (7) user context, which includes information about user profile (such as past performance or topics of interest), and (8) social relation context, which includes data related to social connections within a group structure (such as relative performance and progress).

Regarding how contextual information is gathered, the literature shows several prototypes that gather and manage contextual information in different ways for filtering a particular content to the user [START_REF] Lonsdale | A context awareness architecture for facilitating mobile learning[END_REF]] [Adomavicious and Tuzhilin, 2005] [Verbert et al., 2013]. Literature reviews identify three mechanisms to collect contextual information [Adomavicius andTuzhilin, 2005] [Verbert et al., 2012]: (1) explicit, which relies on manual input from users; (2) implicit, when the information is automatically captured from the environment through different sensors (embedded or not into users' devices); and (3) by inference, in which the contextual information is inferred by analyzing users' interactions with the system or by combining different data. Once the contextual data has been captured, there are two main paradigms that define how to use this information as a filtering parameter for providing recommendations: (1) the context-driven querying and search paradigm, which utilizes the contextual information to filter from a particular corpus of resources, and (2) the contextual preference elicitation and estimation paradigm, which is based on models and learning networks that use contextual data to make the system learn about user preferences [START_REF] Verbert | Context-Aware Recommender Systems for Learning: A Survey and Future Challenges[END_REF].

Regarding what mechanisms are implemented to deliver this content to the user, recent literature provides some insights about the impact of notifications on mobile devices (one of the most common ways to deliver information to the user). Given the overabundance of information that people are exposed daily, one of the key aspects in the design of mobile and pervasive systems consists on capturing user attention [START_REF] Ferscha | Attention Management in Pervasive Computing[END_REF]; alerts or notifications are one of the most common mechanisms for this purpose. Some studies analyze the right moments to issue notifications and propose predictive models to find these moments [START_REF] Ferscha | Attention Management in Pervasive Computing[END_REF][START_REF] Gallacher | [END_REF]. [START_REF] Poppinga | Sensor-Based Identification of Opportune Moments for Triggering Notifications[END_REF] report some mechanisms to improve the receptivity of the user, such as scheduling interruptions, using proper content, or including notifications during "downtimes". A recent study suggests that using reminders that include quizzes or reviews on a particular learning resource, and that are based on learners' records of acquired knowledge (logs of users' performance), supports students to advance on the course [START_REF] Li | Context-aware and Personalization Method in Ubiquitous Learning Log System[END_REF].

MLM is built upon the results of these research studies. First, MLM context definition will include learners' 4Ps (Profile, Preferences, Priorities and previous Performance), as well as information about MOOCs. This fits with 4 of the dimensions proposed by Verbert et al. and with the definitions of context regarding environment, human aspects and social aspects [START_REF] Bellotti | Intelligibility and Accountability: Human Considerations in Context-Aware Systems[END_REF]: (1) time context, considering the availability of the learner and the best times to work according to his preferences; (2) resource context, considering the information about MOOCs and how tasks are organized; (3) user context, considering the human dimension of the context (intentions, perceptions, motivations and daily habits) by taking into account user's profile, priorities and previous performance; and (4) social context, considering other users' performances to enrich the planning and advice provided by MLM. Second, MLM will use both explicit and inference mechanisms to collect contextual information. On the one hand, the user will explicitly provide data to define, for instance, his preferences and priorities. On the other hand, inferential mechanisms will collect data to better understand users' needs and improve the recommendations and planning. All this data will be processed using a context-driven querying and search paradigm, filtering from the corpuses of MOOCs and tips and hints to help users advance on the course. Finally, MLM discards at this time the use of notifications to avoid overwhelming learners. Users will proactively decide when to ask for personalized planning and advice to MLM.

Adaptive learning planners

Adaptivity is a common term in computer science that refers to the capacity of a system (adaptive system) to adjust itself to new conditions and changes [Cristea and De Bra, 2002]. This term should not be mistaken for adaptability, which refers to the capacity of a system (adaptable system) to be tailored by the user, altering for instance its content or functionality [Cristea and De Bra, 2002]. From now on we will only focus on adaptivity and adaptive systems as a prelude to the introduction of MLM, which is an application designed to be adaptive (but not necessarily adaptable).

Research on personal assistants is a mature domain that explores mechanisms to facilitate the accomplishments of tasks through the automated creation of planners, using tools and mechanisms from the artificial intelligence field [START_REF] Mitchell | Experience with a Learning Personal Assistant[END_REF]] [Eder et al., 2003]. In the last decade, researchers in the Technology Enhanced Learning community have been exploring how to take advantage of these personal assistants to support students' progress by defining sequences of learning activities and tasks. [START_REF] Brusilovsky | Course sequencing techniques for large-scale web-based education[END_REF]] described three approaches for applying course sequencing in educational contexts: (1) using it as the core of a course maintenance system; (2) for generating an adaptive courseware for a group of learners; and (3) for dynamically generating a courseware that observes and adapts to students' progress. [START_REF] Karampiperis | Adaptive instructional planning using ontologies[END_REF] proposed applying ontologies and learning object metadata to recommend a sequence of learning resources to be provided to the learner. In their approach, domain concepts and learning resources are modeled as interconnected networks and combined into a "directed acyclic graph". The learning path is obtained from the application of a shortest-path algorithm towards the domain concept to be acquired.

Adaptive learning has a long history of researches contributing to the design and development of personal assistants and recommender systems (RS) which provide personalized instruction tailored to each student's needs [START_REF] Berlanga | Learning Design in Adaptive Educational Hypermedia Systems[END_REF]. Personal assistants and RS are also capable of recommending activity plans adapted to the student's profile and context. For instance, Koper [Koper, 2005] proposed analyzing the social context of the user (user's social interactions) for recommending learning paths formed by sequences of units of learning. The results of applying these RS in lifelong learning scenarios show that the learners' effectiveness increases [START_REF] Janssen | Navigational support in lifelong learning: enhancing effectiveness through indirect social navigation[END_REF]. [START_REF] Hummel | Combining social-based and information-based approaches for personalised recommendation on sequencing learning activities[END_REF] proposed an evolution of Kopers' idea designing a RS that combines social information, learning profiles and characteristics of the learning activities to provide the appropriate learning path to each user.

More recently, research in adaptive planners has been applied to support ubiquitous learning, taking advantage of the affordances of mobile technologies. These planners adapt the learning paths using physical contextual information. For instance, Yau and Joy [START_REF] Yau | Architecture of a Context-aware and Adaptive Learning Schedule for Learning Java[END_REF] proposed the Context-aware and Adaptive Learning Schedule (CALS) framework to support students in their daily routines. This framework was used to develop an application that takes into account students' learning style and current learning context for recommending a particular activity or learning path. In their proposal, students explicitly introduce information to predefine schedules, activity types and time slots, and propose an activity to be performed during this particular time.

MLM takes these studies as a reference to propose learning paths, considering learners' needs and the structure of MOOCs. MLM differs from the current research in personal assistants, RS, and adaptive planners in two main aspects. On the one hand, the context of application is different: MLM proposes scheduling tasks to improve the overall learning experience in a MOOC by guiding the learner, looking for a continuum between traditional and connectivist MOOCs [Gillet, 2013]. On the other hand, MLM considers explicit feedback introduced by the learner. This learners' feedback will be incorporated to make adjustments and refinements in future planning [START_REF] Wang | Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system[END_REF].

MyLearningMentor

This section presents MyLearningMentor (MLM), an application that addresses the lack of support and personalized advice for learners participating in MOOCs and that is built upon existing research on mobile context-aware recommender systems and adaptive learning planners. First, the requirements of MLM are defined. Next, the design of MLM is discussed according to these requirements, including the functional architecture and a brief overview of the planning algorithm. Finally, the current implementation of the application is briefly described. Despite the separation of the requirement analysis, design and implementation in three subsections, the process of building MLM follows an iterative process on these three issues.

Requirements analysis

MLM targets less experienced learners that enroll in MOOCs. In order to gain insights on the work habits and study skills of today's learners (both in face-to-face and in online instruction), a 5-point Likert scale questionnaire was distributed among second-year engineering students from a Spanish university. This questionnaire covered topics such as the selection of a proper workplace to study, the presence of distractions while studying, the habit of studying with peers (face-to-face or remotely), the planning of the time to study (and the need for rescheduling it during the week), and students' experience with online education [Gutiérrez-Rojas et al., 2014a]. The questionnaire was answered by 41 students. These students had extensive experience in face-to-face and blended learning but little experience in online learning. The data extracted from this questionnaire point out that today's students lack many of the study skills and work habits that are necessary for becoming selflearner, which, according to Laplante [Laplante, 2013], is a skill to be developed in order to follow up and complete most MOOCs. In addition, there is a general lack of experience with online courses; the few students that tried online education recognized having difficulties that prevented them from completing online courses.

From the answers to this questionnaire, and considering the overall objectives of MLM, the following five requirements were collected.

The first requirement (REQ1) is that MLM must be offered as a mobile application. According to most of the reports published, MOOC participants are typically in the range between 25-40 years old [Christensen et al., 2013] [Ho et al., 2014], an age in which mobile technologies are having a big impact. As a matter of fact, MOOC participants with mobile devices tend to interact more with their peers, this social component being a key element in MOOCs [START_REF] Dewaard | [END_REF]. In addition, most MOOC platforms are being optimized to display courses in a responsive way, enabling the download of videos to watch them in places with poor connectivity, such as the public transport.

The second requirement (REQ2) is that MLM must be personalized to the different participants that enroll in the different types of MOOCs. Regarding differences between participants, MLM will collect information from them and react accordingly. This information will be classified in 4Ps: Profile (e.g., background, age…), Preferences (e.g., available hours, best time to study…), Priorities (e.g., MOOCs that the participant wants to address with a higher priority) and previous Performance (e.g., tasks that the participant was able to complete and time spent on them). Regarding differences between MOOCs, MLM will take into account the amount of workload and its expected distribution (e.g., sequence of tasks and distribution throughout the MOOC), the nature of each task (mandatory, recommended or optional), and the deadlines (if any).

The third requirement (REQ3) is that MLM must provide an adaptive planner that organizes the tasks that each participant needs to complete in the short-term (e.g., the next week) and in the long-term. The adaptive planner will take into account the 4Ps that define the participant and also the tasks that define each particular MOOC, as described in REQ2. In order to improve the correctness of the planning, the adaptive planner will react when the participant provides new information (e.g., adding information about the completion of tasks as part of the previous Performance) or updates existing one (e.g., updating the available hours for the next week as part of the Preferences), or when new tasks and/or deadlines are set in the MOOC.

The fourth requirement (REQ4) is that MLM must enable learners to publish and curate information about the MOOCs and their tasks (following a social, or crowdsourced, model) so that the adaptive planner can provide a rich planning, as described in REQ3. Although ideally MLM should be able to collect all the information related to the MOOCs from the platforms in which they are deployed, currently most platforms do not offer APIs suitable for this purpose. The alternative of scraping the information from the website is feasible, but cumbersome and error prone, which motivates the appearance of this requirement. It may also happen that teachers decide to publish new tasks during the course, or include changes between two editions of the same course, requiring in both cases the update of the information stored in MLM.

Finally, the fifth requirement (REQ5) is that MLM must provide advice to learners that have problems to follow the MOOCs. This advice will be provided in the form of tips and hints. Tips and hints can cover generic issues that learners should take into account to follow a MOOC, or specific issues related to a particular MOOC. Examples of the former can be tips for increase concentration while working on a task, or for reviewing and assessing peer's work. Examples of the latter can be recommendations of particular references or videos to get an additional explanation of the most difficult contents in a MOOC.

Design

Figure 2 shows the functional architecture of MyLerningMentor (MLM). This architecture follows a client-server model and makes use of external services. These external services are the platforms in which the MOOCs are deployed (e.g., edX, Coursera, MiríadaX, etc.). The client of MLM is a mobile application (MyLearningApp). This design decision is a consequence of requirement REQ1. The server of MLM contains three databases, five services, and several independent processes. MyLearningApp communicates with the services sending requests to a REST interface. The independent processes collect information from the external services populating the databases.

The three databases are: User data, MOOCs data and Tips and Hints data.  MOOC data. This database contains information about the existing MOOCs.

This information will be general information of the MOOC and specific information of the tasks in the MOOC. We consider as a task any activity that the learner can do in the course, such as watching a video, answering a question, reviewing a work, consulting additional links. General information includes course name, number of weeks, expected weekly workload, platform, URL, and knowledge area. Specific information includes name of the task, order in the course, type of task (e.g., video, exercise…), nature of the task (required, recommended, optional) and deadline (if any).  Tips and Hints data. This database contains a set of preloaded tips and hints oriented to provide generic or specific advice to users, as indicated in requirement REQ5. Each tip and hint contains a name, the text with the advice, the category, and, in the case of specific advice, the MOOC to which it refers.

The five services are: Account and profile management, Adaptive planner, MOOC information publisher, MOOC directory, Tips and hints.  Account and profile management. This service stores information about the user in the User data database. That includes the generation of new information and the modification of existing information related to the user account and the user profile. The first time the user downloads and clicks on MyLearningApp this service will be accessed in order to complete the account and profile information.  Adaptive planner. The adaptive planner is the main service in MLM. It provides each learner with a detailed planning of the sequence of tasks to be completed in the short-term (typically a week) and in the long-term, indicating the number of tasks, their order and the best times to complete them (see REQ3). In order to calculate the planning in a personalized way, as stated in REQ2, the adaptive planner collects information from the User data and MOOC data databases (see Figures 2 and3). The adaptive planner collects the 4Ps from the User data database: Profile, Preferences, Priorities and previous Performance. It also collects the sequence of tasks in each MOOC, including their types, deadlines and nature: required, recommended or optional.

 MOOC information publisher. This service enables users to publish two kinds of information in the databases. First, users can publish in the User data database information about their previous performance, indicating the time devoted and if they were able to complete the tasks that the adaptive planner assigned to them on time. This previous performance is taken into account by the adaptive planner in the next iterations in order to adjust the planning provided to each learner. Second, users can complete, update and curate the information about MOOCs stored in the MOOC data database (REQ4). This ensures a higher accuracy in the sequence of tasks for each MOOC and, eventually, a better planning.  MOOC directory. This service enables users to search for MOOCs in the MOOC data database, indicating the URL, or alternatively, the platform and name of the MOOC. Once the MOOC is found, the user can add it to his profile in the User data database and establish the priority for this MOOC. The user can register new MOOCs that are not available in the MOOC data database, but only through the MOOC information publisher.  Tips and hints. This service provides advice to learners in the form of short tips and hints (REQ5), which are collected from the Tips and Hints data database. Tips and hints can be adapted also to the 4Ps of each learner (collecting this information from the User data database), and can be either generic (about MOOCs in general), or specific (about one MOOC in particular).

MLM also incorporates several independent processes that scrape information from the platforms web sites. Due to the different organization of information, every platform needs a different process. These processes run periodically collecting (and updating) information that is stored in the MOOC data database, although they can also be invoked by the MLM services at certain moments.

The planning algorithm followed by the adaptive planner has already been published by the authors and is detailed in depth in [START_REF] Alario-Hoyos | Adaptive planner for facilitating the management of tasks in MOOCs[END_REF]. A summary of the planning algorithm is presented here so that the reader can get a feeling of what the main steps in the planning algorithm are. The planning algorithm starts from the assumption that a learner should complete all the required tasks before their deadlines. Therefore, it prioritizes required tasks over recommended and optional tasks, taking into account the available study time indicated by the learner: 1. For each MOOC in which the learner is enrolled, the algorithm initially allocates the amount of time required to complete the entire sequence of tasks (including required, recommended and optional tasks), multiplied by a factor that represents learner's profile and previous performance; 2. MOOCs are ordered according to the list of priorities given by the learner, and the time allocated for each MOOC is adjusted according to this order; 3. The algorithm generates the sequence of required tasks for each MOOC, computing if it is possible to complete all the required tasks for all the MOOCs according to learner's preferences, recommending withdrawing MOOCs with a lower priority otherwise; 4. The remaining time is allocated for the recommended tasks, starting from those that belong to the MOOC with a higher priority; 5. If there is still time, optional tasks are scheduled following the list of priorities.

Implementation

The implementation of MLM follows an incremental approach. Currently a first version of the client, of the five services, of the three databases and of one of the processes is developed. The mobile client is developed for Android OS, using the Android SDK 4.4.2, the Java language and an Android emulator; Figure 4 shows two screenshots with the planning for a learner, where the left screenshot shows the next tasks that the learner has to complete and the right screenshot shows further information on one particular task.

In the server side, the five services are developed in PHP. The information exchanged between client and server is formatted in JSON documents. The three databases are developed using SQL or MongoDB, depending on the characteristics of the information to be stored. Particularly the User data and the Tips and Hint data databases are developed as relational databases in SQL, while the MOOCs data database is developed using a document-oriented database in MongoDB. The implementation process has resulted in the need for a new database in order to store the relationship between users and the tasks they complete (previous performance) as a document-oriented database in MongoDB.

Finally, an edX scraper built on three Python scripts is currently under development. These scripts are used to access all the MOOCs in edX and to obtain the public information from the courses, their structure, and the tasks learners need to complete. Two of these scripts are based on the Selenium browser automation framework (http://www.seleniumhq.org).

Use cases and practical examples of use

From the definition of the architecture, MLM supports the following main use cases: 1. Registering and completing user profile. The user will interact with the Account and Profile Management service and, as a result, the User data database will be updated. 2. Searching for a MOOC. The user will interact with the MOOC directory service in order to search for a course (or for information about a course) in the MOOC data database. This search may result in invocations to the independent processes to collect the courses (or information about the courses), updating the MOOC data database. 3. Adding a new MOOC. The user will interact with the MOOC directory in order to add a new MOOC to his profile. As a result, the User data database will change. This use case includes use case 2 ("Search for a MOOC") as a previous step to the addition of the MOOC to the user's profile. 4. Completing MOOC information. The user will interact with the MOOC Information Publisher to add more information about a MOOC. As a result, the MOOC data database will be modified. 5. Asking for personal planning. The user will interact with the adaptive planner, which will collect information both from the MOOC data database and the User data database for calculating the personalized planning.

6. Adding feedback about performance. The user will interact with the MOOC Information Publisher adding his performance in the User data database. 7. Asking for tips. The user will interact with the Tips and Hints service which will collect information from the User data and the Tips and Hints databases.

Given the former use cases, this section will focus on the third and fifth ones, explaining them through user stories. Alice and Bob have recently discovered MOOCs and they would like to take advantage of them to enhance their professional careers. Nevertheless, they feel that they do not have enough time to dedicate to MOOCs after work, and that they are not able to properly organize their time by themselves, given the number of tasks that are required to complete the MOOCs they have been scouting. In addition, Alice and Bob have no experience with online education and they feel they may need some mentoring on how to make the most of MOOCs. A friend of them recommends Alice and Bob MyLearningMentor, a new mobile application that offers personalized planning and advice to MOOC learners, and both Alice and Bob decide to install MLM in their smartphones.

After checking the available MOOCs in Coursera, edX and MiríadaX, Alice and Bob decide to start the edX course entitled Mentoring 101, whose URL is urlMentoring101. Alice enrolls in Mentoring 101 directly in edX and then she wants to add this course in MLM to receive personalized planning and advice. Alice logs in MyLearningApp and requests to add a new course, indicating the URL. MyLearningApp sends this request to the server. Within the server, her request is processed by the MOOC Directory service, which searches within the MOOC data database if there is information stored about Mentoring 101. As Alice is the first user interested in this MOOC, there is no information about it in the database. As a consequence, the MOOC Directory asks the suitable scraper (edX scraper) for information about Mentoring 101. The information returned by the scraper is inserted in the MOOC data database. Finally, Mentoring 101 is added as one of Alice's courses in the User data database. Notice that if, later, Bob wants to add Mentoring 101 to his courses, the general information about this MOOC will be already available in MLM and there will be no need to perform the web scraping from the edX platform.

As Alice is eager to start working in the MOOC, she asks for her personal planning to MLM. Her request is processed by the Adaptive Planner, which gets Alice's 4Ps, including the MOOCs she is enrolled, from the User data database. It is noteworthy that the Adaptive Planner also collects, for each MOOC, the sequence of tasks from the MOOC data database (see Figure 3). If there is no information about the tasks (or if outdated), the suitable scraper will be notified in order to obtain the sequence of tasks, updating the corresponding entry in the MOOC data database and returning the sequence of tasks to the Adaptive Planner. Once the Adaptive Planner has the whole sequence of tasks, it will check Alice's status on them (if they are completed or not) in the User data Database. Then, the Adaptive Planner will execute the planning algorithm as explained in section 3.2, returning the personalized planning to Alice.

It is important to note that scraping the information about the tasks in a MOOC is a computationally expensive process. Therefore, it cannot be performed every time a user wants to get his personalized planning. As MOOCs are available anytime and anywhere, the usual approach of updating the MOOC data periodically (e.g., once a week at night) is not suitable. Alternatively, MLM sets both an expiration date and a validity period for the information stored in the database that was scraped from MOOC platforms (this is not applied to the information crowdsourced by the community of users). If Bob asks for his personal planning immediately after Alice, the list of tasks of Mentoring 101 will not be updated (unless they reached their expiration date). However, if the system asks for the list of tasks of Mentoring 101 once their validity expires, their update will be mandatory.

Discussion

The architecture of MLM involves several services that require a more detailed design and analysis. This is for instance the case of the adaptive planner [Alario-Hoyos et al., 2014], the tips and hints service and the scrapers. For this reason an incremental process is followed for implementing the architecture, according to which the different services are refined and improved iteration by iteration. Once all the services are ready, and before releasing the application to such a potentially large community of learners, a preliminary evaluation will be conducted with a limited number of users and courses. These users will be university students with a similar profile to those who completed the questionnaire that led to the capture of requirements for MLM. Then, MLM will be offered as a supporting tool for learners who take part in the MOOCs. The purpose of this evaluation will be to assess the usefulness of MLM and the correctness of the planning and advice provided by this tool. The outcomes of this evaluation will be used to continue to refine the design and implementation of MLM.

One of the most important aspects when providing a personalized planning is to have accurate and updated information about the tasks learners need to carry out, as well as about the sequence they form. MLM follows a mixed approach, combining web scraping and crowdsourcing to get this information. Web scraping is the alternative to the lack of APIs for retrieving information from the MOOC platforms, and alleviates the workload of users writing all the tasks from scratch, as well as the well-known cold start problem in social systems. Nevertheless, this strategy is very sensitive to changes in the design of web interfaces and may cause inconsistencies if information updates are not properly addressed. Crowdsourced information contributes to increasing the accuracy of the data, but relies on the willingness of users. Furthermore, crowdsourced information may need manual checking or community approval before becoming part of the MOOC data database. Gamification or the assignment of special roles (e.g., curator) to expert and proactive users are recurrent strategies to promote the sustainable collaborative knowledge construction in systems that rely on crowsourced information.

The planning and advice MLM provides to users can be delivered following different approaches. Currently, it is the user who explicitly requests the planning and advice to MLM. However, MLM can also be redesigned to introduce mobile alerts reminding users the best moments to work on pending tasks (according to users' preferences and weekly planning). Nevertheless, an excess of alerts can be annoying or disruptive when they occur at inappropriate times and does not necessarily improve the responsiveness and attitude of the user towards the commitment with their duties, as discussed in section 2.2. It is therefore necessary to conduct a study with MOOC learners to find out the best way to deliver the personalized planning and advice through MLM.

Regarding the personalized planning, the algorithm designed to calculate it prioritizes the required tasks of every single MOOC where the user is enrolled; recommended and optional tasks are only allocated if there is enough available time [START_REF] Alario-Hoyos | Adaptive planner for facilitating the management of tasks in MOOCs[END_REF]. As a result, the personalized planning always shows required tasks first, then recommended tasks, and, finally, optional ones. Within each type, tasks are ordered by MOOC personal priority. This design decision aims to ensure that the learner is able to pass all the MOOCs he enrolled (even if he is not able to complete all the recommended and optional tasks). However, other approaches are possible with users selecting the type of approach as an input parameter of the adaptive planner. An alternative approach would be, for instance, allocating time for all the tasks (required, recommended and optional) in the MOOC with a higher priority; and using the remaining available time for the other MOOCs, allowing the user to deepen in the knowledge of the MOOCs he is more eager to follow up.

Regarding the personalized advice, the tips and hints service provides different advice depending on the profile, preferences and previous performance of the learner. In the current design, these tips and hints are preloaded in the tips and hints database. Alternatively, a crowdsourced approach where teachers and peers generate and classify new tips and hints could be implemented. It would then be possible not only to have teachers' advice for a specific MOOC as static information loaded before the start of the course, but also as dynamic information that teachers create and update as a reaction to learners' progress throughout the MOOC.

The search of MOOCs is carried out in MLM through the MOOC directory service, either indicating the URL, or the platform and name of the MOOC. Nevertheless, the user needs to know beforehand the course in which he wants to enroll and register directly through the corresponding MOOC platform. Alternatively, it can be consider the possibility of integrating a richer MOOC search system, such as Class Central (http://class-central.com) or moocrank [Gutiérrez-Rojas et al., 2014b] to help users search and discover new MOOCs that match their expected learning objectives. This way, MLM users could easily discover and enroll in new MOOCs for which they would receive planning and advice, all in one single mobile personal learning environment [START_REF] García-Peñalvo | The impact of a mobile Personal Learning Environment in different educational contexts[END_REF].

The current design of MLM is decoupled from MOOC platforms, with the only exception of the web scraping processes retrieving information about MOOCs and tasks. Nevertheless, it would also be possible to design and implement a tighter integration with certain platforms, so that MLM would be a generic service offered, for instance, to edX learners in the edX platform. The shortcoming of this tight approach is that MLM would not be able to support learners that want to enroll in MOOCs offered by different platforms.

Conclusions and future work

The lack of work habits and study skills is a significant factor that hinders the follow up and completion of MOOCs, affecting particularly learners with little or no experience in online learning. MOOC teachers cannot give personalized support to learners and therefore there is a need for approaches that provide learners with planning and advice to face the challenge of participating in MOOCs and, eventually, become self-learners. MyLearningMentor (MLM) addresses this problem providing personalized planning and advice to learners in MOOCs. Nevertheless, MLM is still in the process of implementation through an iterative construction of the different services, processes and databases that are defined in its architecture. And a proper evaluation with real users is already planned to understand its benefits and impact.

Although MLM was conceived to help less experienced learners that enroll in MOOCs, it needs to be researched if MLM can be useful for learners with other profiles, such as people with study experience but with problems for self-managing their time. Further research is also planned in order to see if MLM can be beneficial in other educational contexts. Examples of these contexts in formal education are faceto-face courses, blended learning courses (e.g., university courses with a strong workload outside the classroom), online (but private) courses, or vocational training. Non-formal educational settings, such as workplace learning and professional development can also serve to assess the usefulness of MLM in different contexts. The particular educational context, as well as the particular user profile will very likely have an impact on the type of planning and advice that MLM has to provide. Finally, MLM is intended to be extended in order to serve as a communication channel between alumni, teachers and other mentors around MOOCs.

In conclusion, MLM is a first approach towards the objective of reducing the education gap between those people that are qualified and those that are not; gap that MOOCs are otherwise contributing to increase, considering the profiles of most of the participants taking advantage of these courses. Moreover, MLM goes beyond current research on MOOCs by considering the affordances of mobile and context-aware technologies to provide a more adaptive environment to improve learners' learning experience.

Figure 1 :

 1 Figure 1: Research domains supporting MLM.

Figure 2 :

 2 Figure 2: Overview of the functional architecture of MLM. This is a client-server architecture that makes use of external services (MOOC platforms). The client is a mobile application. The server includes 5 services, 3 databases, and several processes (3 in this example) that collect information from the external services. Arrows pointing at the databases indicate services or processes that store information in them. Arrows coming out of databases indicate services that retrieve information from them.

Figure 3 :

 3 Figure 3: Input sources of the adaptive planner. From User data database the 4Ps: profile, preferences, priorities and previous performance (time devoted to complete previous tasks). From MOOC data database: sequence of tasks in MOOCs.

Figure 4 :

 4 Figure 4: Screenshots of the adaptive planner obtained from the Android emulator. Left side shows the next tasks to be completed. Right side shows more information for one task, and the user providing feedback about the completion of that task.

Alario-Hoyos C., Estevez-Ayres I., Perez Sanagustin M., Leony D. ...

Acknowledgements

This work has been funded by the Spanish Ministry of Economy and Competitiveness Project TIN2011-28308-C03-01, the Regional Government of Madrid project S2013/ICE-2715, and the postdoctoral fellowship Alliance 4 Universities. The authors would also like to thank Israel Gutiérrez-Rojas for his contributions to the ideas behind MLM and Ricardo García Pericuesta and Carlos de Frutos Plaza for their work implementing different parts of the architecture.


Marion M Simmons

Leigh Thorne

Angel Ortiz-Pelaez

John Spiropoulos

Soteria Georgiadou

Penelope Papasavva-Stylianou

Olivier Andreoletti

Stephen A C Hawkins

C Hawkins

Daniela Meloni

Thorne, Spiropoulos, Hawkins, Cassar Simmons

Georgiadou

Claire Cassar
email: claire.cassar@apha.gov.uk

Transmissible spongiform encephalopathy in goats: is PrP rapid test sensitivity affected by genotype?

Keywords: ELISA, goats, prion protein, scrapie, transmissible spongiform encephalopathy

scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Classical scrapie, the archetype of the transmissible spongiform encephalopathies (TSE) or prion diseases, was first recognized in sheep almost 300 y ago and became endemic in the national flocks of several countries. [START_REF] Detwiler | Scrapie[END_REF][START_REF]Scientific opinion on the scrapie situation in the EU after 10 years of monitoring and control in sheep and goats[END_REF] Classical scrapie was first reported in goats in France in the 1940s. [START_REF] Chelle | Un cas de tremblante chez la chèvre [A case of scrapie in the goat[END_REF] Active surveillance for scrapie in small ruminants has been compulsory in the EU since 2002, with the requirement that EU member states test a random sample of healthy slaughtered animals and fallen stock (TSE Regulation (EC) 999/2001), [START_REF]Regulation (EC) No 999/2001 of the European Parliament and of the Council of 22 May 2001 laying down rules for the prevention, control and eradication of certain transmissible spongiform encephalopathies[END_REF] complementing passive surveillance, that requires the detection of bovine spongiform encephalopathy (BSE) in cattle and scrapie in sheep and goats since 1998 (Decision 98/272/ EC), [START_REF]Commission Decision of 23 April 1998 on epidemio-surveillance for transmissible spongiform encephalopathies and amending Decision 94/474/ EC[END_REF] focusing on animals showing clinical signs compatible with TSE. This TSE Regulation, which evolved through the need to control BSE in cattle, sets out the framework for all the measures related to the prevention and control of scrapie in the EU, including the age of animals to be tested and the tests that are listed as approved for this purpose.

When mandatory active surveillance of small ruminant populations was first introduced, EU-approved rapid tests that performed satisfactorily on bovine tissues (Decision 2000/374/EC) [START_REF]Commission Decision of 5 June 2000 amending Decision 98/272/EC on epidemiosurveillance for transmissible spongiform encephalopathies[END_REF] were provisionally approved for small ruminants and used for the active and passive surveillance of TSE in sheep and goats during 2002-2004. In 2007, an EU evaluation of the diagnostic and analytical sensitivity of these rapid postmortem tests was undertaken using samples from cases of naturally occurring classical and atypical scrapie. [START_REF] Food | Protocol for the evaluation of rapid post mortem tests to detect TSE in small ruminants[END_REF] At this point, some tests were not able to meet all of these requirements for atypical scrapie and were subsequently delisted. All of these evaluations were based on sheep samples, and approval extrapolated to use in "small ruminants."

The currently approved rapid and confirmatory tests are all based on immunodetection methods, using antibodies that recognize PrP Sc , the prion protein, an abnormal isoform of the host protein PrP that is encoded by the PRNP gene. In sheep, polymorphisms in the PRNP gene at codons 136, 141, 154, and 171 are demonstrated to be associated with disease susceptibility. [START_REF] Benestad | Atypical/Nor98 scrapie: properties of the agent, genetics, and epidemiology[END_REF][START_REF] Clouscard | Different allelic effects of the codons 136 and 171 of the prion protein gene in sheep with natural scrapie[END_REF][START_REF] Hunter | Natural scrapie in a closed flock of Cheviot sheep occurs only in specific PrP genotypes[END_REF] The goat PRNP gene is also polymorphic, but the polymorphisms are largely located at different codons from those in sheep, and many of the polymorphisms have been observed in only 1 or 2 countries, and associated with specific breeds. At least 5 of these changes have been suggested to be associated with TSE susceptibility. [START_REF] Panel | Scientific opinion on genetic resistance to transmissible spongiform encephalopathies (TSE) in goats[END_REF][START_REF] Vaccari | State-of-the-art review of goat TSE in the European Union, with special emphasis on PRNP genetics and epidemiology[END_REF] At present, the EU goat population is much more heterogeneous than the sheep population, with more PRNP-related polymorphisms, and with marked breed-related differences.

The ability of the current tests to detect disease-specific PrP Sc generated against these different genetic backgrounds is assumed rather than tested. Many TSE-affected goats have been identified through active surveillance using these tests, and these cases have been readily confirmed by alternative methods. There is no evidence, therefore, to suggest that test specificity is an issue. There has been no systematic assessment, however, of the sensitivity of these tests in goats, to our knowledge.

Data indicate that the performance of both rapid and confirmatory tests for the detection of disease in goats experimentally infected with either scrapie or BSE can be variable as a consequence of host allelic variations. [START_REF] Madsen-Bouterse | PRNP variants in goats reduce sensitivity of detection of PrP(Sc) by immunoassay[END_REF][START_REF] Meloni | EU-approved rapid tests might underestimate bovine spongiform encephalopathy infection in goats[END_REF][START_REF] Papasavva-Stylianou | The effect of polymorphisms at codon 146 of the goat PRNP gene on susceptibility to challenge with classical scrapie by different routes[END_REF] Similar data have been reported previously for sheep. [START_REF] Tongue | The importance of the PrP genotype in active surveillance for ovine scrapie[END_REF] The experimental challenge study [START_REF] Papasavva-Stylianou | The effect of polymorphisms at codon 146 of the goat PRNP gene on susceptibility to challenge with classical scrapie by different routes[END_REF] of scrapie in goats polymorphic at codon 146 showed that the presence of aspartic acid (D) at codon 146 resulted in false-negative results using an EU-approved rapid test. We explored the relative performance of the commonly used commercial tests in scrapie-positive goats with a range of PRNP polymorphisms, to establish whether any other polymorphisms might have an adverse effect on test performance.

Materials and methods

Composition of sample panel

Brain samples were obtained from 36 goats that had been experimentally inoculated intracerebrally with brain tissue from naturally occurring classical scrapie cases originating in Cyprus, as described previously. [START_REF] Panel | Scientific opinion on genetic resistance to transmissible spongiform encephalopathies (TSE) in goats[END_REF][START_REF] Papasavva-Stylianou | The effect of polymorphisms at codon 146 of the goat PRNP gene on susceptibility to challenge with classical scrapie by different routes[END_REF] These goats represented all of the polymorphisms at codon 146 (NN, ND, DD, NS, SS), while being homozygous for the wild-type arginine (RR) at codon 154.

In addition, EU member states were invited to provide examples of positive caprine scrapie cases representative of the genetic composition recognized within their national herd. These (referred to as the mixed EU panel) comprised both field cases and experimental animals from France, Italy, Cyprus, and the UK (Supplementary Table 1). Disease had been confirmed by the contributing laboratory based on the presence of PrP Sc in the brainstem and/or the lymphoreticular tissues using statutory confirmatory tests (immunohistochemistry [IHC] or western blot [WB]). Five goats experimentally challenged with BSE 22 completed the set.

The full PRNP sequence was not available for any animal, so the "wild-type" genotype has been assumed for codons that were not specifically reported by the donating laboratory. The panel comprised samples from 162 animals, representing (in various proportions) the polymorphisms G127S, I142M, N146DS, R154H, R211Q, Q222K, and P240S.

Limitations as a result of the amount of tissue available from single animals mean that no dilution series were used as part of our analysis and therefore no analytical sensitivity estimates were possible. Consequently, our results are not directly comparable to previous formal European Food Safety Authority (EFSA) evaluations of test performance on bovine and ovine samples, but rather give a qualitative assessment of the possible field (i.e., diagnostic) sensitivity of the tests when they are used for disease detection in an active surveillance context. In addition, some animals were at clinical end-stage and others were apparently healthy, thereby representing the range of disease states that might be encountered in the field during active surveillance activities (Supplementary Table 1).

Testing of samples

The mixed EU panel samples were finely chopped and divided into 4 aliquots; the Cypriot experimental panel samples were chopped and divided into 3 aliquots. Each aliquot was tested using 1 of the following 3 rapid test kits: IDEXX HerdChek Small Ruminant conjugate SRB-CC (IDEXX SR; IDEXX Laboratories, Westbrook, ME), IDEXX HerdChek Bovine conjugate CC (IDEXX BOV), and Bio-Rad Sheep/ Goat (Bio-Rad SG; Bio-Rad Laboratories, Hercules, CA). In addition, the Bio-Rad TeSeE was applied to all samples except the Cypriot experimental panel, for which data were already available. [START_REF] Papasavva-Stylianou | The effect of polymorphisms at codon 146 of the goat PRNP gene on susceptibility to challenge with classical scrapie by different routes[END_REF] All assays were carried out according to the manufacturers' instructions. All tests were performed at the Animal and Plant Health Agency (APHA; Weybridge, UK), thereby minimizing operator variability.

Calculation of test sensitivity

The samples in our study represent goats with polymorphisms at codons 127, 142, 146, 154, 211, 222, and 240. Of these polymorphisms, only I142M and R211Q were represented by numbers of cases that were sufficient to use for sensitivity calculations. Estimation of sensitivity of the rapid tests was conducted considering the result of the confirmatory tests (either IHC or WB) as the true disease status of the animal. The sensitivity of the 4 rapid tests considered, namely IDEXX SR, IDEXX BOV, Bio-Rad SG, and Bio-Rad TeSeE, were calculated according to clinical status and genotypes at codon 142 and 211 for the mixed EU panel, and at codon 146 for the Cypriot experimental panel. The 95% confidence intervals (CIs) of the proportion were calculated by applying the Wilson score interval. Comparisons of the sensitivity were conducted by applying the 2-sample test for equality of proportions with continuity correction, and calculation of the CIs using the R package "binom" (Dorai-Raj S. Binom: binomial confidence intervals for several parameterizations. R package v.3.6.0. 2014. Available at: https://rdrr.io/cran/ binom/).

Results

Cypriot experimental panel

The 36 positive cases of the Cypriot experimental panel were subject to the IDEXX HerdChek test (both conjugates) and to the Bio-Rad SG. The IDEXX HerdChek test (both conjugates) performed well regardless of the host genotype (Figs. 1A,1B), and were able to correctly detect all cases except 2 animals that were preclinical intercurrent deaths (one DD and one SS). The Bio-Rad SG (Fig. 1C) detected all cases of ND, NN, and NS genotypes but not those of DD (0 of 8) and SS (0 of 8) genotypes. Using the Bio-Rad TeSeE (Fig. 1D), [START_REF] Papasavva-Stylianou | The effect of polymorphisms at codon 146 of the goat PRNP gene on susceptibility to challenge with classical scrapie by different routes[END_REF] only the NN animals resulted in a consistently high optical density (OD); the other genotypes gave more variable OD results. The presence of aspartic acid (D), and to a lesser extent serine (S), at codon 146, appeared to be a limiting factor for the performance of this test.

European surveillance panel

The sensitivity of each test by clinical status was compared with the observed sensitivity of these tests as reported by other studies (Table 1). This initial comparison did not take genotype into account.

We included 128 positive samples in this analysis: 49 from Cyprus (38%), 42 from the UK (33%), 31 from France (24%), and 6 (representing 4 animals) from Italy (5%). Of the 128 positive samples, 25 were clinically negative (from the UK), 41 were clinically positive (31 from France and 10 from the UK), 7 inconclusive (from the UK), and 55 were of unknown clinical status (49 from Cyprus and 6 from Italy; Supplementary Table 1).

A total of 106 samples were positive by IDEXX SR (82.8%; 95% CI: 75.3-88.4%), 108 by IDEXX BOV (84.4%; 95% CI: 77.1-89.6%), and 94 by Bio-Rad SG (73.4%; 95% CI: 65.1-80.3%). For the Bio-Rad TeSeE, there were results from 73 samples, of which 55 were positive (75.3%. 95% CI: 64.3-83.8%).

When looking at data from all animals, the sensitivity of the 4 tests is not significantly different pairwise, except when comparing IDEXX BOV and Bio-Rad SG (chi-squared test [χ 2 ] = 3.97, df = 1, p = 0.046), albeit marginally. The sensitivity of the 4 tests when applied to samples from clinical positive animals (41) was very high, with all samples correctly classified by both IDEXX tests and the Bio-Rad SG, and only 1 positive sample missed by the Bio-Rad TeSeE (40 of 41). However, test performance varied between rapid tests in samples from clinically negative animals, or those of unknown status, in which the sensitivity of all the tests was consistently lower than in samples from clinical cases. The sensitivity of the Bio-Rad TeSeE in this subset of samples (n = 87) was significantly lower than that of the IDEXX BOV and IDEXX SR (χ 2 = 7, df = 1, p = 0.008; 2-sample test for equality of proportions with continuity correction). Although the sensitivity of the Bio-Rad SG in this subset of samples (87) was lower than that of the IDEXX BOV and IDEXX SR, it was not statistically significantly lower, albeit marginally (p = 0.007). There was no significant difference in the sensitivity between the 2 Bio-Rad tests (p = 0.2).

The sensitivity of the 4 tests across genotypes at codons 142 and 211 (Table 2) was not significantly different, despite the absolute differences. For example, between IDEXX conjugates and Bio-Rad in the I142M genotype, there is no significant difference in the proportion of positive samples detected by the 2 IDEXX conjugates (92% and 100%, respectively) and the Bio-Rad conjugates (68% and 72%, respectively).

Discussion

The 4 rapid tests included in our study were able to detect scrapie in goats in the clinical stages of the disease. However, test performance was more variable for animals that were preclinical or of unknown status.

The early detection of disease in animals with preclinical infection (i.e., field diagnostic sensitivity) relies on the correct sampling of brain target areas in which the initial accumulation of PrP Sc is known to occur, regardless of the test that is applied. The analytical sensitivity aspect of previous formal test evaluations [START_REF]Scientific opinion on analytical sensitivity of approved TSE rapid tests[END_REF] recognized this problem by the inclusion of sample dilution series as a proxy for early disease and/or less accurate sampling. Because of the very long incubation period in TSE, and the reliance on PrP Sc accumulation as a marker of infection, it is accepted that all active surveillance programs in healthy animals, regardless of species, will miss a proportion of animals incubating disease.

When samples from the preclinical cases in our study are considered (i.e., a sample that might be considered representative of field active surveillance), there is a spread of apparent test sensitivity, with the IDEXX HerdChek test formats displaying greater sensitivity than the Bio-Rad SG, and with the Bio-Rad TeSeE giving a lower sensitivity against a gold standard of either IHC or WB. This general ranking is supported by the data from other studies in both sheep and goats. [START_REF] Bozzetta | Comparative performance of three TSE rapid tests for surveillance in healthy sheep affected by scrapie[END_REF][START_REF] Kittelberger | Evaluation of two commercial, rapid, ELISA kits testing for scrapie in retro-pharyngeal lymph nodes in sheep[END_REF][START_REF] Meloni | EU-approved rapid tests might underestimate bovine spongiform encephalopathy infection in goats[END_REF][START_REF] Nappi | Interlaboratory trial on TSE rapid tests for the control of the Italian scrapie surveillance network[END_REF] Our results confirm that the stage of the incubation period at which an infected animal is tested determines the amount of PrP Sc available for detection and consequently the likelihood that any rapid test will correctly identify an infected animal. [START_REF] Arnold | Estimating the temporal relationship between PrP Sc detection and incubation period in experimental bovine spongiform encephalopathy of cattle[END_REF][START_REF] Arnold | Pathogenesis of experimental bovine spongiform encephalopathy (BSE): estimation of tissue infectivity according to incubation period[END_REF][START_REF] Balkema-Buschmann | Pathogenesis of classical and atypical BSE in cattle[END_REF][START_REF] Corbière | The limits of test-based scrapie eradication programs in goats[END_REF][START_REF] González | High prevalence of scrapie in a dairy goat herd: tissue distribution of disease-associated PrP and effect of PRNP genotype and age[END_REF][START_REF] Ortiz-Pelaez | Allelic variants at codon 146 in the PRNP gene show significant differences in the risk for natural scrapie in Cypriot goats[END_REF][START_REF] Reckzeh | Rapid testing leads to the underestimation of the scrapie prevalence in an affected sheep and goat flock[END_REF][START_REF] Tongue | Prevalence of scrapie infection in cull animals from 14 scrapie-affected flocks in Great Britain[END_REF][START_REF] Wells | Preliminary observations on the pathogenesis of experimental bovine spongiform encephalopathy (BSE): an update[END_REF] A considerable proportion of preclinical animals are likely to be missed in large-scale surveillance. We compared sub-aliquots of the same finely chopped tissue samples, allowing the consideration of test performance specifically, without the confounding effect of sample location.

Each test uses a capture antibody or ligand, and then a detection antibody raised against a different epitope of the PrP molecule. Historically, many monoclonal antibodies (mAbs) have been raised against PrP from several animal species. Given the difficulty of raising an antibody against a host protein, synthetic peptides have often been used as the antigen. This means that some antibodies recognize linear epitopes, whereas others, raised against whole protein, may recognize conformational epitopes based on the 3D structure of the abnormal isoform of the protein. Test manufacturers do not publicly declare which antibodies are used in their tests. It is possible that if a polymorphism is present within the epitope recognized by a test antibody, the binding of that antibody and the subsequent sensitivity of the test may be compromised. This effect has already been documented in the context of mAb F99/97.6.1, an antibody that has a binding epitope that aligns with PRNP codons 220-225, [START_REF] Dassanayake | Accumulation profiles of PrP(Sc) in hemal nodes of naturally and experimentally scrapie-infected sheep[END_REF][START_REF] Madsen-Bouterse | PRNP variants in goats reduce sensitivity of detection of PrP(Sc) by immunoassay[END_REF] and is widely used for TSE confirmation in a range of species. Chronic wasting disease-infected mule deer homozygous for phenylalanine at codon 225 (F225F), showed relatively poor staining when immunolabeled with F99, [START_REF] Wolfe | Atypical" chronic wasting disease in PRNP genotype 225FF mule deer[END_REF] as did goats polymorphic for lysine at codon 222 (K222). [START_REF] Madsen-Bouterse | PRNP variants in goats reduce sensitivity of detection of PrP(Sc) by immunoassay[END_REF][START_REF] Vaccari | State-of-the-art review of goat TSE in the European Union, with special emphasis on PRNP genetics and epidemiology[END_REF] Polymorphism at codon 143 has also been reported to result in a reduced Seven of the 29 reported amino acid changes in the goat PRNP gene appear to have worldwide distribution by haplotype ranking, from most common: proline at codon 240 (P240), arginine at codon 143 (R143), serine at codon 127 (S127), histidine at codon 154 (H154), lysine at codon 222 (K222), glutamine at codon 211 (Q211), methionine at codon 142 (M142), and serine or aspartic acid at codon 146 (S/D 146). All of these polymorphisms were represented in the sample panel, but only 2 (M142 and Q211) were available in sufficient numbers to allow statistical analysis. Although polymorphisms at codon 211 had no effect on test outcome, relative test performance was different in animals homozygous or heterozygous for M codon 142 compared to the wild type (II), but this effect was not absolute, or statistically significant. The presence of M at codon 142 does appear to compromise to some extent the sensitivity of the Bio-Rad tests.

The only absolute effect on test performance was seen in the samples from the goats of the experimentally challenged Cypriot panel that were polymorphic for serine or aspartic acid at codon 146, in which the performance of the Bio-Rad tests was profoundly affected. Aspartic acid, in particular, prevented the binding of at least one test antibody and resulted in false-negative results for samples from animals of this genotype. When these samples were further analyzed by western blot (data not shown; Chaplin, Papasavva-Stylianou, Simmons, Georgiadou, pers. comm., 2012) the blots showed a loss of binding, with substantial reduction of immunoreactivity with the N-terminal mAb P4 89-104 amino acid sequence of ovine PrP (which is a BSE-like outcome) in animals that are heterozygous at codon 146 or are homozygous for SS. In principle, this could confound discriminatory testing, but to date no naturally occurring cases of scrapie have been identified in animals of this genotype. [START_REF] Georgiadou | Goats with aspartic acid or serine at codon 146 of the PRNP gene remain scrapie-negative after lifetime exposure in affected herds in Cyprus[END_REF] Our data show that a polymorphism may adversely affect one test and not another, depending on how the polymorphism might affect the linear or conformational epitopes recognized by the mAbs or ligands used in the test, and underline the dangers of extrapolating from other studies. TSE tests should always be assessed specifically for the populations in which they are to be applied, regarding species and the full range of host genetics within a species (where known), to minimize the possibility of false-negative test results.

Figure 1 .

 1 Figure 1. Comparative test performance by genotype for the Cypriot experimental sample panel. The IDEXX SR (A) and the IDEXX BOV (B) tests both performed well regardless of the host genotype, and only failed to detect disease-specific levels of PrP in the 2 animals that were preclinical intercurrent deaths. Bio-Rad SG (C) and Bio-Rad TeSeE (D) data 25 are shown for comparison. Filled circles indicate positive test results from transmissible spongiform encephalopathy (TSE)-positive animals; open circles indicate TSE-negative test results derived from TSE-negative animals; half-filled circles indicate TSE-negative test results associated with TSE-positive animals.

Table 1 .

 1 Test sensitivity (when applied to brain tissue) relative to confirmatory methods. Published data from other studies are summarized in the lower half of the table for ease of comparison. All positive samples from clinically normal as well as suspect and unknown clinical status animals.† No. of test-positive animals/no. of TSE-positive animals. ‡ Confidence intervals of the proportions (sensitivity). §Includes animals positive only in lymphoreticular tissues. ¦Sensitivity is reported as 100% for animals showing clinical signs.

				Test sensitivity
	Reference	n	IDEXX BOV	IDEXX SR	Bio-Rad SG	Bio-Rad TeSeE
	Current study
	Total sample set*	128	108/128 † (84.4%)	106/128 (82.8%)	94/128 (73.4 %)	55/73 (75.3%)
			(77.1-89.6% ‡)	(75.3-88.4%)	(65.1-80.3%)	(64.3-83.8%)
	Clinically affected	41	41/41 (100%)	41/41 (100%)	41/41 (100%)	40/41 (97.4%)
			(91.4-100%)	(91.4-100%)	(91.4-100%)	(87.4-99.5%)
	Preclinical animals or of unknown clinical	87	65/87 (74.7%)	65/87 (74.7%)	53/87 (60.9%)	15/32 (46.8%)
	status		(64.6-82.6%)	(64.6-82.6%)	(50.4-70.5%)	(30.8-63.5%)
	UK herd cull 16	34				56%*
		68				27.9% §
	French surveillance
	Lab 1 8	180			47.3%* §
	Lab 2 8				58.4%* §
	European scrapie cases 21	61		96.7%¦	83.6%¦	85.2%¦

*

Table 2 .

 2 Estimation of test sensitivity (brain).

	Genotype	IDEXX BOV	IDEXX SR	Bio-Rad SG	Bio-Rad TeSeE
	Codon 142 (all goats: 128)
	II142	78/95 (82.1%)
		(73.2-88.5%)

Acknowledgments

A. Ortiz-Pelaez is employed by the European Food Safety Authority (EFSA). The present article is published under the sole responsibility of the authors and may not be considered as an EFSA scientific output. The positions and opinions presented in this article are those of the authors alone and do not necessarily represent the views or scientific works of EFSA.

Funding

This work was funded by the EC and Defra through the activities of the EURL for TSE at APHA. The UMR INRA ENVT 1225 was funded for this work by the EU FEDER/INTERREG (EFA148/16 REDPRION)

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplementary material

Supplementary material for this article is available online.